Add external repair tool step to db repair troubleshooting.

Signed-off-by: Jason Volk <jason@zemos.net>
This commit is contained in:
Jason Volk
2025-09-04 19:24:59 +00:00
parent a164da8b58
commit 3225db0ddd

View File

@@ -99,12 +99,16 @@ to a savage state. This guide has been simplified into a set of universal steps
which everyone can follow from the top until they have recovered or reach the
end. The details and implications will be explained within each step.
> [!NOTE]
> [!TIP]
> All command-line `-O` options can be expressed as environment variables or in
> the config file based on your deployment's requirements. Note that
> `--maintenance` is only available on the command-line, but is equivalent to
> configuring `startup_netburst = false` and `listening = false`.
> [!IMPORTANT]
> Always create a backup of the database before running any operation. This is
> critical for steps 3 and above.
0. Start the server with the following options:
`tuwunel --maintenance -O rocksdb_recovery_mode=0`
@@ -139,7 +143,7 @@ loss, but it is more likely than above that clients may need to clear-cache
3. Start the server in Skip-Any-Corrupted-Record mode:
> [!CAUTION]
> [!WARNING]
> Salvage mode potentially impacting the application's ability to function.
> We cannot provide any further support for users who have entered this mode.
@@ -154,10 +158,15 @@ keys, etc, in a salvage effort and prepare to reinstall.
4. Start the server in repair mode.
> [!CAUTION]
> [!WARNING]
> Salvage mode potentially impacting the application's ability to function.
> We cannot provide any further support for users who have entered this mode.
> [!CAUTION]
> Always create a backup of the database before entering this mode. The repair
> is not configurable and not interactive. It may automatically remove more
> data than anticipated, preventing further salvage efforts.
`tuwunel --maintenance -O rocksdb_repair=true`
For corruption affecting the bulk database tables not covered by any journal.
@@ -167,6 +176,19 @@ records were dropped, such as some historical records which are no longer
essential. Nevertheless the impact of this operation is impossible to assess
and a successful recovery should be used to salvage data prior to reinstall.
5. Utilize an external repair tool.
> [!WARNING]
> Salvage mode potentially impacting the application's ability to function.
> We cannot provide any further support for users who have entered this mode.
1. `git clone https://github.com/facebook/rocksdb`
2. `make -j$(nproc) ldb`
3. `./ldb repair --db=/var/lib/tuwunel/ 2>./repair-log.txt`
For situations when the repair mode in step 4 is not successful or produces
unexpected results.
## Debugging
Note that users should not really be debugging things. If you find yourself