studio/tuwunel
2
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Add external repair tool step to db repair troubleshooting.

Browse Source 
Signed-off-by: Jason Volk <jason@zemos.net>
This commit is contained in:
Jason Volk
2025-09-04 19:24:59 +00:00
parent a164da8b58
commit 3225db0ddd
1 changed files with 25 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
docs/troubleshooting.md
View File

@@ -99,12 +99,16 @@ to a savage state. This guide has been simplified into a set of universal steps
which everyone can follow from the top until they have recovered or reach the which everyone can follow from the top until they have recovered or reach the
end. The details and implications will be explained within each step. end. The details and implications will be explained within each step.
> [!NOTE] > [!TIP]
> All command-line `-O` options can be expressed as environment variables or in > All command-line `-O` options can be expressed as environment variables or in
> the config file based on your deployment's requirements. Note that > the config file based on your deployment's requirements. Note that
> `--maintenance` is only available on the command-line, but is equivalent to > `--maintenance` is only available on the command-line, but is equivalent to
> configuring `startup_netburst = false` and `listening = false`. > configuring `startup_netburst = false` and `listening = false`.
> [!IMPORTANT]
> Always create a backup of the database before running any operation. This is
> critical for steps 3 and above.
0. Start the server with the following options: 0. Start the server with the following options:
`tuwunel --maintenance -O rocksdb_recovery_mode=0` `tuwunel --maintenance -O rocksdb_recovery_mode=0`
@@ -139,7 +143,7 @@ loss, but it is more likely than above that clients may need to clear-cache
3. Start the server in Skip-Any-Corrupted-Record mode: 3. Start the server in Skip-Any-Corrupted-Record mode:
> [!CAUTION] > [!WARNING]
> Salvage mode potentially impacting the application's ability to function. > Salvage mode potentially impacting the application's ability to function.
> We cannot provide any further support for users who have entered this mode. > We cannot provide any further support for users who have entered this mode.
@@ -154,10 +158,15 @@ keys, etc, in a salvage effort and prepare to reinstall.
4. Start the server in repair mode. 4. Start the server in repair mode.
> [!CAUTION] > [!WARNING]
> Salvage mode potentially impacting the application's ability to function. > Salvage mode potentially impacting the application's ability to function.
> We cannot provide any further support for users who have entered this mode. > We cannot provide any further support for users who have entered this mode.
> [!CAUTION]
> Always create a backup of the database before entering this mode. The repair
> is not configurable and not interactive. It may automatically remove more
> data than anticipated, preventing further salvage efforts.
`tuwunel --maintenance -O rocksdb_repair=true` `tuwunel --maintenance -O rocksdb_repair=true`
For corruption affecting the bulk database tables not covered by any journal. For corruption affecting the bulk database tables not covered by any journal.
@@ -167,6 +176,19 @@ records were dropped, such as some historical records which are no longer
essential. Nevertheless the impact of this operation is impossible to assess essential. Nevertheless the impact of this operation is impossible to assess
and a successful recovery should be used to salvage data prior to reinstall. and a successful recovery should be used to salvage data prior to reinstall.
5. Utilize an external repair tool.
> [!WARNING]
> Salvage mode potentially impacting the application's ability to function.
> We cannot provide any further support for users who have entered this mode.
1. `git clone https://github.com/facebook/rocksdb`
2. `make -j$(nproc) ldb`
3. `./ldb repair --db=/var/lib/tuwunel/ 2>./repair-log.txt`
For situations when the repair mode in step 4 is not successful or produces
unexpected results.
## Debugging ## Debugging
Note that users should not really be debugging things. If you find yourself Note that users should not really be debugging things. If you find yourself