forked from Hostea/website
152 lines
8.2 KiB
Markdown
152 lines
8.2 KiB
Markdown
|
+++
|
||
|
|
title = "[tutorial] A gentle introduction to the gitea doctor"
|
||
|
|
date = 2022-06-14
|
||
|
|
description = "The doctor command is useful to check the health of a running Gitea instance. Specially after performing an upgrade."
|
||
|
|
[taxonomies]
|
||
|
|
tags = ['hostea', 'gitea', 'troubleshoot', 'problem', 'tutorial']
|
||
|
|
|
||
|
|
[extra]
|
||
|
|
author = 'dachary'
|
||
|
|
+++
|
||
|
|
|
||
|
|
While helping people with their upgrades [in the Gitea forum](https://discourse.gitea.io/t/migration-from-1-2-to-1-16-8/5309) or [at the Hostea clinic](https://forum.hostea.org/t/gitea-upgrade-from-1-14-1-to-1-16-8/90), I realized that few Gitea admins know about the [`gitea doctor`](https://docs.gitea.io/en-us/command-line/#doctor) command and decided to write this blog post as a gentle introduction.
|
||
|
|
|
||
|
|
### An apple a day keeps the doctor away
|
||
|
|
|
||
|
|
Or in our case, Gitea versions [below 1.11.5](https://github.com/go-gitea/gitea/blob/v1.11.5/cmd/doctor.go). Since then, the `gitea doctor` is available and is designed to run against a specific Gitea version. It would not be a good idea to try to run the doctor from Gitea 1.16 to verify the sanity of a Gitea 1.2 instance: it will be confused by how the database is organized and a number of other details. Historical fun fact: the `gitea doctor` was backported to [Gitea 1.10.5](https://github.com/go-gitea/gitea/blob/v1.10.5/cmd/doctor.go) and [Gitea 1.10.6](https://github.com/go-gitea/gitea/blob/v1.10.6/cmd/doctor.go) and may be of help if you run this particular version and are facing the problem that motivated the backport.
|
||
|
|
|
||
|
|
With each version `gitea doctor` improves and gains new capabilities. For instance, in Gitea 1.17 it becomes aware of [orphaned pull requests](https://github.com/go-gitea/gitea/pull/19731) and is able to fix them. If such a problem exists in Gitea 1.16, it does not know about it.
|
||
|
|
|
||
|
|
### Calling the doctor
|
||
|
|
|
||
|
|
In the following, examples are based on a Gitea 1.16.8 instance you can run as follows:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker run --name gitea -p 3000:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||
|
|
$ docker exec gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||
|
|
$ docker exec gitea mkdir /var/lib/gitea/data/log
|
||
|
|
```
|
||
|
|
|
||
|
|
And then you can go to the [web interface](https://127.0.0.1:3000/) to create a `test` repository, with an initial `README.md` file. When this is done the doctor can be called as follows:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker exec gitea gitea doctor --all
|
||
|
|
[1] Check paths and basic configuration
|
||
|
|
- [I] Configuration File Path: "/etc/gitea/app.ini"
|
||
|
|
- [I] Repository Root Path: "/var/lib/gitea/git/repositories"
|
||
|
|
- [I] Data Root Path: "/var/lib/gitea"
|
||
|
|
- [I] Custom File Root Path: "/var/lib/gitea/custom"
|
||
|
|
- [I] Work directory: "/var/lib/gitea"
|
||
|
|
- [I] Log Root Path: "/var/lib/gitea/data/log"
|
||
|
|
OK
|
||
|
|
[2] Check if there is garbage storage files
|
||
|
|
OK
|
||
|
|
[3] Check Database Version
|
||
|
|
OK
|
||
|
|
[4] Check consistency of database
|
||
|
|
OK
|
||
|
|
[5] Check if user with wrong type exist
|
||
|
|
OK
|
||
|
|
[6] Check if OpenSSH authorized_keys file is up-to-date
|
||
|
|
OK
|
||
|
|
[7] Check if SCRIPT_TYPE is available
|
||
|
|
- [I] ScriptType bash is on the current PATH at /bin/bash
|
||
|
|
OK
|
||
|
|
[8] Check if hook files are up-to-date and executable
|
||
|
|
OK
|
||
|
|
[9] Recalculate Stars number for all user
|
||
|
|
OK
|
||
|
|
[10] Check old archives
|
||
|
|
- [I] 0 old archives in repository need to be deleted
|
||
|
|
OK
|
||
|
|
[11] Enable push options
|
||
|
|
- [I] Checked 1 repositories, 0 need updates.
|
||
|
|
OK
|
||
|
|
[12] Check for incorrectly dumped repo_units (See #16961)
|
||
|
|
- [W] Found 0 broken repo_units
|
||
|
|
OK
|
||
|
|
[13] Recalculate merge bases
|
||
|
|
- [W] 0 PRs with incorrect mergebases of 0 PRs total in 1 repos
|
||
|
|
OK
|
||
|
|
[14] Check git-daemon-export-ok files
|
||
|
|
- [I] Checked 1 repositories, 0 need updates.
|
||
|
|
```
|
||
|
|
|
||
|
|
### What does the doctor know?
|
||
|
|
|
||
|
|
Although the `doctor` can be compared to [fsck(8)](https://en.wikipedia.org/wiki/Fsck), it does not know everything. It took decades for `fsck` to become the ultimate authority on finding problems on file systems and reliably fixing them without losing data. Nowadays, only a handful of people in the world are brave enough to manually attempt a file system recovery when `fsck` cannot recover from a data loss.
|
||
|
|
|
||
|
|
The first `doctor` version is two years old and Gitea admins are still routinely running SQL queries against the database or moving files around when trying to figure out why a Gitea instance is not behaving as it should. It is however worth checking if the doctor does not already have a solution by listing all it can do:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker exec gitea gitea doctor --list
|
||
|
|
Default Name Title
|
||
|
|
* paths Check paths and basic configuration
|
||
|
|
storages Check if there is garbage storage files
|
||
|
|
* check-db-version Check Database Version
|
||
|
|
check-db-consistency Check consistency of database
|
||
|
|
* check-user-type Check if user with wrong type exist
|
||
|
|
* authorized-keys Check if OpenSSH authorized_keys file is up-to-date
|
||
|
|
script-type Check if SCRIPT_TYPE is available
|
||
|
|
hooks Check if hook files are up-to-date and executable
|
||
|
|
recalculate-stars-number Recalculate Stars number for all user
|
||
|
|
check-old-archives Check old archives
|
||
|
|
enable-push-options Enable push options
|
||
|
|
fix-broken-repo-units Check for incorrectly dumped repo_units (See #16961)
|
||
|
|
recalculate-merge-bases Recalculate merge bases
|
||
|
|
check-git-daemon-export-ok Check git-daemon-export-ok files
|
||
|
|
```
|
||
|
|
|
||
|
|
And then call the `check` that looks interesting:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker exec gitea gitea doctor --run authorized-keys
|
||
|
|
[1] Check if OpenSSH authorized_keys file is up-to-date
|
||
|
|
OK
|
||
|
|
```
|
||
|
|
|
||
|
|
The challenge is to figure out which `check` does what and at the moment the best source of information is ... [the sources](https://github.com/go-gitea/gitea/tree/v1.16.8) themselves. The [doctor.go](https://github.com/go-gitea/gitea/blob/v1.16.8/cmd/doctor.go) command is the entry point and [the doctor directory](https://github.com/go-gitea/gitea/tree/v1.16.8/modules/doctor) contains the rest.
|
||
|
|
|
||
|
|
Some `checks` are straightforward to understand, even if you do not know Go, such as [the authorized-keys check](https://github.com/go-gitea/gitea/blob/v1.16.8/modules/doctor/authorizedkeys.go). Others are much more involved and your best chance is to [ask the Gitea chatroom](https://matrix.to/#/#gitea:matrix.org) for help.
|
||
|
|
|
||
|
|
### Is it going to hurt?
|
||
|
|
|
||
|
|
By default the doctor (very much like `fsck -N`) only performs non destructive checks and displays diagnostics, with an indication of how serious the problem is. In the example above, there only are lines with **[I]** (which indicates an information) and **[W]** which indicates a warning that can be ignored but may be worth looking into. Those two warnings are actually just informational and should be labelled as **[I]**, [which has been fixed](https://github.com/go-gitea/gitea/pull/19836) in a more recent version of the doctor.
|
||
|
|
|
||
|
|
Now let's do something bad: remove the permissions from a hook in our repository:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker exec gitea chmod -x /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive
|
||
|
|
```
|
||
|
|
|
||
|
|
Run the doctor with the `check` supposed to find that out:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker exec gitea gitea doctor --run hooks
|
||
|
|
[1] Check if hook files are up-to-date and executable
|
||
|
|
- [W] old hook file /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive is not executable
|
||
|
|
```
|
||
|
|
|
||
|
|
Ask it to fix this with the `--fix` flag:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker exec gitea gitea doctor --run hooks --fix
|
||
|
|
[1] Check if hook files are up-to-date and executable
|
||
|
|
- [W] Regenerated hooks for root/test
|
||
|
|
- [W] old hook file /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive is not executable
|
||
|
|
```
|
||
|
|
|
||
|
|
And run it one last time to check all is well:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ docker exec gitea gitea doctor --run hooks
|
||
|
|
[1] Check if hook files are up-to-date and executable
|
||
|
|
OK
|
||
|
|
```
|
||
|
|
|
||
|
|
Even when the doctor is unable to fix a problem, it can help by showing extensive debug output which can be found, by default, in the `doctor.log` file in the directory from which it runs. Or it can be displayed on the standard output with `--log-file -`, which is most convenient when running in docker.
|
||
|
|
|
||
|
|
### Going further
|
||
|
|
|
||
|
|
If that was helpful to you, I would very much appreciate if you [send me a message on Mastodon](https://mastodon.online/@dachary). It will encourage me to write more blog posts to share what I learn about Gitea. Even better: you could [send a pull request](https://github.com/go-gitea/gitea/pulls) to improve the doctor and help it mature.
|