- troubleshoot

1.17 breaking changes episode 2: preserving a custom gitconfig

2022-07-20T00:00:00+00:00

On June 21st, 2022 1.17.0-rc1 was published and the location of the gitconfig file moved to a new location, which required manual intervention</a>. This change impacted a large number of Gitea installations because the docker image tag latest was set to 1.17.0-rc1</a> by accident. As a result, about 10,000 pulls per hour from the docker hub got the release candidate instead of the expected stable version.

Unfortunately moving the git home directory in 1.17.0-rc1 was implemented in way that created a security problem. The fix that was merged in Gitea</a> to fix it requires moving the gitconfig file and was released July 19th, 2022 in 1.17.0-rc2.

This would have been a minor inconvenience if it only has an impact on adventurous people trying the release candidate in a test environment. But since all Gitea production installations based on the latest tag were inadvertently upgraded to 1.17.0-rc1, the admins who moved their custom .gitconfig will need to move it one more time when upgrading to 1.17.0-rc2.

In 1.17.0-rc2, a custom .gitconfig must be moved manually to the new git home directory</a> as follows:

Figure out the directory where $HOME/.gitconfig</code> must be moved by running the doctor</a>:</li> </ul> $ gitea --work-path /app/gitea -c /data/gitea/conf/app.ini doctor [1] Check paths and basic configuration - [I] Configuration File Path: "/data/gitea/conf/app.ini" - [I] Repository Root Path: "/data/git/repositories" - [I] Data Root Path: "/data/gitea" - [I] Custom File Root Path: "/data/gitea" - [I] Work directory: "/app/gitea" - [I] Log Root Path: "/data/gitea/log" OK </code></pre> Copy the $HOME/.gitconfig</code> file to the Data Root Path/home (which is /data/gitea/home</code> in the example above).</li> </ul>

1.17 breaking changes episode 1: preserving a custom gitconfig 2022-06-22T00:00:00+00:00 Before version 1.17, when Gitea needed to change the git configuration</a>, it modified the $HOME/.gitconfig</code> file. For instance it would set core.quotePath to false</a>: [core] quotePath = false </code></pre> When installing Gitea from docker</a> or rootless</a> or even from binary</a> this $HOME/.gitconfig</code> file belongs to a user that is dedicated to Gitea</a> and not used by anyone else. However, if an Gitea installation was done differently and $HOME/.gitconfig</code> has been customized because it is shared by a user or another application, there is a good chance that manual modifications were done such as: [user] name = Jane Doe email = jane@doe.com </code></pre> It is also possible that the file was modified manually by the Gitea admin for other reasons. In both there is a potential for breakage when upgrading to Gitea >= 1.17 because the location of the file changed. It must be moved manually to the new location as follows: Figure out the directory where $HOME/.gitconfig</code> must be moved by running the doctor</a>:</li> </ul> $ gitea --work-path /app/gitea -c /data/gitea/conf/app.ini doctor [1] Check paths and basic configuration - [I] Configuration File Path: "/data/gitea/conf/app.ini" - [I] Repository Root Path: "/data/git/repositories" - [I] Data Root Path: "/data/gitea" - [I] Custom File Root Path: "/data/gitea" - [I] Work directory: "/app/gitea" - [I] Log Root Path: "/data/gitea/log" OK </code></pre> Copy the $HOME/.gitconfig</code> file to the Repository Root Path (which is /data/git/repositories</code> in the example above).</li> </ul> The reason why this breaking change was introduced is to workaround a rare problem</a> impacting Gitea installations relying on networked volumes. [tutorial] A gentle introduction to the gitea doctor 2022-06-14T00:00:00+00:00 While helping people with their upgrades in the Gitea forum</a> or at the Gna! clinic</a>, I realized that few Gitea admins know about the gitea doctor</code></a> command and decided to write this blog post as a gentle introduction. An apple a day keeps the doctor away#</a > </h3> Or in our case, Gitea versions below 1.11.5</a>. Since then, the gitea doctor</code> is available and is designed to run against a specific Gitea version. It would not be a good idea to try to run the doctor from Gitea 1.16 to verify the sanity of a Gitea 1.2 instance: it will be confused by how the database is organized and a number of other details. Historical fun fact: the gitea doctor</code> was backported to Gitea 1.10.5</a> and Gitea 1.10.6</a> and may be of help if you run this particular version and are facing the problem that motivated the backport. With each version gitea doctor</code> improves and gains new capabilities. For instance, in Gitea 1.17 it becomes aware of orphaned pull requests</a> and is able to fix them. If such a problem exists in Gitea 1.16, it does not know about it. Calling the doctor#</a > </h3> In the following, examples are based on a Gitea 1.16.8 instance you can run as follows: $ docker run --name gitea -p 3000:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless $ docker exec gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com $ docker exec gitea mkdir /var/lib/gitea/data/log </code></pre> And then you can go to the web interface</a> to create a test</code> repository, with an initial README.md</code> file. When this is done the doctor can be called as follows: $ docker exec gitea gitea doctor --all [1] Check paths and basic configuration - [I] Configuration File Path: "/etc/gitea/app.ini" - [I] Repository Root Path: "/var/lib/gitea/git/repositories" - [I] Data Root Path: "/var/lib/gitea" - [I] Custom File Root Path: "/var/lib/gitea/custom" - [I] Work directory: "/var/lib/gitea" - [I] Log Root Path: "/var/lib/gitea/data/log" OK [2] Check if there is garbage storage files OK [3] Check Database Version OK [4] Check consistency of database OK [5] Check if user with wrong type exist OK [6] Check if OpenSSH authorized_keys file is up-to-date OK [7] Check if SCRIPT_TYPE is available - [I] ScriptType bash is on the current PATH at /bin/bash OK [8] Check if hook files are up-to-date and executable OK [9] Recalculate Stars number for all user OK [10] Check old archives - [I] 0 old archives in repository need to be deleted OK [11] Enable push options - [I] Checked 1 repositories, 0 need updates. OK [12] Check for incorrectly dumped repo_units (See #16961) - [W] Found 0 broken repo_units OK [13] Recalculate merge bases - [W] 0 PRs with incorrect mergebases of 0 PRs total in 1 repos OK [14] Check git-daemon-export-ok files - [I] Checked 1 repositories, 0 need updates. </code></pre> What does the doctor know?#</a > </h3> Although the doctor</code> can be compared to fsck(8)</a>, it does not know everything. It took decades for fsck</code> to become the ultimate authority on finding problems on file systems and reliably fixing them without losing data. Nowadays, only a handful of people in the world are brave enough to manually attempt a file system recovery when fsck</code> cannot recover from a data loss. The first doctor</code> version is two years old and Gitea admins are still routinely running SQL queries against the database or moving files around when trying to figure out why a Gitea instance is not behaving as it should. It is however worth checking if the doctor does not already have a solution by listing all it can do: $ docker exec gitea gitea doctor --list Default Name Title * paths Check paths and basic configuration storages Check if there is garbage storage files * check-db-version Check Database Version check-db-consistency Check consistency of database * check-user-type Check if user with wrong type exist * authorized-keys Check if OpenSSH authorized_keys file is up-to-date script-type Check if SCRIPT_TYPE is available hooks Check if hook files are up-to-date and executable recalculate-stars-number Recalculate Stars number for all user check-old-archives Check old archives enable-push-options Enable push options fix-broken-repo-units Check for incorrectly dumped repo_units (See #16961) recalculate-merge-bases Recalculate merge bases check-git-daemon-export-ok Check git-daemon-export-ok files </code></pre> And then call the check</code> that looks interesting: $ docker exec gitea gitea doctor --run authorized-keys [1] Check if OpenSSH authorized_keys file is up-to-date OK </code></pre> The challenge is to figure out which check</code> does what and at the moment the best source of information is ... the sources</a> themselves. The doctor.go</a> command is the entry point and the doctor directory</a> contains the rest. Some checks</code> are straightforward to understand, even if you do not know Go, such as the authorized-keys check</a>. Others are much more involved and your best chance is to ask the Gitea chatroom</a> for help. Is it going to hurt?#</a > </h3> By default the doctor (very much like fsck -N</code>) only performs non destructive checks and displays diagnostics, with an indication of how serious the problem is. In the example above, there only are lines with [I] (which indicates an information) and [W] which indicates a warning that can be ignored but may be worth looking into. Those two warnings are actually just informational and should be labelled as [I], which has been fixed</a> in a more recent version of the doctor. Now let's do something bad: remove the permissions from a hook in our repository: $ docker exec gitea chmod -x /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive </code></pre> Run the doctor with the check</code> supposed to find that out: $ docker exec gitea gitea doctor --run hooks [1] Check if hook files are up-to-date and executable - [W] old hook file /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive is not executable </code></pre> Ask it to fix this with the --fix</code> flag: $ docker exec gitea gitea doctor --run hooks --fix [1] Check if hook files are up-to-date and executable - [W] Regenerated hooks for root/test - [W] old hook file /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive is not executable </code></pre> And run it one last time to check all is well: $ docker exec gitea gitea doctor --run hooks [1] Check if hook files are up-to-date and executable OK </code></pre> Even when the doctor is unable to fix a problem, it can help by showing extensive debug output which can be found, by default, in the doctor.log</code> file in the directory from which it runs. Or it can be displayed on the standard output with --log-file -</code>, which is most convenient when running in docker. Going further#</a > </h3> If that was helpful to you, I would very much appreciate if you send me a message on Mastodon</a>. It will encourage me to write more blog posts to share what I learn about Gitea. Even better: you could send a pull request</a> to improve the doctor and help it mature. [solved] Zombies created by Gitea 2022-06-04T00:00:00+00:00 Gitea can create zombies</a>, for instance if a Git mirror takes too long. When updating a mirror, Gitea relies on the git remote update</code> command which creates a child process, git-remote-https</code>, to fetch data from the remote repository. Gitea has an internal timeout that will kill the child process (e.g. git remote update</code>) when it takes too long but will not kill the grandchild. This grandchild will become an orphan and run forever or until its own timeout expires, which is about two minutes on git version 2.25. $ time git clone https://4.4.4.4 Clonage dans '4.4.4.4'... fatal: impossible d'accéder à 'https://4.4.4.4/': Failed to connect to 4.4.4.4 port 443: Connexion terminée par expiration du délai d'attente real 2m9,753s user 0m0,001s sys 0m0,009s </code></pre> As explained in the diagnostic blog post regarding Gitea zombies</a> there fortunately is a very simple way to avoid this by making sure each Gitea child is a process group leader</a>. That first step was introduced in Gitea 1.17</a> and backported to Gitea 1.16.9</a>. The actual bug fix can now be implemented. Using negative process id to kill children#</a > </h3> When Gitea timeout on a child, it relies on os.Process.Kill</a> which translates into a using the kill(2) system call to send a SIGKILL signal to unconditionally terminate it: kill(pid, SIGKILL)</code>. Using a negative pid with kill(-pid, SIGKILL)</code> will also terminate all processes created by Gitea's child, without Gitea knowing when or why they were created. From the kill(2) manual page: If pid is less than -1, then sig is sent to every process in the process group whose ID is -pid. </blockquote> Which is implemented as follows in the Friendly Forge Format library</a>: syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)</code> </blockquote> Not using the default Go CommandContext#</a > </h3> Since CommandContext</a> does not allow to send a signal to the negative pid of the child process, it has to be implemented by Gitea itself, in a way that is similar to how the Friendly Forge Format library</a> does it: ctxErr := watchCtx(ctx, cmd.Process.Pid) err = cmd.Wait() interruptErr := <-ctxErr // If cmd.Wait returned an error, prefer that. // Otherwise, report any error from the interrupt goroutine. if interruptErr != nil && err == nil { err = interruptErr } </code></pre> Testing the bug is fixed and stays fixed#</a > </h3> Long standing bugs that are difficult to reproduce manually such as this one require robust testing to ensure that: the diagnostic identifying the root cause is correct</li> the bug fix works</li> it does not resurface insidiously because of a subtle regression introduce years later</li> </ul> It is easy to implement as can be seen in the Friendly Forge Format library</a>. In a nutshell: git clone https://4.4.4.4</a> which will hang because of firewall rules</li> wait for the git-remote-https</a> grandchild process to be spawned</li> cancel the context and wait for the goroutine to terminate</a></li> verify the git-remote-https is killed</a></li> </ul> And with that... no more zombies! [diagnostic] Zombies created by Gitea 2022-06-02T00:00:00+00:00 The first issue about zombie processes</a> created by Gitea was reported in 2017 and resurfaced</a> on a regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie: Gitea updates a mirror by spawning the process git remote update</code></li> git remote update</code> spawns yet another process, git fetch</code></li> git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li> Gitea kill the process git remote update</code></li> When killed git remote update</code> does not kill its own child and git fetch</code> becomes an orphaned process which keeps running</li> When git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li> </ul> PID 1 process and waiting on orphans#</a > </h3> This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is /bin/init</code> and when Gitea runs from the default docker image</a> this is s6</code>: they will both wait on orphaned processes and there won't be any zombies. What if Gitea is the only running process?#</a > </h3> But when Gitea runs from the rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way: $ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless $ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com </code></pre> The git</code> command can then be replaced with a script that waits forever: $ ( echo -e '#!/bin/bash\nsleep infinity' ) | docker exec -i --user root gitea tee /usr/bin/git $ docker exec --user root gitea chmod +x /usr/bin/git </code></pre> Trying to create a repository from the web interface will create the conditions for a zombie to show: $ docker exec gitea ps -o ppid,pid,comm,args PPID PID COMMAND COMMAND 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web 1 94 sleep [sleep] 1 99 sleep [sleep] 1 111 sleep [sleep] 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare 164 165 sleep sleep infinity </code></pre> When the git</code> process is killed by Gitea, the sleep</code> child will be orphaned: $ docker exec gitea ps -o ppid,pid,comm,args PPID PID COMMAND COMMAND 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web 1 94 sleep [sleep] 1 99 sleep [sleep] 1 111 sleep [sleep] 1 165 sleep sleep infinity </code></pre> Killing it will turn it into a zombie: $ docker exec gitea kill 165 $ docker exec gitea ps -o ppid,pid,comm,args PPID PID COMMAND COMMAND 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web 1 94 sleep [sleep] 1 99 sleep [sleep] 1 111 sleep [sleep] 1 165 sleep [sleep] </code></pre> Killing a child process and all its children#</a > </h3> There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed. [solved] Gitea 1.15 and up: path not found or permission denied 2022-05-28T00:00:00+00:00 In Gitea 1.15 the app.example.ini</a> file was changed to comment out most of the values</a>. The assumption was that all values exactly matched the defaults in the source code</a>. However, there are differences, for instance for APP_DATA_PATH</a>. Before Gitea 1.15, app.example.ini</code> contained: APP_DATA_PATH = data </code></pre> and the path was relative to the directory from which the Gitea server was running. In Gitea 1.15 up to 1.16, it was commented out: ; APP_DATA_PATH = data </code></pre> and the path was relative to the work path directory, as provided either via the --work-path argument or the GITEA_WORK_DIR</code> environment variable. When a distribution such as voidlinux uses app.example.ini</a> as a base for the Gitea package, this change indirectly creates a regression and an upgrade of Gitea fails with errors</a> such as unable to open level db at data/data/queues/common: mkdir data: permission denied</code>. The regression did not show as soon as Gitea 1.15 became available in voidlinux because the package explicitly set APP_DATA_PATH</code></a>. But this changed when Gitea 1.15.6 was packaged</a> and once the value was commented out, upgrading triggered the problem. This was worked around six month later with the Gitea 1.16.8</a> package. The APP_DATA_PATH</code> directory is not the only one, the [log] ROOT_PATH</code> is another example. There is an ongoing effort</a> to improve the situation in Gitea 1.17. With the downside of introducing breaking changes that will have an impact on all Gitea installations because the content of the app.ini</code> file will be interpreted differently. In the case of APP_DATA_PATH</code>, both: APP_DATA_PATH = data </code></pre> and: ; APP_DATA_PATH = data </code></pre> will be interpreted to be relative to the work path directory, as provided either via the --work-path argument or the GITEA_WORK_DIR</code> environment variable. Every Gitea installation using APP_DATA_PATH = data will need to update the value to be an absolute path such as /var/lib/gitea/data so that it keeps pointing to the expected directory. In order to prepare for the change or ensure the consistency of all path, there fortunately is a very simple solution: always use absolute paths in the app.ini</code> configuration file. [solved] Gitea 1.16.[678] error: fatal: unsafe repository is owned by someone else 2022-05-15T00:00:00+00:00 April 12, 2022 version git v2.35.2</a> was released and addresses a security issue CVE-2022-24765</a>. It was backported to 2.30.3, v2.31.2, v2.32.1, v2.33.2, and v2.34.2 and published by distributions such as Debian GNU/Linux</a>, Alpine</a>. If Gitea runs as user foo</code>, calls a patched Git version and a parent directory of the git repositories is owned by a user other than foo</code>, it will fail with a message such as: Failed to open repository: Git/Data Error: exit status 128 - fatal: unsafe repository ('/data/git/repositories/git/data.git' is owned by someone else) </code></pre> This started to show in the past few weeks to users running the Gitea binary on Windows</a> who also independently installed git v2.36. And then to people running Gitea from snap</a>, on a Synology NAS</a> and then people running from Gitea docker images</a> which is based on Alpine</a>. Workarounds#</a > </h3> If using Gitea docker images</a>: upgrade to Gitea >=1.16.9</a> or 1.17, both have git >=2.36</li> git config --global --replace-all safe.directory '*'</code></li> </ul> </li> If the Gitea binary was installed independently of git, upgrade git to a version that is greater or equal to 2.36</a> and disable the security check entirely with: impersonate the user dedicated to Gitea</a> (usually git)</li> git config --global --replace-all safe.directory '*'</code></li> </ul> </li> </ul> Bug fix#</a > </h3> The bug fix</a> is for Gitea to ensure git config --global --replace-all safe.directory '*'</code> is set on its dedicated user</a> when it initializes. It is effective on the condition that the git CLI version is greater or equal to 2.36</a>. Bug fix rationale#</a > </h3> It is safe to disable the security check in Gitea</a>. It is not vulnerable to CVE-2022-24765</a> because it calls the git CLI after changing its working directory</a> to be the git repository targeted by the command (for instance diff</a>) or a temporary directory. Therefore it will not explore the parent directories looking for a git configuration file. The security check is triggered because the repository is owned by an unexpected user (root instead of git typically) and not because a parent directory is owned by an unexpected user. This, in itself, is a problem worth investigating but it is unrelated and was revealed by the newer security check of git even though it does not match the threat described in CVE-2022-24765</a>. It appears non trivial to enforce a consistent ownership of files and directories, either within docker or outside docker when networked file systems are involved. The Gitea server was not troubled by this inconsistency so far because the permissions allow it to write and read where expected, regardless of the owner. It is not worth looking into but it is ancient and unrelated. Gitea runs under a dedicated user, either when installed from binary</a> or from docker</a> and modifies the global git configuration</a> depending on the git version at initialization time. Fixing the problem can therefore be done by disabling the security check in the global git config file at initialization time</a>. It also requires a minimum version of git 2.36 to be installed, which is the case for Gitea docker images with versions >= 1.16.9</a>. [solved] blank or error 500 page after login 2022-05-08T00:00:00+00:00 The instructions to upgrade a Gitea instance</a> only require three to four steps. They work fine most of the time but the documentation is lacking a "Troubleshooting" section to help out when something goes wrong. Maintaining instructions on how to diagnose and fix upgrade problems is an ambitious undertaking and requires updates every time a new case is discovered. An inventory of the known upgrade issues</a> was started to figure out how to structure such a section in the documentation. The release notes</a> were analyzed all the way back to Gitea 1.9.6</a> and the work is still in progress. Here is a sample of the tips that will be included: Upgrade directly to the latest Gitea version, there is no need to upgrade to intermediate versions.</li> If the upgrade from version x.y to version x.y+2 fails and there is a need to narrow down the problem, try upgrading to the latest minor version of each major version and verify it works.</li> etc.</li> </ul> However, even with the best documentation, someone will eventually run into an new problem and fixing it without compromising the integrity of the data will be challenging. This is best demonstrated by a real world example that was concluded a few days ago. Getting help from the community#</a > </h1> After upgrading a Gitea intsance from 1.9.6 to 1.16.5</a> the tests conducted manually did not uncover any problem. However, after going to production, some users saw a blank page after login and had to manually type the URL of the project they wanted to see in the browser. The person in charge of the upgrade never had to diagnose Gitea problem and reached out in the Gitea forum</a>. Tip: explain the problem in a public forum as early as possible to get help from the community </blockquote> In their post in the forum they explained how they attempted to diagnose the problem and how why they thought that only users created a few years ago were impacted. It was a detailed analysis that was concluded with a partial copy of the logs. It was unfortunately missing key information</a> that was provided only three days later. In the meantime, as they could not figure out the source of the problem, they were on the verge of accepting the loss of all the Gitea database</a> and start over from the repositories. However, once all the details were available, a workaround</a> was suggested in the forum. Tip: focus more on providing detailed facts than exposing the attempted diagnostic </blockquote> There was hope to fix Gitea and in the following days they applied the workaround. They also tried to improve it but without success and eventually accepted a partial data loss as inevitable and reported their success back to the forum</a>. Tip: when getting support from the community, providing feedback is the best token of appreciation </blockquote> Getting professional help#</a > </h1> The Gna! Clinic</a> is a collective of individual and companies that provides professional services to Gitea admins. They are active members of the Gitea community who help out</a> as volunteers. They can also be hired to resolve the more complicated cases. The Gitea instance that was in trouble required more than a few minutes of work and access to the database content for a proper diagnostic. They proposed their assistance</a> but although well received</a>, it was not accepted. When the Gitea admin explained how they chose to resolve the problem on the forum</a>, it confirmed the workaround was viable and the root problem was identified. That was enough to figure out a fix for the underlying bug with a rather simple patch</a> that was merged and backported</a> in the following days. But it happened too late to avoid the data loss. To summarize with a timeline, here is what happened: J+1: The problem is discovered by users who see a blank page after login and a the Gitea admin tries to diagnose the problem</li> J+2: A message is sent to ask for help in the community</li> J+2 to J+6: Three people in the community suggest ideas but the Gitea admin cannot figure out the root cause and is on the verge of accepting the loss of all Gitea data and restart from the git repositories</li> J+6: A workaround is suggested by the community</li> J+7 to J+17: The Gitea admin applies the workaround and only looses part of the Gitea data</li> </ul> And in retrospect, here is what could have happened instead: J+1: The problem is discovered by users who see a blank page after login</li> J+1: The Gitea admin reaches out to someone at the Gna! Clinic</a></li> J+2: The logs of the Gitea instance</a> are analyzed, the root cause diagnosed and a patch</a> is created to fix it.</li> J+3: If necessary a Gitea binary is created with the patch and used as a temporary replacement until the next point release is published with the backport</a>. The Gitea admin runs the patched Gitea binary in the meantime. There is no data loss.</li> </ul> It does not mean all upgrade problems can be resolved so easily. But it shows, with an example, that in some cases it makes sense to get professional help.

- troubleshoot

1.17 breaking changes episode 2: preserving a custom gitconfig

1.17 breaking changes episode 1: preserving a custom gitconfig

[tutorial] A gentle introduction to the gitea doctor

[solved] Zombies created by Gitea

Testing the bug is fixed and stays fixed#</span></a > </h3> Long standing bugs that are difficult to reproduce manually such as this one require robust testing to ensure that:</p> the diagnostic identifying the root cause is correct</li> the bug fix works</li>

[diagnostic] Zombies created by Gitea

[solved] Gitea 1.15 and up: path not found or permission denied

[solved] Gitea 1.16.[678] error: fatal: unsafe repository is owned by someone else

[solved] blank or error 500 page after login