721 lines
99 KiB
XML
721 lines
99 KiB
XML
|
<?xml version="1.0" encoding="UTF-8"?>
|
||
|
|
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
|
||
|
|
<title> - gna</title>
|
||
|
|
<link href="https://gna.org/tags/gna/atom.xml" rel="self" type="application/atom+xml"/>
|
||
|
|
<link href="https://gna.org"/>
|
||
|
|
<generator uri="https://www.getzola.org/">Zola</generator>
|
||
|
|
<updated>2022-07-20T00:00:00+00:00</updated>
|
||
|
|
<id>https://gna.org/tags/gna/atom.xml</id>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>1.17 breaking changes episode 2: preserving a custom gitconfig</title>
|
||
|
|
<published>2022-07-20T00:00:00+00:00</published>
|
||
|
|
<updated>2022-07-20T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/1-17-breaking-episode-2/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/1-17-breaking-episode-2/</id>
|
||
|
|
<content type="html"><p>On June 21st, 2022 1.17.0-rc1 was published and the location of the gitconfig file moved to a new location, <a href="2022-06-23-1.17-breaking-episode-1">which required manual intervention</a>. This change impacted a large number of Gitea installations because the docker image tag <strong>latest</strong> <a href="https://mastodon.online/@hostea/108514134565401798">was set to 1.17.0-rc1</a> by accident. As a result, about 10,000 pulls per hour from the docker hub got the release candidate instead of the expected stable version.</p>
|
||
|
|
<p>Unfortunately moving the git home directory in 1.17.0-rc1 was implemented in way that created a security problem. The <a href="https://github.com/go-gitea/gitea/pull/20114">fix that was merged in Gitea</a> to fix it requires moving the gitconfig file and was released July 19th, 2022 in 1.17.0-rc2.</p>
|
||
|
|
<p>This would have been a minor inconvenience if it only has an impact on adventurous people trying the release candidate in a test environment. But since all Gitea production installations based on the <strong>latest</strong> tag were inadvertently upgraded to 1.17.0-rc1, the admins who moved their custom .gitconfig will need to move it one more time when upgrading to 1.17.0-rc2.</p>
|
||
|
|
<p>In 1.17.0-rc2, a custom .gitconfig must be moved manually to the <a href="https://docs.gitea.io/en-us/config-cheat-sheet/#git-git">new git home directory</a> as follows:</p>
|
||
|
|
<ul>
|
||
|
|
<li>Figure out the directory where <code>$HOME/.gitconfig</code> must be moved by <a href="https://gna.org/blog/gentle-introduction-to-the-doctor/">running the doctor</a>:</li>
|
||
|
|
</ul>
|
||
|
|
<pre data-lang="shell" style="background-color:#2b303b;color:#c0c5ce;" class="language-shell "><code class="language-shell" data-lang="shell"><span>$ gitea --work-path /app/gitea -c /data/gitea/conf/app.ini doctor
|
||
|
|
</span><span>[1] Check paths and basic configuration
|
||
|
|
</span><span> - [I] Configuration File Path: &quot;/data/gitea/conf/app.ini&quot;
|
||
|
|
</span><span> - [I] Repository Root Path: &quot;/data/git/repositories&quot;
|
||
|
|
</span><span> - [I] Data Root Path: &quot;/data/gitea&quot;
|
||
|
|
</span><span> - [I] Custom File Root Path: &quot;/data/gitea&quot;
|
||
|
|
</span><span> - [I] Work directory: &quot;/app/gitea&quot;
|
||
|
|
</span><span> - [I] Log Root Path: &quot;/data/gitea/log&quot;
|
||
|
|
</span><span>OK
|
||
|
|
</span></code></pre>
|
||
|
|
<ul>
|
||
|
|
<li>Copy the <code>$HOME/.gitconfig</code> file to the <strong>Data Root Path</strong>/home (which is <code>/data/gitea/home</code> in the example above).</li>
|
||
|
|
</ul>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>Get a Gitea instance with CI at Gna!</title>
|
||
|
|
<published>2022-07-10T00:00:00+00:00</published>
|
||
|
|
<updated>2022-07-10T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/experimental/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/experimental/</id>
|
||
|
|
<content type="html"><p>Hosting a Gitea instance on Gna! is now possible (but still experimental). It is meant to be a minimum viable product: anyone can <a href="https://hosteadashboard.gna.org">create a new dedicated Gitea instance</a> within minutes and pay for it on a monthly basis with a credit card. It includes a dedicated CI based on <a href="https://woodpecker-ci.org/">Woodpecker</a>. The smallest instance costs 10€ per month (2GB RAM, 10GB disk, 1CPU) and will be a good fit for a freelance up to a team of five people but bigger instances are also available if more RAM, CPU or disk is required.</p>
|
||
|
|
<video width="600" controls>
|
||
|
|
<source src="https://cloud.forgefriends.org/s/xXLcYpsE469tJEj/download?path=&files=hostea-screencast-vm-create.mp4" type="video/mp4" />
|
||
|
|
</video>
|
||
|
|
<p>The service is 100% infrastructure as code, published as <a href="https://lab.enough.community/main/infrastructure/-/tree/9e18ebbf675c8a65d1585d20b4cf6295af6e52ed/playbooks/hosteadashboard">Ansible playbooks within Enough</a>. It can be self-hosted on bare metal (with <a href="https://libvirt.org/">libvirt</a>) or in the cloud (with <a href="https://www.openstack.org/">OpenStack</a>): follow the <a href="https://enough-community.readthedocs.io/en/latest/introduction.html#quick-start">quick start</a>, configure playbooks for <a href="https://enough-community.readthedocs.io/en/latest/services/hostea.html">hostea</a> and <a href="https://enough-community.readthedocs.io/en/latest/services/hosteadashboard.html">the dashboard</a>.</p>
|
||
|
|
<p>The organization supporting Gna! is a <a href="https://forum.gna.org/c/governance-and-decisions/7">horizontal collective</a> of individuals and organizations. The <a href="https://forum.hostea.org/t/decision-revenue-sharing-model/92">revenue sharing model</a> is set to dedicate 25% of the income (more than the profits) to help the Free Software projects Hostea depends on such as Gitea, Enough, Django etc.</p>
|
||
|
|
<h3 id="the-origin">The origin<a class="zola-anchor" href="#the-origin" aria-label="Anchor link for: the-origin"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>In February 2022 the project of running a dedicated Gitea hosting service was <a href="https://discourse.gitea.io/t/a-gitea-hosting-service-under-the-umbrella-of-the-gitea-project/4692">proposed to the Gitea project</a> and other organizations and <a href="https://blog.dachary.org/2022/02/16/project-plans-for-a-hosted-gitea-online-service/">plans were drafted</a>. After a month of discussions it turned out to not be a <a href="https://blog.dachary.org/2022/03/11/the-inconclusive-story-of-four-failed-project-offers/">good match for any of them</a>.</p>
|
||
|
|
<p>It was suggested to create a new project from scratch instead of joining an existing organization. However, in order to be sustainable, a hosting service needs customers willing to pay for the service. And nobody had that kind of skill. It was therefore decided that to terminate the project: there is no point is creating a technical stack that's not going to be used by anyone.</p>
|
||
|
|
<h3 id="a-technical-stack-with-no-users">A technical stack with no users<a class="zola-anchor" href="#a-technical-stack-with-no-users" aria-label="Anchor link for: a-technical-stack-with-no-users"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>The most common mistake technical people do when creating a new piece of software is to overlook the fact that they have absolutely no idea how to let their intended user base know about it. Maybe the reason it happens so often is because it is very difficult to resist the urge of creating something. Because that's what technical people love to do: create things, even when they have no clue if it can be used.</p>
|
||
|
|
<p>It took no longer than two weeks for the people involved in Gna! to decide to build the technical stack to run hostea instead of being reasonable and give up. It was just too tempting.</p>
|
||
|
|
<p>To keep the madness contained and enjoyable, it was decided to set a deadline to July 1st and to <a href="https://gitea.gna.org/Hostea/july-mvp/issues">define precise and realistic technical goals</a>. It turned out to be an enjoyable experience: everyone learned a lot in the process and the outcome is something that can be reproduced. Most MVPs are a brittle pile of hacks designed to last a few weeks and be thrown away. But since a primary goal of the project was to create something self-hostable, it had to implement that feature and therefore be reproducible.</p>
|
||
|
|
<h3 id="a-horizontal-collective-with-a-revenue-sharing-model">A horizontal collective with a revenue sharing model<a class="zola-anchor" href="#a-horizontal-collective-with-a-revenue-sharing-model" aria-label="Anchor link for: a-horizontal-collective-with-a-revenue-sharing-model"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>Another goal of Gna! is to deploy federated forges, even at an experimental stage. Instead of creating a centralized organization to support Hostea, it was decided to create <a href="https://forum.gna.org/c/governance-and-decisions/7">horizontal collective</a>. It feels like a contradiction for a project committed to decentralization to be governed by a centralized organization.</p>
|
||
|
|
<p>The collective is composed of individuals and organizations but, unlike exclusively volunteer based Free Software projects, it is for profit. Customers rent Gitea instances by the month and the income is used to pay for expenses. There is however a difficulty: by nature a horizontal collective cannot be incorporated as it would create a level of hierarchy. The <a href="https://forum.gna.org/t/decision-revenue-sharing-model/92">revenue sharing model</a> had to be set as an informal agreement between members where one of them receives the income and distributes it to the others, depending on their Gna! related expenses.</p>
|
||
|
|
<p>It also requires that 25% of the income (not the profits) is dedicated to help the Free Software projects that Gna! depends on such as Gitea, Enough, Django etc. It can be via a donation, by upstreaming a bug fix or any kind of work that is beneficial to the dependency.</p>
|
||
|
|
<h3 id="dedicated-to-forge-federation">Dedicated to forge federation<a class="zola-anchor" href="#dedicated-to-forge-federation" aria-label="Anchor link for: dedicated-to-forge-federation"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>In the spirit of dogfooding, the people who created the technical stack of Gna! will use it for themselves on a daily basis. Since the focus of the authors is on <a href="https://forgefriends.org/blog/2022/06/30/2022-06-state-forge-federation/">forge federation</a>, they will add federation support in Hostea. This will be their primary motivation to improve and maintain Hostea: it is the only hosting platform where this can happen.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>1.17 breaking changes episode 1: preserving a custom gitconfig</title>
|
||
|
|
<published>2022-06-22T00:00:00+00:00</published>
|
||
|
|
<updated>2022-06-22T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/1-17-breaking-episode-1/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/1-17-breaking-episode-1/</id>
|
||
|
|
<content type="html"><p>Before version 1.17, when Gitea needed to change the <a href="https://git-scm.com/docs/git-config">git configuration</a>, it modified the <code>$HOME/.gitconfig</code> file. For instance it would <a href="https://github.com/go-gitea/gitea/blob/release/v1.16/modules/git/git.go#L174-L177">set core.quotePath to false</a>:</p>
|
||
|
|
<pre data-lang="ini" style="background-color:#2b303b;color:#c0c5ce;" class="language-ini "><code class="language-ini" data-lang="ini"><span style="color:#b48ead;">[core]
|
||
|
|
</span><span> </span><span style="color:#bf616a;">quotePath </span><span>= </span><span style="color:#d08770;">false
|
||
|
|
</span></code></pre>
|
||
|
|
<p>When installing Gitea <a href="https://docs.gitea.io/en-us/install-with-docker/">from docker</a> or <a href="https://docs.gitea.io/en-us/install-with-docker-rootless/">rootless</a> or even <a href="https://docs.gitea.io/en-us/install-from-binary/">from binary</a> this <code>$HOME/.gitconfig</code> file belongs to a user that is <a href="https://docs.gitea.io/en-us/install-from-binary/#prepare-environment">dedicated to Gitea</a> and not used by anyone else.</p>
|
||
|
|
<p>However, if an Gitea installation was done differently and <code>$HOME/.gitconfig</code> has been customized because it is shared by a user or another application, there is a good chance that manual modifications were done such as:</p>
|
||
|
|
<pre data-lang="ini" style="background-color:#2b303b;color:#c0c5ce;" class="language-ini "><code class="language-ini" data-lang="ini"><span style="color:#b48ead;">[user]
|
||
|
|
</span><span> </span><span style="color:#bf616a;">name </span><span>= Jane Doe
|
||
|
|
</span><span> </span><span style="color:#bf616a;">email </span><span>= jane</span><span style="color:#b48ead;">@doe</span><span>.com
|
||
|
|
</span></code></pre>
|
||
|
|
<p>It is also possible that the file was modified manually by the Gitea admin for other reasons. In both there is a <strong>potential for breakage when upgrading to Gitea &gt;= 1.17 because the location of the file changed</strong>. It must be moved manually to the new location as follows:</p>
|
||
|
|
<ul>
|
||
|
|
<li>Figure out the directory where <code>$HOME/.gitconfig</code> must be moved by <a href="https://gna.org/blog/gentle-introduction-to-the-doctor/">running the doctor</a>:</li>
|
||
|
|
</ul>
|
||
|
|
<pre data-lang="shell" style="background-color:#2b303b;color:#c0c5ce;" class="language-shell "><code class="language-shell" data-lang="shell"><span>$ gitea --work-path /app/gitea -c /data/gitea/conf/app.ini doctor
|
||
|
|
</span><span>[1] Check paths and basic configuration
|
||
|
|
</span><span> - [I] Configuration File Path: &quot;/data/gitea/conf/app.ini&quot;
|
||
|
|
</span><span> - [I] Repository Root Path: &quot;/data/git/repositories&quot;
|
||
|
|
</span><span> - [I] Data Root Path: &quot;/data/gitea&quot;
|
||
|
|
</span><span> - [I] Custom File Root Path: &quot;/data/gitea&quot;
|
||
|
|
</span><span> - [I] Work directory: &quot;/app/gitea&quot;
|
||
|
|
</span><span> - [I] Log Root Path: &quot;/data/gitea/log&quot;
|
||
|
|
</span><span>OK
|
||
|
|
</span></code></pre>
|
||
|
|
<ul>
|
||
|
|
<li>Copy the <code>$HOME/.gitconfig</code> file to the <strong>Repository Root Path</strong> (which is <code>/data/git/repositories</code> in the example above).</li>
|
||
|
|
</ul>
|
||
|
|
<p>The reason why this breaking change was introduced is to workaround <a href="https://gna.org/blog/unsafe-repository-is-owned-by-someone-else/">a rare problem</a> impacting Gitea installations relying on networked volumes.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>[tutorial] A gentle introduction to the gitea doctor</title>
|
||
|
|
<published>2022-06-14T00:00:00+00:00</published>
|
||
|
|
<updated>2022-06-14T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/gentle-introduction-to-the-doctor/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/gentle-introduction-to-the-doctor/</id>
|
||
|
|
<content type="html"><p>While helping people with their upgrades <a href="https://discourse.gitea.io/t/migration-from-1-2-to-1-16-8/5309">in the Gitea forum</a> or <a href="https://forum.gna.org/t/gitea-upgrade-from-1-14-1-to-1-16-8/90">at the Gna! clinic</a>, I realized that few Gitea admins know about the <a href="https://docs.gitea.io/en-us/command-line/#doctor"><code>gitea doctor</code></a> command and decided to write this blog post as a gentle introduction.</p>
|
||
|
|
<h3 id="an-apple-a-day-keeps-the-doctor-away">An apple a day keeps the doctor away<a class="zola-anchor" href="#an-apple-a-day-keeps-the-doctor-away" aria-label="Anchor link for: an-apple-a-day-keeps-the-doctor-away"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>Or in our case, Gitea versions <a href="https://github.com/go-gitea/gitea/blob/v1.11.5/cmd/doctor.go">below 1.11.5</a>. Since then, the <code>gitea doctor</code> is available and is designed to run against a specific Gitea version. It would not be a good idea to try to run the doctor from Gitea 1.16 to verify the sanity of a Gitea 1.2 instance: it will be confused by how the database is organized and a number of other details. Historical fun fact: the <code>gitea doctor</code> was backported to <a href="https://github.com/go-gitea/gitea/blob/v1.10.5/cmd/doctor.go">Gitea 1.10.5</a> and <a href="https://github.com/go-gitea/gitea/blob/v1.10.6/cmd/doctor.go">Gitea 1.10.6</a> and may be of help if you run this particular version and are facing the problem that motivated the backport.</p>
|
||
|
|
<p>With each version <code>gitea doctor</code> improves and gains new capabilities. For instance, in Gitea 1.17 it becomes aware of <a href="https://github.com/go-gitea/gitea/pull/19731">orphaned pull requests</a> and is able to fix them. If such a problem exists in Gitea 1.16, it does not know about it.</p>
|
||
|
|
<h3 id="calling-the-doctor">Calling the doctor<a class="zola-anchor" href="#calling-the-doctor" aria-label="Anchor link for: calling-the-doctor"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>In the following, examples are based on a Gitea 1.16.8 instance you can run as follows:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker run</span><span style="color:#bf616a;"> --name</span><span> gitea</span><span style="color:#bf616a;"> -p</span><span> 3000:3000</span><span style="color:#bf616a;"> -e</span><span> GITEA__security__INSTALL_LOCK=true</span><span style="color:#bf616a;"> -d</span><span> gitea/gitea:1.16.8-rootless
|
||
|
|
</span><span style="color:#bf616a;">$</span><span> docker exec gitea gitea admin user create</span><span style="color:#bf616a;"> --admin --username</span><span> root</span><span style="color:#bf616a;"> --password</span><span> admin1234</span><span style="color:#bf616a;"> --email</span><span> root@example.com
|
||
|
|
</span><span style="color:#bf616a;">$</span><span> docker exec gitea mkdir /var/lib/gitea/data/log
|
||
|
|
</span></code></pre>
|
||
|
|
<p>And then you can go to the <a href="https://127.0.0.1:3000/">web interface</a> to create a <code>test</code> repository, with an initial <code>README.md</code> file. When this is done the doctor can be called as follows:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker exec gitea gitea doctor</span><span style="color:#bf616a;"> --all
|
||
|
|
</span><span style="color:#bf616a;">[1]</span><span> Check paths and basic configuration
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Configuration File Path: &quot;</span><span style="color:#a3be8c;">/etc/gitea/app.ini</span><span>&quot;
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Repository Root Path: &quot;</span><span style="color:#a3be8c;">/var/lib/gitea/git/repositories</span><span>&quot;
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Data Root Path: &quot;</span><span style="color:#a3be8c;">/var/lib/gitea</span><span>&quot;
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Custom File Root Path: &quot;</span><span style="color:#a3be8c;">/var/lib/gitea/custom</span><span>&quot;
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Work directory: &quot;</span><span style="color:#a3be8c;">/var/lib/gitea</span><span>&quot;
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Log Root Path: &quot;</span><span style="color:#a3be8c;">/var/lib/gitea/data/log</span><span>&quot;
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[2]</span><span> Check if there is garbage storage files
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[3]</span><span> Check Database Version
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[4]</span><span> Check consistency of database
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[5]</span><span> Check if user with wrong type exist
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[6]</span><span> Check if OpenSSH authorized_keys file is up-to-date
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[7]</span><span> Check if SCRIPT_TYPE is available
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> ScriptType bash is on the current PATH at /bin/bash
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[8]</span><span> Check if hook files are up-to-date and executable
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[9]</span><span> Recalculate Stars number for all user
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[10]</span><span> Check old archives
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> 0 old archives in repository need to be deleted
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[11]</span><span> Enable push options
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Checked 1 repositories, 0 need updates.
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[12]</span><span> Check for incorrectly dumped repo_units (See </span><span style="color:#65737e;">#16961)
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>W</span><span style="color:#b48ead;">]</span><span> Found 0 broken repo_units
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[13]</span><span> Recalculate merge bases
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>W</span><span style="color:#b48ead;">]</span><span> 0 PRs with incorrect mergebases of 0 PRs total in 1 repos
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span><span style="color:#bf616a;">[14]</span><span> Check git-daemon-export-ok files
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>I</span><span style="color:#b48ead;">]</span><span> Checked 1 repositories, 0 need updates.
|
||
|
|
</span></code></pre>
|
||
|
|
<h3 id="what-does-the-doctor-know">What does the doctor know?<a class="zola-anchor" href="#what-does-the-doctor-know" aria-label="Anchor link for: what-does-the-doctor-know"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>Although the <code>doctor</code> can be compared to <a href="https://en.wikipedia.org/wiki/Fsck">fsck(8)</a>, it does not know everything. It took decades for <code>fsck</code> to become the ultimate authority on finding problems on file systems and reliably fixing them without losing data. Nowadays, only a handful of people in the world are brave enough to manually attempt a file system recovery when <code>fsck</code> cannot recover from a data loss.</p>
|
||
|
|
<p>The first <code>doctor</code> version is two years old and Gitea admins are still routinely running SQL queries against the database or moving files around when trying to figure out why a Gitea instance is not behaving as it should. It is however worth checking if the doctor does not already have a solution by listing all it can do:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker exec gitea gitea doctor</span><span style="color:#bf616a;"> --list
|
||
|
|
</span><span style="color:#bf616a;">Default</span><span> Name Title
|
||
|
|
</span><span style="color:#bf616a;">*</span><span> paths Check paths and basic configuration
|
||
|
|
</span><span> </span><span style="color:#bf616a;">storages</span><span> Check if there is garbage storage files
|
||
|
|
</span><span style="color:#bf616a;">*</span><span> check-db-version Check Database Version
|
||
|
|
</span><span> </span><span style="color:#bf616a;">check-db-consistency</span><span> Check consistency of database
|
||
|
|
</span><span style="color:#bf616a;">*</span><span> check-user-type Check if user with wrong type exist
|
||
|
|
</span><span style="color:#bf616a;">*</span><span> authorized-keys Check if OpenSSH authorized_keys file is up-to-date
|
||
|
|
</span><span> </span><span style="color:#bf616a;">script-type</span><span> Check if SCRIPT_TYPE is available
|
||
|
|
</span><span> </span><span style="color:#bf616a;">hooks</span><span> Check if hook files are up-to-date and executable
|
||
|
|
</span><span> </span><span style="color:#bf616a;">recalculate-stars-number</span><span> Recalculate Stars number for all user
|
||
|
|
</span><span> </span><span style="color:#bf616a;">check-old-archives</span><span> Check old archives
|
||
|
|
</span><span> </span><span style="color:#bf616a;">enable-push-options</span><span> Enable push options
|
||
|
|
</span><span> </span><span style="color:#bf616a;">fix-broken-repo-units</span><span> Check for incorrectly dumped repo_units (See </span><span style="color:#65737e;">#16961)
|
||
|
|
</span><span> </span><span style="color:#bf616a;">recalculate-merge-bases</span><span> Recalculate merge bases
|
||
|
|
</span><span> </span><span style="color:#bf616a;">check-git-daemon-export-ok</span><span> Check git-daemon-export-ok files
|
||
|
|
</span></code></pre>
|
||
|
|
<p>And then call the <code>check</code> that looks interesting:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker exec gitea gitea doctor</span><span style="color:#bf616a;"> --run</span><span> authorized-keys
|
||
|
|
</span><span style="color:#bf616a;">[1]</span><span> Check if OpenSSH authorized_keys file is up-to-date
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span></code></pre>
|
||
|
|
<p>The challenge is to figure out which <code>check</code> does what and at the moment the best source of information is ... <a href="https://github.com/go-gitea/gitea/tree/v1.16.8">the sources</a> themselves. The <a href="https://github.com/go-gitea/gitea/blob/v1.16.8/cmd/doctor.go">doctor.go</a> command is the entry point and <a href="https://github.com/go-gitea/gitea/tree/v1.16.8/modules/doctor">the doctor directory</a> contains the rest.</p>
|
||
|
|
<p>Some <code>checks</code> are straightforward to understand, even if you do not know Go, such as <a href="https://github.com/go-gitea/gitea/blob/v1.16.8/modules/doctor/authorizedkeys.go">the authorized-keys check</a>. Others are much more involved and your best chance is to <a href="https://matrix.to/#/#gitea:matrix.org">ask the Gitea chatroom</a> for help.</p>
|
||
|
|
<h3 id="is-it-going-to-hurt">Is it going to hurt?<a class="zola-anchor" href="#is-it-going-to-hurt" aria-label="Anchor link for: is-it-going-to-hurt"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>By default the doctor (very much like <code>fsck -N</code>) only performs non destructive checks and displays diagnostics, with an indication of how serious the problem is. In the example above, there only are lines with <strong>[I]</strong> (which indicates an information) and <strong>[W]</strong> which indicates a warning that can be ignored but may be worth looking into. Those two warnings are actually just informational and should be labelled as <strong>[I]</strong>, <a href="https://github.com/go-gitea/gitea/pull/19836">which has been fixed</a> in a more recent version of the doctor.</p>
|
||
|
|
<p>Now let's do something bad: remove the permissions from a hook in our repository:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker exec gitea chmod</span><span style="color:#bf616a;"> -x</span><span> /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive
|
||
|
|
</span></code></pre>
|
||
|
|
<p>Run the doctor with the <code>check</code> supposed to find that out:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker exec gitea gitea doctor</span><span style="color:#bf616a;"> --run</span><span> hooks
|
||
|
|
</span><span style="color:#bf616a;">[1]</span><span> Check if hook files are up-to-date and executable
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>W</span><span style="color:#b48ead;">]</span><span> old hook file /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive is not executable
|
||
|
|
</span></code></pre>
|
||
|
|
<p>Ask it to fix this with the <code>--fix</code> flag:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker exec gitea gitea doctor</span><span style="color:#bf616a;"> --run</span><span> hooks</span><span style="color:#bf616a;"> --fix
|
||
|
|
</span><span style="color:#bf616a;">[1]</span><span> Check if hook files are up-to-date and executable
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>W</span><span style="color:#b48ead;">]</span><span> Regenerated hooks for root/test
|
||
|
|
</span><span> </span><span style="color:#bf616a;">- </span><span style="color:#b48ead;">[</span><span>W</span><span style="color:#b48ead;">]</span><span> old hook file /var/lib/gitea/git/repositories/root/test.git/hooks/post-receive is not executable
|
||
|
|
</span></code></pre>
|
||
|
|
<p>And run it one last time to check all is well:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">$</span><span> docker exec gitea gitea doctor</span><span style="color:#bf616a;"> --run</span><span> hooks
|
||
|
|
</span><span style="color:#bf616a;">[1]</span><span> Check if hook files are up-to-date and executable
|
||
|
|
</span><span style="color:#bf616a;">OK
|
||
|
|
</span></code></pre>
|
||
|
|
<p>Even when the doctor is unable to fix a problem, it can help by showing extensive debug output which can be found, by default, in the <code>doctor.log</code> file in the directory from which it runs. Or it can be displayed on the standard output with <code>--log-file -</code>, which is most convenient when running in docker.</p>
|
||
|
|
<h3 id="going-further">Going further<a class="zola-anchor" href="#going-further" aria-label="Anchor link for: going-further"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>If that was helpful to you, I would very much appreciate if you <a href="https://mastodon.online/@dachary">send me a message on Mastodon</a>. It will encourage me to write more blog posts to share what I learn about Gitea. Even better: you could <a href="https://github.com/go-gitea/gitea/pulls">send a pull request</a> to improve the doctor and help it mature.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>[solved] Zombies created by Gitea</title>
|
||
|
|
<published>2022-06-04T00:00:00+00:00</published>
|
||
|
|
<updated>2022-06-04T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/zombies-part-2/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/zombies-part-2/</id>
|
||
|
|
<content type="html"><p>Gitea can <a href="/blog/zombies">create zombies</a>, for instance if a Git mirror takes too long. When updating a mirror, Gitea relies on the <code>git remote update</code> command which creates a child process, <code>git-remote-https</code>, to fetch data from the remote repository. Gitea has an internal timeout that will kill the child process (e.g. <code>git remote update</code>) when it takes too long but will not kill the grandchild. This grandchild will become an orphan and run forever or until its own timeout expires, which is about two minutes on git version 2.25.</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ time git clone https://4.4.4.4
|
||
|
|
</span><span>Clonage dans &#39;4.4.4.4&#39;...
|
||
|
|
</span><span>fatal: impossible d&#39;accéder à &#39;https://4.4.4.4/&#39;: Failed to connect to 4.4.4.4 port 443: Connexion terminée par expiration du délai d&#39;attente
|
||
|
|
</span><span>
|
||
|
|
</span><span>real 2m9,753s
|
||
|
|
</span><span>user 0m0,001s
|
||
|
|
</span><span>sys 0m0,009s
|
||
|
|
</span></code></pre>
|
||
|
|
<p>As explained in the <a href="/blog/zombies/#killing-a-child-process-and-all-its-children">diagnostic blog post regarding Gitea zombies</a> there fortunately is a very simple way to avoid this by making sure each Gitea child is a <a href="https://en.wikipedia.org/wiki/Process_group">process group leader</a>. That first step was <a href="https://github.com/go-gitea/gitea/pull/19865">introduced in Gitea 1.17</a> and <a href="https://github.com/go-gitea/gitea/pull/19865">backported to Gitea 1.16.9</a>. The actual bug fix can now be implemented.</p>
|
||
|
|
<h3 id="using-negative-process-id-to-kill-children">Using negative process id to kill children<a class="zola-anchor" href="#using-negative-process-id-to-kill-children" aria-label="Anchor link for: using-negative-process-id-to-kill-children"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>When Gitea timeout on a child, it relies on <a href="https://github.com/golang/go/blob/f8a53df314e4af8cd350eedb0dae77d4c4fc30d0/src/os/exec/exec.go#L650">os.Process.Kill</a> which translates into a using the kill(2) system call to send a SIGKILL signal to unconditionally terminate it: <code>kill(pid, SIGKILL)</code>. Using a negative pid with <code>kill(-pid, SIGKILL)</code> will also terminate all processes created by Gitea's child, without Gitea knowing when or why they were created. From the kill(2) manual page:</p>
|
||
|
|
<blockquote>
|
||
|
|
<p>If pid is less than -1, then sig is sent to every process in the process group whose ID is -pid.</p>
|
||
|
|
</blockquote>
|
||
|
|
<p>Which is implemented as follows in the <a href="https://lab.forgefriends.org/friendlyforgeformat/gofff/-/blob/a9603c7cc934fccd4382b7f4309b75c852742480/util/exec.go#L130">Friendly Forge Format library</a>:</p>
|
||
|
|
<blockquote>
|
||
|
|
<p><code>syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)</code></p>
|
||
|
|
</blockquote>
|
||
|
|
<h3 id="not-using-the-default-go-commandcontext">Not using the default Go CommandContext<a class="zola-anchor" href="#not-using-the-default-go-commandcontext" aria-label="Anchor link for: not-using-the-default-go-commandcontext"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>Since <a href="https://pkg.go.dev/os/exec#CommandContext">CommandContext</a> does not allow to send a signal to the negative pid of the child process, it has to be implemented by Gitea itself, in a way that is similar to how the <a href="https://lab.forgefriends.org/friendlyforgeformat/gofff/-/blob/a9603c7cc934fccd4382b7f4309b75c852742480/util/exec.go#L75-82">Friendly Forge Format library</a> does it:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> ctxErr := watchCtx(ctx, cmd.Process.Pid)
|
||
|
|
</span><span> err = cmd.Wait()
|
||
|
|
</span><span> interruptErr := &lt;-ctxErr
|
||
|
|
</span><span> // If cmd.Wait returned an error, prefer that.
|
||
|
|
</span><span> // Otherwise, report any error from the interrupt goroutine.
|
||
|
|
</span><span> if interruptErr != nil &amp;&amp; err == nil {
|
||
|
|
</span><span> err = interruptErr
|
||
|
|
</span><span> }
|
||
|
|
</span></code></pre>
|
||
|
|
<h3 id="testing-the-bug-is-fixed-and-stays-fixed">Testing the bug is fixed and stays fixed<a class="zola-anchor" href="#testing-the-bug-is-fixed-and-stays-fixed" aria-label="Anchor link for: testing-the-bug-is-fixed-and-stays-fixed"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>Long standing bugs that are difficult to reproduce manually such as this one require robust testing to ensure that:</p>
|
||
|
|
<ul>
|
||
|
|
<li>the diagnostic identifying the root cause is correct</li>
|
||
|
|
<li>the bug fix works</li>
|
||
|
|
<li>it does not resurface insidiously because of a subtle regression introduce years later</li>
|
||
|
|
</ul>
|
||
|
|
<p>It is easy to implement as can be seen in the <a href="https://lab.forgefriends.org/friendlyforgeformat/gofff/-/blob/a9603c7cc934fccd4382b7f4309b75c852742480/util/exec_test.go#L44-76">Friendly Forge Format library</a>. In a nutshell:</p>
|
||
|
|
<ul>
|
||
|
|
<li><a href="https://lab.forgefriends.org/friendlyforgeformat/gofff/-/blob/a9603c7cc934fccd4382b7f4309b75c852742480/util/exec_test.go#L53">git clone https://4.4.4.4</a> which will hang because of firewall rules</li>
|
||
|
|
<li><a href="https://lab.forgefriends.org/friendlyforgeformat/gofff/-/blob/a9603c7cc934fccd4382b7f4309b75c852742480/util/exec_test.go#L60-65">wait for the git-remote-https</a> grandchild process to be spawned</li>
|
||
|
|
<li><a href="https://lab.forgefriends.org/friendlyforgeformat/gofff/-/blob/a9603c7cc934fccd4382b7f4309b75c852742480/util/exec_test.go#L67-68">cancel the context and wait for the goroutine to terminate</a></li>
|
||
|
|
<li><a href="https://lab.forgefriends.org/friendlyforgeformat/gofff/-/blob/a9603c7cc934fccd4382b7f4309b75c852742480/util/exec_test.go#L70-75">verify the git-remote-https is killed</a></li>
|
||
|
|
</ul>
|
||
|
|
<p>And with that... no more zombies!</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>[diagnostic] Zombies created by Gitea</title>
|
||
|
|
<published>2022-06-02T00:00:00+00:00</published>
|
||
|
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/zombies/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/zombies/</id>
|
||
|
|
<content type="html"><p>The first <a href="https://github.com/go-gitea/gitea/issues/3242">issue about zombie processes</a> created by Gitea was reported in 2017 and <a href="https://github.com/go-gitea/gitea/issues/13987">resurfaced</a> on a <a href="https://github.com/go-gitea/gitea/issues/19077">regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie:</p>
|
||
|
|
<ul>
|
||
|
|
<li>Gitea updates a mirror by spawning the process <code>git remote update</code></li>
|
||
|
|
<li><code>git remote update</code> spawns yet another process, <code>git fetch</code></li>
|
||
|
|
<li><code>git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li>
|
||
|
|
<li>Gitea kill the process <code>git remote update</code></li>
|
||
|
|
<li>When killed <code>git remote update</code> does not kill its own child and <code>git fetch</code> becomes an orphaned process which keeps running</li>
|
||
|
|
<li>When <code>git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li>
|
||
|
|
</ul>
|
||
|
|
<h3 id="pid-1-process-and-waiting-on-orphans">PID 1 process and waiting on orphans<a class="zola-anchor" href="#pid-1-process-and-waiting-on-orphans" aria-label="Anchor link for: pid-1-process-and-waiting-on-orphans"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is <code>/bin/init</code> and when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile">default docker image</a> this is <code>s6</code>: they will both wait on orphaned processes and there won't be any zombies.</p>
|
||
|
|
<h3 id="what-if-gitea-is-the-only-running-process">What if Gitea is the only running process?<a class="zola-anchor" href="#what-if-gitea-is-the-only-running-process" aria-label="Anchor link for: what-if-gitea-is-the-only-running-process"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>But when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile.rootless">rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||
|
|
</span><span>$ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||
|
|
</span></code></pre>
|
||
|
|
<p>The <code>git</code> command can then be replaced with a script that waits forever:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ( echo -e &#39;#!/bin/bash\nsleep infinity&#39; ) | docker exec -i --user root gitea tee /usr/bin/git
|
||
|
|
</span><span>$ docker exec --user root gitea chmod +x /usr/bin/git
|
||
|
|
</span></code></pre>
|
||
|
|
<p>Trying to create a repository from the web interface will create the conditions for a zombie to show:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||
|
|
</span><span>PPID PID COMMAND COMMAND
|
||
|
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||
|
|
</span><span> 1 94 sleep [sleep]
|
||
|
|
</span><span> 1 99 sleep [sleep]
|
||
|
|
</span><span> 1 111 sleep [sleep]
|
||
|
|
</span><span> 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare
|
||
|
|
</span><span> 164 165 sleep sleep infinity
|
||
|
|
</span></code></pre>
|
||
|
|
<p>When the <code>git</code> process is killed by Gitea, the <code>sleep</code> child will be orphaned:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||
|
|
</span><span>PPID PID COMMAND COMMAND
|
||
|
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||
|
|
</span><span> 1 94 sleep [sleep]
|
||
|
|
</span><span> 1 99 sleep [sleep]
|
||
|
|
</span><span> 1 111 sleep [sleep]
|
||
|
|
</span><span> 1 165 sleep sleep infinity
|
||
|
|
</span></code></pre>
|
||
|
|
<p>Killing it will turn it into a zombie:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea kill 165
|
||
|
|
</span><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||
|
|
</span><span>PPID PID COMMAND COMMAND
|
||
|
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||
|
|
</span><span> 1 94 sleep [sleep]
|
||
|
|
</span><span> 1 99 sleep [sleep]
|
||
|
|
</span><span> 1 111 sleep [sleep]
|
||
|
|
</span><span> 1 165 sleep [sleep]
|
||
|
|
</span></code></pre>
|
||
|
|
<h3 id="killing-a-child-process-and-all-its-children">Killing a child process and all its children<a class="zola-anchor" href="#killing-a-child-process-and-all-its-children" aria-label="Anchor link for: killing-a-child-process-and-all-its-children"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are <a href="https://en.wikipedia.org/wiki/Process_group">process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
||
|
|
<published>2022-05-28T00:00:00+00:00</published>
|
||
|
|
<updated>2022-05-28T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/path-not-found/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/path-not-found/</id>
|
||
|
|
<content type="html"><p>In Gitea 1.15 the <a href="https://github.com/go-gitea/gitea/blob/cfb4c23a5009b9c236d48ac0bc156577c7d70741/custom/conf/app.example.ini">app.example.ini</a> file was changed to <a href="https://github.com/go-gitea/gitea/commit/4a84022d2559ccfc99960c7c654ee8b9b38664f7">comment out most of the values</a>. The assumption was that all values exactly matched the defaults <a href="https://github.com/go-gitea/gitea/blob/main/modules/setting/setting.go">in the source code</a>. However, there are differences, for instance for <a href="https://github.com/go-gitea/gitea/blob/cfb4c23a5009b9c236d48ac0bc156577c7d70741/modules/setting/setting.go#L771">APP_DATA_PATH</a>. Before Gitea 1.15, <code>app.example.ini</code> contained:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>APP_DATA_PATH = data
|
||
|
|
</span></code></pre>
|
||
|
|
<p>and the path was relative to the <strong>directory from which the Gitea server was running</strong>. In Gitea 1.15 up to 1.16, it was commented out:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>; APP_DATA_PATH = data
|
||
|
|
</span></code></pre>
|
||
|
|
<p>and the path was relative to the <strong>work path directory</strong>, as provided either via the --work-path argument or the <code>GITEA_WORK_DIR</code> environment variable. </p>
|
||
|
|
<p>When a distribution such as voidlinux <a href="https://github.com/void-linux/void-packages/blob/master/srcpkgs/gitea/patches/config.patch">uses app.example.ini</a> as a base for the Gitea package, this change indirectly creates a regression and an upgrade of Gitea <a href="https://github.com/go-gitea/gitea/issues/19367">fails with errors</a> such as <code>unable to open level db at data/data/queues/common: mkdir data: permission denied</code>. The regression did not show as soon as Gitea 1.15 became available in voidlinux because the package <a href="https://github.com/void-linux/void-packages/blob/7fc9190f0e0d557dd5031e68df4e183892d4315b/srcpkgs/gitea/patches/config.patch#L62">explicitly set <code>APP_DATA_PATH</code></a>. But this <a href="https://github.com/void-linux/void-packages/commit/19d986a2cae9ce73d32552ddb62443b5e7fa13e2">changed when Gitea 1.15.6 was packaged</a> and once the value was commented out, upgrading triggered the problem. This was worked around six month later with the <a href="https://github.com/void-linux/void-packages/commit/44b6c96fa12ce9d993c7a2ac9486d892735b7e3a">Gitea 1.16.8</a> package.</p>
|
||
|
|
<p>The <code>APP_DATA_PATH</code> directory is not the only one, the <code>[log] ROOT_PATH</code> is another example. There is an <a href="https://github.com/go-gitea/gitea/pull/19815">ongoing effort</a> to improve the situation in Gitea 1.17. With the downside of introducing breaking changes that will have an impact on all Gitea installations because the content of the <code>app.ini</code> file will be interpreted differently. In the case of <code>APP_DATA_PATH</code>, both:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>APP_DATA_PATH = data
|
||
|
|
</span></code></pre>
|
||
|
|
<p>and:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>; APP_DATA_PATH = data
|
||
|
|
</span></code></pre>
|
||
|
|
<p>will be interpreted to be relative to the <strong>work path directory</strong>, as provided either via the --work-path argument or the <code>GITEA_WORK_DIR</code> environment variable. Every Gitea installation using <strong>APP_DATA_PATH = data</strong> will need to update the value to be an absolute path such as <strong>/var/lib/gitea/data</strong> so that it keeps pointing to the expected directory.</p>
|
||
|
|
<p>In order to prepare for the change or ensure the consistency of all path, there fortunately is a very simple <strong>solution: always use absolute paths in the <code>app.ini</code> configuration file</strong>.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>[solved] Gitea 1.16.[678] error: fatal: unsafe repository is owned by someone else</title>
|
||
|
|
<published>2022-05-15T00:00:00+00:00</published>
|
||
|
|
<updated>2022-05-15T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/unsafe-repository-is-owned-by-someone-else/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/unsafe-repository-is-owned-by-someone-else/</id>
|
||
|
|
<content type="html"><p>April 12, 2022 version <a href="https://lore.kernel.org/git/xmqqv8veb5i6.fsf@gitster.g/">git v2.35.2</a> was released and addresses a security issue <a href="https://github.com/git-for-windows/git/security/advisories/GHSA-vw2c-22j4-2fh2">CVE-2022-24765</a>. It was backported to 2.30.3, v2.31.2, v2.32.1, v2.33.2, and v2.34.2 and published by distributions such as <a href="https://security-tracker.debian.org/tracker/CVE-2022-24765">Debian GNU/Linux</a>, <a href="https://www.alpinelinux.org/releases/">Alpine</a>.</p>
|
||
|
|
<p><strong>If Gitea runs as user <code>foo</code>, calls a patched Git version and a parent directory of the git repositories is owned by a user other than <code>foo</code>, it will fail</strong> with a message such as:</p>
|
||
|
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Failed to open repository: Git/Data Error: exit status 128 - fatal: unsafe repository (&#39;/data/git/repositories/git/data.git&#39; is owned by someone else)
|
||
|
|
</span></code></pre>
|
||
|
|
<p>This started to show in the past few weeks to <a href="https://github.com/go-gitea/gitea/issues/19455">users running the Gitea binary on Windows</a> who also independently installed git v2.36. And then to people running <a href="https://github.com/go-gitea/gitea/issues/19455#issuecomment-1106331149">Gitea from snap</a>, on <a href="https://github.com/go-gitea/gitea/issues/19455#issuecomment-1106312061">a Synology NAS</a> and then people running from <a href="https://github.com/go-gitea/gitea/blob/main/Dockerfile#L2">Gitea docker images</a> which is based on <a href="https://www.alpinelinux.org/releases/">Alpine</a>.</p>
|
||
|
|
<h3 id="workarounds">Workarounds<a class="zola-anchor" href="#workarounds" aria-label="Anchor link for: workarounds"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<ul>
|
||
|
|
<li>If using <a href="https://hub.docker.com/r/gitea/gitea">Gitea docker images</a>:
|
||
|
|
<ul>
|
||
|
|
<li>upgrade to <a href="https://github.com/go-gitea/gitea/pull/19876">Gitea &gt;=1.16.9</a> or 1.17, both have git &gt;=2.36</li>
|
||
|
|
<li><code>git config --global --replace-all safe.directory '*'</code></li>
|
||
|
|
</ul>
|
||
|
|
</li>
|
||
|
|
<li>If the Gitea binary was installed independently of git, upgrade git to a version that is <a href="https://git-scm.com/docs/git-config#Documentation/git-config.txt-safedirectory">greater or equal to 2.36</a> and disable the security check entirely with:
|
||
|
|
<ul>
|
||
|
|
<li>impersonate the <a href="https://docs.gitea.io/en-us/install-from-binary/#recommended-server-configuration">user dedicated to Gitea</a> (usually git)</li>
|
||
|
|
<li><code>git config --global --replace-all safe.directory '*'</code></li>
|
||
|
|
</ul>
|
||
|
|
</li>
|
||
|
|
</ul>
|
||
|
|
<h3 id="bug-fix">Bug fix<a class="zola-anchor" href="#bug-fix" aria-label="Anchor link for: bug-fix"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>The <a href="https://github.com/go-gitea/gitea/pull/19870">bug fix</a> is for Gitea to ensure <code>git config --global --replace-all safe.directory '*'</code> is set on its <a href="https://docs.gitea.io/en-us/install-from-binary/#recommended-server-configuration">dedicated user</a> when it initializes. It is effective on the condition that the git CLI version is <a href="https://git-scm.com/docs/git-config#Documentation/git-config.txt-safedirectory">greater or equal to 2.36</a>.</p>
|
||
|
|
<h3 id="bug-fix-rationale">Bug fix rationale<a class="zola-anchor" href="#bug-fix-rationale" aria-label="Anchor link for: bug-fix-rationale"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h3>
|
||
|
|
<p>It is safe to <a href="https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/50/diffs">disable the security check in Gitea</a>. It is not vulnerable to <strong><a href="https://github.com/git-for-windows/git/security/advisories/GHSA-vw2c-22j4-2fh2">CVE-2022-24765</a></strong> because it calls the git CLI <a href="https://github.com/go-gitea/gitea/blob/main/modules/git/command.go#L160">after changing its working directory</a> to be the git repository targeted by the command (for instance <a href="https://github.com/go-gitea/gitea/blob/main/modules/git/diff.go#L38-L45">diff</a>) or a temporary directory. Therefore <strong>it will not explore the parent directories looking for a git configuration file</strong>.</p>
|
||
|
|
<p>The security check is triggered because the repository is owned by an unexpected user (root instead of git typically) and <strong>not because a parent directory is owned by an unexpected user</strong>. This, in itself, is a problem worth investigating but it is unrelated and was revealed by the newer security check of git even though it does not match the threat described in <strong><a href="https://github.com/git-for-windows/git/security/advisories/GHSA-vw2c-22j4-2fh2">CVE-2022-24765</a></strong>.</p>
|
||
|
|
<p>It appears non trivial to enforce a consistent ownership of files and directories, either within docker or outside docker when networked file systems are involved. The Gitea server was not troubled by this inconsistency so far because the permissions allow it to write and read where expected, regardless of the owner. It is not worth looking into but it is ancient and unrelated.</p>
|
||
|
|
<p>Gitea runs under a dedicated user, either when installed <a href="https://docs.gitea.io/en-us/install-from-binary/#recommended-server-configuration">from binary</a> or from <a href="https://docs.gitea.io/en-us/install-with-docker/">docker</a> and <a href="https://github.com/go-gitea/gitea/blob/main/modules/git/git.go#L196-L207">modifies the global git configuration</a> depending on the git version at initialization time. Fixing the problem can therefore be done by <a href="https://lab.forgefriends.org/forgefriends/forgefriends/-/merge_requests/50/diffs#bcd72ff867cbd1ddd5b6518c3a05b5f1a6021286_209_209">disabling the security check in the global git config file at initialization time</a>. It also requires a minimum version of git 2.36 to be installed, which is the case for Gitea docker images with <a href="https://github.com/go-gitea/gitea/pull/19876">versions &gt;= 1.16.9</a>.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>[solved] blank or error 500 page after login</title>
|
||
|
|
<published>2022-05-08T00:00:00+00:00</published>
|
||
|
|
<updated>2022-05-08T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/blank-or-error-500-page-after-login/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/blank-or-error-500-page-after-login/</id>
|
||
|
|
<content type="html"><p>The <a href="https://docs.gitea.io/en-us/upgrade-from-gitea/#upgrade-from-binary">instructions to upgrade a Gitea instance</a> only require three to four steps. They work fine most of the time but the documentation is lacking a &quot;Troubleshooting&quot; section to help out when something goes wrong. Maintaining instructions on how to diagnose and fix upgrade problems is an ambitious undertaking and requires updates every time a new case is discovered.</p>
|
||
|
|
<p>An <a href="https://forum.gna.org/t/things-to-know-about-gitea-upgrades/39">inventory of the known upgrade issues</a> was started to figure out how to structure such a section in the documentation. The <a href="https://blog.gitea.io/">release notes</a> were analyzed all the way back to <a href="https://github.com/go-gitea/gitea/releases/tag/v1.9.6">Gitea 1.9.6</a> and the work is still in progress. Here is a sample of the tips that will be included:</p>
|
||
|
|
<ul>
|
||
|
|
<li>Upgrade directly to the latest Gitea version, there is no need to upgrade to intermediate versions.</li>
|
||
|
|
<li>If the upgrade from version x.y to version x.y+2 fails and there is a need to narrow down the problem, try upgrading to the latest minor version of each major version and verify it works.</li>
|
||
|
|
<li>etc.</li>
|
||
|
|
</ul>
|
||
|
|
<p>However, even with the best documentation, someone will eventually <strong>run into an new problem</strong> and fixing it without compromising the integrity of the data will be challenging. This is best demonstrated by a real world example that was concluded a few days ago.</p>
|
||
|
|
<h1 id="getting-help-from-the-community">Getting help from the community<a class="zola-anchor" href="#getting-help-from-the-community" aria-label="Anchor link for: getting-help-from-the-community"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h1>
|
||
|
|
<p>After <a href="https://discourse.gitea.io/t/blank-page-after-login/5051">upgrading a Gitea intsance from 1.9.6 to 1.16.5</a> the tests conducted manually did not uncover any problem. However, after going to production, some users saw a blank page after login and had to manually type the URL of the project they wanted to see in the browser. The person in charge of the upgrade never had to diagnose Gitea problem and <a href="https://discourse.gitea.io/t/blank-page-after-login/5051">reached out in the Gitea forum</a>.</p>
|
||
|
|
<blockquote>
|
||
|
|
<p><strong>Tip: explain the problem in a public forum as early as possible to get help from the community</strong></p>
|
||
|
|
</blockquote>
|
||
|
|
<p>In their post in the forum they explained how they attempted to diagnose the problem and how why they thought that only users created a few years ago were impacted. It was a detailed analysis that was concluded with a partial copy of the logs. It was unfortunately missing <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/12">key information</a> that was provided only three days later. In the meantime, as they could not figure out the source of the problem, they were on the <strong>verge of <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/11">accepting the loss of all the Gitea database</a> and start over from the repositories</strong>. However, once all the details were available, <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/13">a workaround</a> was suggested in the forum.</p>
|
||
|
|
<blockquote>
|
||
|
|
<p><strong>Tip: focus more on providing detailed facts than exposing the attempted diagnostic</strong></p>
|
||
|
|
</blockquote>
|
||
|
|
<p>There was hope to fix Gitea and in the following days they applied the workaround. They also tried to improve it but without success and eventually accepted a <strong>partial data loss</strong> as inevitable and <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/14">reported their success back to the forum</a>.</p>
|
||
|
|
<blockquote>
|
||
|
|
<p><strong>Tip: when getting support from the community, providing feedback is the best token of appreciation</strong></p>
|
||
|
|
</blockquote>
|
||
|
|
<h1 id="getting-professional-help">Getting professional help<a class="zola-anchor" href="#getting-professional-help" aria-label="Anchor link for: getting-professional-help"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h1>
|
||
|
|
<p>The <a href="https://gna.org/gitea-clinic/">Gna! Clinic</a> is a collective of individual and companies that provides professional services to Gitea admins. They are active members of the Gitea community who <a href="https://discourse.gitea.io/u/dachary/activity">help out</a> as volunteers. They can also be hired to resolve the more complicated cases.</p>
|
||
|
|
<p>The Gitea instance that was in trouble required more than a few minutes of work and access to the database content for a proper diagnostic. They <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/13">proposed their assistance</a> but although <a href="https://discourse.gitea.io/t/user-research-about-gitea-upgrade-experiences-call-for-volunteers/5063/2">well received</a>, it was not accepted.</p>
|
||
|
|
<p>When the Gitea admin explained how they chose to resolve the problem <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/14">on the forum</a>, it confirmed the workaround was viable and the root problem was identified. That was enough to figure out a fix for the underlying bug with <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/17">a rather simple patch</a> that was merged <a href="https://github.com/go-gitea/gitea/pull/19629">and backported</a> in the following days. But it happened too late to avoid the data loss.</p>
|
||
|
|
<p>To summarize with a timeline, here is what happened:</p>
|
||
|
|
<ul>
|
||
|
|
<li>J+1: The <strong>problem is discovered</strong> by users who see a blank page after login and a the Gitea admin tries to diagnose the problem</li>
|
||
|
|
<li>J+2: A message is sent <strong>to ask for help in the community</strong></li>
|
||
|
|
<li>J+2 to J+6: Three people in the community suggest ideas but <strong>the Gitea admin cannot figure out the root cause and is on the verge of accepting the loss of all Gitea data</strong> and restart from the git repositories</li>
|
||
|
|
<li>J+6: A <strong>workaround is suggested by the community</strong></li>
|
||
|
|
<li>J+7 to J+17: The Gitea admin applies the <strong>workaround and only looses part of the Gitea data</strong></li>
|
||
|
|
</ul>
|
||
|
|
<p>And in retrospect, here is what could have happened instead:</p>
|
||
|
|
<ul>
|
||
|
|
<li>J+1: The <strong>problem is discovered</strong> by users who see a blank page after login</li>
|
||
|
|
<li>J+1: The Gitea admin <strong><a href="https://gna.org/gitea-clinic/">reaches out to someone at the Gna! Clinic</a></strong></li>
|
||
|
|
<li>J+2: The <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/12">logs of the Gitea instance</a> are analyzed, <strong>the root cause diagnosed</strong> and <a href="https://discourse.gitea.io/t/blank-page-after-login/5051/17">a patch</a> is created to fix it.</li>
|
||
|
|
<li>J+3: If necessary a Gitea binary is created with the patch and used as a temporary replacement until the next point release is published with <a href="https://github.com/go-gitea/gitea/pull/19629">the backport</a>. The Gitea admin runs the patched Gitea binary in the meantime. <strong>There is no data loss</strong>.</li>
|
||
|
|
</ul>
|
||
|
|
<p>It does not mean all upgrade problems can be resolved so easily. But it shows, with an example, that in some cases it makes sense to get professional help.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
<entry xml:lang="en">
|
||
|
|
<title>Project plans for a hosted Gitea online service</title>
|
||
|
|
<published>2022-04-18T00:00:00+00:00</published>
|
||
|
|
<updated>2022-04-18T00:00:00+00:00</updated>
|
||
|
|
<link href="https://gna.org/blog/project-plans-for-hosted-gitea-online-service/" type="text/html"/>
|
||
|
|
<id>https://gna.org/blog/project-plans-for-hosted-gitea-online-service/</id>
|
||
|
|
<content type="html"><p><em>This post was originally published on <a href="https://blog.dachary.org/2022/02/16/project-plans-for-a-hosted-gitea-online-service/">Loïc Dachary's
|
||
|
|
blog</a>.</em></p>
|
||
|
|
<hr />
|
||
|
|
<p>When an organization asks me about Gitea, I would like to direct them to
|
||
|
|
a provider where they can rent an instance and just use it, in the same
|
||
|
|
way they can go to https://discourse.org for a forum, or
|
||
|
|
https://nextcloud.com for storage. Instead of waiting for that to
|
||
|
|
happen, <a href="https://batsense.net/about/">Aravinth</a> and
|
||
|
|
<a href="https://dachary.org/">myself</a> decided to do something about it, in a
|
||
|
|
way that is in line with our shared values: transparency and Free
|
||
|
|
Software.</p>
|
||
|
|
<p>After doing some research we found counter examples that showed the
|
||
|
|
pitfalls to avoid. GitLab because its business model heavily relies on
|
||
|
|
selling proprietary licenses. CiviCRM because setting it up is complex
|
||
|
|
and requires training: users can't figure it out on their own. Gitea
|
||
|
|
images provided by Digital Ocean because they do not include security
|
||
|
|
upgrades. MySQL configured and run by AWS because of the vendor lock-in
|
||
|
|
that makes it impossible to self-host.</p>
|
||
|
|
<p>We concluded that an online service such as Gitea can be hosted in a
|
||
|
|
sustainable way as long as:</p>
|
||
|
|
<ul>
|
||
|
|
<li>It is well maintained and upgrades itself</li>
|
||
|
|
<li>It can be self-hosted</li>
|
||
|
|
<li>The service can automatically be restored from backups when the
|
||
|
|
underlying resources fail</li>
|
||
|
|
</ul>
|
||
|
|
<p>GitHub and GitLab make it look like there is a market around software
|
||
|
|
forges. It is however impossible to figure out if this market exists
|
||
|
|
only because it is based on proprietary software. How many of these
|
||
|
|
customers would pay for a Free Software hosted Gitea instance?</p>
|
||
|
|
<p>Even if these customers do exist, a new service provider would have to
|
||
|
|
figure out how to convince them to subscribe. The technical development
|
||
|
|
of the service can be done within weeks but building a sustainable
|
||
|
|
business takes much longer. Again, there were examples of what can go
|
||
|
|
wrong, for instance ElasticSearch. After years of work developing a
|
||
|
|
successful online service and a customer base, AWS entered the
|
||
|
|
competition and started to take it away from them.</p>
|
||
|
|
<p>The sustainability of the Free Software ecosystem is a new and very
|
||
|
|
difficult problem to solve. It is discussed more than it ever was in the
|
||
|
|
wake of security breaches originating from widely used and yet abandoned
|
||
|
|
library or disillusioned Free Software authors self-sabotaging their
|
||
|
|
next release, and everyone has diverging opinions. It falls on each Free
|
||
|
|
Software author to spend time to think about their own projects because
|
||
|
|
there are no handbook or good examples to follow. That is what Aravinth
|
||
|
|
and myself did to find a semblance of clarity and decide how to go about
|
||
|
|
this hosted Gitea service idea.</p>
|
||
|
|
<h1 id="sustaining-free-software-online-services">Sustaining Free Software online services<a class="zola-anchor" href="#sustaining-free-software-online-services" aria-label="Anchor link for: sustaining-free-software-online-services"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h1>
|
||
|
|
<h2 id="more-mature-online-services-mean-less-opportunities-to-sell-services">More mature online services mean less opportunities to sell services<a class="zola-anchor" href="#more-mature-online-services-mean-less-opportunities-to-sell-services" aria-label="Anchor link for: more-mature-online-services-mean-less-opportunities-to-sell-services"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h2>
|
||
|
|
<p>Ideally the software running an online service is rock solid and bugs
|
||
|
|
are so rare that it can run unattended. This is true of
|
||
|
|
https://wordpress.org and it is not uncommon for an instance to run for
|
||
|
|
years while upgrading themselves to get security patches. The cost of
|
||
|
|
maintaining such a service is negligible and hosting companies can offer
|
||
|
|
it for free to their customers. They make their profit by renting the
|
||
|
|
machines on which the service runs.</p>
|
||
|
|
<p>When the software is not as mature, it is more expensive to run. Bugs
|
||
|
|
need fixing, upgrades require manual intervention, users must be trained
|
||
|
|
to overcome the complexity of the user experience, etc. Well known
|
||
|
|
examples are Discourse or CiviCRM for which companies sell services to
|
||
|
|
overcome these issues.</p>
|
||
|
|
<p>But when an organization is both the author of the software and the
|
||
|
|
provider of paid services to compensate for its lack of maturity, it
|
||
|
|
creates a conflict of interest. Should they focus their effort on making
|
||
|
|
the software more mature, they would harm a business model that is based
|
||
|
|
on this very lack of maturity. For instance, if the author of a software
|
||
|
|
also sells training courses, they are not motivated to solve user
|
||
|
|
experience issues. If they did, it would lower the need for training
|
||
|
|
courses and reduce their income.</p>
|
||
|
|
<h2 id="free-software-online-services-in-the-wake-of-the-sustainability">Free Software online services in the wake of the sustainability<a class="zola-anchor" href="#free-software-online-services-in-the-wake-of-the-sustainability" aria-label="Anchor link for: free-software-online-services-in-the-wake-of-the-sustainability"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h2>
|
||
|
|
<p>crisis</p>
|
||
|
|
<p>Nowadays all Free Software authors struggle to get enough resources to
|
||
|
|
produce a steady stream of releases, even when the project is very
|
||
|
|
popular. This sustainability problem is getting more and more attention
|
||
|
|
as the number of Free Software projects in use world wide keeps growing.
|
||
|
|
Even the simplest online service relies on thousands of Free Software
|
||
|
|
projects, each of which needs work to keep going. Accidents caused by
|
||
|
|
poorly maintained Free Software projects become more frequent.</p>
|
||
|
|
<p>This Free Software sustainability crisis is barely emerging and very
|
||
|
|
much resembles ecological problems such as climate change. In both cases
|
||
|
|
it is very difficult to figure out how to properly care for the
|
||
|
|
resources that are consumed. After decades of advocacy, it is generally
|
||
|
|
accepted that fossil energy won't last forever but there still is a long
|
||
|
|
way to go. It will also take a long time for the Free Software community
|
||
|
|
to answer this simple question: how to sustain an ever growing library
|
||
|
|
of Free Software?</p>
|
||
|
|
<p>Luckily, it is relatively simpler to solve that problem for an online
|
||
|
|
service because it has users. They can be reminded that their assistance
|
||
|
|
is needed to keep the project afloat, for instance by a donation. A
|
||
|
|
proposition that would be much more difficult to make for the author of
|
||
|
|
a cryptographic library. Convincing users to pay for an online service
|
||
|
|
has the best chance of success when the author of the software is also
|
||
|
|
the service provider. This is the business model of Discourse and
|
||
|
|
Weblate, but it is relatively fragile because nothing stands in the way
|
||
|
|
of the competition.</p>
|
||
|
|
<p>A few years ago ElasticSearch successfully developed an online service
|
||
|
|
offering. But when AWS entered the competition and was better at
|
||
|
|
marketing it, ElasticSearch quickly realized they would likely go out of
|
||
|
|
business. They tried to fight back by <a href="https://www.elastic.co/blog/licensing-change">changing their
|
||
|
|
license</a>, which was the
|
||
|
|
wrong answer to a real problem. Discourse or Weblate are also likely to
|
||
|
|
face competition from hosting companies in the future and they may not
|
||
|
|
survive it.</p>
|
||
|
|
<p>In the end, the durable source of income for a Free Software online
|
||
|
|
service is to rent the resources (CPU/RAM/network/disk) it needs to run.
|
||
|
|
In other words only hosting companies can make a profit when running
|
||
|
|
such an online service. And for that reason they also need to share part
|
||
|
|
of the profits to ensure the sustainability of the Free Software service
|
||
|
|
their customers need.</p>
|
||
|
|
<h1 id="online-services-vendor-lock-in-is-cured-by-free-software">Online services vendor lock-in is cured by Free Software<a class="zola-anchor" href="#online-services-vendor-lock-in-is-cured-by-free-software" aria-label="Anchor link for: online-services-vendor-lock-in-is-cured-by-free-software"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h1>
|
||
|
|
<p>When hosting companies offer online services they also provide upgrades
|
||
|
|
and transparent recovery when the hardware fails. But none of them allow
|
||
|
|
the service to be self-hosted. When their price policy change, or when
|
||
|
|
the term of services ban users from a given country, migrating the
|
||
|
|
service elsewhere it costly and difficult. For instance when AWS runs
|
||
|
|
MySQL for their customers, they allow to download the data but not the
|
||
|
|
software that runs the proprietary AWS interface used to configure and
|
||
|
|
control the server. Another example is GitHub where the content of the
|
||
|
|
git repository can be downloaded but the code that runs GitHub itself is
|
||
|
|
not Free Software.</p>
|
||
|
|
<p>If a customer cannot run the same software as their service provider,
|
||
|
|
they are locked-in, even if they can download their data. It is a common
|
||
|
|
misconception to think that there is no vendor lock-in as long as it is
|
||
|
|
possible to download the data in an standard format. Migrating the data
|
||
|
|
from one software to another is, more often than not, time consuming and
|
||
|
|
costly to a point that it is effectively a blocker. A GitHub salesperson
|
||
|
|
would argue that it is possible for people to run GitHub Enterprise on
|
||
|
|
their own hardware. But the vendor lock-in is still present via the
|
||
|
|
proprietary license contract. The user experience, maintenance and
|
||
|
|
upgrades are still exclusively controlled by GitHub.</p>
|
||
|
|
<p>To guarantee their independence, the customers of an online service need
|
||
|
|
to be able to:</p>
|
||
|
|
<ul>
|
||
|
|
<li>Download their data</li>
|
||
|
|
<li>Run the exact same Free Software as their service provider</li>
|
||
|
|
<li>Run the exact same Free Software infrastructure as code as their
|
||
|
|
service provider</li>
|
||
|
|
</ul>
|
||
|
|
<p>The requirement regarding Free Software infrastructure as code refers
|
||
|
|
to, for instance, the AWS control panel and all that is behind it when
|
||
|
|
creating a new MySQL service. It includes whatever a competitor would
|
||
|
|
need to run the same online service. An example would be
|
||
|
|
https://enough.community, an infrastructure as code dedicated to
|
||
|
|
creating the services needed by whistleblowers and human rights
|
||
|
|
defenders. It consumes resources rented by hosting providers, assembles
|
||
|
|
disks and machines, setup monitoring and intrusion detection, installs
|
||
|
|
various online services and upgrades them.</p>
|
||
|
|
<p>The availability of the software that creates the infrastructure is not
|
||
|
|
only useful to the competitors of a service provider. It also benefits a
|
||
|
|
non-profit that wants to provide (for instance) Wordpress instances to
|
||
|
|
its members. Without it they would need to create something from scratch
|
||
|
|
using building blocks such as CiviCRM. Even though such building blocks
|
||
|
|
exist, this is a significant undertaking and effectively a blocker.</p>
|
||
|
|
<h1 id="federated-online-services-and-durability">Federated online services and durability<a class="zola-anchor" href="#federated-online-services-and-durability" aria-label="Anchor link for: federated-online-services-and-durability"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h1>
|
||
|
|
<p>All self-hosted services are in danger of losing the data they contain.
|
||
|
|
When a Wordpress service is hosted in a home and the machine dies, it
|
||
|
|
must be restored from backups... when there are backups. Hosting
|
||
|
|
companies ensure the durability of the data with their own backup
|
||
|
|
system. It creates a dilemma for people who are looking into self
|
||
|
|
hosting: independence is desirable, but is it worth taking the risk of
|
||
|
|
data loss?</p>
|
||
|
|
<p>Federated online services do not suffer from this problem, because they
|
||
|
|
can mirror each other. A Gitea instance that is federated with another
|
||
|
|
will mirror copies of software projects found on its peers. Should one
|
||
|
|
instance be destroyed, mirrored projects can be resurrected from the
|
||
|
|
federated instance. Not only is it a practical way to ensure the (rare)
|
||
|
|
failure of an entire datacenter, it also helps with the (more frequent)
|
||
|
|
destruction of self-hosted machines. Contrary to backups that require
|
||
|
|
special attention, the replication involved in federated online service
|
||
|
|
is built in and works continuously. There is no need for an extra backup
|
||
|
|
service that is very rarely used and therefore likely to fail when
|
||
|
|
needed.</p>
|
||
|
|
<p>Federated services are not yet mainstream and Gitea is one of the rare
|
||
|
|
services that started to implement the concept. In the interim,
|
||
|
|
customers of an online hosting service will need to worry about backups
|
||
|
|
to ensure the durability of their data. But the ultimate solution for
|
||
|
|
them won't be the emergence of an ideal backup infrastructure, it will
|
||
|
|
be replication (via federated services) that will continuously ensure
|
||
|
|
the durability of their data.</p>
|
||
|
|
<h1 id="paths-forward">Paths forward<a class="zola-anchor" href="#paths-forward" aria-label="Anchor link for: paths-forward"
|
||
|
|
><span class="anchor-icon">#</span></a
|
||
|
|
>
|
||
|
|
</h1>
|
||
|
|
<p>The Gitea project itself, following the footsteps of Discourse or
|
||
|
|
Weblate, could provide a hosting service. Part of its current user base
|
||
|
|
may become customers and there does not seem to be any blocker to make
|
||
|
|
that happen. As with most successful Free Software project, people
|
||
|
|
working on Gitea daily are already very busy and cannot engage in such a
|
||
|
|
long term project. But Aravinth and myself can, if they will have us.</p>
|
||
|
|
<p>Another path forward would be to wrap Gitea into a bundle that existing
|
||
|
|
hosting companies could easily use to provide such a service to their
|
||
|
|
customers. The biggest hosting companies are unlikely to be interested:
|
||
|
|
if Digital Ocean was to provide upgrades on top of their existing Gitea
|
||
|
|
image, they are more likely to rely on their internal staff to implement
|
||
|
|
that from scratch, as proprietary software integrated into their
|
||
|
|
existing infrastructure. But smaller hosting companies such as
|
||
|
|
https://Octopuce.fr or https://Easter-Eggs.com, who already deploy Gitea
|
||
|
|
instances for their customers, would use it if, for instance, it helped
|
||
|
|
with the upgrades. They would then kindly be reminded to give back a
|
||
|
|
share of their profits in order to sustain the development of the
|
||
|
|
service they deploy.</p>
|
||
|
|
<p>Finally it would also be possible to follow the example of GitLab in the
|
||
|
|
early days (before it turned to proprietary software) or Codeberg and
|
||
|
|
offer a free shared forge hosting service to build a user base. After a
|
||
|
|
few years, a percentage of the user base would convert to being paid
|
||
|
|
customers or donors to sustain the activity and part of the income would
|
||
|
|
be used to sustain the development of the service.</p>
|
||
|
|
</content>
|
||
|
|
</entry>
|
||
|
|
</feed>
|