new deploy: 2022-06-02T10:52:01+00:00
dachary
parent
f8d917c244
commit
52b11b2bac
|
|
@ -4,8 +4,78 @@
|
||||||
<link href="https://hostea.org/blog/atom.xml" rel="self" type="application/atom+xml"/>
|
<link href="https://hostea.org/blog/atom.xml" rel="self" type="application/atom+xml"/>
|
||||||
<link href="https://hostea.org/blog/"/>
|
<link href="https://hostea.org/blog/"/>
|
||||||
<generator uri="https://www.getzola.org/">Zola</generator>
|
<generator uri="https://www.getzola.org/">Zola</generator>
|
||||||
<updated>2022-05-28T00:00:00+00:00</updated>
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
<id>https://hostea.org/blog/atom.xml</id>
|
<id>https://hostea.org/blog/atom.xml</id>
|
||||||
|
<entry xml:lang="en">
|
||||||
|
<title>[solved] Zombies created by Gitea version <= 1.16.8</title>
|
||||||
|
<published>2022-06-02T00:00:00+00:00</published>
|
||||||
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
|
<link href="https://hostea.org/blog/zombies/" type="text/html"/>
|
||||||
|
<id>https://hostea.org/blog/zombies/</id>
|
||||||
|
<content type="html"><p><strong>TL;DR: run Gitea version &gt;= 1.16.9 to avoid zombies</strong></p>
|
||||||
|
<p>The first <a href="https://github.com/go-gitea/gitea/issues/3242">issue about zombie processes</a> created by Gitea was reported in 2017 and <a href="https://github.com/go-gitea/gitea/issues/13987">resurfaced</a> on a <a href="https://github.com/go-gitea/gitea/issues/19077">regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Gitea updates a mirror by spawning the process <code>git remote update</code></li>
|
||||||
|
<li><code>git remote update</code> spawns yet another process, <code>git fetch</code></li>
|
||||||
|
<li><code>git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li>
|
||||||
|
<li>Gitea kill the process <code>git remote update</code></li>
|
||||||
|
<li>When killed <code>git remote update</code> does not kill its own child and <code>git fetch</code> becomes an orphaned process which keeps running</li>
|
||||||
|
<li>When <code>git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="pid-1-process-and-waiting-on-orphans">PID 1 process and waiting on orphans<a class="zola-anchor" href="#pid-1-process-and-waiting-on-orphans" aria-label="Anchor link for: pid-1-process-and-waiting-on-orphans"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is <code>/bin/init</code> and when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile">default docker image</a> this is <code>s6</code>: they will both wait on orphaned processes and there won't be any zombies.</p>
|
||||||
|
<h3 id="what-if-gitea-is-the-only-running-process">What if Gitea is the only running process?<a class="zola-anchor" href="#what-if-gitea-is-the-only-running-process" aria-label="Anchor link for: what-if-gitea-is-the-only-running-process"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>But when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile.rootless">rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||||||
|
</span><span>$ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||||||
|
</span></code></pre>
|
||||||
|
<p>The <code>git</code> command can then be replaced with a script that waits forever:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ( echo -e &#39;#!/bin/bash\nsleep infinity&#39; ) | docker exec -i --user root gitea tee /usr/bin/git
|
||||||
|
</span><span>$ docker exec --user root gitea chmod +x /usr/bin/git
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Trying to create a repository from the web interface will create the conditions for a zombie to show:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare
|
||||||
|
</span><span> 164 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>When the <code>git</code> process is killed by Gita, the <code>sleep</code> child will be orphaned:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Killing it will turn it into a zombie:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea kill 165
|
||||||
|
</span><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep [sleep]
|
||||||
|
</span></code></pre>
|
||||||
|
<h3 id="killing-a-child-process-and-all-its-children">Killing a child process and all its children<a class="zola-anchor" href="#killing-a-child-process-and-all-its-children" aria-label="Anchor link for: killing-a-child-process-and-all-its-children"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are <a href="https://en.wikipedia.org/wiki/Process_group">process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed.</p>
|
||||||
|
<p>A <a href="https://github.com/go-gitea/gitea/pull/19865">patch was introduced in Gitea 1.17</a> and backported to Gitea 1.16.9 so that all <code>git</code> commands are created as process group leaders and solve this problem for good.</p>
|
||||||
|
</content>
|
||||||
|
</entry>
|
||||||
<entry xml:lang="en">
|
<entry xml:lang="en">
|
||||||
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
||||||
<published>2022-05-28T00:00:00+00:00</published>
|
<published>2022-05-28T00:00:00+00:00</published>
|
||||||
|
|
|
||||||
|
|
@ -203,6 +203,41 @@
|
||||||
|
|
||||||
<ul class="blog__list">
|
<ul class="blog__list">
|
||||||
|
|
||||||
|
<li class="blog__post-item">
|
||||||
|
<a href="https://hostea.org/blog/zombies/" class="blog__post-link">
|
||||||
|
<h2 class="blog__post-title">[solved] Zombies created by Gitea version <= 1.16.8</h2>
|
||||||
|
<p class="blog__post-meta">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<a href="https://dachary.org" class="post__author">Loïc Dachary</a>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
· 2
|
||||||
|
June
|
||||||
|
|
||||||
|
,
|
||||||
|
2022 · <b>4 min read</b>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
|
||||||
|
<p class="blog__post-description">An increasing number of zombies processes are created by Gitea because it only kills its direct children on timeout. </p>
|
||||||
|
</a>
|
||||||
|
<div class="blog__post-tag-container">
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/hostea">#hostea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/gitea">#gitea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/troubleshoot">#troubleshoot</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/problem">#problem</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
|
||||||
<li class="blog__post-item">
|
<li class="blog__post-item">
|
||||||
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
||||||
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,360 @@
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<meta name="viewport" content="width=device-width" />
|
||||||
|
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png" />
|
||||||
|
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png" />
|
||||||
|
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png" />
|
||||||
|
<link rel="manifest" href="/site.webmanifest" />
|
||||||
|
<link rel="stylesheet" href="https://hostea.org/main.css" />
|
||||||
|
<link
|
||||||
|
rel="stylesheet"
|
||||||
|
media="screen and (max-width: 1300px)"
|
||||||
|
href="https://hostea.org/mobile.css"
|
||||||
|
/>
|
||||||
|
|
||||||
|
<meta name="referrer" content="no-referrer-when-downgrade" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<link rel="stylesheet" href="https://hostea.org/main.css" />
|
||||||
|
<link
|
||||||
|
rel="stylesheet"
|
||||||
|
media="screen and (max-width: 1300px)"
|
||||||
|
href="https://hostea.org/mobile.css"
|
||||||
|
/>
|
||||||
|
|
||||||
|
<meta name="referrer" content="no-referrer-when-downgrade" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
|
||||||
|
<title>[solved] Zombies created by Gitea version <= 1.16.8 | Hostea: Managed Gitea Hosting </title>
|
||||||
|
<meta name="referrer" content="no-referrer-when-downgrade" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
|
||||||
|
<meta name="description" content="An increasing number of zombies processes are created by Gitea because it only kills its direct children on timeout." />
|
||||||
|
|
||||||
|
|
||||||
|
<meta property="og:title" content="[solved] Zombies created by Gitea version <= 1.16.8 | Hostea: Managed Gitea Hosting " />
|
||||||
|
<meta property="og:type" content="article" />
|
||||||
|
<meta property="og:url" content="https://hostea.org" />
|
||||||
|
|
||||||
|
<meta property="og:description" content="An increasing number of zombies processes are created by Gitea because it only kills its direct children on timeout." />
|
||||||
|
<meta
|
||||||
|
property="og:site_name"
|
||||||
|
content="[solved] Zombies created by Gitea version <= 1.16.8 | Hostea: Managed Gitea Hosting "
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="57x57"
|
||||||
|
href="https://hostea.org/apple-icon-57x57.png?h=c21de14cfdf862a6472ae977557fa048a7c36d39337e61d3274705e9bd8e857f"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="60x60"
|
||||||
|
href="https://hostea.org/apple-icon-60x60.png?h=67089d9025a52d0d1ddce450078c7acefe2c150a2427dec9f5e13c6314f74281"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="72x72"
|
||||||
|
href="https://hostea.org/apple-icon-72x72.png?h=70725943de8884804f9da28202ced0ad6fed483ae9cf8f6d874aa133e30cb693"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="76x76"
|
||||||
|
href="https://hostea.org/apple-icon-76x76.png?h=1e6e8072df3b21bdcea254a42aac6e993611e845f91ddd79f6f35a6c441710a5"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="114x114"
|
||||||
|
href="https://hostea.org/apple-icon-114x114.png?h=c20099f8190ed3962fab5726c5594857a871cdb3ee98439343c622cd3727fed6"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="120x120"
|
||||||
|
href="https://hostea.org/apple-icon-120x120.png?h=4df78e402e60b58c6d44764678bdd737b5b6a836aeb85fb75fa49f706f7e8c81"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="144x144"
|
||||||
|
href="https://hostea.org/apple-icon-144x144.png?h=0c44e6655d714f89ee95cc151032d1f0dc3204bd24d1ca2ee9d94692d4ede84d"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="152x152"
|
||||||
|
href="https://hostea.org/apple-icon-152x152.png?h=157918f883ff95d4eeb6452d0ebb61ca5e21ea0dcac1aefe825f3e2f3999052f"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="apple-touch-icon"
|
||||||
|
sizes="180x180"
|
||||||
|
href="https://hostea.org/apple-icon-180x180.png?h=7d5c16d379b7db6d8ea5aae64921d7162b84f543763acd8fc7c107f80a600213"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="icon"
|
||||||
|
type="image/png"
|
||||||
|
sizes="192x192"
|
||||||
|
href="https://hostea.org/android-icon-192x192.png?h=095e3835b082dba07f606c33fa6f71bcd671a71e987b0ab2e46dcddceef52b9c"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="icon"
|
||||||
|
type="image/png"
|
||||||
|
sizes="32x32"
|
||||||
|
href="https://hostea.org/favicon-32x32.png?h=d7cd5d6390d58e729cd1f3564add60e9d8b63f54482a7f4cb5a66bb4780dfb05"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="icon"
|
||||||
|
type="image/png"
|
||||||
|
sizes="96x96"
|
||||||
|
href="https://hostea.org/favicon-96x96.png?h=5e01ce966b1d7ed88e0b01226d74ad8aaa65cea839073eb1ec6e115e76f3b2db"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="icon"
|
||||||
|
type="image/png"
|
||||||
|
sizes="16x16"
|
||||||
|
href="https://hostea.org/favicon-16x16.png?h=442e55b5177a8b501f75401b6b61bddace8d1ef8d91dab611fb1993293682ba5"
|
||||||
|
/>
|
||||||
|
<link
|
||||||
|
rel="manifest"
|
||||||
|
href="https://hostea.org/manifest.json?h=27eca3e8297eb7ff340deb3849b210185a459b3845456aa4d0036f6d966b3518"
|
||||||
|
/>
|
||||||
|
<meta name="msapplication-TileColor" content="#ffffff" />
|
||||||
|
<meta
|
||||||
|
name="msapplication-TileImage"
|
||||||
|
content="https://hostea.org/ms-icon-144x144.png?h=8170ab51b871b84b8f98bd03cf441afdffb2998b7dfffb04abb7ebf5deeb1f94"
|
||||||
|
/>
|
||||||
|
<meta name="theme-color" content="#ffffff" />
|
||||||
|
|
||||||
|
|
||||||
|
</head>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
</head>
|
||||||
|
<body class="base">
|
||||||
|
<header>
|
||||||
|
<nav class="nav__container">
|
||||||
|
<input type="checkbox" class="nav__toggle" id="nav__toggle" />
|
||||||
|
|
||||||
|
<div class="nav__header">
|
||||||
|
<a class="nav__logo-container" href="/">
|
||||||
|
|
||||||
|
<img src="https://hostea.org/android-icon-48x48.png?h=5115cfa26ec433a1f436236b2842c138d9d17f0c5a6376e3102c14e949dae1cb"
|
||||||
|
alt="Hostea temporary logo"/>
|
||||||
|
<p class="nav__home-btn">
|
||||||
|
ostea
|
||||||
|
</p>
|
||||||
|
</a>
|
||||||
|
<label class="nav__hamburger-menu" for="nav__toggle">
|
||||||
|
<span class="nav__hamburger-inner"></span>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="nav__spacer"></div>
|
||||||
|
|
||||||
|
<div class="nav__link-group">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<div class="nav__link-container">
|
||||||
|
<a class="nav__link" rel="noreferrer" href="/about/">About</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
<div class="nav__link-container">
|
||||||
|
<a class="nav__link" rel="noreferrer" href="/blog/">Blog</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
<div class="nav__link-container">
|
||||||
|
<a class="nav__link" rel="noreferrer" href="/contact/">Contact</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
<div class="nav__link-container">
|
||||||
|
<a class="nav__link" rel="noreferrer" href="/gitea-clinic/">Gitea Clinic</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
<div class="nav__link-container">
|
||||||
|
<a class="nav__link" rel="noreferrer" href="/talks/">Talks</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
</header>
|
||||||
|
<!-- See ../sass/main.scss. Required for pushing footer to the very
|
||||||
|
bottom of the page -->
|
||||||
|
<div class="main__content-container">
|
||||||
|
<main>
|
||||||
|
|
||||||
|
|
||||||
|
<div class="page__container">
|
||||||
|
<h1 class="page__group-title">[solved] Zombies created by Gitea version <= 1.16.8</h1>
|
||||||
|
<p class="blog__post-meta">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<a href="https://dachary.org" class="post__author">Loïc Dachary</a>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
· 2
|
||||||
|
June
|
||||||
|
|
||||||
|
,
|
||||||
|
2022 · <b>4 min read</b>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
|
||||||
|
<div class="blog__content">
|
||||||
|
<p><strong>TL;DR: run Gitea version >= 1.16.9 to avoid zombies</strong></p>
|
||||||
|
<p>The first <a href="https://github.com/go-gitea/gitea/issues/3242">issue about zombie processes</a> created by Gitea was reported in 2017 and <a href="https://github.com/go-gitea/gitea/issues/13987">resurfaced</a> on a <a href="https://github.com/go-gitea/gitea/issues/19077">regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Gitea updates a mirror by spawning the process <code>git remote update</code></li>
|
||||||
|
<li><code>git remote update</code> spawns yet another process, <code>git fetch</code></li>
|
||||||
|
<li><code>git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li>
|
||||||
|
<li>Gitea kill the process <code>git remote update</code></li>
|
||||||
|
<li>When killed <code>git remote update</code> does not kill its own child and <code>git fetch</code> becomes an orphaned process which keeps running</li>
|
||||||
|
<li>When <code>git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="pid-1-process-and-waiting-on-orphans">PID 1 process and waiting on orphans<a class="zola-anchor" href="#pid-1-process-and-waiting-on-orphans" aria-label="Anchor link for: pid-1-process-and-waiting-on-orphans"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is <code>/bin/init</code> and when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile">default docker image</a> this is <code>s6</code>: they will both wait on orphaned processes and there won't be any zombies.</p>
|
||||||
|
<h3 id="what-if-gitea-is-the-only-running-process">What if Gitea is the only running process?<a class="zola-anchor" href="#what-if-gitea-is-the-only-running-process" aria-label="Anchor link for: what-if-gitea-is-the-only-running-process"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>But when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile.rootless">rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||||||
|
</span><span>$ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||||||
|
</span></code></pre>
|
||||||
|
<p>The <code>git</code> command can then be replaced with a script that waits forever:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ( echo -e '#!/bin/bash\nsleep infinity' ) | docker exec -i --user root gitea tee /usr/bin/git
|
||||||
|
</span><span>$ docker exec --user root gitea chmod +x /usr/bin/git
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Trying to create a repository from the web interface will create the conditions for a zombie to show:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare
|
||||||
|
</span><span> 164 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>When the <code>git</code> process is killed by Gita, the <code>sleep</code> child will be orphaned:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Killing it will turn it into a zombie:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea kill 165
|
||||||
|
</span><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep [sleep]
|
||||||
|
</span></code></pre>
|
||||||
|
<h3 id="killing-a-child-process-and-all-its-children">Killing a child process and all its children<a class="zola-anchor" href="#killing-a-child-process-and-all-its-children" aria-label="Anchor link for: killing-a-child-process-and-all-its-children"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are <a href="https://en.wikipedia.org/wiki/Process_group">process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed.</p>
|
||||||
|
<p>A <a href="https://github.com/go-gitea/gitea/pull/19865">patch was introduced in Gitea 1.17</a> and backported to Gitea 1.16.9 so that all <code>git</code> commands are created as process group leaders and solve this problem for good.</p>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
<br>
|
||||||
|
<br>
|
||||||
|
<div class="blog__post-tag-container">
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/hostea">#hostea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/gitea">#gitea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/troubleshoot">#troubleshoot</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/problem">#problem</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
</main>
|
||||||
|
<footer>
|
||||||
|
<div class="footer__container">
|
||||||
|
<!-- <div class="footer__column"> --->
|
||||||
|
<p class="footer__column license__conatiner">
|
||||||
|
All text <a
|
||||||
|
class="license__link"
|
||||||
|
rel="noreferrer"
|
||||||
|
href="http://creativecommons.org/licenses/by-sa/4.0/"
|
||||||
|
target="_blank"
|
||||||
|
> CC-BY-SA </a
|
||||||
|
>
|
||||||
|
& code
|
||||||
|
<a
|
||||||
|
class="license__link"
|
||||||
|
rel="noreferrer"
|
||||||
|
href="https://www.gnu.org/licenses/agpl-3.0.en.html"
|
||||||
|
target="_blank"
|
||||||
|
> AGPL </a
|
||||||
|
>
|
||||||
|
|
|
||||||
|
<a
|
||||||
|
class="license__link"
|
||||||
|
rel="noreferrer"
|
||||||
|
href="https://www.eff.org/issues/do-not-track/amp/"
|
||||||
|
target="_blank"
|
||||||
|
> No AMP </a
|
||||||
|
>
|
||||||
|
</p>
|
||||||
|
<!-- </div> -->
|
||||||
|
<div class="footer__column--center">
|
||||||
|
<a href="/blog/atom.xml" target="_blank" rel="noopener" title="RSS">
|
||||||
|
<img
|
||||||
|
src="https://hostea.org/icons/rss.svg?h=f6cd584bdbcd2eb4d1b8b84c9cf083ef45f772167c33fdcee754b35ae8ff4c7d"
|
||||||
|
class="footer__icon"
|
||||||
|
alt="Email icon"
|
||||||
|
/>
|
||||||
|
</a>
|
||||||
|
</div>
|
||||||
|
<div class="footer__column">
|
||||||
|
<a href="/about" title="About">About</a>
|
||||||
|
<a href="/coc" title="Code of Conduct">CoC</a>
|
||||||
|
<span class="footer__column-divider--mobile-only">|</span>
|
||||||
|
<a href="/legalese" title="Legalese">Legalese</a>
|
||||||
|
<a href="/privacy-policy" title="Privacy Policy">Privacy</a>
|
||||||
|
<span class="footer__column-divider--mobile-only">|</span>
|
||||||
|
<a
|
||||||
|
href="https://stats.uptimerobot.com/EQ7VJHWylx"
|
||||||
|
rel="noreferrer"
|
||||||
|
target="_blank"
|
||||||
|
title="Status"
|
||||||
|
>Status</a
|
||||||
|
>
|
||||||
|
<a href="/tos" title="Terms of Service">ToS</a>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</footer>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -25,6 +25,10 @@
|
||||||
<loc>https://hostea.org/blog/unsafe-repository-is-owned-by-someone-else/</loc>
|
<loc>https://hostea.org/blog/unsafe-repository-is-owned-by-someone-else/</loc>
|
||||||
<lastmod>2022-05-15</lastmod>
|
<lastmod>2022-05-15</lastmod>
|
||||||
</url>
|
</url>
|
||||||
|
<url>
|
||||||
|
<loc>https://hostea.org/blog/zombies/</loc>
|
||||||
|
<lastmod>2022-06-02</lastmod>
|
||||||
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>https://hostea.org/coc/</loc>
|
<loc>https://hostea.org/coc/</loc>
|
||||||
</url>
|
</url>
|
||||||
|
|
|
||||||
|
|
@ -4,8 +4,78 @@
|
||||||
<link href="https://hostea.org/tags/gitea/atom.xml" rel="self" type="application/atom+xml"/>
|
<link href="https://hostea.org/tags/gitea/atom.xml" rel="self" type="application/atom+xml"/>
|
||||||
<link href="https://hostea.org"/>
|
<link href="https://hostea.org"/>
|
||||||
<generator uri="https://www.getzola.org/">Zola</generator>
|
<generator uri="https://www.getzola.org/">Zola</generator>
|
||||||
<updated>2022-05-28T00:00:00+00:00</updated>
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
<id>https://hostea.org/tags/gitea/atom.xml</id>
|
<id>https://hostea.org/tags/gitea/atom.xml</id>
|
||||||
|
<entry xml:lang="en">
|
||||||
|
<title>[solved] Zombies created by Gitea version <= 1.16.8</title>
|
||||||
|
<published>2022-06-02T00:00:00+00:00</published>
|
||||||
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
|
<link href="https://hostea.org/blog/zombies/" type="text/html"/>
|
||||||
|
<id>https://hostea.org/blog/zombies/</id>
|
||||||
|
<content type="html"><p><strong>TL;DR: run Gitea version &gt;= 1.16.9 to avoid zombies</strong></p>
|
||||||
|
<p>The first <a href="https://github.com/go-gitea/gitea/issues/3242">issue about zombie processes</a> created by Gitea was reported in 2017 and <a href="https://github.com/go-gitea/gitea/issues/13987">resurfaced</a> on a <a href="https://github.com/go-gitea/gitea/issues/19077">regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Gitea updates a mirror by spawning the process <code>git remote update</code></li>
|
||||||
|
<li><code>git remote update</code> spawns yet another process, <code>git fetch</code></li>
|
||||||
|
<li><code>git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li>
|
||||||
|
<li>Gitea kill the process <code>git remote update</code></li>
|
||||||
|
<li>When killed <code>git remote update</code> does not kill its own child and <code>git fetch</code> becomes an orphaned process which keeps running</li>
|
||||||
|
<li>When <code>git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="pid-1-process-and-waiting-on-orphans">PID 1 process and waiting on orphans<a class="zola-anchor" href="#pid-1-process-and-waiting-on-orphans" aria-label="Anchor link for: pid-1-process-and-waiting-on-orphans"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is <code>/bin/init</code> and when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile">default docker image</a> this is <code>s6</code>: they will both wait on orphaned processes and there won't be any zombies.</p>
|
||||||
|
<h3 id="what-if-gitea-is-the-only-running-process">What if Gitea is the only running process?<a class="zola-anchor" href="#what-if-gitea-is-the-only-running-process" aria-label="Anchor link for: what-if-gitea-is-the-only-running-process"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>But when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile.rootless">rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||||||
|
</span><span>$ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||||||
|
</span></code></pre>
|
||||||
|
<p>The <code>git</code> command can then be replaced with a script that waits forever:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ( echo -e &#39;#!/bin/bash\nsleep infinity&#39; ) | docker exec -i --user root gitea tee /usr/bin/git
|
||||||
|
</span><span>$ docker exec --user root gitea chmod +x /usr/bin/git
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Trying to create a repository from the web interface will create the conditions for a zombie to show:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare
|
||||||
|
</span><span> 164 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>When the <code>git</code> process is killed by Gita, the <code>sleep</code> child will be orphaned:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Killing it will turn it into a zombie:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea kill 165
|
||||||
|
</span><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep [sleep]
|
||||||
|
</span></code></pre>
|
||||||
|
<h3 id="killing-a-child-process-and-all-its-children">Killing a child process and all its children<a class="zola-anchor" href="#killing-a-child-process-and-all-its-children" aria-label="Anchor link for: killing-a-child-process-and-all-its-children"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are <a href="https://en.wikipedia.org/wiki/Process_group">process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed.</p>
|
||||||
|
<p>A <a href="https://github.com/go-gitea/gitea/pull/19865">patch was introduced in Gitea 1.17</a> and backported to Gitea 1.16.9 so that all <code>git</code> commands are created as process group leaders and solve this problem for good.</p>
|
||||||
|
</content>
|
||||||
|
</entry>
|
||||||
<entry xml:lang="en">
|
<entry xml:lang="en">
|
||||||
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
||||||
<published>2022-05-28T00:00:00+00:00</published>
|
<published>2022-05-28T00:00:00+00:00</published>
|
||||||
|
|
|
||||||
|
|
@ -209,6 +209,32 @@
|
||||||
</a></div>
|
</a></div>
|
||||||
<ul class="blog__list">
|
<ul class="blog__list">
|
||||||
|
|
||||||
|
<li class="blog__post-item">
|
||||||
|
<a href="https://hostea.org/blog/zombies/" class="blog__post-link">
|
||||||
|
<h2 class="blog__post-title">[solved] Zombies created by Gitea version <= 1.16.8</h2>
|
||||||
|
<p class="blog__post-meta">
|
||||||
|
2
|
||||||
|
June
|
||||||
|
|
||||||
|
,
|
||||||
|
2022 · <b>4 min read</b>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<p class="blog__post-description">An increasing number of zombies processes are created by Gitea because it only kills its direct children on timeout. </p>
|
||||||
|
</a>
|
||||||
|
<div class="blog__post-tag-container">
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/hostea">#hostea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/gitea">#gitea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/troubleshoot">#troubleshoot</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/problem">#problem</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
|
||||||
<li class="blog__post-item">
|
<li class="blog__post-item">
|
||||||
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
||||||
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
||||||
|
|
|
||||||
|
|
@ -4,8 +4,78 @@
|
||||||
<link href="https://hostea.org/tags/hostea/atom.xml" rel="self" type="application/atom+xml"/>
|
<link href="https://hostea.org/tags/hostea/atom.xml" rel="self" type="application/atom+xml"/>
|
||||||
<link href="https://hostea.org"/>
|
<link href="https://hostea.org"/>
|
||||||
<generator uri="https://www.getzola.org/">Zola</generator>
|
<generator uri="https://www.getzola.org/">Zola</generator>
|
||||||
<updated>2022-05-28T00:00:00+00:00</updated>
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
<id>https://hostea.org/tags/hostea/atom.xml</id>
|
<id>https://hostea.org/tags/hostea/atom.xml</id>
|
||||||
|
<entry xml:lang="en">
|
||||||
|
<title>[solved] Zombies created by Gitea version <= 1.16.8</title>
|
||||||
|
<published>2022-06-02T00:00:00+00:00</published>
|
||||||
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
|
<link href="https://hostea.org/blog/zombies/" type="text/html"/>
|
||||||
|
<id>https://hostea.org/blog/zombies/</id>
|
||||||
|
<content type="html"><p><strong>TL;DR: run Gitea version &gt;= 1.16.9 to avoid zombies</strong></p>
|
||||||
|
<p>The first <a href="https://github.com/go-gitea/gitea/issues/3242">issue about zombie processes</a> created by Gitea was reported in 2017 and <a href="https://github.com/go-gitea/gitea/issues/13987">resurfaced</a> on a <a href="https://github.com/go-gitea/gitea/issues/19077">regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Gitea updates a mirror by spawning the process <code>git remote update</code></li>
|
||||||
|
<li><code>git remote update</code> spawns yet another process, <code>git fetch</code></li>
|
||||||
|
<li><code>git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li>
|
||||||
|
<li>Gitea kill the process <code>git remote update</code></li>
|
||||||
|
<li>When killed <code>git remote update</code> does not kill its own child and <code>git fetch</code> becomes an orphaned process which keeps running</li>
|
||||||
|
<li>When <code>git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="pid-1-process-and-waiting-on-orphans">PID 1 process and waiting on orphans<a class="zola-anchor" href="#pid-1-process-and-waiting-on-orphans" aria-label="Anchor link for: pid-1-process-and-waiting-on-orphans"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is <code>/bin/init</code> and when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile">default docker image</a> this is <code>s6</code>: they will both wait on orphaned processes and there won't be any zombies.</p>
|
||||||
|
<h3 id="what-if-gitea-is-the-only-running-process">What if Gitea is the only running process?<a class="zola-anchor" href="#what-if-gitea-is-the-only-running-process" aria-label="Anchor link for: what-if-gitea-is-the-only-running-process"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>But when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile.rootless">rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||||||
|
</span><span>$ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||||||
|
</span></code></pre>
|
||||||
|
<p>The <code>git</code> command can then be replaced with a script that waits forever:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ( echo -e &#39;#!/bin/bash\nsleep infinity&#39; ) | docker exec -i --user root gitea tee /usr/bin/git
|
||||||
|
</span><span>$ docker exec --user root gitea chmod +x /usr/bin/git
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Trying to create a repository from the web interface will create the conditions for a zombie to show:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare
|
||||||
|
</span><span> 164 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>When the <code>git</code> process is killed by Gita, the <code>sleep</code> child will be orphaned:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Killing it will turn it into a zombie:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea kill 165
|
||||||
|
</span><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep [sleep]
|
||||||
|
</span></code></pre>
|
||||||
|
<h3 id="killing-a-child-process-and-all-its-children">Killing a child process and all its children<a class="zola-anchor" href="#killing-a-child-process-and-all-its-children" aria-label="Anchor link for: killing-a-child-process-and-all-its-children"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are <a href="https://en.wikipedia.org/wiki/Process_group">process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed.</p>
|
||||||
|
<p>A <a href="https://github.com/go-gitea/gitea/pull/19865">patch was introduced in Gitea 1.17</a> and backported to Gitea 1.16.9 so that all <code>git</code> commands are created as process group leaders and solve this problem for good.</p>
|
||||||
|
</content>
|
||||||
|
</entry>
|
||||||
<entry xml:lang="en">
|
<entry xml:lang="en">
|
||||||
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
||||||
<published>2022-05-28T00:00:00+00:00</published>
|
<published>2022-05-28T00:00:00+00:00</published>
|
||||||
|
|
|
||||||
|
|
@ -209,6 +209,32 @@
|
||||||
</a></div>
|
</a></div>
|
||||||
<ul class="blog__list">
|
<ul class="blog__list">
|
||||||
|
|
||||||
|
<li class="blog__post-item">
|
||||||
|
<a href="https://hostea.org/blog/zombies/" class="blog__post-link">
|
||||||
|
<h2 class="blog__post-title">[solved] Zombies created by Gitea version <= 1.16.8</h2>
|
||||||
|
<p class="blog__post-meta">
|
||||||
|
2
|
||||||
|
June
|
||||||
|
|
||||||
|
,
|
||||||
|
2022 · <b>4 min read</b>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<p class="blog__post-description">An increasing number of zombies processes are created by Gitea because it only kills its direct children on timeout. </p>
|
||||||
|
</a>
|
||||||
|
<div class="blog__post-tag-container">
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/hostea">#hostea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/gitea">#gitea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/troubleshoot">#troubleshoot</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/problem">#problem</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
|
||||||
<li class="blog__post-item">
|
<li class="blog__post-item">
|
||||||
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
||||||
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
||||||
|
|
|
||||||
|
|
@ -269,7 +269,7 @@
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
<span class="tag__meta">4 entries</span>
|
<span class="tag__meta">5 entries</span>
|
||||||
</a>
|
</a>
|
||||||
<a class="tag__rss-link" href="https://hostea.org/tags/gitea/atom.xml" target="_blank" rel="noopener" title="RSS">
|
<a class="tag__rss-link" href="https://hostea.org/tags/gitea/atom.xml" target="_blank" rel="noopener" title="RSS">
|
||||||
<img
|
<img
|
||||||
|
|
@ -287,7 +287,7 @@
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
<span class="tag__meta">5 entries</span>
|
<span class="tag__meta">6 entries</span>
|
||||||
</a>
|
</a>
|
||||||
<a class="tag__rss-link" href="https://hostea.org/tags/hostea/atom.xml" target="_blank" rel="noopener" title="RSS">
|
<a class="tag__rss-link" href="https://hostea.org/tags/hostea/atom.xml" target="_blank" rel="noopener" title="RSS">
|
||||||
<img
|
<img
|
||||||
|
|
@ -305,7 +305,7 @@
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
<span class="tag__meta">3 entries</span>
|
<span class="tag__meta">4 entries</span>
|
||||||
</a>
|
</a>
|
||||||
<a class="tag__rss-link" href="https://hostea.org/tags/problem/atom.xml" target="_blank" rel="noopener" title="RSS">
|
<a class="tag__rss-link" href="https://hostea.org/tags/problem/atom.xml" target="_blank" rel="noopener" title="RSS">
|
||||||
<img
|
<img
|
||||||
|
|
@ -323,7 +323,7 @@
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
<span class="tag__meta">3 entries</span>
|
<span class="tag__meta">4 entries</span>
|
||||||
</a>
|
</a>
|
||||||
<a class="tag__rss-link" href="https://hostea.org/tags/troubleshoot/atom.xml" target="_blank" rel="noopener" title="RSS">
|
<a class="tag__rss-link" href="https://hostea.org/tags/troubleshoot/atom.xml" target="_blank" rel="noopener" title="RSS">
|
||||||
<img
|
<img
|
||||||
|
|
|
||||||
|
|
@ -4,8 +4,78 @@
|
||||||
<link href="https://hostea.org/tags/problem/atom.xml" rel="self" type="application/atom+xml"/>
|
<link href="https://hostea.org/tags/problem/atom.xml" rel="self" type="application/atom+xml"/>
|
||||||
<link href="https://hostea.org"/>
|
<link href="https://hostea.org"/>
|
||||||
<generator uri="https://www.getzola.org/">Zola</generator>
|
<generator uri="https://www.getzola.org/">Zola</generator>
|
||||||
<updated>2022-05-28T00:00:00+00:00</updated>
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
<id>https://hostea.org/tags/problem/atom.xml</id>
|
<id>https://hostea.org/tags/problem/atom.xml</id>
|
||||||
|
<entry xml:lang="en">
|
||||||
|
<title>[solved] Zombies created by Gitea version <= 1.16.8</title>
|
||||||
|
<published>2022-06-02T00:00:00+00:00</published>
|
||||||
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
|
<link href="https://hostea.org/blog/zombies/" type="text/html"/>
|
||||||
|
<id>https://hostea.org/blog/zombies/</id>
|
||||||
|
<content type="html"><p><strong>TL;DR: run Gitea version &gt;= 1.16.9 to avoid zombies</strong></p>
|
||||||
|
<p>The first <a href="https://github.com/go-gitea/gitea/issues/3242">issue about zombie processes</a> created by Gitea was reported in 2017 and <a href="https://github.com/go-gitea/gitea/issues/13987">resurfaced</a> on a <a href="https://github.com/go-gitea/gitea/issues/19077">regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Gitea updates a mirror by spawning the process <code>git remote update</code></li>
|
||||||
|
<li><code>git remote update</code> spawns yet another process, <code>git fetch</code></li>
|
||||||
|
<li><code>git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li>
|
||||||
|
<li>Gitea kill the process <code>git remote update</code></li>
|
||||||
|
<li>When killed <code>git remote update</code> does not kill its own child and <code>git fetch</code> becomes an orphaned process which keeps running</li>
|
||||||
|
<li>When <code>git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="pid-1-process-and-waiting-on-orphans">PID 1 process and waiting on orphans<a class="zola-anchor" href="#pid-1-process-and-waiting-on-orphans" aria-label="Anchor link for: pid-1-process-and-waiting-on-orphans"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is <code>/bin/init</code> and when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile">default docker image</a> this is <code>s6</code>: they will both wait on orphaned processes and there won't be any zombies.</p>
|
||||||
|
<h3 id="what-if-gitea-is-the-only-running-process">What if Gitea is the only running process?<a class="zola-anchor" href="#what-if-gitea-is-the-only-running-process" aria-label="Anchor link for: what-if-gitea-is-the-only-running-process"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>But when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile.rootless">rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||||||
|
</span><span>$ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||||||
|
</span></code></pre>
|
||||||
|
<p>The <code>git</code> command can then be replaced with a script that waits forever:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ( echo -e &#39;#!/bin/bash\nsleep infinity&#39; ) | docker exec -i --user root gitea tee /usr/bin/git
|
||||||
|
</span><span>$ docker exec --user root gitea chmod +x /usr/bin/git
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Trying to create a repository from the web interface will create the conditions for a zombie to show:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare
|
||||||
|
</span><span> 164 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>When the <code>git</code> process is killed by Gita, the <code>sleep</code> child will be orphaned:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Killing it will turn it into a zombie:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea kill 165
|
||||||
|
</span><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep [sleep]
|
||||||
|
</span></code></pre>
|
||||||
|
<h3 id="killing-a-child-process-and-all-its-children">Killing a child process and all its children<a class="zola-anchor" href="#killing-a-child-process-and-all-its-children" aria-label="Anchor link for: killing-a-child-process-and-all-its-children"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are <a href="https://en.wikipedia.org/wiki/Process_group">process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed.</p>
|
||||||
|
<p>A <a href="https://github.com/go-gitea/gitea/pull/19865">patch was introduced in Gitea 1.17</a> and backported to Gitea 1.16.9 so that all <code>git</code> commands are created as process group leaders and solve this problem for good.</p>
|
||||||
|
</content>
|
||||||
|
</entry>
|
||||||
<entry xml:lang="en">
|
<entry xml:lang="en">
|
||||||
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
||||||
<published>2022-05-28T00:00:00+00:00</published>
|
<published>2022-05-28T00:00:00+00:00</published>
|
||||||
|
|
|
||||||
|
|
@ -209,6 +209,32 @@
|
||||||
</a></div>
|
</a></div>
|
||||||
<ul class="blog__list">
|
<ul class="blog__list">
|
||||||
|
|
||||||
|
<li class="blog__post-item">
|
||||||
|
<a href="https://hostea.org/blog/zombies/" class="blog__post-link">
|
||||||
|
<h2 class="blog__post-title">[solved] Zombies created by Gitea version <= 1.16.8</h2>
|
||||||
|
<p class="blog__post-meta">
|
||||||
|
2
|
||||||
|
June
|
||||||
|
|
||||||
|
,
|
||||||
|
2022 · <b>4 min read</b>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<p class="blog__post-description">An increasing number of zombies processes are created by Gitea because it only kills its direct children on timeout. </p>
|
||||||
|
</a>
|
||||||
|
<div class="blog__post-tag-container">
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/hostea">#hostea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/gitea">#gitea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/troubleshoot">#troubleshoot</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/problem">#problem</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
|
||||||
<li class="blog__post-item">
|
<li class="blog__post-item">
|
||||||
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
||||||
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
||||||
|
|
|
||||||
|
|
@ -4,8 +4,78 @@
|
||||||
<link href="https://hostea.org/tags/troubleshoot/atom.xml" rel="self" type="application/atom+xml"/>
|
<link href="https://hostea.org/tags/troubleshoot/atom.xml" rel="self" type="application/atom+xml"/>
|
||||||
<link href="https://hostea.org"/>
|
<link href="https://hostea.org"/>
|
||||||
<generator uri="https://www.getzola.org/">Zola</generator>
|
<generator uri="https://www.getzola.org/">Zola</generator>
|
||||||
<updated>2022-05-28T00:00:00+00:00</updated>
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
<id>https://hostea.org/tags/troubleshoot/atom.xml</id>
|
<id>https://hostea.org/tags/troubleshoot/atom.xml</id>
|
||||||
|
<entry xml:lang="en">
|
||||||
|
<title>[solved] Zombies created by Gitea version <= 1.16.8</title>
|
||||||
|
<published>2022-06-02T00:00:00+00:00</published>
|
||||||
|
<updated>2022-06-02T00:00:00+00:00</updated>
|
||||||
|
<link href="https://hostea.org/blog/zombies/" type="text/html"/>
|
||||||
|
<id>https://hostea.org/blog/zombies/</id>
|
||||||
|
<content type="html"><p><strong>TL;DR: run Gitea version &gt;= 1.16.9 to avoid zombies</strong></p>
|
||||||
|
<p>The first <a href="https://github.com/go-gitea/gitea/issues/3242">issue about zombie processes</a> created by Gitea was reported in 2017 and <a href="https://github.com/go-gitea/gitea/issues/13987">resurfaced</a> on a <a href="https://github.com/go-gitea/gitea/issues/19077">regular basis</a>. Although it does not look pretty, zombie processes are leftovers that do not consume resources and never caused any kind of harm. Here is one scenario that will create a zombie:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Gitea updates a mirror by spawning the process <code>git remote update</code></li>
|
||||||
|
<li><code>git remote update</code> spawns yet another process, <code>git fetch</code></li>
|
||||||
|
<li><code>git fetch</code> is stuck, for instance because of network problems, and Gitea eventually times out</li>
|
||||||
|
<li>Gitea kill the process <code>git remote update</code></li>
|
||||||
|
<li>When killed <code>git remote update</code> does not kill its own child and <code>git fetch</code> becomes an orphaned process which keeps running</li>
|
||||||
|
<li>When <code>git fetch</code> eventually completes it becomes a zombie because its original parent is no longer around to wait on it</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="pid-1-process-and-waiting-on-orphans">PID 1 process and waiting on orphans<a class="zola-anchor" href="#pid-1-process-and-waiting-on-orphans" aria-label="Anchor link for: pid-1-process-and-waiting-on-orphans"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>This scenario is not unique to Gitea and it is such a common pattern that safeguards have been implemented to mitigate the proliferation of zombies. Orphaned process are automatically attached to the process with PID 1, which is expected to wait on every process, whether it created them or not. When Gitea is installed from binary on GNU/Linux this is <code>/bin/init</code> and when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile">default docker image</a> this is <code>s6</code>: they will both wait on orphaned processes and there won't be any zombies.</p>
|
||||||
|
<h3 id="what-if-gitea-is-the-only-running-process">What if Gitea is the only running process?<a class="zola-anchor" href="#what-if-gitea-is-the-only-running-process" aria-label="Anchor link for: what-if-gitea-is-the-only-running-process"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>But when Gitea runs from the <a href="https://github.com/go-gitea/gitea/blob/6171ea7d318c0ca8714bc6efd6a97ea4b495eb6d/Dockerfile.rootless">rootless docker image</a>, Gitea is the only process running in the container. Orphaned processes will have Gitea as a parent but will not wait on them and they will stay in a zombie state forever. To reproduce this problem in a minimal way:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker run --name gitea -p 8080:3000 -e GITEA__security__INSTALL_LOCK=true -d gitea/gitea:1.16.8-rootless
|
||||||
|
</span><span>$ docker exec --user 1000 gitea gitea admin user create --admin --username root --password admin1234 --email root@example.com
|
||||||
|
</span></code></pre>
|
||||||
|
<p>The <code>git</code> command can then be replaced with a script that waits forever:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ( echo -e &#39;#!/bin/bash\nsleep infinity&#39; ) | docker exec -i --user root gitea tee /usr/bin/git
|
||||||
|
</span><span>$ docker exec --user root gitea chmod +x /usr/bin/git
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Trying to create a repository from the web interface will create the conditions for a zombie to show:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 164 git {git} /bin/bash /usr/bin/git -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true init --bare
|
||||||
|
</span><span> 164 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>When the <code>git</code> process is killed by Gita, the <code>sleep</code> child will be orphaned:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep sleep infinity
|
||||||
|
</span></code></pre>
|
||||||
|
<p>Killing it will turn it into a zombie:</p>
|
||||||
|
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ docker exec gitea kill 165
|
||||||
|
</span><span>$ docker exec gitea ps -o ppid,pid,comm,args
|
||||||
|
</span><span>PPID PID COMMAND COMMAND
|
||||||
|
</span><span> 0 1 gitea /usr/local/bin/gitea -c /etc/gitea/app.ini web
|
||||||
|
</span><span> 1 94 sleep [sleep]
|
||||||
|
</span><span> 1 99 sleep [sleep]
|
||||||
|
</span><span> 1 111 sleep [sleep]
|
||||||
|
</span><span> 1 165 sleep [sleep]
|
||||||
|
</span></code></pre>
|
||||||
|
<h3 id="killing-a-child-process-and-all-its-children">Killing a child process and all its children<a class="zola-anchor" href="#killing-a-child-process-and-all-its-children" aria-label="Anchor link for: killing-a-child-process-and-all-its-children"
|
||||||
|
><span class="anchor-icon">#</span></a
|
||||||
|
>
|
||||||
|
</h3>
|
||||||
|
<p>There should be no need for an admin running Gitea to worry about those gory details, it should be taken care of regardless of the environment Gitea runs in. Fortunately there is a very simple way to avoid the creation of zombies by ensuring that all Gitea child process are <a href="https://en.wikipedia.org/wiki/Process_group">process group leaders</a>. In a nutshell it means that when the child is killed all its children and grand children are also killed.</p>
|
||||||
|
<p>A <a href="https://github.com/go-gitea/gitea/pull/19865">patch was introduced in Gitea 1.17</a> and backported to Gitea 1.16.9 so that all <code>git</code> commands are created as process group leaders and solve this problem for good.</p>
|
||||||
|
</content>
|
||||||
|
</entry>
|
||||||
<entry xml:lang="en">
|
<entry xml:lang="en">
|
||||||
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
<title>[solved] Gitea 1.15 and up: path not found or permission denied</title>
|
||||||
<published>2022-05-28T00:00:00+00:00</published>
|
<published>2022-05-28T00:00:00+00:00</published>
|
||||||
|
|
|
||||||
|
|
@ -209,6 +209,32 @@
|
||||||
</a></div>
|
</a></div>
|
||||||
<ul class="blog__list">
|
<ul class="blog__list">
|
||||||
|
|
||||||
|
<li class="blog__post-item">
|
||||||
|
<a href="https://hostea.org/blog/zombies/" class="blog__post-link">
|
||||||
|
<h2 class="blog__post-title">[solved] Zombies created by Gitea version <= 1.16.8</h2>
|
||||||
|
<p class="blog__post-meta">
|
||||||
|
2
|
||||||
|
June
|
||||||
|
|
||||||
|
,
|
||||||
|
2022 · <b>4 min read</b>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<p class="blog__post-description">An increasing number of zombies processes are created by Gitea because it only kills its direct children on timeout. </p>
|
||||||
|
</a>
|
||||||
|
<div class="blog__post-tag-container">
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/hostea">#hostea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/gitea">#gitea</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/troubleshoot">#troubleshoot</a>
|
||||||
|
|
||||||
|
<a class="blog__post-tag" href="/tags/problem">#problem</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
|
||||||
<li class="blog__post-item">
|
<li class="blog__post-item">
|
||||||
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
<a href="https://hostea.org/blog/path-not-found/" class="blog__post-link">
|
||||||
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
<h2 class="blog__post-title">[solved] Gitea 1.15 and up: path not found or permission denied</h2>
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue