Hostea
/
dashboard
2
2
Fork 1
Code Issues 11 Pull Requests Projects Releases Wiki Activity

Backend integration #1

Merged
realaravinth merged 14 commits from wip-enough into master 2022-06-25 13:11:56 +00:00
Conversation 5 Commits 14 Files Changed 25
realaravinth commented 2022-06-24 15:13:35 +00:00
Owner

Structure

  • infrastructure/utils.py: contains utilities to interact with the infrastructure Git repository
  • infrastructure/templates/infrastructure: contains configuration templates, stolen from the enough community docs

@dachary: When you have the time, please review infrastructure.utils.Infra and the templates in infrastructure/templates/infrasture

## Structure - infrastructure/utils.py: contains utilities to interact with the infrastructure Git repository - infrastructure/templates/infrastructure: contains configuration templates, stolen from [the enough community docs](https://enough-community.readthedocs.io/en/latest/services/hostea.html) @dachary: When you have the time, please review `infrastructure.utils.Infra` and the templates in `infrastructure/templates/infrasture`
realaravinth added 5 commits 2022-06-24 15:13:35 +00:00
ci/woodpecker/push/woodpecker Pipeline failed Details
ci/woodpecker/pr/woodpecker Pipeline failed Details
9f55a8ced7
feat: docs: hostea infrastructure config parameters
realaravinth changed title from Backend integration to WIP: Backend integration 2022-06-24 15:13:44 +00:00
realaravinth requested review from dachary 2022-06-24 15:13:56 +00:00
realaravinth self-assigned this 2022-06-24 15:14:19 +00:00
realaravinth commented 2022-06-24 15:14:55 +00:00
Poster
Owner

The CI failure is expected, I have to setup a Gitea service within the CI to properly run tests. Will implement tomorrow.

The CI failure is expected, I have to setup a Gitea service within the CI to properly run tests. Will implement tomorrow.
dachary requested changes 2022-06-24 15:35:41 +00:00
infrastructure/templates/infrastructure/sh/hostscripts/create.sh Outdated
@ -0,0 +1,2 @@
enough --domain d.hostea.org host create {{subdomain}}-host
dachary commented 2022-06-24 15:35:30 +00:00
Owner

d.hostea.org is correct but it should be set to d.$HOSTEA_DOMAIN (when HOSTEA_DOMAIN should be made available at installation time in the settings file created by the playbook).

**d.hostea.org** is correct but it should be set to **d.$HOSTEA_DOMAIN** (when HOSTEA_DOMAIN should be made available at installation time [in the settings file created by the playbook](https://gitea.hostea.org/Hostea/july-mvp/issues/36#issuecomment-664)).
👍 1
infrastructure/templates/infrastructure/yml/gitea.yml Outdated
@ -0,0 +5,4 @@
# Public hostname of the Gitea instance
#
#
gitea_host: "gitea.{{ domain }}"
dachary commented 2022-06-24 15:25:11 +00:00
Owner

Note: Enough will replace this {{ domain }} with the actual value, the dashboard does not need to do it. But the gitea. must be replaced by subdomain which is presumably the name asssigned to the instance by the dashboard.

Note: Enough will replace this {{ domain }} with the actual value, the dashboard does not need to do it. But the **gitea.** must be replaced by **subdomain** which is presumably the name asssigned to the instance by the dashboard.
👍 1
infrastructure/templates/infrastructure/yml/gitea.yml Outdated
@ -0,0 +12,4 @@
# Mailer from
#
#
gitea_mailer_from: "noreply@enough.community"
dachary commented 2022-06-24 15:32:12 +00:00
Owner

This must be replaced with noreply@{{ domain }}. It is set to enough.community to facilitate tests because the SMTP relay is very picky about non-existent domain names.

This must be replaced with **noreply@{{ domain }}**. It is set to **enough.community** to facilitate tests because the SMTP relay is very picky about non-existent domain names.
👍 1
infrastructure/templates/infrastructure/yml/gitea.yml Outdated
@ -0,0 +38,4 @@
#
# Admin user password
#
#gitea_password: etquofEtseudett
dachary commented 2022-06-24 15:26:17 +00:00
Owner

This must be set by the dashboard to a password that is then communicated to the user so they can login.

This must be set by the dashboard to a password that is then communicated to the user so they can login.
👍 1
infrastructure/templates/infrastructure/yml/gitea.yml Outdated
@ -0,0 +44,4 @@
#
# Admin user email
#
gitea_email: contact@enough.community
dachary commented 2022-06-24 15:27:59 +00:00
Owner

This must be set to the email of the user so they can receive notifications.

This must be set to the email of the user so they can receive notifications.
👍 1
infrastructure/templates/infrastructure/yml/gitea.yml Outdated
@ -0,0 +50,4 @@
#
# Unique hostname of the woodpecker server relative to {{ domain }}
#
woodpecker_hostname: "woodpecker"
dachary commented 2022-06-24 15:30:34 +00:00
Owner

This should probably be derived from the subdomain, like subdomainci?

This should probably be derived from the **subdomain**, like **subdomain**ci?
realaravinth commented 2022-06-24 15:38:37 +00:00
Poster
Owner

can we do sub-subdomain? If the user's Gitea is available at mygitea.hostea.org, then the CI at woodpecker.mygitea.hostea.org?

can we do sub-subdomain? If the user's Gitea is available at mygitea.hostea.org, then the CI at `woodpecker.mygitea.hostea.org`?
dachary commented 2022-06-24 16:26:06 +00:00
Owner

I'd rather not try my luck with this 😓 But it would be prettier.

I'd rather not try my luck with this 😓 But it would be prettier.
realaravinth commented 2022-06-25 10:20:26 +00:00
Poster
Owner

Acknowledged, post-MVP goal then :)

Acknowledged, post-MVP goal then :)
dachary marked this conversation as resolved
infrastructure/templates/infrastructure/yml/gitea.yml Outdated
@ -0,0 +68,4 @@
#
# Woodpecker shared agent secret `openssl rand -hex 32`
#
#woodpecker_agent_secret: c2cd326f7104c2ca93e6d22da0b28d1e33b4aeaa071c08945e0ade576b3192ce
dachary commented 2022-06-24 15:26:51 +00:00
Owner

This must be set by the dashboard but the user does not need to know about it.

This must be set by the dashboard but the user does not need to know about it.
realaravinth commented 2022-06-24 15:39:22 +00:00
Poster
Owner

Thanks for the quick and comprehensive view! I'll implement the changes you've suggested.

Thanks for the quick and comprehensive view! I'll implement the changes you've suggested.
dachary reviewed 2022-06-24 16:21:39 +00:00
requirements.txt
@ -15,2 +15,4 @@
django-phonenumber-field==6.3.0
djangorestframework==3.13.1
gitdb==4.0.9
GitPython==3.1.27
dachary commented 2022-06-24 16:21:39 +00:00
Owner

I tend to prefer using http://amoffat.github.io/sh/ like so👍

        self.git = sh.git.bake(_cwd=self.git_directory.name)
        self.git.init()

        p = self.gitea.projects.get(self.user, self.project)
        url = p.http_url_to_repo_with_auth
        self.git.config('http.sslVerify', 'false')
        self.git.remote.add.origin(url)
        self.git.fetch()

because (i) I know the git CLI and learning the python modules API has no appeal (ii) they struggle to catch up with what git does.

My 2cts ;-)

I tend to prefer using http://amoffat.github.io/sh/ like so👍 ``` self.git = sh.git.bake(_cwd=self.git_directory.name) self.git.init() p = self.gitea.projects.get(self.user, self.project) url = p.http_url_to_repo_with_auth self.git.config('http.sslVerify', 'false') self.git.remote.add.origin(url) self.git.fetch() ``` because (i) I know the git CLI and learning the python modules API has no appeal (ii) they struggle to catch up with what git does. My 2cts ;-)
realaravinth commented 2022-06-25 10:21:22 +00:00
Poster
Owner

Cool library, thanks for the suggestion. Works perfectly for our use case, I'll switch in a bit.

Cool library, thanks for the suggestion. Works perfectly for our use case, I'll switch in a bit.
realaravinth commented 2022-06-25 12:26:03 +00:00
Poster
Owner

PLease see here for create a new VM with subdomain=add_vm commit.

  • woodpecker_agent_secret is dynamically generated
  • d.$HOSTA_DOMAIN in hostscript is configurable via settings.py
  • gitea_password is generated and returned to user
  • gitea_email is set to user's email
PLease see [here](https://git.batsense.net/realaravinth/dummy-hostea-dash-test/commit/0474ea8f5f20d2f7c359fe80ebbc3e4337595309) for create a new VM with `subdomain=add_vm` commit. - `woodpecker_agent_secret` is dynamically generated - `d.$HOSTA_DOMAIN` in hostscript is configurable via settings.py - `gitea_password` is generated and returned to user - `gitea_email` is set to user's email
realaravinth added 8 commits 2022-06-25 12:33:34 +00:00
beb4b29c49
feat: pass template configuration, map VM sizes, generate secrets 
return gitea passwd, git pull before writing and push after add/rm
ci/woodpecker/push/woodpecker Pipeline failed Details
ci/woodpecker/pr/woodpecker Pipeline failed Details
871a05ddd3
feat: payment check before creation and save gitea passwd in DB
realaravinth requested review from dachary 2022-06-25 12:33:52 +00:00
realaravinth commented 2022-06-25 12:35:51 +00:00
Poster
Owner

Also, support for dashboard/local_settings.py is implemented.

This PR doesn't implement tests, I'll implement them tomorrow. If in hurry, please feel free to merge this PR.

Also, support for `dashboard/local_settings.py` is implemented. --- This PR doesn't implement tests, I'll implement them tomorrow. If in hurry, please feel free to merge this PR.
realaravinth commented 2022-06-25 12:41:31 +00:00
Poster
Owner

@dachary: the current version of Infra.add_vm method isn't idempotent, running it twice will create two commits as woodpecker_agent_secret and gitea_password will be overwritten. In such cases, will enough update the already created instance's woodpecker_agent_secretandgitea_password`?

I'll implement checks tomorrow to ensure that things aren't being overwritten

@dachary: the current version of `Infra.add_vm `method isn't idempotent, running it twice will create two commits as `woodpecker_agent_secret` and `gitea_password will be overwritten. In such cases, will enough update the already created instance's `woodpecker_agent_secret` and `gitea_password`? I'll implement checks tomorrow to ensure that things aren't being overwritten
realaravinth added 1 commit 2022-06-25 12:55:32 +00:00
ci/woodpecker/push/woodpecker Pipeline failed Details
ci/woodpecker/pr/woodpecker Pipeline failed Details
26b7ea3ef2
fix & rm: create/rm hostscripts, rm HOSTEA_DOMAIN
realaravinth changed title from WIP: Backend integration to Backend integration 2022-06-25 13:11:32 +00:00
realaravinth merged commit f0ee46e045 into master 2022-06-25 13:11:56 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
dachary
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
realaravinth
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Hostea/dashboard#1
There is no content yet.
Delete Branch "wip-enough"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?