131 lines
7.4 KiB
Markdown
131 lines
7.4 KiB
Markdown
|
I'd like to begin this month's Klankbord Digest with a server update.
|
||
|
|
Let's backtrack a bit to understand how the current circumstances
|
||
|
|
arose. The server began as a prototype on a Mac Pro Trash Bin computer
|
||
|
|
which I borrowed from a university. It had 16 GB of RAM and for this
|
||
|
|
reason, it was possible to run loads of software on the machine. For
|
||
|
|
this purpose, I used Incus, which is a Linux Container Management
|
||
|
|
System. It's a fork of LXD, which is made by Canonical. Canonical
|
||
|
|
decided to make LXD proprietary a while ago, and so Incus was the
|
||
|
|
natural alternative. So, I set up a bunch of containers inside of this
|
||
|
|
machine and these were capable of communicating with each other. There
|
||
|
|
was an advantage to using Incus over using LXD. For example, it was
|
||
|
|
possible to launch different operating systems on top of the host
|
||
|
|
operating system. And this was no longer easily doable with LXD.
|
||
|
|
|
||
|
|
I thought it would be a good idea to install Arch Linux on the Trash
|
||
|
|
Bin as the host operating system. This was probably not a good idea,
|
||
|
|
because Arch Linux offers very up-to-date software and is constantly
|
||
|
|
changing. If software is not kept up-to-date, things break. And when
|
||
|
|
the software is updated, sometimes things break. In any case, I wanted
|
||
|
|
to be able to launch different operating systems because various
|
||
|
|
pieces of software that I wanted to run on the machine were
|
||
|
|
well-suited to those operating systems. For example, I wanted to run
|
||
|
|
an OpenProject instance for collaborative project management. In the
|
||
|
|
past I attempted to build this Ruby on the Rails application from
|
||
|
|
source inside a Ubuntu container. I found that that simply didn't work
|
||
|
|
in the long-run. The background jobs processor wasn't working at
|
||
|
|
all. Everything got clogged up, and it crashed. Slowly. And that was
|
||
|
|
the end of that, because the system was simply unusable. Thus, running
|
||
|
|
OpenProject in a Debian environment, without running Debian on the
|
||
|
|
machine itself, was the main reason why I wanted to run
|
||
|
|
Incus. However, the OpenProject instance will not make a
|
||
|
|
reappearance. Partly because it used 1.6 GB of memory. This was not an
|
||
|
|
issue on the Trashbin, but was an issue on the next server.
|
||
|
|
|
||
|
|
Which brings me to a discussion of the next machine: a second-hand
|
||
|
|
Supermicro, which I picked up from v2_ around Christmas. This machine,
|
||
|
|
it turns out, used 90 watts of electricity per hour, which is a lot.
|
||
|
|
For whatever reason, the fan was running at full blast all the time.
|
||
|
|
Everyone who saw it said it shouldn't be doing that. But it was doing
|
||
|
|
it and it wasn't clear why. In any case, the task, as I saw it, before
|
||
|
|
I understood how much electricity this thing was consuming, was to
|
||
|
|
transfer the containers from the Trashbin to the Supermicro
|
||
|
|
machine. There were many containers and I wasn't going to use all of
|
||
|
|
them. Using a laptop and a VPN and an Ethernet cable, I was able to
|
||
|
|
transfer the required containers to the Supermicro
|
||
|
|
server. Technically, when running different operating systems with
|
||
|
|
Incus it's better to use KVMs, or Kernel Virtual Machines. As I wasn't
|
||
|
|
doing that it was already suffering from mis-configuration issues.
|
||
|
|
|
||
|
|
Furthermore, I decided to install Arch Linux on the Supermicro machine
|
||
|
|
as well. There was a bit of complexity involved in booting the server,
|
||
|
|
because it needed to be network booted, and that was something I had
|
||
|
|
not previously encountered. It was quite the adventure but I set it up
|
||
|
|
hastily. When I installed the operating system, it was an `fdisk`
|
||
|
|
situation. I set aside eight GB of SWAP space and a 120 GB root
|
||
|
|
partition, which makes limited sense. No way was there going to be 120
|
||
|
|
GB worth of software on the machine. More likely: it was going to run
|
||
|
|
out of RAM. At least things were up and running at this point. There
|
||
|
|
was the OpenProject. There was a live coding system, Flok. There was
|
||
|
|
a SFTP server. There was a container for the websites (why not just
|
||
|
|
use `/var/www/`...?). There was a Gitea instance and a Gitea runner.
|
||
|
|
There was a container for the PostgreSQL databases. And, finally, one
|
||
|
|
for the calendar also. These were all online at one point and using
|
||
|
|
up a lot of memory. The Supermicro server only had four gigabytes of
|
||
|
|
memory. Unsurprisingly, it could not handle all of this and
|
||
|
|
crashed. Therefore migrating some of the data / processes elsewhere
|
||
|
|
was required. The primary reason for this was to reduce memory
|
||
|
|
consumption. Accordingly the [calendar](https://calendar.klank.school)
|
||
|
|
moved to a VPS in Graz, Austria. This is now online and should stay
|
||
|
|
online as long as the VPS is going!
|
||
|
|
|
||
|
|
In any case, the supermicro server joined the Varia hub. Very
|
||
|
|
exciting. Everything was routed through a VPN, and I had to
|
||
|
|
reconfigure various web servers inside several containers in order to
|
||
|
|
make it work as expected. One advantage of using system containers on
|
||
|
|
a per-application basis is that a more granular picture of web
|
||
|
|
requests can be assembled. Each container had its own web server as
|
||
|
|
depicted in the following diagram.
|
||
|
|
|
||
|
|
```
|
||
|
|
live.klank.school
|
||
|
|
o
|
||
|
|
|
|
||
|
|
| code.klank.school
|
||
|
|
| o
|
||
|
|
| |
|
||
|
|
+-----------+---------+-------------------------------------+
|
||
|
|
| | | |
|
||
|
|
| +-------+---------+-----+ +-------------------------+ |
|
||
|
|
| | B | | | | B | |
|
||
|
|
| | O +---+---------+--+ | | O +-------+ +-------+ | |
|
||
|
|
| | X | | | | | | X | Nginx | | Gitea | | |
|
||
|
|
| K | | o HAProxy o--+--+---+----+---o---+--+---o | | |
|
||
|
|
| L | O | | | | | T | | | | | |
|
||
|
|
| A | N +---+------------+ | | H +-------+ +-------+ | |
|
||
|
|
| N | E | | | R | |
|
||
|
|
| K +-------+---------------+ | E | |
|
||
|
|
| | | E | |
|
||
|
|
| S | +-------------------------+ |
|
||
|
|
| E +-------+------------------+ |
|
||
|
|
| R | B | | each BOX is a container |
|
||
|
|
| V | O +--+----+ +------+ | |
|
||
|
|
| E | X | | | | Flok | | o---o is a web request |
|
||
|
|
| R | | o----+---+-o | | |
|
||
|
|
| | T | Nginx | | | | |
|
||
|
|
| | W +-------+ +------+ | |
|
||
|
|
| | O | |
|
||
|
|
| +--------------------------+ |
|
||
|
|
+-----------------------------------------------------------+
|
||
|
|
|
||
|
|
```
|
||
|
|
|
||
|
|
Despite the advantages of using containers, the decision was made to
|
||
|
|
stop using them. Some might say to use Docker instead, but I don't
|
||
|
|
think it makes a difference. There's no need to containerise all the
|
||
|
|
services on a machine. Excessive complexity and surprising
|
||
|
|
dependencies are a downside of containers. Thus, I preferred to keep
|
||
|
|
it simple because the environments were becoming a messy labyrinth of
|
||
|
|
proxy devices, system containers and storage units. I understood what
|
||
|
|
was going on but no-one else did. I wanted to change this, but
|
||
|
|
couldn't find quite the right moment to do so. And didn't. So I simply
|
||
|
|
pulled the plug on the Supermicro server.
|
||
|
|
|
||
|
|
To be honest, I was having a bit of a strange week when I made this
|
||
|
|
decision. However, it was the right decision to make, along with
|
||
|
|
changing the DNS provider. March 8th was a deadline which I set for
|
||
|
|
myself to change the DNS provider. This was insofar that, for the past
|
||
|
|
couple of years, March 8th has been a day of International
|
||
|
|
Trans*Feminist anti-colonial Counter-cloud action. Thus, in an effort
|
||
|
|
to defund the cloud, there's a new DNS provider.
|