Based on my experience, horizontal scaling1 for small-scale community hosting2 seems to be the most practical strategy. You can just combine whatever you have lying around (old laptops, cloud servers, phones, toasters) to host apps for your neighbourhood or city, without investing in specialised or expensive hardware.
Making use of multiple servers eventually creates a maintenance problem: how do you manage access for both your users and administrators for all your applications and servers? This is usually solved with LDAP or Ansible (or other configuration management tools). LDAP is difficult to understand and configure. Ansible roles are fragile and difficult to maintain.
I never really felt those solutions were sustainable for community hosters. You end up with a never ending maintenance and training TODO list just so you can keep things up and running.
You also can’t avoid that but the core of the problem for community hosters is that you need people with specialist skills to do this work and you don’t have a lot of those. You also typically have a lack of capacity for training. This is how you end up with a low beach-factor3 for your critical internal infrastructure for which no one will pay you to improve or maintain.
I wanted to share that there is also an approach to solving this with Rauthy. It does not solve the never ending TODO list but it makes it less of a critical point of failure because it allows people without specialist skills or training to participate in maintaining the system.
All you need is some bog standard system administrator skills and you’re good to go. That’s important because community hosters can’t afford to have participants sitting constantly behind a computer doing unpaid internal infrastructure work.
If we want to ditch Big Tech then we’re definitely going to need a lot more witches. In other words, we need more community hosters. One of our permanent tasks as community hosters ourselves is to motivate those who want to get involved in this struggle and who are willing to become system administrators. That demands a fresh look at the tools of our trade and their orientation and purpose. We need tools and working methods which are more inviting to newcomers.
This approach could potentially reduce maintenance and training work for community hosters. It could also potentially reduce technical barriers for new system administrators who can get into servers without immediately needing to rabbit hole a random internet article on public key cryptography which is often demoralising.
Configuring single sign-on for web application access with Rauthy is nothing new or special and is covered elsewhere on the interwebz. You have a single username/password for all applications and you connect the app and Rauthy through some standard client configuration.
Here’s where it starts to get interesting (and scary). Rauthy can also manage access to servers themselves via SSH and most (if not all) system level services using the PAM module.
The documentation is excellent. I would absolutely recommend you read that later because it covers a lot of the security concerns that you will need to keep in mind if you choose to go down this path.
Let’s however consider a practical example for now. If you have 5 servers and you want to have SSH access to all of them with the same SSH public key and user account, here’s what you’d roughly need to do.
root or postgres.Now, you can log in on all 5 servers without any further configuration.
What is happening under the hood is that your SSH config is using PAM
to authenticate your username and request your public SSH key from
Rauthy via /usr/sbin/rauthy-authorized-keys (just like
.ssh/authorized_keys).
When you run the Rauthy PAM install script, it will ask you if you
want to allow users who are in the rauthy-wheel group to
have sudo access. If you enable this, your Rauthy enabled
system users can do root escalation.
The password you need to type in can be generated on demand in the regular user Rauthy web UI. This means that you do not have to configure a root password for your user. It also means you don’t have to worry your password will get stolen over time. It’s generated by Rauthy as an ephemeral, auto-expiring password.
You can configure groups which can be attached to the user. These groups can be independent on the server or merged into existing groups. See the PAM Groups documentation for more.
Any user can self-service their own SSH public keys via the Rauthy regular user web UI. No editing files, no visiting multiple servers, running an Ansible role, querying LDAP, etc. etc. You just log into a web UI and click some buttons. As long as you have entered the SSH public key in the web UI and have the SSH private key, then you can enter the server.
As a Rauthy admin, you can configure a specific shell for a PAM user,
specifying /usr/bin/nologin to restrict access completely.
This is a radically different “one click” solution to a member of your
hosting collective taking a break vs. running a slow and fragile Ansible
role across multiple servers which breaks and costs you 3 hours to
fix.
The PAM/NSS machinery are literally core to how a *nix
system authenticates and resolves metadata about a system user. This
means you can connect your Rauthy enabled sytem users to most other
server tools such as SFTP for file sharing and hosting.
If you don’t want to use SSH keys for access, you can use ephemeral passwords in the same manner. SSH to the machine, generate your password, type it in at the prompt, annnddd you’re in. I understand that this can also be connected to USB passkeys for even less software configuration.
I think ephemeral passwords for server access are an especially nice consideration for new system administrators who might still be completely and utterly confused by SSH keys. New administrators can graduate from passwords to keys when they’re ready.
If your Rauthy instance goes down, don’t worry. PAM will also query your system configuration. So, you just need to have a contingency root account (which could be shared) for remote access or prioritise physical access to the servers to maintain them in failure scenarios.
I’d love to hear if you experiment further with this or want to share constructive critique. You can ping me on fedi.