services
Below is a catalog of everything that I run, categorized roughly by its purpose.
Version Control
- Gitea is the main service I use for version control
- Forgejo is something I've been experimenting with. Currently, it serves as a mirror, for public-facing things (e.g. structurizr diagram fragments).
- GitLab uses too many resources and does too many things for me to consider it to be a good candidate for my needs, but I keep it running as a mirror of certain Gitea repos just to say I have experience with GitLab.
CI/CD
- Agola has been my CI + CD solution of choice until recently. My needs have outgrown its capabilities, but I haven't migrated over all of the jobs because it still works for those purposes.
- Concourse is my CI solution for anything I'm writing these days.
- Argo CD handles the deployments once CI passes.
Documentation
- Dendron, a FOSS alternative to Obsidian. I have a CI/CD job that builds a static site on each push, and serves the output files via httpd.
- Backstage to wrangle documentation for all of my various projects and keep it in one central location
- Structurizr Lite ok honestly I don't even know why this is even still running, but it's there
- Stucturizr OnPrem for rapid iteration of structurizr diagrams. It hasn't reached feature parity with the cli, so it's rare I do heavy diagramming with it.
- Structurizr Mini for actually serving structurizr diagrams to users.
- Forgejo Pages as a FOSS alternative to Github Pages. Very often I've found it useful to make a repo's readme a discrete website.
Logging
- Grafana Alloy as a log collection agent. Previously, I used Promtail.
- Loki as a log ingester for Promtail.
Monitoring/Observability
- Jaeger for tracing; utilized mainly by traefik
- Prometheus as a centralized way to collect metrics
- Alertmanager for alerting
- cAdvisor to expose container-level metrics
- node_exporter to expose node-level metrics
- redis_exporter to expose metrics about redis instances
- rabbitmq_exporter to expose metrics about RabbitMQ
- kafka_exporter to expose metrics about Kafka
- transmission_exporter to expose torrent metrics
- mc-monitor to expose metrics about Minecraft worlds I run
- minecraft-exporter to expose player-level metrics in minecraft worlds. I have a writeup here about how I used this data to catch cheaters.
Visualization
- Grafana, the primary interface for visualizing everything observable.
- dvizz for easily visualizing the association of docker swarm nodes, services, and tasks.
- swarm-dashboard for visualizing docker swarm tasks, useful for when Grafana is having issues.
- swarmpit as a standalone way to view docker swarm metrics at-a-glance, as well as reading associated logs. Also mainly used when Grafana is having issues.
- rebrow for when
redis-cli
isn't enough - pgAdmin for when I need to manually interface with postgres. I don't like having tools installed on my workstation, so something accessible over the web is great.
- Redpanda Console as a single-pane-of-glass view for insight into Kafka operations.
Networking
- Traefik for reverse proxying and load balancing
- Nginx is run as a discrete sidecar to many services. Often, I've found it's faster to copy an existing nginx config than it is to rewrite it to work with traefik, and the overhead of running the nginx container is low.
- step-ca for generating HTTPS certificates - some services don't recognize self-signed certificates as "valid", so adding step-ca as a trusted CA and using its certificates instead is a handy workaround.
- CoreDNS for answering queries about
home.lab
domain and its subdomains - Technitium as a general-purpose DNS blackhole for ads and tracking
Messaging
- Kafka as the main pub/sub message queue. I do lots of web scraping, so most projects I spin up use this for delegation of tasks to scrapers.
- Apicurio Registry as a schema registry for Kafka.
- RabbitMQ as a simple message queue for various services that require it. I prefer to use Kafka where possible, but not everything supports it.
Databases
- postgres as a traditional SQL data store. It's rare that I write something that requires SQL, but it's my go-to for any service that requires a SQL data store.
- Cassandra as a storage backend for Jaeger.
- CouchDB as a general-purpose document store. Whenever I spin up a new project and I need a database, I use this.
- InfluxDB as a storage backend for Swarmpit.
- ArangoDB stores social media info that I've scraped.
Storage
- Minio for general object storage. Very frequently I'll need to make some file accessible over the internet, so I just drop it into a bucket and let Minio manage its lifecycle.
- Mimir as a storage backend for Prometheus
- Registry as a centralized location for pushing and pulling docker images
- Liget for a low-overhead nuget server.
- redis as a non-critical key-value store. It's proven to be most useful for quickly iterating upon object schema, when sharing information between processes.
- etcd as a key-value store for data I actually care about persisting in case of disaster
- Samba is used for sharing drives between nodes. Mainly for configuration, but also for media collections
File Sharing (P2P)
- Picoshare for easy short-term storage and sharing of files
- flood is used as a layer of abstraction for all automated services that need a torrent client, and as a UI when necessary
- transmission is behind flood. I've found it to be the most performant of all clients, when the total torrent volume is in the thousands.
- deluge is used for manually downloading torrents
Media
I have a full writeup here talking about how my media stack is configured.
- jellyfin is the primary way that I interface with my media collection
- emby is the best option for Samsung smart TVs, so I begrudgingly run it despite it not meeting my use-case or adhering to my philosophy.
- jellyseerr, a UI for managing requests for my media library
- jackett for better querying to trackers
- prowlarr for syncing indexers between radarr and sonarr
- sonarr for organizing tv shows
- radarr for organizing movies
- flaresolverr for bypassing cloudflare rate-limiting
- Airsonic Advanced for streaming my music library to other devices
Gaming
I run a cracked minecraft server, you can read about my setup here.
- infrared as a reverse proxy. I've found it to suit my needs better than mc-router despite not having as much flexibility.
- docker-mc-proxy to facilitate switching between minecraft worlds
- docker-minecraft-server for both Limbo worlds and the actual game worlds
- docker-mc-backup in order to back up my minecraft worlds
- RCON web admin as a container in order to delegate admin access without having to worry about cracked account impersonation.
Other
- zookeeper to coordinate Kafka and other things
- docker-postgres-backup-local for backing up postgres
- taiga for project/task management. It's pretty bad honestly, but I haven't gotten around to replacing it yet.
- string-is, for manipulating text without having to send it to some random person's webserver in order to do so.
- telegram bot api because I run a telegram bot, and the additional features granted by running your own instance are useful
- whoami because it has been proven immeasurably useful for debugging what a server "thinks" a request IP address is.