diff --git a/content/lab-notes/k8s.md b/content/lab-notes/k8s.md index 9faaedc..9d8f1a6 100755 --- a/content/lab-notes/k8s.md +++ b/content/lab-notes/k8s.md @@ -1,134 +1,92 @@ --- -title: Mira's attempts to understand Kuwubernetes +title: Mira attempts to lock the fuck in and figure out how to k8s date: 2025-04-07 --- +# Current Setup -# Install k3s, helm +Auxin (HTPC): + - OS: NixOS + - Service Runtume:Docker Compose + - Services: + - Syncthing + - Jellyfin + - Caddy (as reverse proxy only) + - Kodi + - Service Storage: + - Bind Mounts to NFS on NAS +Lipotropin (NAS): +- OS: Proxmox +- Services Runtime: K3S +- Services: + - adminer + - caddy + - copyparty + - forgejo + - jackett + - mariadb + - ntfy + - paperless+gotenberg+tika + - qbittorrent+gluetun + - radarr + - redis + - slskd +- Service Storage: + - NFS to spinning rust on same system +- Baremetal storage: + - 6 HDDs in BTRFS5 (50TB raw, 38TB usable) +Motilin (working hostname, Unused) -- run `curl -sfL https://get.k3s.io | sh - on first node -- `curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash` +Other Client Devices: +- Access files over NFS/SMB +- Access services via hostname (local), or URL (external) -`export KUBECONFIG=/etc/rancher/k3s/k3s.yaml` +---- -```yaml +# Endn't Point +Auxin +- Exclusive Services: + - Kodi +- Distributed Services via K8S +- Service Storage: ???? +- Baremetal storage: + - Various HDDs as available +Motilin + Lipotropin +- Distributed Services via K8S +Client Devices +- Still need to be able to access files over NFS/SMB +- Still need to access services via hostname (local), or URL (external) --- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - app: adminer - name: adminer -spec: - replicas: 1 - selector: - matchLabels: - app: adminer - template: - metadata: - labels: - app: adminer - spec: - containers: - - image: adminer:latest - name: adminer - ports: - - containerPort: 8080 ---- -apiVersion: v1 -kind: Service -metadata: - name: adminer-svc -spec: - type: ClusterIP - selector: - app: adminer - ports: - - port: 8080 - targetPort: 8080 ---- -apiVersion: networking.k8s.io/v1 -kind: Ingress -metadata: - name: adminer-http - annotations: - traefik.ingress.kubernetes.io/router.entrypoints: web -spec: - rules: - - host: adminer.lipotropin.lan - http: - paths: - - path: / - pathType: Prefix - backend: - service: - name: adminer-svc - port: - number: 8080 -``` - -- Use `kubectl apply -f ` to deploy a yaml file - -The above two result in the following - -``` -$ kubectl describe svc -Name: adminer -Namespace: default -Labels: io.kompose.service=adminer -Annotations: -Selector: io.kompose.service=adminer -Type: ClusterIP -IP Family Policy: SingleStack -IP Families: IPv4 -IP: 10.43.229.81 -IPs: 10.43.229.81 -Port: 8090 8090/TCP -TargetPort: 8080/TCP -Endpoints: 10.42.0.10:8080 -Session Affinity: None -Internal Traffic Policy: Cluster -Events: -``` - -Adminer can be accesed at `10.42.0.10:8080` - -> [!TODO] how tf do I access this from 127.0.0.1 or outside? +# Still need to figure out: +- How to handle jellyfin requiring GPU access + - nodes not guaranteed to have GPU, or may have heterogenous GPUs, need to lock to nodes with GPUs +- How do I handle failover + - current setup has SSH and HTTP traffic go to auxin, which proxies to lipotropin as needed + - how would I handle auxin going down + - ~~where the fuck would ssh go in general, it's not something reverse proxyable~~ + - Move SSH to router, use proxyjump from there + - see if Auto proxy and fallback are options + - can we do crowdsec and fail2ban on openwrt + - if I fucked up a config, how could I recover without incurring downtime +- How the *fuck* do I handle storage + - how do I do concurrent access, which services even allow that + - how the ever loving fuck do I make syncthing work + - ~~the intent is graceful failover, but can I do load balancing?~~ + - I do not have a good reason to have multiple instances of, cap replicas at 1 + - at best, jellyfin might benefit from multiple GPUs but also nobody is using your instance + - some services use SQLite, is there a way to mitigate concurrency issues with that + - on that note SQLite shits itself when FS access is too slow (eg NFS or ISCSI), how deal with that + - How to gradually migrate to new setup from 6 BTRFS drives on one machine + - The end goal is at least 3 nodes, but at some point there's only going to be 1 or 2 + - how would I minimize buying more storage/spending more money, while maintaining some redundancy + - is the best option just `btrfs remove` -> move drive -> format drive -> copy files from btrfs to new drive? + - new storage setup should be easy to add to (at a minimum, no homogenous drives) + - Is tiered storage something I can make use of, could I grab an SSD and use it as cache somehow + - how to give client devices and services access to the same filesystems + - eg (accessing `/downloads` from `qbittorrent`, `sonarr` and my phone via NFS/SMB) --- +# Rook: does it solve my storage issues? -- Deployment: seems to translate to docker service -- Service: seems to handle connecting to the deployment? -- NodePort: something to do with routing - - `kubectl expose deployment --port= --target-port= --type=NodePort` - -`kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml` -`export KUBECONFIG=/etc/rancher/k3s/k3s.yaml` - ---- - -# TODO: - -- [x] cyberchef -- [x] adminer -- [x] soulseek -- [x] postgres -- [x] mariadb -- [x] Ntfy -- [x] gluetun? -- [x] Jackett -- [x] qbittorrent -- [x] radarr -- [ ] sonarr -- [ ] syncthing -- [ ] forgejo -- [ ] kuma -- [ ] paperless-ngx - - [ ] goternberg - - [ ] redis - - [ ] tika -- [ ] jellyfin -- [ ] Website -- [ ] crowdsec -- [ ] dufs -- [ ] minecraft -- [ ] flaresolverr? +- Needs raw devices/partitions + - requires migration +# Longhorn: will it do shit? \ No newline at end of file diff --git a/content/links.html b/content/links.html index 2f6a142..25bee42 100755 --- a/content/links.html +++ b/content/links.html @@ -12,12 +12,12 @@ title: Links and Socials! - She/They + She/Her
-I don't know who you are or how you found this, but while you're here, feel - free to add me everywhere +I don't know who you are or how you found this, but while you're here, feel free to add me + everywhere