You have a lot of responses here, but I’ll tell what k8s actually is, since a lot of people seem to get this wrong.

Just like k8s, docker has many tools. Although docker is packaged in a way, that it looks like it’s just 1 tool. This is docker desktop. Under the hood there is docker engine that is really a runtime and image management service and API. You can look at this more if you wanted. There is containerd, runc, cri-o. These were all created so that different implementations can all talk to this API in a standard way and work.

Moving on to k8s. K8s is a way to scale these containers to run in different ways and scale horizontally. There are ways to even scale nodes vertically and horizontally to allow for more or less resources to place these containers on. This means k8s is very event driven and utilizes a lot of APIs to communicate and take action.

You said that you are doing kubectl apply constantly and you say feels wrong. In reality, this is correct. Under the hood you are talking with the k8s control plane and it’s taking that manifest and storing it. Other services are communicating with the control plane to understand what they have to do. In fact you can apply a directory of manifests, so you don’t have to specify each file individually.

Again there are many tools you can use to manage k8s. It is an orchestration system to manage pods and run them. You get to pick what tool you want to use. If you want something you can do from a git repo, you can use something like argocd or flux. This is considered to be gitops and more declarative. If you need a templating implementation, there are many, like helm, json net, and kustomize (although not a full templating language). These can help you define your manifests in a more repeatable and meaningful way, but you can always apply these using the same tools (kubectl, argocd, flux, etc…)

There are many services that can run in k8s that will solve one problem or another and these tools scale themselves, since they mostly all use the same designs that keep scalability in mind. I kept things very simple, but try out vanilla k8s first to understand what is going on. It’s great that you are questioning these things as it shows you understand there is probably something better that you can do. Now you just need to find the tools that are right for you. Ask what you hate or dislike about what you are doing and find a way to solve that and if there are any tools that can help. https://landscape.cncf.io/ is a good place to start to see what tools exist.

Anyway, good luck on your adventure. K8s is an enterprise tool after all and it’s not really meant for something like a home lab. It’s an orchestration system and NOT a platform that you can just start running stuff on without some effort. Getting it up and running is day 1 operations. Managing it and keeping it running is day 2 operations.

The answer to the first two questions are helm charts. They are collections of parametrized yaml files and the most popular way to install things into k8s. You just need one config file for each helm release (values.yaml).

If you want to go declarative with gitops rather than imperative, check ArgoCD or flux.

Hey there,

I made a similar journey a few years ago. But I only have one home server and do not run my services in high availability (HA). As @[email protected] mentioned, to run a service in HA, you need more than “just scaling up”. You need to exactly know what talks when to whom. For example, database entries or file writes will be difficult when scaling up a service not ready for HA.

Here are my solutions for your challenges:

• No, you are not supposed to run kubectl apply -f for each file. I would strongly recommend helm. Then you just have to run helm install per service. If you want to write each service by yourself, you will end up with multiple .yaml files. I do it this way. Normally, you create one repository per service, which holds all YAML files. Alternatively, you could use a predefined Helm Chart and just customize the settings. This is comparable to DockerHub.
• If you want to deploy to a cluster, you just have to deploy to one server. If in your .yaml configuration multiple replicas are defined, k8s will automatically balance these replicas on multiple servers and split the entire load on all servers in the same cluster. If you just look for configuration examples, look into Helm Charts. Often service provide examples only for Docker (and Docker Compose) and not for K8s.
• As I see it, you only have to run a single line of install script on your first server and afterward join the cluster with the second server. Then you have k3s deployed. Traefik will be installed alongside k3s. If you want to access the dashboard of Traefik and install rancher and longhorn, yes, you will have to run multiple installations. Since you already have experience with Ansible, I suggest putting everything for the “base installation” into one playbook and then executing this playbook one.

Changelog:

• Removeing k3s install command. If you want to use it, look it up on the official website. Do not copy paste the command from a random user on lemmy ;) Thanks to @[email protected] for bringing up this topic.

I’m the creator of ansible k3s playbooks, event if I not more active maintener. But there’s a big community : give a try and contrib https://github.com/k3s-io/k3s-ansible