It wasn’t for nothing, you got some learning out of the experience and a story to tell. Good luck with the new system, maybe hold upgrading that to testing for a while, there’s plenty to break and fix even without extra quirks from non-stable distribution :)
Have fun and feel free to ask for help again, I and others will be around to share what we’ve learned on our journeys.
You’re not worrying for nothing. Losing wall power will shut down the drives and as usb-cradle is generally slower than “proper” drive bus it’s more likely that some write operation is going on when power is lost and that’ll potentially cause data corruption. Obviously not every power outage will cause issues, but I’d say it’s a higher risk with USB-drives than with drives on a SATA/m.2 bus.
But no matter what your setup is, raid is not a backup. All kinds of things can happen which cause loss of data and you should plan accordingly. If all you have is two drives on usb-cradles I might choose to use one of them as a offline backup disk and one for ‘live’ data so that it’s more likely that at least one of the drives is functional even after power issues or whatever, but that approach has it’s own problems too.
There’s a walkman model which is pretty much just that which runs some flavour of android but I don’t know who they think their customer base is as the pricing is absolutely stupid. Top of the line model has gold plating and a nice 4k price tag. Also it apparently has ‘oxygen free copper’ and other audiophile bullshit, but no FM tuner.
And then there’s a ton of similar products from China but no idea which models (if any) are actually useful.
I had a two (or maybe a bit less) bitcoins on my wallet back in the day. I sold them for ~20€.
Well, the touchscreen part and maybe a bit more, had the same reaction on many directors at Nokia at the time. I don’t know if they feel like an idiot, but at least you’re not alone.
Cameras don’t stop anyone, but I still have few recording my yard. It’s more of a hobby and I’m planning to integrate person detection on those to home automation but for me it’s also a small piece of peace on my mind. Should someone steal my car trailer (or a car) I’d have some footage for the police and insurance. Also a while ago we had a decent storm around and we weren’t at home so it was nice that I could check for possible damages remotely.
But absolute majority of time I don’t even think about them. I don’t have any notifications enabled, I’m not interested about neighbors cat running across our yard or getting interruptions every time someone on the family comes or goes. And while Frigate has some AI things built in, the whole thing runs locally. There’s no way I’d install nest or some other camera which sends/stores data to anywhere which isn’t 100% in my control.
I haven’t tested lvm on arm-based systems, but if it doesn’t automatically locate them you could try pvscan/vgscan/lvscan.
Rootfs location is passed via kernel parameter, for example my grub.cfg has “set root=‘hd4,msdos1’”. That’s used by kernel and initramfs to locate the root filesystem and once ‘actual’ init process starts it already has access to rootfs and thus access to fstab. Initramfs update doesn’t affect on this case, however verifying kernel boot parameters might be a good idea.
Just in case you end up with reinstallation, I’d suggest using stable release for installation. Then, if you want, you can upgrade that to testing (and have all the fun that comes with it) pretty easily. But if you want something more like rolling release, Debian testing isn’t really it as it updates in cycles just like the stable releases, it just has a bit newer (and potentially broken) versions until the current testing is frozen and eventually released as new stable and the cycle starts again. Sid (unstable) version is more like a rolling release, but that comes even more fun quirks than testing.
I’ve used all (stable/testing/unstable) as a daily driver at some point but today I don’t care about rolling releases nor bleeding edge versions of packages, I don’t have time nor interest anymore to tinker with my computers just for the sake of it. Things just need to work and stay out of my way and thus I’m running either Debian stable or Mint Debian edition. My gaming rig has Bazzite on it and it’s been fine so far but it’s pretty fresh installation so I can’t really tell how it works in the long run.
We still have handful of those around at work. 2000, XP and maybe some embedded variant of 98 too still somewhere. They are controlling some non-critical but still useful industrial stuff with stupidly large price tag to replace.
Specially XP is still going to be around for quite a while in industrial settings where the production line is controlled via single computer and replacing it would mean replacing the whole line with price tag potentially in millions. And those aren’t even that old machines, their planning and manufacturing just takes “a while” due to certifications and everything.
I’d argue that if the plan is to run Debian testing it’s at the very least beneficial, if not mandatory, to learn some basics of the terminal. Debian doesn’t ship with sudo by default, so it’s either logging in directly as root or ‘su’. Instead of vim (which I’d personally use) I’d suggest nano, but with live setup it’s also possible to use mousepad or whatever gui editor happens to be available.
I suppose it’d be possible to use gparted or something to dig up the same information over GUI but I don’t have debian testing (nor any other live distro) at hand to see what’s available on it. I’m pretty sure at least stable debian installs with UUIDs by default, but I haven’t used installer from testing in a “while” so it might be different.
The way I’d try to solve this kind of problem would be to manually mount stuff from busybox and start bash from there to get “normal” environment running and then fix fstab, but it’s not the most beginner friendly way and requires some prior knowledge.
Do you happen to have any USB (or other) drives attached? Optical drive maybe? In the first text block kernel suggests it found ‘sdc’ device which, assuming you only have ssd and hdd plugged in and you haven’t used other drives in the system, should not exist. It’s likely your fstab is broken somehow, maybe a bug in daily image, but hard to tell for sure. Other possibility is that you still have remnants of Mint on EFI/whatever and it’s causing issues, but assuming you wiped the drives during installation that’s unlikely.
Busybox is pretty limited, so it might be better to start the system with a live-image on a USB and verify your /etc/fstab -file. It should look something like this (yours will have more lines, this is from a single-drive, single-partition host in my garage):
# / was on /dev/sda1 during installation
UUID=e93ec6c1-8326-470a-956c-468565c35af9 / ext4 errors=remount-ro 0 1
# swap was on /dev/sda5 during installation
UUID=19f7f728-962f-413c-a637-2929450fbb09 none swap sw 0 0
If your fstab has things like /dev/sda1 instead of UUID it’s fine, but those entries are likely pointing to wrong devices. My current drive is /dev/sde instead of comments on fstab mentioning /dev/sda. With the live-image running you can get all the drives from the system running ‘lsblk’ and from there (or running ‘fdisk -l /dev/sdX’ as root, replace sdX with actual device) you can find out which partition should be mounted to what. Then run ‘blkid /dev/sdXN’ (again, replace sdXN with sda1 or whatever you have) and you’ll get UUID of that partition. Then edit fstab accordingly and reboot.
If you’re talking about just moving the physical volumes (as in the actual hard drives) as is to another computer they’re automatically scanned and ready to go in majority of modern distributions. No need to export/import anything. This is obviously assuming your boot drive isn’t a part of volume group and you have healthy drives at your hands. You can test this with any live-distribution, just boot from USB into a new operating system and verify your physical volumes/volume groups from that.
If you want to move the volume group to a new set of disks simplest way would be to add physical drive(s) to volume group and then removing the old drive(s) from it after data has been copied. Search for pvmove and vgreduce. This obviously requires a working system, if your data drive has already failed it’s a whole another circus.
Microsoft deserves all the crap they’ve ordered, but skipping 9 on versioning was pretty smart move on their part. There’s still a ton of older software which just checks if windows version matches ‘windows 9’ to include both 95 and 98 (and all their variants). If 8.1 was released as 9 it would’ve broken a lot of compatibility which at least then was a big deal for Windows. And it still is, but now it seems that they’ll happily break everything from their most known product.
Even if your router acts as an DNS proxy it shouldn’t overload any pihole installation unless you have a crapload of devices doing millions of queries per hour. My pihole manages all my devices (20-30 individual things) without any problems and even if I hit some rate limit it’s going to be a change to default configuration, not a immovable object on your way. Based on quick glance over that reddit thread a new router might be a good option, but that’s another easy-ish task to accomplish. I use mikrotik device and I’m pretty happy with it but there’s a ton of good options.
For hiring someone to coach you I can see quite a few of potential issues. People who claim to know what they’re talking about but don’t really have the knowledge, straight up scammers obviously, mismatch in personal chemistry which will make learning unnecessarily difficult or even impossible, some people just aren’t good at teaching even if they do know their stuff and so on. By all means, use your money however you like, but I personally strongly advice against it unless you can get some courses on (preferably local) reputable vendor. You can look for online courses too, cisco has a ton of courses on networking, redhat has plenty of linux courses and other big players have their own training and even certificates if you want to go that far.
For pihole you don’t need support from router. It’s convenient if you can adjust dhcp-server settings so that pihole will automatically cover your whole network, but it’s not a requirement, you can just manually set each device to use pihole as DNS server. All you need is a static IP address outside your DHCP -pool. For spesific router configurations, you can ask those too, just include spesific model and possibly screenshots from your router interface.
That iMac of yours is more than enough to get you going. If you plan to run multiple things on it it might be good idea to look for hypervisors like proxmox or ovirt, but basic qemu+libvirt -setup on pretty much any linux-installation will work just fine too.
For the 3rd part, your concerns are mostly about networking and setting up pihole/other servers on your local network will gain you knowledge on how to manage that as well. Also, you can set up nextcloud/immich/whatever locally at first, get familiar with them and then allow access from the internet either via bitwarden or other tunneling or directly over public network. Latter has obviously way bigger threat models than using VPN and accessing stuff that way, but gladly the networking side of things is somewhat it’s own beast from the servers so you can build everything local only at first and then figure out what’s the best approach for you with remote access.
However right now I’m simply feeling overwhelmed and blocked.
I could explain to you in pretty decent detail how to build a setup which could cover pretty much every imaginable scenario for a home gamer, but that would also be suitable to serve a mid-sized company who’ll have multiple people on duty to manage the servers, storages, security, networking and other stuff. Also it’d cost roughly as much as a decent house. That’s close to the ‘big picture’ you’re looking for and equally overwhelming than your current situation. I’ve been earning my living with this stuff for quite a while now and there’s still a ton of things I’m at a very much beginner level. Maybe the difference now vs starting this is that I actually have some idea on things which I don’t know and thus I know when to learn more/ask from more experienced team members.
Just like eating an elephant, this field requires that you take it piece by piece. You’ll learn new things to build both your setup and your knowledge further, but if you try to eat it all at once it just doesn’t happen. First you need to decide a simple goal on what you want to get out of self hosting. DNS-based ad-blocking on your network is pretty neat and setting up pihole will get you started. Also with that you don’t need to allow any external connections to your network. Plus if something goes wrong you can easily just return to where you started from and try again. Setting your own router with DHCP, caching DNS and other stuff is pretty neat too and it’s also pretty simple to isolate from the rest of the network so you’ll have your ‘normal’ stuff still working while you learn for new things. Whatever it is, set up a relatively simple goal to work for. Then you can start to ask questions like ‘is raspberry pi 4 suitable for this’ or ‘what subnet I should use for my homelab’ or even ‘how to install debian on a old laptop to run pihole’.
Or if you really insist on going to the deep end, go to library and pick up TCP/IP Network Administration from O’reilly (altough that might be a bit outdated by now) or something similar and dig in. The o’reilly one has a bit over 700 pages to go trough. There’s equally in-depth books for linux administration, firewalls, network security and so on. Annas archive will most likely have some decent books too if you don’t care about legal issues and want to go trough brick-sized books as pdfs.
Some random thoughts about your points:
It’s a pretty damn big picture you’re looking at. Networking, backups, hypervisors, storage solutions, security and a lot of other topics are each big enough that you can make a career out of any of those alone. Obviously you don’t need to know everything about everything but as you learn more you’ll find more and more stuff to learn so I’d say there’s no practical way to learn ‘big picture’ just over a few hours of ‘lessons’. Also there’s a ton of variations on what one might consider as ‘self hosting’. Some will have setup comparable to decent sized company, others will have a single raspberry pi on top of their router.
Same goes here, it’s a pretty big field to go trough. The best setup for me is most likely very different from the best setup for you. Also with real world constraints (money, bandwidth, space available, electricity price…) the best setup is practically quaranteed to be some kind of compromise. Also, at least in my opinion, it makes sense to start with what you already have or can cheaply get, so that you’ll get something out of the system with as little investment as possible even if the first iteration might be a bit janky. Also your needs will likely change over time so the ‘optimal’ configuration for today might be wildly different from the configuration tomorrow.
This goes hand-in-hand with first point. You need to understand some basic networking, backup scenarios and proper threat mitigation against security threats, hardware failures, power outages and so on. Also there’s no ‘initial setup’ after which the system is complete as, again, your needs will change over time.
That’s why we’re here. Just describe your problems in a reasonably sized chunks. Don’t ask how to build a homelab but instead ask for something more spesific which doesn’t have a crapload of variables to figure out before getting to the actual problem.
For the money part, I’ve done stuff like this for companies (getting suitable hardware for their needs, setting it up, offering support…) as a freelancer and at least in here that’ll cost you 80-150€/h commercially. Even as a hobbyist I personally wouldn’t take that kind of contract as I heavily doubt that you’re willing to throw thousands of euros on the table (as properly going trough your list will take quite some time). However, if you can narrow things down and ask for something spesific I’ll happily reply to you around here for free if I happen to have time and/or knowledge about the matter.
So, figure out what you want from the system right now, what’s the first thing you want to build. It might be a hypervisor so you can keep experimenting with virtual machines, it might be a pihole for your network or something else, but you’ll need a pretty spesific goal. Then you can come back and ask more spesific questions and get deeper into the rabbit hole. Also, specially if you’re starting from scratch, there’s no such thing as a perfect setup. I’m working on a decent sized company with offices around the globe and even with those resources there’s still compromises with pretty much everything as cooling capacity, bandwidth, financial, man hours and other things aren’t infinite.
Just a few days ago I tried to feed my home automation logs to copilot in hopes that it might find a reason why my controller jams randomly multiple times per hour. It confidently claimed that as my noise level reported by controller is -100dB (so basically there’s absolutely nothing else on that frequency around, pretty much as good as it can get) it’s the problem and I should physically move the controller to less noisy area. A decent advice in itself, it might actually help on a lot of cases, but in my scenario it’s a completely wrong rabbit hole to dig in. I might still move the thing around to get better reception on some devices but it doesn’t explain why the whole controller freezes for several minutes on random intervals.
Get a REALLY good on-site support deal for your hardware. You also may need to adjust TTL values on your network stack.