• 0 Posts
  • 33 Comments
Joined 20 days ago
cake
Cake day: September 14th, 2025

help-circle
  • I don’t know if there’s a term for them, but Bacula (and I think AMANDA might fall into this camp, but I haven’t looked at it in ages) are oriented more towards…“institutional” backup. Like, there’s a dedicated backup server, maybe dedicated offline media like tapes, the backup server needs to drive the backup, etc).

    There are some things that rsnapshot, rdiff-backup, duplicity, and so forth won’t do.

    • At least some of them (rdiff-backup, for one) won’t dedup files with different names. If a file is unchanged, it won’t use extra storage, but it won’t identify different identical files at different locations. This usually isn’t all that important for a single host, other than maybe if you rename files, but if you’re backing up many different hosts, as in an institutional setting, they likely files in common. They aren’t intended to back up multiple hosts to a single, shared repository.

    • Pull-only. I think that it might be possible to run some of the above three in “pull” mode, where the backup server connects and gets the backup, but where they don’t have the ability to write to the backup server. This may be desirable if you’re concerned about a host being compromised, but not the backup server, since it means that an attacker can’t go dick with your backups. Think of those cybercriminals who encrypt data at a company and wipe other copies and then demand a ransom for an unlock key. But the “institutional” backup systems are going to be aimed at having the backup server drive all this, and have the backup server have access to log into the individual hosts and pull the backups over.

    • Dedup for non-identical files. Note that restic can do this. While files might not be identical, they might share some common elements, and one might want to try to take advantage of that in backup storage.

    • rdiff-backup and rsnapshot don’t do encryption (though duplicity does). If one intends to use storage not under one’s physical control (e.g. “cloud backup”), this might be a concern.

    • No “full” backups. Some backup programs follow a scheme where one periodically does a backup that stores a full copy of the data, and then stores “incremental” backups from the last full backup. All rsnapshot, rdiff-backup, and duplicity are always-incremental, and are aimed at storing their backups on a single destination filesystem. A split between “full” and “incremental” is probably something you want if you’re using, say, tape storage and having backups that span multiple tapes, since it controls how many pieces of media you have to dig up to perform a restore.

    • I don’t know how Bacula or AMANDA handle it, if at all, but if you have a DBMS like PostgreSQL or MySQL or the like, it may be constantly receiving writes. This means that you can’t get an atomic snapshot of the database, which is critical if you want to be reliably backing up the storage. I don’t know what the convention is here, but I’d guess either using filesystem-level atomic snapshot support (e.g. btrfs) or requiring the backup system to be aware of the DBMS and instructing it to suspend modification while it does the backup. rsnapshot, rdiff-backup, and duplicity aren’t going to do anything like that.

    I’d agree that using the more-heavyweight, “institutional” backup programs can make sense for some use cases, like if you’re backing up many workstations or something.


  • Because every “file” in the snapshot is either a file or a hard link to an identical version of that file in another snapshot.) So this can be a problem if you store many snapshots of many files.

    I think that you may be thinking of rsnapshot rather than rdiff-backup which has that behavior; both use rsync.

    But I’m not sure why you’d be concerned about this behavior.

    Are you worried about inode exhaustion on the destination filesystem?


  • slow

    rsync is pretty fast, frankly. Once it’s run once, if you have -a or -t passed, it’ll synchronize mtimes. If the modification time and filesize matches, by default, rsync won’t look at a file further, so subsequent runs will be pretty fast. You can’t really beat that for speed unless you have some sort of monitoring system in place (like, filesystem-level support for identifying modifications).



  • sed can do a bunch of things, but I overwhelmingly use it for a single operation in a pipeline: the s// operation. I think that that’s worth knowing.

    sed 's/foo/bar/'  
    

    will replace all the first text in each line matching the regex “foo” with “bar”.

    That’ll already handle a lot of cases, but a few other helpful sub-uses:

    sed 's/foo/bar/g'  
    

    will replace all text matching regex “foo” with “bar”, even if there are more than one per line

    sed 's/\([0-9a-f]*\)/0x\1/g  
    

    will take the text inside the backslash-escaped parens and put that matched text back in the replacement text, where one has ‘\1’. In the above example, that’s finding all hexadecimal strings and prefixing them with ‘0x’

    If you want to match a literal “/”, the easiest way to do it is to just use a different separator; if you use something other than a “/” as separator after the “s”, sed will expect that later in the expression too, like this:

    sed 's%/%SLASH%g  
    

    will replace all instances of a “/” in the text with “SLASH”.


  • I would generally argue that rsync is not a backup solution.

    Yeah, if you want to use rsync specifically for backups, you’re probably better-off using something like rdiff-backup, which makes use of rsync to generate backups and store them efficiently, and drive it from something like backupninja, which will run the task periodically and notify you if it fails.

    rsync: one-way synchronization

    unison: bidirectional synchronization

    git: synchronization of text files with good interactive merging.

    rdiff-backup: rsync-based backups. I used to use this and moved to restic, as the backupninja target for rdiff-backup has kind of fallen into disrepair.

    That doesn’t mean “don’t use rsync”. I mean, rsync’s a fine tool. It’s just…not really a backup program on its own.



  • tal@olio.cafetoSelfhosted@lemmy.worldhow do I find process that leads to oom?
    link
    fedilink
    English
    arrow-up
    19
    arrow-down
    1
    ·
    edit-2
    2 days ago

    OOMs happen because your system is out of memory.

    You asked how to know which process is responsible. There is no correct answer to which process is “wrong” in using more memory — all one can say is that processes are in aggregate asking for too much memory. The kernel tries to “blame” a process and will kill it, as you’ve seen, to let your system continue to function, but ultimately, you may know better than it which is acting in a way you don’t want.

    It should log something to the kernel log when it OOM kills something.

    It may be that you simply don’t have enough memory to do what you want to do. You could take a glance in top (sort by memory usage with shift-M). You might be able to get by by adding more paging (swap) space. You can do this with a paging file if it’s problematic to create a paging partition.

    EDIT: I don’t know if there’s a way to get a dump of processes that are using memory at exactly the instant of the OOM, but if you want to get an idea of what memory usage looks at at that time, you can certainly do something like leave a top -o %MEM -b >log.txt process running to get a snapshot every two seconds of process memory use. top will print a timestamp at the top of each entry, and between the timestamped OOM entry in the kernel log and the timestamped dump, you should be able to look at what’s using memory.

    There are also various other packages for logging resource usage that provide less information, but also don’t use so much space, if you want to view historical resource usage. sysstat is what I usually use, with the sar command to view logged data, though that’s very elderly. Things like that won’t dump a list of all processes, but they will let you know if, over a given period of time, a server is running low on available memory.


  • tal@olio.cafetoTechnology@lemmy.world60hz Displays are a slideshow
    link
    fedilink
    English
    arrow-up
    3
    ·
    edit-2
    2 days ago

    I’m the other way. I’d rather have battery life on cell phones, and turn the refresh rate down.

    On a desktop, where the power usage is basically irrelevant, then sure, I’ll crank the refresh rate way up. One of the most-immediately-noticeable things is the mouse pointer, and that doesn’t exist on touch interfaces.


  • Hmm.

    So for some software, you can just increase the price.

    But…I wonder what that will do to the cost of video games. Typically, those are closer to one-off releases, not packages where new releases exist and are regularly purchased or subscriptions are in place.

    I’d expect this to increase the cost of maintenance, if there are legal obligations on publishers to monitor, notify, and deploy security fixes for their software and upstream. You’d think that it might encourage vendors to EOL software sooner; pull it off Steam or the like, mark as no longer supported.

    Maybe there are some exemptions somewhere that affect those.



  • I have a feeling its mostly due to some audio and video hardware that has some real longevity. I’ve got a VHS+minidv player that I am transferring old videos from using FireWire (well, for the minidv. VHS is s-video capture).

    Yeah, that’s a thought…though honestly, unless whatever someone is doing requires real-time processing and adding latency is a problem, they can probably pass it through some other old device that can speak both Firewire and something else.

    Probably the m-audio delta 1010

    That doesn’t have a Firewire interface, does it? I thought I had one of those.

    checks

    Oh, I’m thinking of the 1010LT, not the 1010. That lives on a PCI card.


  • If it’s Linux, sounds like it should just work out of box, at least for a while longer.

    https://www.tomshardware.com/news/linux-to-support-firewire-until-2029

    Linux to Support Firewire Until 2029

    The ancient connectivity standard still has years of life ahead of it.

    Firewire is getting a new lease on life and will have extended support up to 2029 on Linux operating systems. Phoronix reports that a Linux maintainer Takashi Sakamoto has volunteered to oversee the Firewire subsystem for Linux during this time, and will work on Firewire’s core functions and sound drivers for the remaining few that still use the connectivity standard.

    Further, Takashi Sakamoto says that his work will help users transition from Firewire to more modern technology standards (like perhaps USB 2.0). Apparently, Firewire still has a dedicated fanbase that is big enough to warrant six more years of support. But we suspect this will be the final stretch for Firewire support, surrounding Linux operating systems. Once 2029 comes around, there’s a good chance Firewire will finally be dropped from the Linux kernel altogether.




  • Is your concern compromise of your data or loss of the server?

    My guess is that most burglaries don’t wind up with people trying to make use of the data on computers.

    As to loss, I mean, do an off-site backup of stuff that you can’t handle losing and in the unlikely case that it gets stolen, be prepared to replace hardware.

    If you just want to keep the hardware out of sight and create a minimal barrier, you can get locking, ventillated racks. I don’t know how cost-effective that is; I’d think that that might cost more than the expected value of the loss from theft. If a computer costs $1000 and you have a 1% chance of it being stolen, you should not spend more than $10 on prevention in terms of reducing cost of hardware loss, even if that method is 100% effective.





  • I mean, I’m listing it because I believe that it’s something that has some value that could be done with the information. But it’s a “are the benefits worth the costs” thing? let’s say that you need to pay $800 and wear a specific set of glasses everywhere. Gotta maintain a charge on them. And while they’re maybe discrete compared to a smartphone, I assume that people in a role where they’re prominent (diplomacy, business deal-cutting, etc) probably know what they look like and do, so I imagine that any relationship-building that might come from showing that you can remember someone’s name and personal details (“how are Margaret and the kids?”) would likely be somewhat undermined if they know that you’re walking around with the equivalent of your Rolodex in front of your eyeballs. Plus, some people might not like others running around with recording gear (especially in some of the roles listed).

    I’m sure that there are a nonzero number of people who would wear them, but I’m hesitant to believe that as they exist today, they’d be a major success.

    I think that some of the people who are building some of these things grew up with Snow Crash and it was an influence on them. Google went out and made Google Earth; Snow Crash had a piece of software called Earth that did more-or-less the same thing (albeit with more layers and data sources than Google Earth does today). Snow Crash had the Metaverse with VR goggles and such; Zuckerberg very badly wanted to make it real, and made a VR world and VR hardware and called it the Metaverse. Snow Crash predicts people wearing augmented reality gear, but also talks about some of the social issues inherent with doing so; it didn’t expect everyone to start running around with them:

    Someone in this overpass, somewhere, is bouncing a laser beam off Hiro’s face. It’s annoying. Without being too obvious about it, he changes his course slightly, wanders over to a point downwind of a trash fire that’s burning in a steel drum. Now he’s standing in the middle of a plume of diluted smoke that he can smell but can’t quite see.

    It’s a gargoyle, standing in the dimness next to a shanty. Just in case he’s not already conspicuous enough, he’s wearing a suit. Hiro starts walking toward him. Gargoyles represent the embarrassing side of the Central Intelligence Corporation. Instead of using laptops, they wear their computers on their bodies, broken up into separate modules that hang on the waist, on the back, on the headset. They serve as human surveillance devices, recording everything that happens around them. Nothing looks stupider, these getups are the modern-day equivalent of the slide-rule scabbard or the calculator pouch on the belt, marking the user as belonging to a class that is at once above and far below human society. They are a boon to Hiro because they embody the worst stereotype of the CIC stringer. They draw all of the attention. The payoff for this self-imposed ostracism is that you can be in the Metaverse all the time, and gather intelligence all the time.

    The CIC brass can’t stand these guys because they upload staggering quantities of useless information to the database, on the off chance that some of it will eventually be useful. It’s like writing down the license number of every car you see on your way to work each morning, just in case one of them will be involved in a hit-and-run accident. Even the CIC database can only hold so much garbage. So, usually, these habitual gargoyles get kicked out of CIC before too long.

    This guy hasn’t been kicked out yet. And to judge from the quality of his equipment – which is very expensive – he’s been at it for a while. So he must be pretty good.

    If so, what’s he doing hanging around this place?

    “Hiro Protagonist,” the gargoyle says as Hiro finally tracks him down in the darkness beside a shanty. “CIC stringer for eleven months. Specializing in the Industry. Former hacker, security guard, pizza deliverer, concert promoter.” He sort of mumbles it, not wanting Hiro to waste his time reciting a bunch of known facts.

    The laser that kept jabbing Hiro in the eye was shot out of this guy’s computer, from a peripheral device that sits above his goggles in the middle of his forehead. A long-range retinal scanner. If you turn toward him with your eyes open, the laser shoots out, penetrates your iris, tenderest of sphincters, and scans your retina. The results are shot back to CIC, which has a database of several tens of millions of scanned retinas. Within a few seconds, if you’re in the database already, the owner finds out who you are. If you’re not already in the database, well, you are now.

    Of course, the user has to have access privileges. And once he gets your identity, he has to have more access privileges to find out personal information about you. This guy, apparently, has a lot of access privileges. A lot more than Hiro.

    “Name’s Lagos,” the gargoyle says.

    So this is the guy. Hiro considers asking him what the hell he’s doing here. He’d love to take him out for a drink, talk to him about how the Librarian was coded. But he’s pissed off. Lagos is being rude to him (gargoyles are rude by definition).

    “You here on the Raven thing? Or just that fuzz-grunge tip you’ve been working on for the last, uh, thirty-six days approximately?” Lagos says.

    Gargoyles are no fun to talk to. They never finish a sentence. They are adrift in a laser-drawn world, scanning retinas in all directions, doing background checks on everyone within a thousand yards, seeing everything in visual light, infrared, millimeter wave radar, and ultrasound all at once. You think they’re talking to you, but they’re actually poring over the credit record of some stranger on the other side of the room, or identifying the make and model of airplanes flying overhead. For all he knows, Lagos is standing there measuring the length of Hiro’s cock through his trousers while they pretend to make conversation.

    I think that Stephenson probably did a reasonable job there of highlighting some of the likely social issues that come with having wearable computers with always-active sensors running.