PocketOS founder blames ‘Cursor running Anthropic's flagship Claude Opus 4.6’ plus Railway’s infrastructure for data disaster.
  • fum@lemmy.worldEnglish
    3 hours

    This is absolutely hilarious. “AI” users getting what they deserve chef’s kiss

  • ZILtoid1991@lemmy.worldEnglish
    5 hours

    Always keep offline backup copies of your important data regardless of using AI slop to look over it! No, I don’t care that “optical media is obsolete and e-waste!”, or that “tapes are a 100 year old obsolete technology compared to cheap SSDs from TEMU!”.

    • PolarKraken@lemmy.dbzer0.comEnglish
      5 hours

      Optical media? Is that a viable part of backup strategies? I would expect tapes for sure, sounds like you know more than me.

      • ZILtoid1991@lemmy.worldEnglish
        4 hours
        1. Better than not having an offline copy.
        2. Write-only, ransomware cannot delete/encrypt it.
        3. Drives are still cheap.

        Downside is having techbros talk you about laser rot, how internal drives are obstructing the optimal airflow in GAMING PC cases, and how Gabe Newell is based and stuff.

        • PolarKraken@lemmy.dbzer0.comEnglish
          3 hours

          Great points! Lotta my optical media use also included hot summers in cars lol, nothing like an archival use.

      • katze@lemmy.4d2.orgEnglish
        4 hours

        A quality disc can last 10 years or more. At a company I used to work at the backups were burned to discs coated with gold. They had 15 year old discs that still worked.

        • PolarKraken@lemmy.dbzer0.comEnglish
          2 hours

          Dang that’s rad, had no idea (about it being used in such a way, I guess I mean, not too hard to imagine discs lasting that long).

  • Wispy2891@lemmy.worldEnglish
    4 hours

    To me it seems more criminal that the cloud provider has a “nuclear button” feature via the API that destroys everything including the backups with a single call and no confirmation whatsoever. What if the key gets accidentally leaked and someone wants to have fun?

  • prodaccess@lemmy.worldEnglish
    6 hours

    Like all interesting outages, there are probably multiple key action items.

    I’m also curious why deleting a “staging volume” would affect prod. I don’t know Railway, but it seems like a bad architectural design.

    • PolarKraken@lemmy.dbzer0.comEnglish
      4 hours

      Sounds like a responsible strategy to draw back from a lot of this. It’s all so…effervescently remade, the “ecosystem”, every few months.

      For me the takeaway comes from time I spent in some safety-critical parts of engineering and personal hobbies. Ultimately relying on people to make good decisions ~all of the time isn’t enough to prevent disaster, if something like disaster is on the line.

      Systems must be engineered to remove possibilities for accidentally bad, in-the-moment human decisions, where it counts. Thoughtfully. This is the weird same-shape but exactly-opposite doppelganger of that set of best practices.

      When the systems are using ~opaque automations that behave like humans (w.r.t. some decision-making and unreliable expectations of behavior) - and then relying on people making the right calls on top of that ever-shifting set of capabilities - I mean c’mon lol.

      This is gonna happen a lot, while the carrot of go-faster remains dangling so unignorably (because it’s in front of everyone, everyone working anywhere near the stuff). Until we look around and take a broader view. Which will be learned the same way we learned to make safety regulations, but I largely doubt our ability to respond in a similar way.

      The money will eventually respond, of course, but that’s always a poor and late proxy for what ought to be done.

      Sidenote, for aspiring engineers, take heart!

      It will be you who ends up tasked with unburying from all the technical debt incurred, truly. A practice steeped in the ancient wizardly traditions of yore. Spending a career on that and building something better.

      It will be necessary, the work begins roughly a while ago lol but more fully when things settle somewhat. Many large and slow organizations are right now very engaged in simply unburying themselves from the technical debt of a previous hype cycle, AKA now making use of all the data they collected (badly, via go-fast charlatans) during the “Big Data! You’ll be left behind if you don’t collect extreme amounts of data, it’s cheapish and everyone else is doing it!” era.

    • Sundray@lemmus.orgEnglish
      4 hours

      “If your prod can be deleted by your AI, it should be.”

  • skisnow@lemmy.caEnglish
    4 hours

    Wow there’s a lot of people in this thread defending the LLM. “They just didn’t set it up right” gtfo

    • RalfWausE_der_zwote@feddit.orgEnglish
      2 hours

      Why? They are right: If the LLM could nuke that system, so could the stupid but motivated intern. Their setup just sucked… a lot.

  • WhatsHerBucket@lemmy.worldEnglish
    11 hours

    “That’s ok, it will be great in robots with lethal weapons. What could go wrong? It’ll be the greatest killing machine, like you’ve never seen before”. 🫲 🍊 🫱

    • Napster153@lemmy.worldEnglish
      8 hours

      Can we make sure to make Ted Farro suffers worse this time?

      Being reduced to a mutant blob for, say, a few extra thousand years and maybe put in a zoo or something?

      • Pman@lemmy.orgEnglish
        8 hours

        Nah but that’s what he wanted, he is the truest form of tech bro, destroy the world, refuse to accept consequences of his actions, weaseled his way out of the situation and managed to, in the wake of unimaginable human suffering, get more power over people and has a god complex tell me this isn’t some or all the characteristics of people like Peter Theil, Elon Musk, Mark Zuckerberg, Sundar Pichai, Bill Gates, hell even Tim Cook and Steve Jobs before him. Punishment doesn’t stop this sort of behavior but removing the possibility of someone having that level of control over others is the only way but the richest and most powerful have always sought ways of amassing more power not realizing that that leads to worse off situations for everyone including themselves, Horizon did great encapsulating that trait in Faro, but be it him, the people behind Skynet, the Matrix or whatever other tech dystopia that tech bros seem pathologically unable to not try to make happen in the worst way possible is only the beginning, they seem to forget that even with advanced tech that serves their needs and wants, which won’t help their mental health, the people lower down on the rungs of society have brains, wants and needs, and they have more expertise in all sorts of things than the 1% are except for mass exploitation. This inevitably goes wrong one of a few ways, either everyone dies from the tech, or so many that societal collapse is inevitable not great and even if society survives it can’t functionally reconstitute itself; 2 they win and kill off or supress enough of society that the society becomes less productive and instead of fighting the powerful they flee or don’t participate in wealth generating for the rich were they don’t have to, maybe to rise up again later or the economy of the region just ignores them completely and the government protects themselves from their people more than anything else, or 3rd your revolution with terror campaigns against any and all who can be credibly accused of being part of the former tyrants. In all 3 cases the richer people end up poorer overall because wealth flees or dies in autocracy.

  • BlackLaZoR@lemmy.worldEnglish
    8 hours

    Learning from mistakes of people dumber than you isn’t a thing these days. Prepare for one AI disaster after another

  • percent@infosec.pubEnglish
    11 hours

    Seems like they were operating with a pile of bad practices, then threw AI into the mix.

    Neural networks are approximation algorithms. There’s a reason LLMs are generally more productive with statically typed languages, TDD, etc. They need those feedback loops and guard rails, or they’ll just carry on as if assuming they never make mistakes (which tends to have a compounding effect).

    If you want to use AI safely, you should be more defensive about it. It will fuck up; plan accordingly.

    • Kage520@lemmy.worldEnglish
      10 hours

      There really should be a certification course for using AI safely. I’m slop coding a hobby app and I’m shocked at how much it FEELS like it can do, because it can do amazing things, yet fails in the strangest ways. When it feels like it can get away with it, it forgets earlier discussions and moves on without it. So you can spend time hammering out a whole section of code, then move on, and AI will rip out everything that references that code and think of a different way in the moment and code that in instead. It won’t be the same. It probably won’t work, or at least won’t pass all test cases. But if you aren’t paying attention and keep coding, your original part of the project is no longer functioning and you won’t understand why. But every step of the way it’s confident in its answers and you won’t suspect that it fundamentally no longer understands the project.

      • ExFed@programming.devEnglish
        9 hours

        As someone who started writing software over 20 years ago (yikes I feel old), I feel like a lot of the best practices I’ve come to appreciate are really just strategies for mitigating future pain or boring/uninspiring work. When you eliminate most of the cost of rewriting everything from scratch by a machine that feels nothing, then “best practices” kinda lose their meaning.

        Edit: confusing sentence order.

        • Rooster326@programming.devEnglish
          9 hours

          I feel like a lot of the best practices I’ve come to appreciate are really just strategies for mitigating future pain or boring/uninspiring work.

          And now you know the difference between Intelligence and Wisdom.

          Also everything has a cost. The only time something has no cost is when you decide your life, your time, is meaningless.

      • mark@programming.devEnglish
        10 hours

        yup and when you DO catch it spitting out nonsense. it"ll say “oh you right, let me change that”… 🙄 like, why do I have to tell you that you’re wrong about something? You should already know it’s wrong and fix it without me ever pointing it out.

        • Rooster326@programming.devEnglish
          9 hours

          But it didn’t even understand it was wrong

          It can’t understand that. It can’t understand anything

          The Human-feedbaxk algorithm dictates humans prefer to receive an apology so it does.

        • SparroHawc@lemmy.zipEnglish
          9 hours

          That’s because it doesn’t really ‘know’ things in the same way you and I do. It’s much more like having a gut reaction to something and then spitting it out as truth; LLMs don’t really have the capability to ruminate about something. The one pass through their neural network is all they get unless it’s a ‘reasoning’ model that then has multiple passes as it generates an approximation of train-of-thought - but even then, its output is still a series of approximations.

          When its training data had something resembling corrections in it, the most likely text that came afterwards was ‘oh you’re right, let me fix that’ - so that’s what the LLM outputs. That’s all there is to it.

      • Rooster326@programming.devEnglish
        10 hours

        There is a course. It’s called experience. Common sense.

        All that any 4 hour YouTube/LinkedIn learning would-do would-be to perpetuate this idea that developers aren’t necessary. Take this course, buy these tokens and become A based God

  • dbtng@eviltoast.orgEnglish
    9 hours

    3-2-1
    Its really common for companies to not have an offsite backup. My own employer only offsites the customer data, not our core biz stuff. And I setup the offsite replication. It did not exist until I built it. (Proxmox Backup Server is tha best!)

    • ClownStatue@piefed.socialEnglish
      6 hours

      Seems like, if nothing else, Ai might finally force corporate accountants to acknowledge that the cost of a good backup strategy far outweighs the cost of losing all your data because some MBA thought he could write a product update himself with Claude code.

  • Fmstrat@lemmy.worldEnglish
    14 hours

    This guy.

    The PocketOS boss puts greater blame on Railway’s architecture than on the deranged AI agent for the database’s irretrievable destruction. Briefly, the cloud provider’s API allows for destructive action without confirmation, it stores backups on the same volume as the source data, and “wiping a volume deletes all backups.” Crane also points out that CLI tokens have blanket permissions across environments.

    Oh look, they have project level tokens: https://docs.railway.com/integrations/api#project-token

    They chose to give it full account access, including to production. But ohhhh nooooo it’s not MYYYY fault!

      • Fmstrat@lemmy.worldEnglish
        13 hours

        Oh yes, I skipped that part. Railway specifically explains their solutions are self-managed. If they were doing pgdumps to the same volume, that’s on them.

        If Railway loses business over this, they may have a libel claim. They’d never do it, but it wouldn’t be invalid.

        • el_abuelo@programming.devEnglish
          13 hours

          “It wouldn’t be invalid” isn’t the worst double negative in the world but it would be valid to say that it was unpleasant to read it when you could have used a less misdirecting choice of prose that wouldn’t have had such a negative effect on my reading comprehension. That is to say that I could have enjoyed it less but I certainly didnt enjoy it as much as i could have if you hadn’t used the double negative when a single positive wasn’t any further from reach.

          • Fmstrat@lemmy.worldEnglish
            12 hours

            I used a litote on purpose to soften the meaning. As for your overall reply, not bad.

          • Sims@lemmy.mlEnglish
            12 hours

            word people angry. me love. me have more. MOORH !!

          • mark@programming.devEnglish
            10 hours

            yes… lol people on HackerNews tend to do this a lot and it really does get annoying. it forces the reader to process what you’re trying to say unnecessarily.

      • Bilb!@lemmy.mlEnglish
        13 hours

        That’s doesn’t even really qualify as a backup. A snapshot, maybe.

  • LordCrom@lemmy.worldEnglish
    12 hours

    This was the exact plot of Silicon Valley when Son of Anton deleted the entire codebase as the most efficient way to remove bugs.