• zeca@lemmy.mlEnglish
    13 minutes

    I never understood how a statistical word-predicting model was expected to be obedient in the first place… of course we can train the model to say yes rather than no to command-sounding phrases, but thats a rather shallow mechanism.

  • cley_faye@lemmy.worldEnglish
    34 minutes

    I’m sure this will be fixed with an ever increasing context window and more “plz be nice” inserted left and right.

  • panda_abyss@lemmy.caEnglish
    6 hours

    Maybe, just maybe, don’t let your chat bot make executive decisions independently.

  • [deleted]@piefed.worldEnglish
    24 hours

    “Researchers find more defective chatbots that don’t follow instructions because glorified text completion doesn’t actually know or understand things.”

    It isn’t evade or ignoring. It is a fucking sentence autocomplete on steroids.

    • Cellari@lemmy.worldEnglish
      24 hours

      And then companies will just feed it more wild data from the users thinking that it will fix it eventually

  • pixxelkick@lemmy.worldEnglish
    23 hours

    They dont lol

    Pretty much always this is just the fact cheaper, especially free, chatbots, have very limited context windows.

    Which means the initial restrictions you set like “dont do this, dont touch that” etc get dropped, the LLM no longer has them loaded. But it does have in the past history the very clear and urgent directives of it trying to do this task, its important, so it’ll do whatever it autocompletes its gotta do to accomplish the task. And then… fucks something up.

    When you react to their fuck up, it *reloads the context back in

    So now the LLM has in its history just this:

    1. It doing a thing against the rules
    2. The user yelling at it
    3. The users now getting loaded after that on top

    So now the LLM is going to autocomplete its generated text on top being very apologetic and going on about how it’ll never happen again.

    Thats all there is to it.

    • cley_faye@lemmy.worldEnglish
      30 minutes

      Thats all there is to it.

      Not really. Even with (theoretical) infinite context windows, things would end up getting diluted. It’s a statistic machine; no matter how complex we make them look. Even with all the safeguards in place, as these grows larger and larger, each “directive” would end up being less represented in the next token.

      People can keep trying to hammer with a screwdriver all they want and keep being impressed when the bent nail is almost flush, though. I’m just enjoying the show from the side at this point.

    • village604@adultswim.fanEnglish
      2 hours

      It’s not just cheap agents. I’ve witnessed paid MS Copilot give a decade old depreciated Microsoft product in response to a single sentence prompt, then when called out a non-existent Microsoft product, then finally giving the right answer after being called out a second time.

      • pixxelkick@lemmy.worldEnglish
        2 hours

        LLMs are not good at answering fact based questions, fundamentally. Unless its an incredibly well known answer that has never changed (like a math or physics question), they dont magically “know” things.

        However, they’re way better at summarizing and reasoning.

        Give them access to playwright web search capability via MCP tooling to go research info, find the answer(s), and then produce output based on the results, and now you can get something useful.

        “Whats the best way to do (task)” << prone to failure, functional of how esoteric it is.

        “Research for me the top 3 best ways to do (task), report on your results and include your sources you found” << actually useful output, assuming you have something like playwright installed for it.

        • village604@adultswim.fanEnglish
          44 minutes

          A user on here built what appears to be a layer over the LLM that runs the query through several other processes first in an attempt to answer the question before it gets to the LLM, and I think it’s brilliant.

          They get bonus points because they made it so the reasoning the LLM uses is given to you. Although I haven’t fully gone through the documentation yet.

    • MalReynolds@slrpnk.netEnglish
      21 hours

      Cheap fuckers cheaping out, shocker (context is (V)RAM). AI speedrunning enshittification, who’d of thunk.

      • pixxelkick@lemmy.worldEnglish
        20 hours

        Uh… no its just the free models being free, theyre lower cost intentionally to provide free options for people who dont wanna pay subscription fees.

        (context is (V)RAM)

        Eh sort of, its more operating costs, the larger the context size the more expensive the model is to run, literally in terms of power consumption.

        Keep in mind we are on the scale of fractions of cents here, but multiply that by millions of users and it adds up fast.

        But the end result is that the agent will fuck stuff up, and will even quickly /forget/ it fucked that up if you dont catch it asap

        A lot of them have a context window that can be wiped out within like, 2 minutes of steady busywork…

        • Log in | Sign up@lemmy.worldEnglish
          3 hours

          I love how your response to the catastrophic results of stupidly trusting ai is “pay more money to ai companies”.

          Sane person’s response: don’t trust llms.

          • pixxelkick@lemmy.worldEnglish
            2 hours

            What are you talking about.

            No? I never said that.

            I just explained /why/ it happened, I literally nowhere in my post said, or implied, someone should pay for more expensive models. What are you smoking?

            You just have to be aware they have very short memory when using a cheap model and assume anything you wrote 1 minute ago has already left its memory, which is why they produce pretty dumb output if you try and depend on that… so… dont depend on that.

            • Log in | Sign up@lemmy.worldEnglish
              1 hour

              Everyone else who has any sense: llms are shit and you shouldn’t trust them with executive power.

              You: just the cheap ones.

              Me: no, all of them. What kind of lunatic trusts control of anything important to a fundamentally stochastic process?

  • XLE@piefed.socialEnglish
    24 hours

    The language in the linked post is disinformation. AI does not “scheme,” but that’s the wording the post uses for its duration. “Scheming” implies competence from a person. This post is evidence of a dysfunctional piece of software failing to work properly, made by apparently increasingly incompetent developers.

    Upon looking a little closer, this is a fearmonger website devoted to overinflating claims of AI power while ignoring real-life present-day harms. They claim to be inspired by Sam Bankman-Fried’s Effective Altruism scam. They show pictures of beautiful beaches but fail to mention AI’s environmental harms. Their paranoid demands, if enacted, would calcify Big Tech’s monopoly on AI and help nobody affected by its abuses on the planet.

    • RiverRabbits@lemmy.blahaj.zoneEnglish
      24 minutes

      thanks for this reply to the post and the clarification! The name of the website contains “longterm”, possibly in reference to “longtermism”, another framing of the effective altruism scam used to justify killing people today for some nebulous and flimsy “longterm minimising of deaths” because they assume their shitty text-predict machine will be a superior intelligence somehow.

      Anyways, it’s important to know their language to detect their bullshit quickly.

  • Codpiece@feddit.ukEnglish
    23 hours

    Sounds more like “Media find anti-AI angle that helps them get paid more for ad impressions”.