Apple has recently overhauled its entire M-Series chip plans, scrapping the launch of the M6 Pro and M6 Max processors and jumping straight to the M7 series. While the base M6 SoC is expected to launch, Apple is moving into the M7 series without the M6 Pro/Max variants, with plans to offer some...
  • eicker@lemmy.worldEnglish
    3 hours

    1.5 TB of unified memory sounds less like a computer and more like Apple preparing for the moment your local AI starts asking for a raise. Plot twist: by 2028 the RAM upgrade still costs more than the rest of the machine combined.

    • AA5B@lemmy.worldEnglish
      55 minutes

      Remember this is “unified”, it’s not like you can upgrade, nor is it available in the “cheap” packaging we’re used to.

      You’ll get whatever Apple puts on the SoC, and you’ll be happy with it

      • eicker@lemmy.worldEnglish
        52 minutes

        The upside is that unified memory is genuinely different from traditional RAM. The CPU, GPU and Neural Engine all share the same memory pool, so data doesn’t need to be copied back and forth. That reduces latency, improves efficiency and lets AI models, graphics and other workloads access much larger datasets. It also uses less power and saves board space. The downside is obvious: because it’s integrated into the chip, you have to choose the right amount upfront, since it can’t be upgraded later.

  • khepri@lemmy.worldEnglish
    3 hours

    There’s going to be a massive boom in local llms if we get there. Already I have replaced ~80% of my paid token usage with a small local model running on my 64gb macbook pro, but being able to run a full-fidelity multi-hundred-billion parameter llm locally would be a game changer for my use case at least.

  • Phoenixz@lemmy.caEnglish
    4 hours

    You’ll only be able to afford 640KB, but it CAN go to 1.5TB!

    • Rai@lemmy.dbzer0.comEnglish
      7 hours

      Making a RAM drive to load a few minutes of rolling video game footage so it isn’t constantly writing to my SSD or HDD was one of the most “the future is now” things I’ve done lately… and it’s not a new concept, I just never considered it before.

  • Mereo@lemmy.caEnglish
    18 hours

    Basically Apple will be building the perfect computers to run local LLMs.

    • SuspiciousCarrot78@aussie.zoneEnglish
      2 hours

      I guess it depends on your definition of perfect - cheap, good or fast.

      This thing is probably going to cost at least $20K USD.

      Edit:

      “Next year’s base M7 processor is expected to arrive in the first half of 2027 and will also upgrade memory bandwidth to about 240 GB/s.”

      That’s…really fucking slow. What’s the goal here - CGI, engineering sims, game dev etc? 1.5TB is cool but at 240GB/s that will crawl for AI use.

      Comparison: this is about $100K, for 7.2TB/s, 252GB VRAM (+500GB system ram, so closer to 750GB total)

      https://www.nvidia.com/en-us/products/workstations/dgx-station/

    • foodandart@lemmy.zipEnglish
      18 hours

      One can only hope that it totally breaks the AI/LLM at industrial scale, so businesses can run their own AI systems with their own data sets.

      No more of this fucking datacanter horseshit.

      • Whostosay@sh.itjust.worksEnglish
        17 hours

        It’s pretty obvious at this point that the data centers are for storing massive amounts of video.

      • mysteryhumpf@feddit.orgEnglish
        15 hours

        Local LLMs are cool but also pretty slow compared to cloud. If you have to wait half an hour for your Feature while coding you might still opt for the cloud agent.

        • SuspiciousCarrot78@aussie.zoneEnglish
          3 hours

          Yeah. But they’re slow because most of us are GPU peasants. If someone were willing to drop $3-5K on a rig, they could probably run decent, dense models at greater than cloud speeds. Hell, with enough black magic, they could do it with less, but they’d have to go deep into the weeds.

          OTOH, $3-5K buys you a shit ton on Open Router, Claude, Chat, Lumo etc.

          The game is entirely rigged for “you will own nothing and be happy about it”.

        • f314@lemmy.worldEnglish
          13 hours

          Yes, they are slower. However, I think that the pricing we’re going to see from the cloud providers might be enough to deter quite a lot of people. At least I hope so:

          The fact that we’re already used to blazing speed generation kinda sucks. Local models are a much more sustainable way of unlocking the benefits of LLMs than giant ecosystem- and community-destroying data centers.

          • mysteryhumpf@feddit.orgEnglish
            9 hours

            I also hope that don’t get me wrong, but as I said: Waiting for the LLM agent to finish coding is currently a bottleneck in software development, they don’t pay high salaries for watching the AI code, they will prefer faster agents even if they are expensive, because they are not only paying the AI Company but also the software engineer overseeing them.

            • foodandart@lemmy.zipEnglish
              5 hours

              I think that is only going to last as long as the AI providers are willing to operate at a loss. The issue is even with the newer higher price points rolled out this year, they’re still losing money. The slower AI machines may be the answer once the REAL profit earning price for the use tokens hits the market. I forsee lots of alternative work going on while the small LLM’s are cooking the data. We will have to see once these machines start to roll out, what the use for LLMs will be and how it’s applied. I am hopeful.

          • mysteryhumpf@feddit.orgEnglish
            14 hours

            Yes ofc I ran Gemma 4 for example, but compare that to the speed of Gemini in the cloud the difference is massive.

            • irate944@piefed.socialEnglish
              12 hours

              How much RAM do you have and which version of the model did you run?

              Local LLMs can be just as fast as long your device clears the requirements. If you noticed a huge difference, there’s a really good chance that you tried to use a model that requires more RAM than you have

              • mysteryhumpf@feddit.orgEnglish
                9 hours

                I ran Gemma 4 31 B quantized so it fits in my RAM. The decoding speed was decent, but if you look at the newest models for example Gemini flash 3.5 they have a decoding speed of 280 token per second, they generate an entire page before my Mac locally generates a sentence.

                • irate944@piefed.socialEnglish
                  8 hours

                  That is a bit too much for your hardware, even the Q4_0. You needed a smaller version (26B likely would suit you better. It would be faster and is a MoE)

        • trougnouf@lemmy.worldEnglish
          14 hours

          Actually they can be much faster given sufficient VRAM and not a lot of concurrent users.

    • ranzispa@mander.xyzEnglish
      12 hours

      Nope, still running single core. Probably runs worse than on older CPUs which were optimized for single core clock speed.

      • boonhet@sopuli.xyzEnglish
        9 hours

        IPC in modern CPUs is up compared to 2007 or whatever though

  • ryper@lemmy.caEnglish
    18 hours

    A Mac with 1.5TB RAM would be expensive at the best of times. In 2027/2028 it might approach $50k, or even get into 6 figures.

    • h0rnman@lemmy.dbzer0.comEnglish
      8 hours

      With the Apple tax, I’d expect 1.5TB to run you closer to $200k. Enterprise prices are that high today, so if trends continue it’s going to be bad

    • lepinkainen@lemmy.worldEnglish
      14 hours

      Big household name gaming companies have devs who burn 20k in tokens A MONTH.

      A 50k machine that can run a model locally with 0 monthly costs will pay for itself in 3 months.

    • foodandart@lemmy.zipEnglish
      18 hours

      It shouldn’t be, if Apple gets that import exemption from the Chinese memory manufacturer (ChangXin Memory Technologies) they are asking for. Apparently they’re already testing the CXMT chips to put in the phones sold in China, freeing up the orders/stock they’ve sourced already from “safe” sources, to go into their products sold in the rest of the world.

      Smart move if they can finagle it.

      Hopefully they can get it through before the fuckwits in the administration understand how effectively it can threaten the big AI players that Trump seems to be sniffing around.

      You absolutely BET that he will scuttle any trade deal if it interferes with his own personal agenda WRT his investments in AI.

      He’s that much a greedy cunt.

      • Fizz@lemmy.nzEnglish
        11 hours

        apple sells ram at double market rate in the cheapest of times

      • starblursd@lemmy.zipEnglish
        15 hours

        Or the more likely option that they get the cheaper ram from China and then double dip and use the ram scarcity excuse for higher prices