Anthropic uncovered 10,000 vulnerabilities through Project Glasswing, driving urgent patching efforts and stronger cyber defenses.
  • fiat_lux@lemmy.zipEnglish
    14 days

    Let’s not bury that image content.

    23,019 potential vulnerability candidates -> 1,900 Reviewed by external security firms -> 1726 confirmed positive -> 467 reported to maintainers

    Why only review 1900? How were these chosen? Were the 1259 that were not reported to maintainers just duplicates or were they even valid?

    23,019 potential vulnerability candidates -> 1,129 reported direct to maintainers by Anthropic, at their request (May contain false positives)

    They just spammed the maintainers with these without reviewing them?

    1129 + 467 = 1596 total reported to maintainers -> 1451 acknowledged by maintainers

    Does acknowledged mean they said they received the report or does it mean they validated the report? Because it looks a lot like “received”, when accounting for that prior 1259 gap and the fact the bulk of them weren’t reviewed prior to sending.

    Subsequent analysis of these vulnerability candidates has identified that 1,726 are valid true positives. As many as 1,094 flaws are assessed to be either high- or critical-severity.

    But that 1726 was reduced to 467 come reporting time. Which makes that 17% hit rate possibly… 4.7%?

    MYTHOS IS TOO POWERFUL TO RELEASE /s

    • Elting@piefed.socialEnglish
      14 days

      Yeah they re-release this “news” with fudged numbers every so often it seems, and the media just eats it up and shits it back out with about as much time and care as they took auditing the software.

      • fiat_lux@lemmy.zipEnglish
        14 days

        I assume this article is slop. It contradicts 10k high sev by paragraph 3, not even Anthropic claims it in their media release, which contains even sadder numbers.

    • givesomefucks@lemmy.worldEnglish
      14 days

      Why only review 1900?

      Because of how time works…

      It just throws out thousands of redflags, then humans need to review those bits.

      But it’s a “one time thing” what matters more than the first time it’s run, is how many new redflags it generates over a time period.

      I’d also be interested to see how this performs versus the same amount of manpower hours of code review.

      Like, they’re gonna find shit, even if looking at random code.

      So ideally we’d need to take an identical code as and audit it:

      1. AI telling humans where to look

      2. Humans looking randomly

      3. Humans telling humans where to look

      And I feel like once again we’re gonna see that the only real benefit of “AI” as we know it, is doing the initial basic steps. It’s heavily biased to false redflags, but we want that because a false positive is just wasting some hours looking into it, a false negative means it goes uncaught.

      If it had a 100% identify rate, it would just mean it’s definitely not catching everything.

      But again, that confusion is because people keep acting like it can replace humans instead of assisting them.