Chatbots provided incorrect, conflicting medical advice, researchers found: “Despite all the hype, AI just isn’t ready to take on the role of the physician.”
“In an extreme case, two users sent very similar messages describing symptoms of a subarachnoid hemorrhage but were given opposite advice,” the study’s authors wrote. “One user was told to lie down in a dark room, and the other user was given the correct recommendation to seek emergency care.”
How much of that is the chat bot itself versus humans just being horrible at self reporting symptoms?
That is why “bedside manner” is so important. Connect the dots and ask follow up questions for clarifications or just look at a person and assume they are wrong. Obviously there are some BIG problems with that (ask any black woman, for example) but… humans are horrible at reporting symptoms.
Which gets back to how “AI” is actually an incredible tool (especially in this case when it is mostly a human language interface to a search engine) but you still need domain experts in the loop to understand what questions to ask and whether the resulting answer makes any sense at all.
Yet, instead, people do the equivalent of just raw dogging whatever the first response on stack overflow is.
Rawdogging the first response from stack overflow to try and fix a coding issue isn’f going to kill someone.
It is if your software goes anywhere near infrastructure or safety.
Which is literally what musk and the oligarchs were arguing as a way to “fix” Air Traffic Control. And that is far from the first time tech charlatans have wanted to “disrupt” an industry.
Someone who uses stack overflow to solve a problem will be doing testing to confirm it worked as part of an overall development workflow.
Using an LLM as a doctor is like vibe coding, where there is no testing or quality control.
So… they wouldn’t be raw dogging stack overflow? Because raw dogging the code you get from a rando off stack overflow is a bad idea?
Because you can just as easily use generative AI as a component in test driven development. But the people pushing to “make coders more efficient” are looking at firing people. And they continue to not want to add the guard rails that would mean they fire 1 engineer instead of 5.