“Language models don’t apply to us because this is not a language problem,” Nesterenko explained. “If you ask it to actually create a blueprint, it has no training data for that. It has no context for that…” Instead, Quilter built what Nesterenko describes as a “game” where the AI agent makes sequential decisions — place this component here, route this trace there — and receives feedback based on whether the resulting design satisfies electromagnetic, thermal, and manufacturing constraints… The approach mirrors DeepMind’s progression with its Go-playing systems.
This is kind of interesting and cool, and it’s not a hallucinating LLM. I’ve designed a couple of simple circuit boards, and running traces can be sort of zen, but it is tedious and would be maddening as a job, so I can only imagine what the process must be like on complex projects from scratch. Definitely some hype levels coming from the company that give me pause, but it seems like an actual useful task for a machine learning algorithm.
as someone who used to work on “expert models” i’m excited that not everyone has abandoned them for “what if we just had a model that knows everything (that doesn’t exist) and costs a billion dollars to run”
But you know how people are already comparing vibe coding to 40k where “priests” pray to computers and hope if they do the exact same thing they’ll get the same result they want?
If we start walking down this road of even the chat or not understanding why what it did was better…
Serious unintended consequences are going to be inevitable.
Like, I swear nobody knows the paperclip story anymore.
Instrumental convergence posits that an intelligent agent with seemingly harmless but unbounded goals can act in surprisingly harmful ways. For example, a sufficiently intelligent program with the sole, unconstrained goal of solving a complex mathematics problem like the Riemann hypothesis could attempt to turn the Earth (and in principle other celestial bodies) into additional computing infrastructure to succeed in its calculations.[2]
I mean, we can make a very very solid argument that much of our current problems are caused by high level stock trading being done by algorithms who’s only instruction is “make numbers go up”.
This shit aint even hypothetical anymore, it’s just instead of “make as many paperclips” we told it “make more money than you did yesterday”.
Which is why we’re burning down the planet to make billionaires even more money
This is kind of interesting and cool, and it’s not a hallucinating LLM. I’ve designed a couple of simple circuit boards, and running traces can be sort of zen, but it is tedious and would be maddening as a job, so I can only imagine what the process must be like on complex projects from scratch. Definitely some hype levels coming from the company that give me pause, but it seems like an actual useful task for a machine learning algorithm.
as someone who used to work on “expert models” i’m excited that not everyone has abandoned them for “what if we just had a model that knows everything (that doesn’t exist) and costs a billion dollars to run”
Yeah…
But you know how people are already comparing vibe coding to 40k where “priests” pray to computers and hope if they do the exact same thing they’ll get the same result they want?
If we start walking down this road of even the chat or not understanding why what it did was better…
Serious unintended consequences are going to be inevitable.
Like, I swear nobody knows the paperclip story anymore.
https://en.wikipedia.org/wiki/Instrumental_convergence
I mean, we can make a very very solid argument that much of our current problems are caused by high level stock trading being done by algorithms who’s only instruction is “make numbers go up”.
This shit aint even hypothetical anymore, it’s just instead of “make as many paperclips” we told it “make more money than you did yesterday”.
Which is why we’re burning down the planet to make billionaires even more money
I was going to ask how this is different than a Reinforcement Learning algorithm but then they called out Deep Minds Alpha-Go