• 0 Posts
  • 117 Comments
Joined 2 years ago
Cake day: April 6th, 2024








  • If it gets it wrong the first time I rarely reprompt. I know I can get it to fix it, but it’s usually faster for me to do it because I already figured out where and what to do the fix. Low key think it’s just a ploy to get us to burn more tokens. Sure correcting it means it writes a few lines to the memory file, but it’s only a matter of time before it trips over that context as well.


  • I have similar problems whenever I send it to investigate a bug and the local runtime is inside a container. It cannot reliably translate paths without the help of an IDE. Hell, it even occasionally mangles API paths if I have it prefixed elsewhere in the codebase (despite having Claude.md etc, your context needs to be pure for it to be reliable). Having it fix a Dockerfile is comically bad.



  • Everything listed should be done before ever getting into code along with business and product partners.

    Ehh, it really depends on where the risk is and the problem is LLMs can’t evaluate for that unless you feed it everything. Some projects need code experiments before you settle on an architecture, but that’s only if you’re a pioneer (which frankly is where the money is at).


  • In my experience there are three ways to be successful with this tool:

    • write something that already exists so it doesn’t need to think
    • do all the thinking for it upfront (hello waterfall development)
    • work in very small iterations that doesn’t require any leaps of logic. Don’t reprompt when it gets something wrong, instead reshape the code so it can only get it right

    The issue with debugging is that it doesn’t actually think. LLMs pattern match to a chain of thought based on signals, not reasoning. For it to debug you need good signals in your code that explicitly tell what it is doing and the LLMs do not write code with that level of observability by default.

    Edit: one of my workflows that I had success with is as follows:

    • write a gherkin feature file describing desired functionality, maybe have the LLM create multiple scenarios after I defined one to copy from
    • tell the LLM to write tests using those feature files, does an okay job but needs help making tests run in parallel.
    • if the feature is simple, ask the LLM to make a plan and review it
    • if the feature is complex then stub out the implementation in code and add TODOs, then direct the LLM to plan. Giving explicit goals in the code itself reduces token consumption and yield better plans