How to tackle unreliability of coding assistants

Generative AI, specifically Large Language Models (LLMs), has gained widespread attention. As a software developer at Thoughtworks, I am intrigued by the possibilities but uncertain about its long-term implications for our profession. In this role, my colleagues and I are exploring how this technology will impact software delivery practices. In terms of toolchains that use LLMs for coding assistance, they can assist with tasks such as finding information, generating code, explaining code, and transforming code into other formats. Interaction modes include chat interfaces, in-line assistance, and CLI prompt composition. The quality and composition of the prompt, as well as the properties of the model, such as training data, size, and origin, also play a role in the effectiveness of these tools. Currently, combinations of chat interaction and in-line assistance are commonly used, but there is growing experimentation with prompt composition tools. The usefulness of these tools depends on factors such as prevalent tech stacks, the simplicity of problems, the size of suggestions, the experience of the developer, and the margin for errors. It is important to have an understanding of the generated suggestions and to review them for quality and correctness.

https://martinfowler.com/articles/exploring-gen-ai.html#memo-08

To top