feat: 10 gains from gpt-augmented swe
10 ways that GPT tooling improves software engineering productivity
In this article, I will describe ten ways that GPT tooling can increase software engineer productivity. This article does not argue for a positive ROI to GPT augmentation. I make a non-quantitative benefits assessment while ignoring costs. The article demonstrates that software engineering organizations have the potential for large productivity gains, but further research investment is required to precisely identify the quantitative net gain.
While I refrain from precise quantitative measurement, I will broadly comment that in my experience GPT-4 saves me substantial time with no loss in quality. It seems able to solve the vast majority of my technical questions within a few prompts or a few minutes.
Tech companies more generally stand to experience even greater benefits because GPT augmentation can enhance not only SWE productivity, but also marketing, research, and other practice areas.
1 - Enhanced Search
Googling is a fundamental developer skill and it involves at least three pain points that GPT tooling solves. First, Google searches match by keyword and advanced searching requires knowledge of special operators and search syntax.
Second, developers must have ahead-of-time knowledge of quality content providers in order to perform an efficient search. For example, software developers must know to be on the lookout for content from providers like Stack Overflow, MDN, CSS-Tricks, or the official documentation sites related to the current code under search.
Third, even in the happy case where a developer is well-equipped with the prior skills, a developer will often still have to manually browse many web pages or content sections in order to find one or more distinct and relevant code snippets or bits of technical guidance.
GPT tooling solves all of the above. GPT-4 does not require ahead-of-time knowledge of content providers, it does not merely filter results by keyword, and it can immediately consolidate distinct and relevant code snippets and bits of technical guidance from many sources.
Notice that enhanced search is not merely a software engineering benefit. It benefits most employees in most knowledge-based roles. Researchers and software engineers perhaps stand to benefit disproportionately.
2 - Code Generation from a Template
I can show GPT-4 a code component then show GPT-4 a schema specification, such as a Prisma schema file, then ask GPT-4 to produce an analogous code component for a different model. This is something I have done to great success in rapidly producing CRUD services and routes for many distinct models, as well as UI components like graphs and tables.
Sometimes GPT-4 will naively produce the correct final code, but often it will produce code that is only largely correct and some tweaking is required. For example, it might produce a graph where it automatically picked one property to graph, but I actually want to graph a different property. GPT still results in a massive productivity enhancement and substantial time saved despite the frequent need for developer revision of GPT-drafted code.
3 - Type, Factory, and Flat File Translation
Given a static JSON file, GPT can automatically extract typed schemas in a variety of programming languages, and for use with a variety of libraries that may use a domain-specific language, like gRPC or Prisma.
GPT can also do the reverse operation. Given a type, it can generate sample JSON, XML, CSV, HTML, plain text, or other flat-file content.
Finally, it can generate factory functions based on such type information, and it can also generate types from factory functions. It can also update your code to include type annotations. Even if your code is already typed, GPT can sometimes enhance the type to be more precise.
4 - Code Generation for Unit Tests and Validation
Given a factory function, we already mentioned that GPT tools can automatically refactor the code to use types. Similarly, GPT tools can also add type annotations to many non-factory functions. Beyond type annotations, GPT can automatically generate validation functions, potentially including custom exceptions and appropriate logs, as well as unit tests for the functions under augmentation.
5 - Cross-Language Code Translation
Migrating across languages is a time-consuming task for developers. GPT is exceedingly performant at this task.
6 - Normalized Quality Assessment
In the quality assessment of code, developers are used to hard measures like lint error density. These metrics often become numerous and have various caveats and adjustments across languages, libraries, and codebases. When metrics get rolled up for reporting, these details are often consolidated into soft metrics like red-yellow-green project identification. Subject to the usual constraints and caveats, GPT can quickly analyze an entire codebase for quality and suggest technical improvements.
7 - Design Discovery, Draft, and Revision
Given a problem statement, GPT can draft a narrative technical solution or design document. These drafts can include technical suggestions such as particular programming languages and code libraries suggested for use, the technical manner of their use including code snippets, and reasoned benefits, costs, and alternatives to the solution suggested.
Given a design document that has already been prepared by an individual contributor, GPT can suggest improved wording or other edits to the document. GPT can also provide a few-sentence summary for a long design document.
8 - Code Documentation
GPT can explain code, saving engineers hours of writing and maintaining documentation. This is not merely a time-savings benefit, but also a job quality and business efficiency benefit, because writing documentation is often a low-value and low-satisfaction task for engineers.
GPT can also be used to standardize, draft, revise, and improve post-mortem documentation, white papers, and other semi-technical documents besides pure code documentation.
9 - Ticket Hygiene
Writing user stories, acceptance criteria, defect descriptions, or other work tickets and requirement tickets is often an area of weakness for engineers. An engineer can provide a terse explanation for GPT and GPT can reformat this terse explanation into a well-formated user story with acceptance criteria following a proper given/when/then syntax or other syntactical frameworks of choice. GPT can occasionally even suggest related test cases or solution code snippets immediately at ticket creation time.
10 - Peer Review Hygiene
GPT can suggest commit messages based on committed code. It can draft a pull request description based on pull requested code. It can suggest a pull request title and a branch name. It can also perform some automatic code review and suggest some code improvements, much of which has already been discussed such as the enhancement of types.
For reviewers, if you find a pull request with a poor description, you can also run GPT as a reviewer to quickly obtain a relevant summary description of the code.
GPT can also generate narrative or summary release notes in plain text, markdown format, HTML format, and so on. It’s worth noting that GPT now has several plugins and integrations with Slack and other platforms.
11 - GPT-Augmented CLI
This feature isn’t quite out yet but it will be out soon and I think it’s another win for engineering productivity. Today, engineers can use bash autocompletion for some productivity benefits using the command line.
GitHub Copilot X integrates GPT-4 into the IDE and also into the CLI, supercharging autocompletion with GPT-powered text prediction. This means engineers will be able to access many of the existing benefits of GPT with even lower latency and friction because they will be able to use GPT without leaving their IDE or CLI.