Programming in the Age of AI
AI changes how developers work, but production-quality software still requires engineering judgment, testing, and human oversight.
Faster code still needs engineering
An AI-generated application can look finished before anyone has checked its behavior carefully. The interface loads, the login screen appears, and a few records move into a database.
Then a user retries a failed request and creates a duplicate transaction. Another user discovers they can retrieve someone else’s records by changing an identifier. A deployment changes the database schema, and the previous version of the application can no longer run.
Those are hypothetical failures, but they describe the distance between a convincing demonstration and software people can depend on. A demo shows that a selected path works. Production software must also handle misuse, partial failures, maintenance, and recovery.
AI makes generating code easier. That does not make software development experience optional. Someone still has to understand what the application should do, recognize what the implementation actually does, and decide whether the evidence is strong enough to release it.
That person does not need to type every line. They do need to be able to explain, test, and maintain the code they accept.
What AI changes about the developer’s job
A coding assistant uses a language model to generate proposed code and explanations from the context it receives. An agent adds the ability to take actions: inspect files, edit a repository, run commands, and work through several steps. These capabilities make delegation useful; they do not establish that the resulting software meets your requirements.
A developer using AI may spend less time writing repetitive code or constructing an initial test fixture. More attention can go toward describing behavior precisely, choosing interfaces, reviewing changes, investigating unexpected results, and checking that the pieces work together. Some tasks become faster. Others acquire a new cost: correcting a plausible implementation that took the wrong approach.
Google’s 2025 DevOps Research and Assessment (DORA) research found that greater AI adoption was associated with higher software delivery throughput, but lower delivery stability. The findings pair increased delivery with more delivery problems. The authors emphasize automated testing, version control, fast feedback, and architectures whose components can change independently. These survey-based associations do not prove that AI caused a particular team’s outcome.[1]
This is why counting generated lines is a poor measure of success. Measure the time needed to deliver an accepted change, including review and rework. Track defects after release, recovery time, and whether users received something useful.
Real successes involve real engineering
The strongest examples here involve professional teams using AI on existing software, with review and verification built into the work.
Google: changing identifiers across an existing codebase
In a Google Ads migration from 32-bit to 64-bit identifiers, Google reported that 80% of the code modifications in accepted changes were AI-authored. Engineers estimated a 50% reduction in total migration time.[2]
An expert identified the migration target, and existing analysis tools located the relevant code and dependencies. AI generated proposed edits. Compilation and unit tests checked those edits, engineers corrected mistakes, and the owners of the affected code reviewed the changes before acceptance.[2]
The time saving was an engineer-reported estimate, not a randomized measurement. The case shows AI helping with a large migration while engineers still analyzed the affected files and reviewed the changes.
Amazon: modernizing production Java applications
Amazon reported migrating tens of thousands of production applications from Java 8 or 11 to Java 17 with assistance from Amazon Q Developer. It also reported $260 million in annual infrastructure savings associated with the performance improvements of the upgraded applications.[3]
Those are Amazon’s figures, not independently audited findings. The infrastructure savings came from running the modernized applications more efficiently; they should not be confused with money saved simply by generating code. This was maintenance of an existing production fleet, not autonomous creation of thousands of new products.[3]
Professional developers: measured gains, with limits
A paper published in June 2025 analyzed randomized field experiments involving 4,867 developers at Microsoft, Accenture, and an unnamed Fortune 100 company. The experiments took place in 2022–2023 and tested GitHub Copilot’s code-completion assistant, not today’s multi-step agents.[11] The authors’ preferred pooled estimate was a 26.08% increase in weekly completed tasks among developers whose adoption of the assistant resulted from being offered access.[4][11]
The outcome was completed work, measured primarily through pull requests, rather than time saved on each task. Because some developers offered access did not use the tool, the authors adjusted the estimate for adoption. The individual experiments were noisy, and measuring code quality was difficult. The result does not establish a comparable improvement in production reliability.[11]
The participants were developers working within organizations. The finding supports AI-assisted development; it does not establish that programming competence can be removed from the process.
There is also evidence against assuming every use will help. METR’s early-2025 randomized study found that 16 experienced open-source developers took 19% longer on tasks when allowed to use AI, despite believing it made them faster.[5]
That result needs its date attached. METR now says the historical findings no longer reflect current model impact. Its February 2026 follow-up reported that selection effects and difficulties measuring concurrent agent work made the new productivity estimate unreliable. Evidence that this field is changing rapidly.[5][6]
The practical lesson is to measure your own workflow, rather than treating either a favorable study or an unfavorable one as a universal answer.
These studies measure productivity in particular settings. They do not test whether an inexperienced person can safely replace an engineering team. The argument for programming competence comes from the work of specifying behavior, evaluating changes, and operating the software.
Why programming knowledge still matters
A technically capable person can describe a feature without knowing how to implement it safely. That gap matters when the feature involves permissions, sensitive information, money, or shared infrastructure.
Consider an instruction to “add secure document downloads.” An assistant might produce a working endpoint. A developer needs to ask whether authorization is checked for each document, whether tenant boundaries hold, whether file paths can escape the storage directory, and what the audit log records. None of those requirements is satisfied merely because the browser downloads a file.
Experience helps you notice missing requirements and choose meaningful tests. It also helps you recognize when a proposed solution adds unnecessary complexity or violates an existing design.
Beginners can use AI to learn and build useful tools. As the consequences of failure grow, they need enough understanding to evaluate the work or an experienced collaborator who can. Established platform controls can reduce the risks, but they cannot decide whether the application meets its requirements.
If you cannot explain a consequential change, reproduce its behavior, or investigate its failure, you are not ready to approve it without additional review. A well-written prompt cannot substitute for that understanding.
A workflow that keeps the developer involved
The following is a recommended development workflow, not a guarantee supplied by any product. It applies whether you use an editor assistant, a terminal agent, or a cloud service that returns a proposed change.
1. Specify the behavior before requesting the implementation
Write a short specification in the repository. Describe the user’s problem, expected behavior, non-goals, interfaces, and constraints. Include security and privacy requirements, compatibility expectations, and examples of what acceptance looks like.
For a document-download feature, the specification might say:
An authenticated user may download only documents belonging to their organization. Guessing another document identifier must not reveal its contents or metadata. Failed authorization attempts must produce the agreed audit event without logging credentials or document contents.
Ask the assistant to identify ambiguities and missing cases before it edits code and then have it ask you questions about its findings. After this loop, resolve ambiguities and missing cases yourself or with the person responsible for the requirement.
Keep the specification short if the change is small, but make the acceptance criteria concrete enough for another person to check.
2. Make the design decisions deliberately
Inspect the existing code and establish a baseline build and test result. Decide where the change belongs before asking an agent to implement it.
Keep responsibilities separate. Put authorization policy somewhere it can be tested consistently. Use explicit interfaces between components, limit privileges, and reuse established patterns unless there is a reason to change them. Prefer a design the team can understand over a framework-heavy solution that merely looks sophisticated.
Ask AI to compare approaches and expose trade-offs. The developer chooses the approach. Anthropic’s Claude Code guidance explicitly recommends separating research and planning from implementation to avoid solving the wrong problem.[7]
3. Delegate a change small enough to review
Give the assistant a bounded task, relevant repository context, acceptance criteria, and limits on what it may change. For example:
Implement the document authorization check and its tests. Follow the existing permission model. Do not change the database schema or add dependencies. Stop and ask if the current interface cannot meet the requirement.
A small diff is easier to understand than a complete generated subsystem. If you use several agents, give each a clear boundary and isolated working state, then review their integration. Parallel output still needs someone to check that the interfaces agree.
OpenAI’s Codex workflow cookbook recommends keeping work small enough to implement and verify independently. Its example workflow also makes approval gates explicit.[8]
4. Use the test-driven development cycle
Test-driven development, or TDD, gives the assistant a concrete feedback loop:
- Red: write a test for the approved behavior and run it. Confirm that it fails because the behavior is missing, rather than because the test environment is broken.
- Green: implement the smallest coherent change that passes the test.
- Refactor: improve the structure without changing the behavior, then run the tests again.
For the download example, begin with an allowed request and a denied cross-organization request. Review the expected outcomes yourself. Add boundary and failure cases as the design requires.
AI can help write both tests and implementation. That creates a trap: the two can share the same mistaken assumption. Derive expected results from the specification, trusted fixtures, and existing contracts, not from whatever the generated code happens to do.
Do not allow the assistant to delete a failing assertion or relax the requirement simply to make the suite green. TDD supplies feedback. It does not prove that the requirements are complete or that the software is secure.
5. Review evidence and the complete change
Read the implementation and the tests. Inspect dependencies, configuration changes, and error handling. Run the relevant regression suite as well as the new tests. Exercise integration boundaries and the actual user journey; a mocked unit test cannot establish that a deployed permission check works.
Ask for commands and results, not an assurance that “everything passes.” Anthropic’s guidance makes this distinction explicit: the assistant should show test output or other evidence rather than assert success.[7]
Another AI reviewer can be useful, especially for spotting omissions. It remains another fallible review, not independent proof. A human must decide whether the code and the verification cover the intended behavior.
6. Update documentation and verify delivery checks
Update the README, affected API documentation, configuration instructions, and operational notes in the same change. Record significant design decisions and migration requirements. Documentation should describe the software that exists, not the assistant’s plan for it.
Run automated build, type, lint, test, and security checks appropriate to the project in continuous integration, or CI. Check the results for the final revision. A local test run before the last edit does not validate the code you are about to merge, and a skipped CI job did not execute its tests.
For delivery, prepare a reproducible release artifact and clear instructions for configuration and deployment. The next developer or operator should not need the original chat history to run the application.
7. Release with an owner and a recovery plan
Exercise the release candidate in a production-like staging environment using safe representative data. Check integrations, migrations, logs, and important user journeys.
Before release, decide who authorizes deployment, what success looks like, when to stop, and how to recover. Retain a known-good artifact. Treat database recovery separately: reverting application code may not undo a destructive schema change or restore lost data.
Deployment controls can enforce some of these boundaries. GitHub Actions environments, for example, support required reviewers and can withhold environment secrets until approval. Availability depends on the plan and repository visibility, and bypass settings need attention.[10]
After deployment, inspect the actual service behavior. Watch errors and agreed operational signals, and verify that the intended change reached users. Delivery ends with a working release and a usable handoff, not a successful code-generation session.
Staying in the loop does not mean approving every keystroke
A developer can remain responsible throughout the workflow without watching every command. Approve the goal and design, define the agent’s scope, inspect evidence at meaningful checkpoints, and require a human decision before consequential changes reach production.
Give the agent only the access it needs, and check what source code or data the service receives under your organization’s policy. Keep credentials and sensitive production data out of prompts and test fixtures. Use enforceable permissions and sandbox boundaries where available. An instruction such as “never touch production” is not an access control; check the tool’s actual approval settings.
Learn the work, not just the prompting
AI can shorten the path from an idea to code. When learning with it, use its suggestions as material to investigate. Check how the code behaves, change the tests, and trace failures until you understand them.
Before approving a consequential change, be able to explain what it does and how you checked it, or bring in someone who can. Users will depend on the software long after the generation session is over.
Conclusions
Software development skills matter more than ever in the age of AI. While the future is bright, no matter what model, agent, or harness you use, keep your eye on tools progress the field is advancing rapidly.
Sources
[1] Google Cloud: 2025 DORA report
[2] Accelerating code migrations with AI
[5] Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - METR
[6] We are Changing our Developer Productivity Experiment Design - METR
[7] Best practices for Claude Code - Claude Code Docs
[8] Iterating Development Workflows with Codex
[10] Deployments and environments - GitHub Docs
[11] Full paper: The Effects of Generative AI on High-Skilled Work