Artificial intelligence is changing software development at remarkable speed. Developers can now use AI assistants to generate code, write tests, explain unfamiliar code, identify potential vulnerabilities, create documentation, refactor functions, and even suggest architectural approaches. These capabilities can dramatically increase developer productivity.
But greater development speed introduces a new question: Who is responsible for software quality when part of the software is created with AI assistance?
The answer cannot simply be “the AI” or “the developer.” AI-assisted development requires a broader quality governance model—one that combines human accountability, automated controls, risk management, security practices, and continuous evaluation.
AI can accelerate software development, but without appropriate governance, it can also accelerate defects, vulnerabilities, technical debt, and poor engineering decisions.
AI Changes the Software Quality Equation
Traditional software development already involves complex quality challenges. Teams must manage requirements, architecture, code, testing, security, performance, dependencies, and deployment.
AI adds another layer.
An AI coding assistant may generate a working solution in seconds, but the speed of generation does not guarantee that the solution is correct, secure, maintainable, or appropriate for the organization’s requirements.
NIST’s DevSecOps guidance makes a similar point: AI can improve development efficiency and potentially software quality, but AI-generated content still needs human monitoring and validation, along with processes that make its accuracy and trustworthiness verifiable.
This changes the role of quality engineering. Instead of simply asking whether developers are producing good code, organizations must also ask how AI is being used to produce that code and what controls surround its use.
AI Can Amplify Existing Process Problems
One of the biggest risks of AI-assisted development is that it can make weak processes faster.
Consider a team with poorly defined requirements. Previously, developers might spend hours interpreting those requirements before writing code. With AI assistance, they may generate an implementation almost immediately—but based on the same ambiguity.
The result is not necessarily better software. It may simply be a faster path from an unclear requirement to a flawed implementation.
The same principle applies to testing. If a team has incomplete test strategies, AI can generate hundreds of tests that fail to address the most important risks.
AI is therefore an amplifier. It can magnify good engineering practices, but it can also magnify bad ones.
This is why governance should begin with the software development process itself rather than being added after AI tools have already been adopted.
Human Accountability Still Matters
AI-generated code may look authoritative because it is presented quickly and confidently. But developers remain responsible for understanding and approving the code that enters the product.
Human review should not become a ceremonial step where someone simply accepts an AI-generated pull request. Reviewers need enough context to evaluate whether the code satisfies requirements, follows architectural standards, handles edge cases, and introduces security or privacy risks.
This does not mean humans must manually inspect every character produced by AI. Instead, organizations should define risk-based human oversight.
For low-risk changes, automated checks may provide most of the necessary validation. For high-risk changes—such as authentication, financial calculations, healthcare workflows, authorization, encryption, or sensitive data processing—stronger review and testing should be mandatory.
The principle is simple: the greater the potential impact of an AI-assisted change, the stronger the required oversight should be.
Establish Clear AI Development Policies
Organizations need practical rules for how developers can use AI.
A policy should answer questions such as:
- Which AI tools are approved?
- What company information can be entered into AI systems?
- Can confidential source code be submitted to external models?
- How should AI-generated code be reviewed?
- When is additional security testing required?
- How should AI assistance be documented?
- Who owns the final decision to accept AI-generated output?
These policies should not be designed merely to restrict developers. Overly restrictive policies can encourage employees to use unapproved tools secretly, creating “shadow AI” and reducing organizational visibility.
Instead, governance should provide safe, approved pathways for AI adoption.
Build Quality Gates Around AI-Assisted Code
AI should not bypass existing engineering controls.
Every AI-assisted change should still pass the organization’s normal quality gates, including appropriate combinations of:
- Code review
- Unit and integration testing
- Static analysis
- Dependency scanning
- Security testing
- Secret detection
- API and contract testing
- Performance testing
- End-to-end testing
- Compliance checks
Organizations should also consider additional controls specifically for AI-assisted development.
For example, repositories can record whether AI tools were used during development, while automated pipelines can apply stronger analysis to changes affecting sensitive components.
The objective is not to punish AI-assisted coding. It is to ensure that the speed of code generation does not exceed the organization’s ability to validate the resulting software.
Govern the Software Supply Chain
AI-generated code does not exist in isolation. It may introduce third-party libraries, copied patterns, dependencies, APIs, or implementation approaches whose origins are not immediately obvious.
This makes software supply-chain governance increasingly important.
NIST’s Secure Software Development Framework work for generative AI specifically extends secure-development practices to AI-related development and recommends integrating those practices into the software lifecycle.
Organizations should therefore maintain visibility into dependencies, licenses, vulnerabilities, provenance, and approved components.
An AI assistant should never become an invisible route around established software supply-chain controls.
Measure Quality, Not AI Productivity Alone
AI adoption is often measured through productivity metrics: lines of code generated, coding time saved, number of pull requests completed, or developer throughput.
These measurements can be useful, but they do not tell the whole story.
A team that produces twice as much code but introduces twice as many production defects has not necessarily improved.
Organizations should balance productivity metrics with quality indicators such as:
- Production defect rates
- Vulnerability rates
- Change failure rate
- Escaped defects
- Test effectiveness
- Mean time to detect and resolve problems
- Technical debt
- Reliability and availability
- Customer-reported issues
The important question is not simply “How much faster are developers with AI?”
It is “Are we delivering better software faster?”
Use Risk-Based AI Governance
Not every AI-assisted development activity carries the same level of risk.
Generating a simple internal utility is different from generating code that processes financial transactions. Writing documentation is different from designing an authorization mechanism.
A mature governance framework should classify AI-assisted work according to potential impact.
NIST’s AI Risk Management Framework organizes AI risk management around four functions: Govern, Map, Measure, and Manage. It emphasizes governance as a cross-cutting activity that should influence risk management throughout the AI system lifecycle.
This approach can be adapted to software engineering.
Low-risk activities may require standard automated controls. Medium-risk changes can require additional review and testing. High-risk changes may require security specialists, architecture review, documented validation, and explicit approval.
Risk-based governance prevents organizations from creating unnecessary bureaucracy while still protecting critical systems.
Create an Audit Trail
As AI becomes embedded in development workflows, organizations will increasingly need to understand how software was produced.
This does not necessarily mean recording every prompt a developer sends to an AI assistant. Instead, organizations should maintain meaningful evidence around important decisions.
For significant systems, useful records may include:
- Which AI tools were approved
- Which models or services were used
- Who reviewed significant AI-assisted changes
- What validation was performed
- Which security checks were completed
- What exceptions were granted
- Which risks were identified and accepted
This creates accountability and makes investigations easier when something goes wrong.
Make Governance Continuous
AI tools, models, coding practices, and risks will continue to evolve. A policy created once and never revisited will quickly become outdated.
Governance therefore needs a feedback loop.
Organizations should regularly review AI-related incidents, failed tests, security findings, developer experiences, customer issues, and changes in AI capabilities. Governance rules should then evolve based on what the organization learns.
NIST’s AI RMF is designed around continuous risk management rather than a one-time assessment, and its current resources explicitly connect governance with the full AI lifecycle.
This mindset is essential because today’s low-risk AI capability may become tomorrow’s high-impact capability.
Conclusion
AI-assisted development is not a reason to abandon traditional software quality practices. It is a reason to make them stronger.
AI can write code faster, generate tests faster, identify potential problems faster, and help developers solve unfamiliar technical challenges. But speed without governance can create risk at unprecedented scale.
The future of software quality will therefore depend less on whether organizations use AI and more on how responsibly they govern its use.
Successful organizations will establish clear AI policies, maintain human accountability, apply risk-based controls, strengthen software supply-chain practices, automate quality gates, measure meaningful outcomes, and continuously evaluate their development processes.
The goal is not to keep AI out of software engineering. Nor is it to allow AI to operate without oversight.
The goal is to create a development environment where AI accelerates engineering excellence without becoming an accelerator for engineering mistakes.
In the age of AI-assisted development, software quality is no longer just a testing responsibility. It is a governance responsibility shared across the entire organization.
