Quality of Engineering: How to Measure It in Your Team
Quality of Engineering explained: how to assess maintainability, testing, reliability, delivery and team decisions, with a practical checklist.
The release dashboard is green, but a small customer request still takes weeks to ship. Engineers hesitate to touch particular modules, support keeps reporting familiar failures, and nobody can explain why the previous team made a critical design choice. Passing tests hasn't made the platform easy to run or safe to change.
For founders, chief technology officers (CTOs) and operations leaders, the useful question isn't how much code the team produces. It's whether the quality of engineering lets the business change its software without repeatedly disrupting customers. The answer requires a feedback system that measures both failure prevention and delivery flow.
Engineering Quality at a Glance
Engineering quality scorecard
| Dimension | What to Measure | Healthy Signal | Warning Sign |
|---|---|---|---|
| Maintainability | Time and files touched for a typical change | Small changes stay small and new engineers ship within weeks | Every fix touches many modules |
| Testing | Escaped defects and coverage of critical paths | Failures are caught before release | Customers find most bugs first |
| Reliability | Incident count, error rate and time to restore | Incidents are rare, short and have a clear owner | The same incidents repeat with no owner |
| Delivery practices | Release frequency, lead time and change failure rate | Releases are small and routine | Releases are batched into freezes |
| Team decision-making | Who approves what and whether reasons are recorded | Each release has a named owner and a written reason | Decisions live in chat or in one person's memory |
These dimensions answer different questions. Maintainability concerns the effort required to change the system. Testing examines failure detection, while reliability measures what happens under real operating conditions.
Delivery practices determine whether changes reach production safely. Team decision-making establishes who accepts the remaining risk and whether another team could understand that decision later. No single number captures all of this.
Read the signals together
A falling incident count might reflect better engineering, but it might also mean the team stopped releasing. Faster delivery might indicate simpler architecture, or it might mean engineers removed checks without replacing their protection. Each indicator starts an investigation; it doesn't settle one.
The same framework applies to an enterprise platform, a private equity (PE)-backed business and managed-service businesses (MSPs), meaning non-IT service firms that run a multi-party platform. Their priorities differ, but each needs software that can change without losing operational control.
Practical rule: A quality measure becomes useful when it changes a decision, not when it improves a dashboard.
Establish a working baseline
Score each dimension from one to five against the last three releases. Treat this as an internal review exercise, not an industry benchmark, and attach a concrete example to every score.
Mark the lowest score rather than averaging everything into a reassuring total. That weakness determines the first investigation: better tests won't resolve unclear release ownership, and faster deployment won't repair modules nobody understands.
Maintainability and How Safely Code Can Change
Maintainability is the cost of changing software safely after its first release. A practical assessment starts with an ordinary request: change an approval rule, add a customer field or adjust a booking workflow. Record the elapsed time from a clear requirement to merged code, then inspect the files and modules touched.
The pattern matters more than the raw file count. A generated schema update may legitimately touch many files; a pricing-rule adjustment that changes authentication and reporting deserves closer inspection. Separate necessary breadth from accidental coupling.

Inspect the cost of ordinary changes
A short repository review provides a useful starting point:
- Pick the last five changes. Include routine work, not just the cleanest examples.
- Count the files and modules touched. Note whether each edit belongs to the requested behaviour.
- Identify urgent corrections. Record changes that needed an immediate fix and the assumption that failed.
- List avoided modules. Ask engineers which parts they won't edit without help, and why.
Then check dependency age and onboarding friction. Libraries several major versions behind need an upgrade assessment, especially when security fixes are no longer available. A new engineer's first safe change tests whether documentation, local setup and review practices work together.
Neither signal proves poor architecture alone. An upgrade may be blocked by a necessary integration, while onboarding may stall because access permissions haven't been granted. Record the actual obstacle before prescribing a rewrite.
Examine unfamiliar code more closely
For code inherited from a previous vendor or written with artificial intelligence (AI) assistance, include changes nobody on the current team originally authored. Inspect interface assumptions, configuration, error handling and business rules that exist only in implementation details.
An empirical study of 3,800 real-world issues across three AI coding tools found that more than 67% were functional bugs, while 37.3% originated in application programming interface (API), integration or configuration errors. These findings concern the tools studied, not a predicted defect rate for a particular codebase.
Complexity and duplication measurements help only when engineers use them to choose work. A duplicated rule might justify consolidation; a complex but stable calculation might justify stronger tests instead. A healthy codebase has clear module boundaries, short architecture notes and a shared map of where business rules live.
Testing That Catches Failures Before Customers Do
A refund flow can pass every check in staging and still fail the first time a customer cancels part of an order. Testing quality concerns which failures the team can detect before customers encounter them, and a large suite that exercises the wrong paths protects little. Requirements, test design and realistic operating conditions matter as much as automation.
A report on the 2002 US National Institute of Standards and Technology (NIST) study states that inadequate software testing cost the US economy approximately $59.5 billion annually, about 0.6% of gross domestic product, that more than half of defects were discovered late in development or after release, and that improved testing infrastructure could have eliminated approximately $22.2 billion of the annual loss. The figures are from 2002 and describe late discovery, not current losses.
Match each layer to the failure it should expose
Every check answers a different question, and each leaves a different blind spot.
| Test layer | Failure it catches | What it cannot prove |
|---|---|---|
| Unit tests | Broken isolated logic: rounding, eligibility rules, state transitions | That services work together |
| Integration tests | Faults across services, databases and external boundaries, including failed responses and partial writes | That a full customer journey completes |
| End-to-end tests | Breaks in critical journeys through the deployed application | Which component caused the break |
| Contract tests | Incompatible expectations at shared interfaces | Behaviour outside the agreed contract |
| Monitoring | Operating conditions and customer failures that pre-release tests missed | That the fault would have been caught earlier |
The trade-off is maintenance and diagnostic effort. Broad checks find more but explain less; narrow checks explain quickly but see less. Treat the suite as a feedback system rather than a fixed asset: each failure that slips through tells you which layer should have caught it, and monitoring gaps tell you which journeys were never really tested.
Count what reaches customers
Track escaped defects per release, meaning bugs discovered after launch, alongside coverage of critical paths such as payments, onboarding and approvals. Classify escapes by severity and cause so a spelling correction doesn't carry the same weight as an incorrect payment.
Consider a hypothetical team reporting 85% code coverage while still shipping payment bugs. Its tests exercise getters and setters but never attempt a refund after partial fulfilment. The percentage describes executed code, not protection against the business failure.
Coverage is a map of execution, not evidence that the right questions were asked.
Requirements also shift between releases, and a check that verified last quarter's rule may verify nothing today. Projectify's guide to keeping traceability accurate as requirements change covers how to maintain the connection between a business rule and the checks intended to verify it.
Close the gap review with three steps:
- List the three flows where failure would hurt most.
- For each, confirm a test would fail if the important behaviour broke.
- Where practical, introduce the fault in an isolated environment and verify the check detects it.
Reliability After Release and Who Gets Paged
Reliability concerns what the system does when real users, data and traffic arrive. A successful deployment doesn't establish that a customer can complete a booking, receive an approval or recover from an interrupted payment. The operating team needs evidence from those journeys.
A 2025 systematic review of bugs in AI-generated code analysed 72 studies published between January 2020 and April 2025 and reported that system bugs and reliability bugs each appeared in approximately 26–29% of the reviewed studies. Those percentages describe how often each category recurred across the reviewed literature, which makes the list of categories a practical checklist of failure modes to watch after launch.

Measure customer impact and restoration
Use incident count and severity to distinguish inconvenience from blocked operations. Measure error rate on key journeys alongside server responses, since a request can return successfully while the customer receives an incorrect outcome.
Mean time to restore (MTTR) measures how long restoration takes across incidents. Define the start and end consistently, and inspect individual incidents alongside the average. Otherwise, short interruptions can conceal a prolonged failure.
Set service level objectives (SLOs) around important customer outcomes. An associated error budget defines the tolerated level of failure and establishes when the team should prioritise reliability over additional feature risk. Quality here works as a feedback system: budget spend throttles new release risk when the failures users experience exceed what was agreed, and business owners should accept that trade-off before an incident forces the conversation.
Investigate repeated recovery
A worker stops processing jobs every month, and restarts keep the queue moving. Each incident closes in minutes, so restoration time looks healthy. Customers still face recurring delays because nobody owns the underlying resource leak. The metric records the restart, while the defect that caused it stays open.
| Log entry | Superficial interpretation | Useful follow-up |
|---|---|---|
| Service restored promptly | Response process works | Was lost or delayed work reconciled? |
| Familiar fault returns | Another isolated incident | Is a corrective task assigned? |
| Alert closes after restart | System is healthy | Has the customer journey recovered? |
An actionable alert needs an owner, a runbook and an escalation path. Noisy alerts teach responders that notifications don't reliably require action. For a concrete runbook example, Mailbeam's guidance to fix stuck verification code emails addresses an onboarding dependency that can block access even when the main application appears healthy.
Grail Computer Labs provides production support after a rebuild, using Sentry errors and traces and product issues captured with Jam to create maintenance tasks in GC Labs' Software Factory. That connection between operational evidence and assigned engineering work is central to application management services.
Low-traffic products can't establish reliability confidently from sparse observations. Start with the journey that matters most, combine production evidence with controlled checks, and resolve three questions: which journey matters most, which error would hurt most, and who gets paged?
Delivery Practices That Speed Releases Without Losing Control
A pipeline can look fast on a dashboard while changes sit unowned in an approval queue for days. Read as a feedback system, delivery metrics balance two signals: lag indicators such as escaped defects and rollback frequency show whether detection works, while flow indicators such as queue age and lead time show what that detection costs.
| Metric | What It Measures | Useful Internal Baseline | Warning Sign |
|---|---|---|---|
| Deployment frequency | How often a change reaches production | Cadence actually sustained across your last three releases | Releases held for monthly or quarterly freezes |
| Lead time for changes | Time from commit to running in production | Where the hours accumulate between commit and deploy | Weeks spent waiting in review queues |
| Change failure rate | Share of releases that need a fix or rollback | Which failure types recur in your own rollbacks | Hotfixes after most releases |
| Time to restore service | Time taken to recover from a failed release | Longest recovery across your recent failed releases | Days to recover, or no tested rollback |
The DevOps Research and Assessment (DORA) metrics describe delivery speed and stability together. External reference bands only carry meaning with their original definitions and service context attached, so a consistent internal baseline beats an unsupported target, especially on an inherited platform. Measure where work waits, not just how long engineers spend implementing it. A fast build pipeline will not resolve an approval queue that nobody owns.

Make each quality gate earn its place
A 2025 industrial study involving 238 practitioners across ten projects found that an AI-assisted code-review tool improved bug detection and awareness of quality practices, while average pull-request closure time rose from 5 hours 52 minutes to 8 hours 20 minutes, with practitioners reporting faulty reviews, unnecessary corrections and irrelevant comments. That is a trade-off each team has to measure in its own review queue.
A control can catch valuable defects while adding real waiting time. Removing a noisy check can improve flow without weakening protection, provided another check covers the same risk. Score each gate on evidence:
- Failure caught. Identify the defect class or requirement it protects.
- Valid work blocked. Track false alarms and the effort needed to resolve them.
- Clearance owner. Name who can investigate, tune or override the check.
- Demonstrated value. Retain examples of meaningful defects caught, including incidents plausibly prevented.
- Deferred findings. Check whether reviewers fix problems or push them into an unowned backlog.
A gate with no documented prevented incident can still earn its place. Some checks address rare but serious risks, and their justification should state that risk explicitly rather than resting on habit.
Reduce release risk without hiding delay
Reverting application code will not undo incompatible data transformations, so database changes need staged compatibility steps written into the release plan, particularly during legacy system migration. The same logic shapes the rest of the route to production. Small, reviewable merges limit the scope of investigation when something breaks. Automated tests should gate the merge where they provide reliable protection, a repeatable deployment pipeline reduces manual variation, and a tested rollback or roll-forward procedure gives responders a credible recovery route.
Control design: Pair review latency and queue age with escaped defects, incidents and rollback frequency before adding or removing a gate.
Measure waiting separately from active review. A growing queue usually points to unclear ownership or oversized changes, and either fix is cheaper than another tool. Controlled flow is the target.
Team Decisions That Keep Quality Accountable
The Boeing 737 MAX failures show why component checks cannot establish system safety on their own. A Harvard Law School Forum account of the Boeing 737 MAX covers the crashes of 29 October 2018 and 10 March 2019, which killed 346 people, the aircraft's approximately 20-month grounding, and failures involving organisational breakdowns, weak safety oversight, communication with the US Federal Aviation Administration and insufficient pilot input.
Ordinary product defects are not an aviation disaster, and teams shouldn't treat them as one. The transferable lesson is narrower: quality lives in the decisions, evidence and handovers around the code. A component can pass every check and still contribute to a failing system, because the failure sits in who approved what, on which evidence, and who owned the risk afterwards.
Make AI approval boundaries explicit
An agent assembles evidence and drafts a claims decision for an analyst to review. The analyst signs off whenever the claim exceeds a business-approved threshold or lacks required evidence, and the record preserves both the proposed decision and the analyst's reason for accepting or changing it. In GC Labs' systems, people approve decisions affecting customers, money, privacy or compliance, while software handles repeatable checks and routine work. Engineers stay in the loop wherever product judgement, risk ownership or customer context is required, and the same separation runs through an agentic software development life cycle.
A workflow like this earns expansion only while it stays explainable. The owner should describe it in plain language, the exception queue should stay small enough to review and well categorised, and the approval boundary should be explicit and accepted by real business stakeholders. A growing exception backlog signals a capacity or design problem in the workflow, so treat it as a cue to resize the work rather than a reason to route around review.
Accountability requires a named person with enough context and authority to say no. Agents execute while humans decide.
Record the decision, not the entire conversation
Significant changes need a short decision record that survives a team or vendor transition:
- Requirement and rationale. State the intended business outcome and the trade-off accepted.
- Verification evidence. Identify the relevant checks, results and unresolved limitations.
- Approval. Record the named approver and date, including any conditions attached to release.
Keep the record close to the change. A concise explanation attached to the requirement, implementation and release approval beats a lengthy document nobody maintains. Decision records add overhead, so reserve fuller treatment for meaningful product or risk choices.
The record also closes the feedback loop. When an escaped defect traces back to a missing approval or an untested assumption, it shows which handover failed, so the team can fix that point instead of adding another gate that slows every change.
Run a Quality Review Before Your Next Release
Run a bounded review before a significant release or a handover to another team. Use repository history, release records and incident logs rather than recollection. The following exercise produces an investigation list, not a certification of safety.
- Score the dimensions. Rate maintainability, testing, reliability, delivery practices and decision-making against the last three releases, attaching evidence to each judgement.
- Count production corrections. Inspect rollbacks, hotfixes and escaped defects across the last ten releases, separating severity and cause.
- Trace an ordinary change. Time its route from commit to production, distinguishing implementation, active review and waiting.
- Assign incident sources. List the top three recurring sources and the person responsible for corrective work.
- Inspect approvals. Confirm that significant release decisions retain a recorded reason and supporting evidence.
- Mark unsafe modules. Identify areas nobody can change confidently and describe the missing knowledge or protection.
Choose improvement or rebuilding from evidence
GC Labs improves the current platform when the foundation is sound enough, the core architecture can support the next stage, production data and business logic are trustworthy, failures sit in identifiable parts, and a reliable release process can be restored quickly. Before rebuilding an inherited platform, GC Labs audits the code, data, infrastructure and product documents left by the previous vendor.
Rebuilding isn't automatically the safer choice. It can discard undocumented business behaviour and introduce migration risk. The decision should explain which constraints cannot reasonably be removed through targeted improvement.
Treat delivery examples as bounded evidence
An Australian healthcare platform was stuck in development for 8 months with no clear deadline in sight; GC Labs took over from the previous vendor, rebuilt the platform and had it live in production within 30 days; by day 40, 100 business customers had been onboarded.
That was a rebuild rather than a repair, and it's one reported project, not a general benchmark. The useful decision is which intervention restores safe change, dependable operation and clear ownership for the platform under review.
Grail Computer Labs specialises in software rescue, replacing underperforming vendors and moving stalled AI and software projects into production, with continuous delivery by the same team afterwards. Bring one critical system or product brief and book a call through Grail Computer Labs to identify the safest useful starting point. An outcome engagement delivers one pre-defined, measurable outcome in 4–8 weeks, with a 15-day trial in which qualified clients can stop within the first 15 days without paying.