QA Q&A

I was recently asked for my opinion about the trend toward eliminating QA as a unique team, instead merging it into the responsibility of the engineers. I've had experience with both structures, and success with both. I'll try to offer a balanced opinion:

Traditional QA

The traditional arrangement is to have completely distinct engineering and QA teams; they may even work for different companies, in different countries.

Pro

• QA staff really knows QA
• Engineers (who are more costly than QA staff) spend all their time writing code
• Peace of mind: independent auditors are validating software before launch

Con

• Engineers are allowed to be lazy (they "throw their code over the wall" and aren't responsible for making sure it works)
• Increased cycle time and waste work: bouncing issues back and forth takes time and communication effort. Plus, lazy engineers introduce more bugs which then take time to fix.

Implicit QA

A recent movement aims to do away with formal QA teams, and make engineers responsible for their own software quality. I'll call it "implicit QA."
Pro

• Engineers assume ownership of software quality, so they're motivated to get it right the first time
• Rapid development cycles: no waiting for QA validation before code can be deployed
• Users are engaged and their feedback is more relevant

Con

• More defects are released to production (Maybe! I haven't systematically studied this. Does anyone have data?)
• Requires engineers to wear many hats and take more responsibility; some may balk

Philosophical Distinction

The only true measure of software quality is user satisfaction. The question is, to what extent can that be predicted before shipping? Traditional QA uses defect counts as the proxy for user complaints, and refuses to ship until all defects are eliminated. Implicit QA asserts that only users can reliably report their own concerns, and rather than trying to catch every defect before shipping, aims to optimize the process of finding and addressing bugs in the wild. You can hear echoes of the Waterfall vs Agile debate here. 

Situational Considerations

An important factor in this decision is the impact that a defect would have if it's released to the wild. The severity of a bug can be placed on a spectrum: 

life-threatening > threat to software company's business > threat to individual client's business/mission > major user inconvenience > minor user inconvenience

This magnitude, divided by the resolution speed, yields the impact of a bug in the wild. In aggregate, the potential impact of all potential bugs indicates the risk presented by low software quality. The higher this risk, the more attention needs to paid to QA. This attention can be paid either to reducing the number of bugs (traditional QA) or to improving resolution time (implicit QA). Or, both. Both philosophies encourage improvements of both of these dimensions, but each philosophy prioritizes one over the other.

Making the problem smaller

In either model, the number of bugs in production and the risk therein can be significantly reduced through automated testing. All modern languages have extensive frameworks for automated testing, against everything from low-level APIs up through front-end UX. The more extensive the test coverage, the fewer bugs make it out of dev. Test-Driven Development (TDD) is the ultimate expression of this approach, but isn't strictly necessary for achieving good test coverage. Pair programming also helps, by having two sets of eyes on the code before it even gets committed.

Recommendation

Either approach is good, and the choice of methodology is less important than hiring good people (devs and/or QA) and aligning them to the mission. Any organization should choose an approach that makes sense for the mission, integrates well with the engineering process, and feels right for the user relationship. Still, I usually lean toward the "implicit" approach, heavily mitigated through automated testing. You'll get faster release cycles and more rapid feature evolution, and a closer connection to the users. Nobody likes bugs, but (IMHO) most communities can tolerate a few defects in exchange for rapid innovation.