[Reading time: 3 minutes 54 seconds]
There’s something that has been brewing in my mind for a while: a different way of looking at software QA.
Talking about it is a bit funny, in that it’s subtle. Some of you may read this and find nothing new (and you really might not – I won’t delude myself to imagine I’m the first or only person to have thought of this).
But it’s easy to overlook the crucial subtleties and just go “well, duh”. It’s kind of like embracing agile development. What can be so hard about iterating in two-week sprints? Turns out, a lot is hard about it, and the difficulty is all in the details.
So, this is the traditional view of QA:
QA, according to this drawing, is an activity which you do in a certain phase, and then you’re finished.
You might be tempted to scoff at this picture and denounce it as old-fashioned. I couldn’t fault you: it is. This is an embodiment of the famous waterfall model of course.
Insidiously, though, many teams still operate according to this model, whether they’re conscious of it or not.
Firstly, equating QA with testing. And secondly, considering testing as something to be done at a particular stage in the development process.
Even in orgs which consider themselves progressive, this sort of view may sneak in.
Now, the more modern thinking is to “shift left” your QA activities: perform them earlier in the process, instead of pretty close to the end, after development is finished.
Of course, these days you’ll probably also include automated testing in your toolchain, so the “shape” of testing will have changed. Instead of it being a lengthy human-driven activity, now test creation will be done by humans.
Test execution will happen very quickly, and not be confined to one particular period in the development flow. Instead, various tests may run very close to the actual implementation of a function.
And indeed before the first line of code is written, long before testing, QA already begins in the form of reviews or even just informal conversations.
So, QA becomes this diffuse thing happening in different forms and different intensity during the entire development process.
What’s more: if you squint at it a certain way, some aspects of monitoring are essentially testing; and indeed some organisations are now using testing in production.
So, in essence, QA has taken on a funny new shape, which I like to call the DevOps Testing T.
It features QA all through the development process, and indeed also after. The intensity of QA activities varies of course. In the beginning, when there just isn’t much there yet, it might be at lower intensity (but higher importance). As the body of actual product grows, so does the intensity of QA.
You can see here that there is a very short, focused, high-intensity block of testing (this is why this block is narrow but high – thanks to test automation it’s a lot of effort in a short timespan).
QA efforts start low, gradually ramping up Q&A effort which culminates in a very brief, intense test phase and then it slowly moves into testing in production.
This may look like something very simple and obvious. And in a sense, it is. However it’s actually very difficult to achieve in practice. There are so many interlocking systems, feedback loops and flows of information that the whole thing may become daunting to set up and maintain.
But once more, don’t worry. This is just the end state.
In reality, you would start with one part of that system, get it built out and your feedback loops in place. And then take care of the next aspect. Just start with what lends itself, do it step by step, and if you look back after a year you’ll be surprised at how far you’ve come.
And you’ll never want to work another way again. I know, because I’ve walked this path.
By the way, if you look at it, this is really an embodiment of Gene Kim’s 3 ways:
- systems thinking, in that QA is now an activity that encompasses the entire development and production system
- amplifying feedback loops, in that you get many of them in many different places and phases of development and operation
- continuous improvement and learning, as shown by the fact that these feedback loops never stop – you even receive feedback from production, potentially forever
I’m really happy with this image and the idea it expresses: it gives us a concrete goal, an expression of what a modern development process in a DevOps context should look like.