Review Apps

Testing is very important, it helps us to increase stability, but in an iteration we often build up a load of work into our master/dev branch and then have a stabilisation period, if the testers are going very well then we are kind of always slightly unstable anything just added is not tested and often not looked at by the PO/BA who requested it until a demo or it is released. If the test team is struggling to keep up then what we end up with is a wave of instability, often leading to iterations more focused on bug fixing.

Surely one of the points of agile is to be able to release at any point, having a couple of develop iterations then a stabilisation one is kind of like having one really big iteration. If I remember correctly then the idea of an iteration is that is releasable every iteration at a minimum.

So how to solve this? Review apps! We started this practice when we moved to docker using gitlab, basically docker allowed us to commission environments much faster so we could deploy the application much more easily. So every pull request that gets created can be tested by a tester and reviewed by the person who asked for the change prior to it being merged, this significantly helps to increase the stability of the app.

It can be achieved without docker in my thinking by just having an environment for each person of interest, for example a tester can have their own environment and just deploy the builds that happen automatically on every branch to their environment for testing, then they just add a tag saying that they approve this.

There can be some issues due to things like needing a lot of data in a system to test or perhaps database migrations. These can be stumbling blocks to getting this working, but there are ways round this through good seed data, or by building tools to quickly push test data into the system. It may seem like this could take a lot of time but in my opinion it is worth the effort.

Going forward we are going to look at adding automated smoke testing to the review apps as well, if every pull request is tested, smoke tested and reviewed by the person asking for it hopefully this should lead to us having an extremely stable and releasable master/dev branch as well as helping to guarantee we are building what was originally asked for.