Here at Sagepay we work in weekly iterations, i.e. the aim at the end of every weekly iteration is to deliver something new to the user; whether it be a developer, a customer or the manager of a business.
To help with this, we use Jenkins as our continuous integration tool. Once development is complete (we work on branches), the new code is merged to master. This will trigger builds on the pipeline that will deploy to the development environment and run the e2e tests (also on dev). The deployment to the QA environment is manual; I do this when one particular story is complete and is ‘releasable’. I run some basic manual tests on QA to make sure basic flows are working as expected (I have a checklist!).
What I have learnt when you are working in this way is that you have to have 100% confidence in the automated tests that are on the pipeline and the coverage that these tests have. Otherwise, it would not be possible to release weekly and QA would almost always block a release if it were manual.
This leads on to pair programming. When we have decided which stories we are going to work on in a sprint, I pair with a developer and start discussing what tests we should have for this particular story. This usually starts off by looking at the acceptance criteria as a baseline. We write some basic tests, e2e tests are easiest to write first and make them fail. Unit tests/integration tests follow. I will be writing a separate blog post about the different types of testing and the purpose of each level of testing. Once these new tests have passed and regression tests have passed, I am confident that the new code we will release will not break anything.
One thing to note: my focus is not to test everything and anything. It is to focus on what matters to the user and what they will do. How are they using the application? That way you ensure you test what is necessary and what will give you the most useful feedback.
TDD (test driven development) and BDD (behaviour driven development) are usually used hand in hand in an agile environment. However, sometimes it seems like they are used in a way that they mean the same thing which is not true. TDD/BDD what’s the difference eh? Well, to me here it is:
TDD – the focus of writing and making tests pass in the implementation phase helps make the development process faster and more efficient (in terms of faster and useful feedback). Always write tests first when you can (sometimes it is difficult to write tests or it may not make sense to test something so small – I will write another blog post about this).
BDD – real user behaviour and interactions drive tests and development. You are focused on what the user does, whoever the user may be. You will consider the standard user on one end and the extreme user on the other (i.e. someone who will click every button on your site or do strange things like doing a GET on a URL that should only be doing a POST).
So I’ve decided to start a testing blog to share thoughts about different types of testing (or maybe sometimes have a moan!) and how to go about testing in general.
I’ve been in the testing/QA world since July 2011 and graduated this year with a degree in Computer Science. I have worked at these places
- Velti (company has gone down..) – this was an agency that was focused on delivering android/iOS apps to clients such as o2, Vodafone and PlayStation. I also worked on the Panasonic In flight entertainment system application (not sure if this went live in the end) which was pretty cool. The testing here was all manual and using the waterfall approach
- Sagepay – A payment service provider. Pretty much focused on test automation and continuous delivery, little manual testing. The system is built using microservices so there is definitely been a change in my approach to testing – will discuss this in other blog posts