And in order to create as much value as possible we need feedback, fast feedback. So this change in the way we develop software also brought change in the way we test it. One of the principles of Agile Development is getting feedback as early as possible which in turn means we need to showcase our product to stakeholders as soon as possible. Scrum for example aims at having potentially releasable code at the end of every iteration (Sprint).
To do this we also need shorter test-cycles, gone are the days were we developed a feature and then handed it over to test just to get back a list of issues (e.g. the new feature broke old functionality etc.). Automated testing allows us to cut down the time from finishing development of a feature to feedback about how well it works and whether it breaks old functionality. In this context there are three Methodologies around automated testing and subsequently also delivery of new features to our customers:
- Continuous Integration
- Continuous Delivery
- Continuous Deployment
On the following lines I want to give a short overview of these three practices and their differences.
Continuous IntegrationAs explained above, using Agile Development Methodologies we focus on getting stuff out the door much quicker than before in order to get early feedback on what we delivered to be able to create products/features that actually hold value.
Testing is a crucial part of that as we don't want to deliver every 1 to 4 weeks only to discover that the little exploratory testing we did before handing it of to SIT wasn't enough and spend half of the following iteration on fixing what we broke with the last code drop.
This is where Continuous Integration comes in, unit tests, automated integration and acceptance tests can be run periodically or even on every commit to a code repository. This allows us to get early feedback on whether we broke some old functionality during development of the next high-priority feature for our client.
At the end of an iteration the continuous integraton workflow is started once more to check if everything passes and then the code is deployed to the first external system (e.g. SIT) where there are manual tests run on the codebase then moves on to the next stage (e.g. UAT) and is then finally deployed to production.
The important difference to the subsequent practices is that in Continuous Integration the actual deployment process is usually done manually.
Continuous DeliveryContinuous Delivery adds one more step to the Continuous Integration flow shown above: Deployment to Production. It requires your Continuous Integration to be thorough enough so that you feel comfortable releasing the changes to the product at any time all the tests pass. This allows the product to be deployed at any time in its lifecycle with the push of a button.
In short: Continuous Delivery aims to have every change be deployable at any time.
Martin Fowler put up a 4-point checklist on his blog that define continuous delivery:
- Your software is deployable throughout its lifecycle
- Your team prioritizes keeping the software deployable over working on new features
- Anybody can get fast, automated feedback on the production readiness of their systems any time somebody makes a change to them
- You can perform push-button deployments of any version of the software to any environment on demand
Continuous DeploymentContinuous Deployment goes one step further, once you have a working Continuous delivery workflow you're in good shape to go the extra mile toward Continuous Deployment.
Continuous Deployment removes the last manual step in the Continuous Delivery pipeline. Code is not deployed on demand but rather at any point the Continuous Integration flow passes (i.e. all tests pass).
Continuous Integration is sort of the "odd one out" in this comparison as it is a fundemantal part of both Continuous Delivery and Continuous Deployment without which they wouldn't exist. The latter two put an even bigger emphasis on the automated tests as they need to be rock-solid in order to prove the system is working without reasonable doubt. Some companies even re-trained their test teams to write automated tests instead of executing their tests manually.
So, which one's better? Well, there's no real answer for that one. It basically comes down to personal and customer preference.
I for one would probably go with Continuous Delivery in a Customer project as it gives them the choice on whether they want the new version deployed or not. I'd maybe even provide them with a simple frontend which lets them press the button themselves not just make the decision that the button should be pressed. Thus putting them in the driver's seat :).
In a product development environment I'd probably opt for Continuous Deployment as it provides an even shorter feedback loop for the developers.
Have you had experiences with these techniques? Let us know in the comments!