Have you ever heard of or worked on a project where everything seems like it is going downhill? Deadlines are not met, clients are not happy, developers are not happy, everybody is blaming everybody else and nothing is going according to plan. While not every project is destined to be a great success, there are software methodologies and practices that can dramatically increase the chance of success of a project and make the development more pleasurable experience.
One of such practices is Continuous Integration. It was originally adopted as the extreme programming practice and its main purpose is to prevent integration problems and to avoid “integration hell”.
What is Continuous Integration?
Continuous Integration is the practice of continuously integrating the changes made to the project and testing them accordingly at least on a daily basis or more frequently.
Martin Fowler explains it nicely in his article on Continuous Integration:
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
Automating your build, test and deploy processes can alleviate much of the headaches and problems commonly happening on projects. Having a reliable method of doing things, and integrating changes more frequently ensures that errors can be found sooner than later. Having a blocking issue that appeared right on the demo as the result of some part of the code you implemented a few months earlier and haven’t had a proper chance to test against other parts of your system is not a pleasant event at all.
Benefits of Continuous Integration
Using the CI approach can be beneficial for any team or project. Benefits include:
Reduced integration risk. Projects more often than not include multiple people working on the separate tasks or parts of the code. When the time comes to merge all of the separate parts of the project there is a great risk of bugs appearing and software not working as planned. Depending on how bad the problem really is, debugging and solving the issue can be really painful and can potentially mean a lot of changes to the code. Integrating on a daily basis or even more frequently can help finding these problems and solving them while they are not as hazardous.
Higher code quality. Not having to worry about the problems, and focusing more on the functionality of the code results in a higher quality product.
The code in version control works. If you have committed something that breaks the build, you and your team get the notice immediately and the problem is fixed before anyone else pulls the “broken” code.
Reduced friction between team members. Having the impartial system in place reduces the frequency of quarrels between team members.
The quality of life improvement for testers. Having different versions and builds of the code can help isolate and trace bugs efficiently, and it makes life easier for the QA team.
Less time deploying. Deploying projects can be very tedious and time-consuming, and automating that process makes perfect sense.
Increased confidence and morale. People that don’t work for fear of breaking something, are more likely to produce better results and can focus their energy and concentration on producing instead of worrying about potential consequences of their actions.
A side effect of all these benefits is that new team members will have much easier time getting into the project. Having a clear vision of the building process can greatly speed up adaptation of the newest dev on the team.
Having said all this, you might be wondering what are the requirements for the installing of the CI system for your needs. There are a few requirements, but they must be fulfilled for the CI to make sense, or be implemented at all.
The first requirement is having the version control system. There is no way around it and there shouldn’t be a way around it. Version control systems are the reliable and proven method to centralize and preserve changes made to your project over time and thus an invaluable asset to any team or project.
Another requirement is a spare server or workstation, or at the very least a virtual machine. Having a clean machine to build your system on is of the essential importance. If you don’t want to mess with either servers or virtual machines, there are many hosted CI tool solutions that abstract the maintenance from the whole process and offer easier scalability. Disadvantages of the hosted systems are usually the lack of configuration options that self-hosted tools offer.
If you opt in for the self-hosted variant, you will need to install one of the many available continuous integration tools. Technically, the CI tool is not required per se, like the IDE is not required for software development but it would be significantly harder to implement Continuous Integration without the help of one. The most widely known and used CI tools include Jenkins, TeamCity, Hudson, Bamboo, Go… Some of the tools are open source, and others offer trials and premium packages. Choosing the right one depends mostly on your project needs, preference and budget.
CI server (or build server) is the software tool that centralizes all your CI operations and provides a reliable and stable environment for you to build your projects on. CI servers are highly configurable and adjustable to be able to build a variety of projects for different platforms. Running builds and tests are the basic features of every build server.
The most important things to consider when using CI server is to have a clean machine prepared for its installation. Having a neutral environment, untainted by unnecessary tools, environment variables, and other configurations, is crucial for the successful usage of the CI server and CI overall. If it’s not possible to install the CI server physical machine, you can set up a virtual environment and use it as the last resort. Using development machines without setting up virtual environments will probably leave you with false assumptions and results, and you will have the same problems when deploying as you would without using the CI server.
Typically CI server uses a version control system like Subversion or Git or any other to pull your project files. It monitors your project’s repository and on the successful commit it pulls the changes and performs the tasks you defined previously for that project. Upon completion of the tasks, CI server sends feedback to the relevant project members with the details of the build. Checking out the latest version of your project, running the build scripts, running the tests, and sending notifications are the most basic functionalities of the CI servers. Besides these, features like code analysis, code coverage, code quality reports, agent pooling, pipelines, build comparisons, IDE integration, third party tools support and many others make the CI servers very flexible and comfortable to use.
For the more detailed list of CI servers and their features, you can check the wiki article here.
Everyone is responsible
Even more important than hardware or software requirements is the ability of the team to take the responsible approach to implementation of the CI. In order to use CI effectively, developers must change their day-to-day software development habits. These habits are considered as good software development practices that should be applied even without continuous integration system in place.
There are six practices that help individuals and teams running CI on a project:
- Commit code frequently
- Don’t commit broken code
- Write unit tests
- Fix broken builds immediately
- All tests must pass
- Avoid getting broken code
That means you should not be checking in the latest changes just before the lunch time or just before you head home. You should wait for the build report to be sure the build is successful, and if it is not you should be the one fixing it. On the other hand, you should always be on a lookout for the broken builds, even if it wasn’t your fault because checking out broken build or checking in over it can prolong the fixing process.
Going a step further
Typical CI lifecycle consists of building the project, unit testing, deploying to stage and acceptance testing. Once the project successfully passes all of these stages, it is ready for the deployment to the production environment.
Delivering a project to the production environment can be done automatically like the rest of the stages, but due to the business concerns, it could be suitable to do it manually. In the first case, we are talking about Continuous Delivery, and in the second Continuous Deployment.
The graph below shows the differences between these three terms.
What prevents teams from using the CI?
There are some concerns that might discourage people from using the CI on their projects.
- Increased project maintenance overhead.
You already need to build, test and deploy your systems. If you don’t manage those processes with the CI, you will be managed by those processes. Once the project gets to the certain complexity, it becomes incredibly hard to manage.
- Too much change.
People might be unwilling to make so many changes to implement the CI on their legacy system. If that is the case, CI could be easily introduced to the system one part at a time or as a daily routine. Once the team members are comfortable with the results, CI can be introduced to the other parts of the system or more frequently.
- Hardware/Software costs.
Costs of the server machine and software tools to implement the CI could be perceived as the unnecessary by both developers and management on the project, but those costs are marginal compared to the costs of the maintenance and finding the problems later down the line.
- Developers are already doing all these operations.
Certainly, but they could be investing more time on the functionality and solving business problems, instead of wasting it on repetitive tasks, which by the way, can be done more reliably by the machine.
- Project too small.
Some projects might not be suited for the CI but even the small projects can be automated. Transparency of the development process and centralization of the project resources and previous builds are also the things that even the smallest project can benefit from.
As the software development gravitates more towards the agile practices it becomes evident that implementation of the continuous integration system on projects is somewhat of an imperative. Infrequent software releases are the thing of the past and most of the leading and successful companies are already actively using these techniques with good results, Amazon being one of the leaders by making changes to production every 11.6 seconds (May 2011).
Setting up the continuous integration system for your project could be the costly and time-consuming task, but it pays for itself many times over once it is up and running. It may not guarantee the success of your project, but it raises the chances of success by a significant margin, and you will most definitely have a more pleasurable working experience and satisfied customers.
More resources on Continuous Integration can be found on Thoughtworks company’s website, and if you want to go even further I recommend you read Continuous Delivery by Jez Humble and David Farley and Continuous Integration by Paul M. Duvall.
What are your experiences when using Continuous Integration? Feel free to share your thoughts in the comment section below.
Reached this far? You should follow me on G+. I post updates there regularly.