Have you ever heard of or worked on a project where everything seems like it is going downhill?
Deadlines are not met, clients are not happy, developers are not happy, everybody is blaming everybody else and nothing is going according to plan.
While not every project is destined to be a great success, there are software methodologies and practices that can dramatically increase the chance of success of a project and make the development more pleasurable experience.
One of such practices is Continuous Integration. It was originally adopted as the extreme programming practice and its main purpose is to prevent integration problems and to avoid “integration hell”.
In this article you will learn:
- What is Continuous Integration?
- Benefits of using Continuous Integration
- More about Continuous Integration servers
- Your responsibility when using Continuous Integration
- Continuous Integration vs Continuous Delivery
- Potential pitfalls
This knowledge will make you more proficient than most people that blindly walk into the CI world. After you’ve learned about what is the Continuos Integration heare, head to the top Continuous Integration tools to choose the one that fits you best.
Continuous Integration is the practice of continuously integrating the changes made to the project and testing them accordingly at least on a daily basis or more frequently.
Martin Fowler explains it nicely in his article on Continuous Integration:
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
Automating your build, test and deploy processes can alleviate much of the headaches and problems commonly happening on projects. Having a reliable method of doing things, and integrating changes more frequently ensures that errors can be found sooner than later. Having a blocking issue that appeared right on the demo as the result of some part of the code you implemented a few months earlier and haven’t had a proper chance to test against other parts of your system is not a pleasant event at all.
We can all agree that having a blocking issue that appeared right on the demo is not a pleasent experience. CI can help with that. A lot.
Here is how the basic CI cycle looks like.
Using the CI is beneficial for many reasons.
Reduced integration risk. More often than not, working on projects means multiple people are working on the separate tasks or parts of the code. The more people, the riskier the integration. Depending on how bad the problem really is, debugging and solving the issue can be really painful and can potentially mean a lot of changes to the code. Integrating on a daily basis or even more frequently can help reduce these kinds of problems to a minimum.
Higher code quality. Not having to worry about the problems, and focusing more on the functionality of the code results in a higher quality product.
The code in version control works. If you have committed something that breaks the build, you and your team get the notice immediately and the problem is fixed before anyone else pulls the “broken” code.
Reduced friction between team members. Having the impartial system in place reduces the frequency of quarrels between team members.
The quality of life improvement for testers. Having different versions and builds of the code can help isolate and trace bugs efficiently, and it makes life easier for the QA team.
Less time deploying. Deploying projects can be very tedious and time-consuming, and automating that process makes perfect sense.
Increased confidence and morale. People that don’t work for fear of breaking something, are more likely to produce better results and can focus their energy and concentration on producing instead of worrying about potential consequences of their actions.
One side effect of all these benefits is that new team members will have much easier time getting into the project. Having a clear vision of the building process can greatly speed up adaptation of the newest dev on the team.
Having said all this, you might be wondering what are the requirements for the installing of the CI system for your needs. There are a few requirements, but they must be fulfilled for the CI to make sense, or be implemented at all.
The first requirement is having the version control system. There is no way around it and there shouldn’t be a way around it. Version control systems are the reliable and proven method to centralize and preserve changes made to your project over time and thus an invaluable asset to any team or project.
If you are using onsite solutions another requirement is to have a spare server or workstation or at the very least a virtual machine. Having a clean machine to build your system on is of the essential importance.
If you don’t want to mess with either servers or virtual machines, there are many hosted CI tool solutions that abstract the maintenance from the whole process and offer easier scalability. Disadvantages of the hosted systems are usually the lack of configuration options that self-hosted tools offer.
If you opt in for the self-hosted variant, you will need to install one of the many available continuous integration tools.
Technically, the CI tool is not required per se, like the IDE is not required for software development but it would be significantly harder to implement Continuous Integration without the help of one.
The most widely known and used CI tools include Jenkins, TeamCity, Bamboo, Go… Read more about the different CI tools available.
CI server (or build server) is the software tool that centralizes all your CI operations and provides a reliable and stable environment for you to build your projects on. CI servers are highly configurable and adjustable to be able to build a variety of projects for different platforms. Running builds and tests are the basic features of every build server.
The most important things to consider when using CI server is to have a clean machine prepared for its installation. Having a neutral environment, untainted by unnecessary tools, environment variables, and other configurations, is crucial for the successful usage of the CI server and CI overall. If it’s not possible to install the CI server physical machine, you can set up a virtual environment and use it as the last resort.
Using development machines without setting up virtual environments will probably leave you with false assumptions and results, and you will have the same problems when deploying as you would without using the CI server.
Typically CI server uses a version control system like Subversion or Git or any other to pull your project files. It monitors your project’s repository and on the successful commit it pulls the changes and performs the tasks you defined previously for that project. Upon completion of the tasks, CI server sends feedback to the relevant project members with the details of the build. Checking out the latest version of your project, running the build scripts, running the tests, and sending notifications are the most basic functionalities of the CI servers.
Besides these, features like code analysis, code coverage, code quality reports, agent pooling, pipelines, build comparisons, IDE integration, third party tools support and many others make the CI servers very flexible and comfortable to use.
Even more important than hardware or software requirements is the ability of the team to take the responsible approach to implementation of the CI. In order to use CI effectively, developers must change their day-to-day software development habits. These habits are considered as good software development practices that should be applied even without continuous integration system in place.
There are six practices that help individuals and teams running CI on a project:
- Commit code frequently
- Don’t commit broken code
- Write unit tests
- Fix broken builds immediately
- All tests must pass
- Avoid getting broken code
That means you should not be checking in the latest changes just before the lunch time or just before you head home. You should wait for the build report to be sure the build is successful, and if it is not you should be the one fixing it.
On the other hand, you should always be on a lookout for the broken builds, even if it wasn’t your fault because checking out broken build or checking in over it can prolong the fixing process.
Typical CI lifecycle consists of building the project, unit testing, deploying to stage and acceptance testing. Once the project successfully passes all of these stages, it is ready for the deployment to the production environment.
Delivering a project to the production environment can be done automatically like the rest of the stages, but due to the business concerns, it could be suitable to do it manually. In the first case, we are talking about Continuous Delivery, and in the second Continuous Deployment.
The graph below shows the differences between these three terms.
There are some concerns that might discourage people from using the CI on their projects.
- Increased project maintenance overhead.
You already need to build, test and deploy your systems. If you don’t manage those processes with the CI, you will be managed by those processes. Once the project gets to the certain complexity, it becomes incredibly hard to manage.
- Too much change.
People might be unwilling to make so many changes to implement the CI on their legacy system. If that is the case, CI could be easily introduced to the system one part at a time or as a daily routine. Once the team members are comfortable with the results, CI can be introduced to the other parts of the system or more frequently.
- Hardware/Software costs.
Costs of the server machine and software tools to implement the CI could be perceived as the unnecessary by both developers and management on the project, but those costs are marginal compared to the costs of the maintenance and finding the problems later down the line.
- Developers are already doing all these operations.
Certainly, but they could be investing more time on the functionality and solving business problems, instead of wasting it on repetitive tasks, which by the way, can be done more reliably by the machine.
- Project too small.
Some projects might not be suited for the CI but even the small projects can be automated. Especially with the plethora of online Continuous Integration tools that are very easy to setup.
Transparency of the development process and centralization of the project resources and previous builds are also the things that even the smallest project can benefit from.
As the software development gravitates more towards the agile practices it becomes evident that implementation of the continuous integration system on projects is somewhat of an imperative.
Infrequent software releases are the thing of the past and most of the leading and successful companies are already actively using these techniques with good results, Amazon being one of the leaders by making changes to production every 11.6 seconds (May 2011).
Setting up the continuous integration system for your project could be potentially costly and time-consuming task (doesn’t have to be), but it pays for itself many times over once it is up and running.
Now that you are well informed about all the aspects of the Continuous Integration, you can check out my top Continuous Integration tools list.
What are your experiences when using Continuous Integration? Feel free to share your thoughts in the comment section below.