Implementing Continuous Integration into our Development Practice

We all know that many organizations still adhere to old development practice with complete lack of current agile development methodology as per current technology & trends. To be more productive and agile development organization, it has to use the latest methodologies, processes, Technology and Tool. This work is daunting and one-size can't fit all the organization and it is really difficult to prescribe something specific for an organization. The best we can do is to identify the key areas of improvement and work on those one by one. Then, feed the identified idea through an agile process and let it churn over the period of time. We want to make sure, we re-evaluate the process at least every six months and feed in the lesson-learnt into the guideline.

One of the identified area of improvement is Continuous Integration (CI). I couldn't find any cook-book or one-size-fits-all prescription on this topic, so I'm trying to dig into this topic and come up with some kind of recommendation to enable the Continuous Integration practice into the organization. In my opinion, it is OK, to be not 100% perfect at first time time. I want that my team understands what I am recommending and I have their buy-in into this effort. I would also like to have an active discussion or response on this topic. Before we go further, I would like to define the background and scope of this effort. 

Background and Scope

This approach is for a mid size organization with many legacy applications, with a mix of old legacy applications and modern SaaS applications. Recommended approach of solution delivery is through SaaS approach with minimal configuration and stick to out of the box functionality thus reducing implementation, maintenance and upgrade cost. We want to challenge ourselves in identifying key products which will enable our business rather than implementing an incompatible product with lots of customization. With this it is clear that we are not a hard-core development shop. But, not all solutions could be available in the market or feasible due to incurred acquisition cost. Thus requiring development of custom solutions based on open standards or proprietary standards. Along with this, there are tons of other legacy applications which will keep their position till something planned or unplanned happens.

Before we move ahead we should create a high level methodology, process or guideline, which will help us take informed decision. So, I am trying to stick to following guideline and recommendation:

1. We don't want to re-invent CI, but follow the best practices of industry
2. Make sure to choose only what best fits into our organization
3. Developers are at the core of CI, so make sure, they are in agreement
4. How to socialize this practice across different projects
5. Facilitate an environment where CI practice can evolve within the organization
6. We don't want to over-do this effort. This effort should be limited to the proportion of our development compared to a complex development activity

Define Continuous Integration (CI)

"Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day." - wiki

Continuous Integration relies on following principles:

1. Maintain a code repository
2. Automate the build
3. Make the build self-testing
4. Everyone commits to the baseline every day
5. Every commit (to the baseline) should be built
6. Keep the build fast
7. Test in a clone of the production environment
8. Make it easy to get the latest deliverable
9. Everyone can see the results of the latest build
10. Automate deployment

The definition and list of CI principles should give us a high level overview of what all things encompasses CI practice. If we are able to implement these principles properly then our objective would be complete. So, if my understanding and logic is correct then it would be good idea to go through each principles and examine them carefully to understand what they are, whether we need them or not, what technology do we need to enable them, etc. With this, it seems that we have reached to a obvious conclusion that we need to do a gap analysis on these topics, which will help us identify key area where we need to focus and spend time.

Gap Analysis

We want to examine each CI Principles individually and find the gap within our implementation. As we run down with each principle, we have to do following task:

It is necessary and important that we create a survey of these questionnaire and distributed to all development delivery resources. I mean, Developers, Lead, Designers, Architects, etc.

1. How well the principle is implemented in the organization, compared to the organization's need?
2. Which Tools or Technologies is used currently to enable this principle?
3. Does the existing Tools or Technologies satisfactorily implemented or used?
4. Do we recommend some other Tools or Technologies ?
5. If a survey could not be done then individual principles could be analyzed properly

CI Enablement

In order to introduce the CI practice. Following artifacts and details should be well documented and distributed through proper channel.

CI Process

A proper official document will be created that will describe the complete process of CI practice. In other words, it will describe the full life-cycle from code creation till Release Version distribution.

Socializing 

A wiki should be created for content distribution. This is to facilitate active contribution from anyone from the team and get feedback on overall process. We don't want to introduce this into a document which no one bothers to open. Wiki content could be delivered in document format, but always as a secondary distribution channel.

What Next

1. I will wait for all stakeholders to respond on this. Since, I don't know, what would be the feedback and pulse of the team, further steps would be decided after the active response. Assuming that people are in agreement to a certain extent, we will continue with the following steps. NOTE: These are time bound materials and I will not be waiting forever. I would assume no response as a sign of agreement.
2. Actively create and distribute the Survey
3. Re-work on survey, if the results could not be analyzed properly or need more tweaking.
4. If a survey is not possible then each principles should be analyzed properly.
5. Clearly define the Gap within each CI Principles. This should be based on - "What do we have today and where do we want to go tomorrow?"
6. Identify all current Technologies and Tools
7. Identify all future/recommended Tools and Technologies
8. Now is the time for decision. By this time we would have already analyzed most of the things, if not, re-iterate the things which needs to be done.
9. Identify the candidate Tool and Technology for each CI Principle
10. Identify, define, describe and create a CI Process wiki & document
11. Create CI Governance
12. I'm sure, by the time we are working on these items, we will have many clear thoughts on this, which will guide us.