Pitfalls to avoid on your DevOps journey

22 Mar 2016

Working at a number of clients across the banking, investment and media industries has given me the insights and experience to identify a number of bad practices to avoid when adopting DevOps

DevOps can bring many benefits, even more when part of a wider transformation program of change that combines people, processes and technology. As the rush to embrace it gathers momentum, it’s easy to become caught up in the flurry of DevOpsDays, MeetUps and other events that seem to take place weekly or daily across the world. In parallel to this, I’ve experienced what DevOps and Continuous Delivery involves at the coalface, meeting and working with clients every week, and I’ve come to the realisation that there are some misguided common assumptions that IT organisations can often make.

Ignore legacy at your peril

DevOps folks love the cool new stuff. You know, new development languages, frameworks, expandable infrastructure, build kit on demand, creating stateless, micro-service architectures to spin up and down at will. But the reality is that most enterprise organisations have ‘legacy’ applications and infrastructure that have to be maintained until the new world order within IT arrives. These application architectures often store application state and logic in different layers of the stack and you can’t just tear it down and build it up again as you wish. These are often complex multi inter-dependent components running in long lived infrastructures. Often one wrong deployment and the whole system collapses. They are not designed for a simian army to randomly tear down components to test the resilience of the system. Netflix’s Chaos Monkey approach will not work in these environments. There’s a ton of infrastructure out there that simply isn’t architected in this way.

What makes a system legacy anyway?

In my opinion, a legacy system is one that can’t be regression tested. If the system can’t be tested effectively and efficiently, i.e. automated testing that runs in less than one hour (or shorter) cycles, then how can you quickly adapt these systems and make changes to it without validating it still works? All systems need to be architected to adapt and change. Therefore, for these ‘legacy’ systems you need to adopt a different strategy to implement DevOps practices. You need to work with the tools that you have and automate the deployment and testing of these systems. It may be worth investing in new automation and testing frameworks and/or adopting open source equivalents. You need to make sure that the all the components of these systems have automated deployment and testing processes in place. Furthermore, you need to make sure the workflow is in place to order the deployment and testing processes. Once this tenet of DevOps is in place you can then select slices functionality to implement in new architectures in different infrastructures, and still ensure the system functions as a whole whilst having the feedbacks loops in-place to make sure you are safely adapting the system.

Don’t forget the big picture

It’s also essential to understand the processes that exist around all the different components and how to deploy them. There are different teams and different owners of the components and you need to understand how these work together, and their organisational structure. Awareness is needed of the wider political landscape and technology environment. Often when you undertake DevOps projects, you need to engage with multiple teams of different systems, for example; the Windows, Active Directory, Linux, Storage, DB and Middleware teams. Added to this, they can also be duplicated within different business units and are all doing very similar, but different, things. You need to engage with these teams and you can’t just use the tooling you want. It is key to understand the organisation and where the power lies.

You need to establish a set guidelines on teams making decisions – therefore you need to understand how architecturally systems are put together, and also how these are owned politically and organisationally. You need to navigate the organisation and address the architecture constraints as and when you find them. Often decisions that were made years ago or were ill thought out need to be challenged, and you find yourself spending time on educating and evangelising on why this change is required.

Don’t let your passion spill over into arrogance

The topic of DevOps generates huge enthusiasm amongst many, and it’s great that we have a number of fantastic luminaries who trot the globe spreading the word. Pitch your passion just right and you can win others over, inspire and enable them. But crank the noise and energy too much and there’s a risk you become dogmatic, blinkered and inflexible to listening to other’s opinions. Often teams have been working in a particular way, for a reason, and telling a team that they should change should be delivered in a tactile way. Passion isn’t actually a requisite for DevOps success – do things correctly and focus on problem solving. You don’t need to know the history of DevOps to be a good practitioner, just like you don’t need to know when penicillin was invented to be a good doctor; same goes for DevOps. Ease off on the arrogance and superiority and spend time on listening, understanding and being approachable.

Build out DevOps capabilities organically

I have yet seen a command and control DevOps centre of excellence that works effectively to proliferate DevOps practices throughout the enterprise. By building out a separate team of ‘DevOps’ employees you are promoting Conway’s law; the systems that you design will be constrained by the copies of the communication structure within your organisation. Labelling a team of people that they are now ‘DevOps’ rarely works. It can suggest to others that this team is elevated and superior to other teams within the organisation, with special privileges and carte blanche to do what they want. Instead, you should identify full stack projects that you want to implement with DevOps practices. Take that team, and educate them on the new tooling and implementation best practices to use on the project. Once successfully delivered, only then the relevant practitioners will identify themselves and they become the seeds that enable the same behaviours and quality in further projects. When the number of projects scale, that is when you need to implement decision guidelines and governance.

Get back to your grass roots

As a technologist, I love the changes that DevOps is bringing to the industry. I like to solve technical problems, I always have done. The growth of DevOps has brought an explosion of automation and containerisation tools to the market and I like to understand which are the best for any given situation. However, to me DevOps practices are not a new thing. I like to work with different teams to solve a problem. DevOps allows common sense to prevail, to continually improve the way we work as an industry. Processes to create workflows, provision infrastructure, deploy, test and release application have been around for many years. It’s just now the tooling has become so sophisticated we can automate it all and make everyone involved inclusive in the process. We just need to work out how these tools work for our particular system in our unique environment. And that’s the fun part.