The Case Against Certification

Posted on Feb 21 2015
Picture source, licence

[Updated on 26/02/2015 to reflect the fact that SAFe and Scrum do require exams]

After the recent announcement of the DevOps Institute, I expressed a certain degree of anger and consternation on Twitter. Continuing from where I left off in my previous post (where I wondered out-loud who it was who had decided they knew best about DevOps), I thought now would be a good time to set out what my problem is with the certification they offer.

What Are You Even Teaching?

The Institute will offer training. I like training. I think it is useful. I've had some myself, and delivered quite a few courses too (and I still do). Although I'm pretty left-leaning and think capitalism isn't all its cracked up to be, I certainly have no problem with people making money from DevOps in general or DevOps training in particular. If the group had come out and said 'we're going to offer awesome training courses - look at our fantastic content!' I would probably have been very supportive, and may even have pointed people in their direction.

However, the Institute sees certification as essential, and it is front and centre in their marketing. From the homepage:

The DevOps Institute is bringing enterprise level DevOps training and certification to the IT market.

And from Why the DevOps Institute (emphasis mine):

The founders of the DevOps Institute believe that for DevOps to reach its true potential best practices will have to be codified.

And (emphasis mine again):

Enterprises will require certifications to not only show a return on their investment in training, but more importantly a certification in a DOI course will signify that the applicant has been trained in the highest quality, best practices based training available.

I actually went digging to try and find out what their courses offer. But there are virtually no details about the content. Which in turn means we have very little idea as to what things people will be certified for, making the value of the certification even more dubious. Their website spends way more time talking about why certification is important rather than actually telling us what they teach or what the certificate attendees will get will signify!

Pay Cash, Tick Box, Success!

Like Scrum or the Scaled Agile Framework (SAFe) certification, attending the whole course is required to be certified, but unlike those bullshit programs, the DevOps Institute at least makes you sit an exam, and like Scrum or SAFe, the DevOps Institute also makes you sit an exam. From the Institute website:

Class culminates with an independent Foundation exam. Successfully passing (65%) the 60 minute exam, consisting of 40 multiple-choice questions.

So, better than Scrum or SAFe certification then, right? Well, no. DevOps is a cultural movement. How do you assess an individuals understanding of, and ability to participate in a piece of organisational and cultural change by asking a bunch of multiple-choice questions?

I do have some respect for some sorts of certification in IT, but very little of it. Cisco certification for example assesses individuals ability to understand, install, and maintain Cisco hardware. There are lots of clear yes or no type questions you can ask about setting up a router. If I want to install some Cisco networking equipment, I'm likely to look for someone who knows their stuff, and in the world of Cisco, that means I'm looking at their accreditation. But I see this in the same class as making sure the person who installs my gas boiler knows their stuff. In the UK for example their is a program to ensure that gas engineers are properly trained and do things right.

At GeeCon a couple of years ago, I saw Joel Spolsky talk about the cultural anthropology of StackOverflow. In the video, he discussed what it was about those StackExchange sites that worked, and those that didn't. He highlighted the relative failure of the Project Management StackExchange site. He said that for these sites to work well, the questions that get asked have to have fairly obvious right or wrong answers. But the questions on the program management site all too often get answered with 'it depends'. Multiple-choice questions will work well in the same way for verifying if someone knows the clear answer to very specific questions, but for DevOps? We're definitely in 'it depends' territory now.

DevOps is about bringing people closer together. It's about breaking down silos, using different behaviors combined with the appropriate tooling to help bring focus on the end goal. It certainly has very little in common with installing networking equipment. As William Berry put it so well on Twitter:

And again:

Some put across the argument that certificates are more a way to prove that you attended some training, and to help companies feel that they have got their money's worth (as the Institute themselves say Enterprises will require certifications to … show a return on their investment in training). That argument certainly doesn't ring true to me. Proving someone attended training just tells me that they could sit still for 8 hours a day. If companies are doing this to ensure that their people actually went to a course they paid for, and need this because they don't trust their employees, well then I'm sure I don't need to tell you how screwed up that is.

But I Got A Sir-Tiffy-Cat!

So, will I have any faith that someone with such a certificate knows anything about DevOps? No - such a certificate will give me just as much faith in their abilities as Scrum or SAFe certification would. In fact, I would actively question the motivations of someone promoting the fact that they had such a certificate. Many of my friends and colleagues have gone through SAFe and Scrum training, either because a client insisted on it or just out of interest. None of them to my knowledge put this on their CV or spoke about it except to share stories about how pointless the exercise was.

A cynic might wonder aloud of the use of certification is a way for trainers to set up a closed shop and make more money as a result. I wonder what the process (and costs) are like to become part of their Registered Education Partner program for example? This is the sort of thing that has plagued the Scrum community, and undermined many of the sensible things coming from out of that space. The problem is that there is always going to be a conflict of interest when the same group that sets up the certification process controls who can deliver certified training, at the same time as benefiting financially from delivering training themselves.

So, in summary, go ahead and make money from DevOps with my blessing (not that anyone is asking for it). Offer training, set up whatever institute you want. But please, don't claim this sort of certification is going to fix the problems we face in the Industry. The DevOps Institute could have decided to differentiate itself from the wealth of other organisations and companies out there offering solutions in the DevOps space by offering awesome training courses from smart people who understand what DevOps is, and are passionate about sharing that with the world. Instead they seemed to choose certification as their differentiator. And that just leaves me shaking my head.

You may also be interested by a couple of posts from Martin Fowler (full disclosure - Martin and I work at the same company) on the general topic of IT Certification, where he discusses the correlation between certification and competency (say that ten times fast) and his early thoughts on certification in the agile space.

Back to Blog.