COBIT is a useful IT capability assessment tool

| | Comments (12)

I recently evaluated a consulting client's IT infrastructure and operational capabilities using COBIT, which is an assessment standard originally developed in the UK and now used worldwide. I found COBIT to be very useful for this task at the management/process level, although it doesn't really get into the technical details. Here's how I used it, and how you might find it useful too...

Description

COBIT identifies a set of 34 "processes" that the authors consider essential for IT success. These 34 processes are grouped into 4 major categories:

  • Planning & Organization
  • Acquisition & Implementation
  • Delivery & Support
  • Monitoring

For each of these processes, there's a numerical "maturity level" evaluation on a 0-5 scale:

  • 0 -- Non-Existent -- Management processes are not applied at all
  • 1 -- Initial -- Processes are ad hoc and disorganised
  • 2 -- Repeatable -- Processes follow a regular pattern
  • 3 -- Defined -- Processes are documented and communicated
  • 4 -- Managed -- Processes are monitored and measured
  • 5 -- Optimised -- Best practices are followed and automated

You can use this scale for each process to evaluate a number of different things:

  • What level your organization is currently at
  • What level your organization should be (or wants to be)
  • What level is considered "best practice" in your industry
  • What level the best of your competitors have achieved

This is totally a self-applied tool that you can use within your organization. There's no requirement that an outside agency administer the assessment and certify the results or anything like that (though there are plenty of consultants who would be happy to do so).

You can find a great variety of free information, discussion, and guidance about the tool on the COBIT web site.

Of all the material on the web site, I found the Management Guidelines booklet to be the most useful; it's available as a free PDF after registration. Besides general information on how to use COBIT, the Management Guidelines includes a very useful 2-page spread for each of the 34 processes. On the left-hand side of the spread, it talks about what this particular process is all about, and give examples of what you'd see in an organization that was really doing this process right, as well as key goal and performance indicators that you can track to see how well you're handling this process. On the right-hand side of the spread, there are sample statements to help you figure out which of the 0-5 capability level applies to your current situation.

For example, one of the "Delivery & Support" processes is "Manage the Configuration". For that process, they give the following sample statements to illustrate various capability maturity levels:


  • Level 0 -- Non-Existent -- Management does not have an appreciation of the benefits of having a process in place that is capable of reporting on and managing the IT infrastructure, for either hardware or software configurations.
  • Level 1 -- Initial/Ad Hoc-- The need for configuration management is recognised. Basic configuration management tasks, such as maintaining inventories of hardware and software, are performed on an individual basis. No standard practices are applied.
  • Level 2 -- Repeatable but Intuitive -- Management is aware of the benefits of controlling the IT configuration but there is implicit reliance on technical personnel knowledge and expertise. Configuration management tools are being employed to a certain degree, but differ among platforms. Moreover, no standard working practices have been defined. Configuration data content is limited and not used by interrelated processes, such as change management and problem management.
  • Level 3 ...
  • Level 4 ...
  • Level 5 ...

Application

For this particular consulting engagement, the client's goal was to figure out where they were and where they ought to be, from an IT operational standpoint. I used COBIT as a framework for discussion with each member of the client's team. I prepared a simple survey spreadsheet with a page for each staff member, which listed each of the COBIT processes and allowed them to specify their assessment of "Current" and "Target" levels for each process.

I met with each staff member individually to complete their survey spreadsheet interactively, so that I could discuss the issues with them and answer questions about interpretation and application to their particular environment (and thus, hopefully, get data that was more readily comparable between individuals), and so that I could capture any additional insightful comments that they made about each process.

When all the survey interviews were finished, I created a spreadsheet that summarized the survey data from all of the interviews. This summary let us see things like what folks felt the current and target levels were for each process, as well as how much agreement (or disagreement) there was about those levels. This summary is being used by the organization as a framework for further debate and discussion within the organization about what level they're at, why they're at that level, what level they wish to be at, and how to get from where they are to where they want to be.

Limitations

COBIT does a good job of addressing the high-level processes that an IT organization needs to have a handle on, but it doesn't get into any of the technical details. For instance there's one process called "Manage Data", in the "Delivery & Support" category, which covers everything like filesystem organization and naming, backup procedures and mechanisms, capacity management, and so forth. If you want to dig into the technical details of your operation, then you'll need additional tools, such as Geoff Halprin's SA-BOK (System Administration Book of Knowledge).

12 Comments

I mentioned this blog entry on the SAGE-Members mailing list, and Marius Strom (a professional IT auditor) posted an interesting followup:

Date: Wed, 20 Jul 2005 08:20:36 -0700
From: Marius Strom <marius <at> marius.org>
To: Brent Chapman <Brent <at> GreatCircle.COM>
Cc: sage-members <at> sage.org
Subject: Re: [SAGE] COBIT: useful IT capability assessment tool
FWIW, if you're interested in using CobiT-like things to evaluate IT
Infrastructure, you may also be interested in ITIL/MOF as well.
And also, for background, CobiT is loosely based on COSO (which all of
you folks who've had to deal with Sarbanes-Oxley in the past few years
probably know a bit about).
-- 
                       /------------------------------------------------->
Marius Strom           | Always carry a short length of fibre-optic cable.
Professional Geek      | If you get lost, then you can drop it on the
IT Auditor             | ground, wait 10 minutes, and ask the backhoe
http://www.marius.org/ | operator how to get back to civilization.
                       \-------------| Mike Andrews |-------------------->

Hi Brent
Interesting idea - i would like to try this out on my IT dept - would you be willing to supply a copy of your excel doc to me?
Many thanks
Scott

Hi, i am wondering if i could take a look at your survey questions please?
Is 34 high-level questions of current and future targets enough to draw conclusion as where the company stand in the industry?

thanks alot
nata

Hi, i am wondering if i could take a look at your survey questions please?

As explained above, for each of the 34 COBIT-identified "processes", I asked 2 questions:

  • Where are we?
  • Where do we want to be?

I can't send you the survey itself, because it draws heavily on the copyrighted COBIT material. Much of that material is available for free from their website, though, after registration; see the original entry above for the links.

Is 34 high-level questions of current and future targets enough to draw conclusion as where the company stand in the industry?

Are you going to learn everything from these 34 questions? No, of course not. However, it's a good way to start the discussion, and gives you a framework to make sure you don't inadvertently miss anything vital.

Sometimes the most interesting things you learn from a survey like this are where the disagreements among the participants lie; that can give you insight into areas where different folks have different goals or different views of current status, which need to be resolved.

Hi Brent,

Thank you for your prompt reply. I really appreciate it. I have another question.

You only asked 2 questions for each 34 subsections of CobiT. What about the maturity model, KPI&KGI, what about determining the 7 cobit risk criteria?

I mean, if you only ask those 2 questions (where are we and where we want to be), it would be rather too-high-level survey, wouldn't it?

I hope you can advice me on this.

Thank you

You only asked 2 questions for each 34 subsections of CobiT. What about the maturity model, KPI&KGI, what about determining the 7 cobit risk criteria?

I mean, if you only ask those 2 questions (where are we and where we want to be), it would be rather too-high-level survey, wouldn't it?

It's only a starting point, but that's what this particular client requested. As I said, "For this particular consulting engagement, the client's goal was to figure out where they were and where they ought to be, from an IT operational standpoint". How to get from here to there was beyond the scope of this particular consulting assignment. The client used the survey results to identify and resolve disagreements among management and staff (and thus get everybody on the same page about which were the most pressing problems, and how they were going to be addressed), and to determine which areas to focus on first in their upcoming budgeting/planning cycle.

This survey was a quick and effective way to tap into the collective insight of the IT team, to figure out current status and vision for the 34 COBIT-covered areas. It was a consensus-building tool. Even after using the survey to figure out what to work on, they still had to figure out when and how; the survey didn't address that at all.

So, no, this survey wasn't intended as the be-all and end-all of COBIT analyses; it was a simplified starting point for a company which had never even heard of COBIT before.

-Brent

Hi Brent,

Thanks alot for your explanation. I now can start on making the survey questions.

Thank you!

Hi Brent,

Where do we get data about point "What level is considered "best practice" in your industry",

or do you have the maturity score in telecommunication industry in the world?

thx,

Dedy

Where do we get data about point "What level is considered "best practice" in your industry", or do you have the maturity score in telecommunication industry in the world?

Unfortunately, I don't have that data. Some of the larger consultancies might have it, if they've worked with a sufficient number of clients in the industry, but they'd probably consider it proprietary information.

Even without such detailed comparative data, I've found COBIT to be a useful tool for framing discussions within a firm. It gives them a way to come to a shared understanding of where they are, and to discuss and debate where they want to be. Knowing where your peers and competitors are is nice, but not essential; it's often useful enough just to know where you are yourself.

-Brent

Hi there, I'd just like to take this time to tell you personally how much I loved reading your blog.
I believe that everyone would benefit from reading this as it has a slew of wonderful information and knowledge that could be gained in reading your post.
So, I'd just like to tell you thanks for taking the time out of your busy schedule to write this valuable post as I'm certain that I am not the only one that gained a great deal of information out of reading this.
I will be checking back on this blog periodically to check and see if you've written more great posts, as I'm sure a lot of other people will be too.
Take care, can't wait to see what else you post!

HI Brent,

Recently i was assigned a task to do an auditing for ICT Strategic Plan (ISP) by using COBIT as a tool. But to focus only at Plan and Organise domain. which comprised of 10 steps.

How could i come up with the question survey. Could you give and example of the questions?

Thanks

Recently i was assigned a task to do an auditing for ICT Strategic Plan (ISP) by using COBIT as a tool. But to focus only at Plan and Organise domain. which comprised of 10 steps.

How could i come up with the question survey. Could you give and example of the questions?

As I said, I asked all of my survey participants (i.e., all of my client's tech staff) only two questions about each of the 34 COBIT "processes":

  • Where are we?
  • Where do we want to be?

Their answers to each of those two questions, for each of the 34 COBIT processes, were on the 0-5 scale outlined in the COBIT documentation. For each process, the COBIT documentation gives examples of what each rating means (i.e., what a "3" means, versus a "0" or a "5").

I can't send you the survey itself that I created, because it draws heavily on the copyrighted COBIT material. Much of that material is available for free from their website, though, after registration; see the original entry above for the links.

It sounds like you only want to cover 10 of the 34 areas, but that you might want to ask more than my 2 questions for each of those 10 areas. To construct your survey, I'd create a form with a page for each of the 10 areas; each page for a particular area would include a list of the questions you want to ask about that area, as well as background info (drawn from the COBIT documentation) to help your survey respondants understand the question and the detailed meanings of the various answers that they can select from.

Good luck!

-Brent

Pages

About this Entry Archives

This page contains a single entry by Brent Chapman published on July 15, 2005 3:16 PM.

Network Automation BoF scheduled for USENIX conf next week in Anaheim was the previous entry in this blog.

Overviews of incident/outage management principles and practices is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Mailing List

Creative Commons License
This weblog is licensed under a Creative Commons License.
Powered by Movable Type 4.12