Certified Agile: The PMI-ACP Exam

I sat for the Project Management Institute’s Agile Certified Practitioner (PMI-ACP) exam earlier this week. The PMI-ACP tests your understanding of common Agile development methods, values and practices. It focuses on basic Agile principles, and on Scrum and XP in detail, as well as fundamentals of Lean and Kanban.

Unlike the PMP, there is no Book of Knowledge which defines best practices and a process framework for this certification. Instead there is a certification content outline that explains at a high level the tools, techniques, knowledge and skills that you will be expected to know and will be tested on, and a reference list of books to read which includes some of the usual suspects. Out of this list I’d recommend reading Mike Cohn’s books on Agile Estimating and Planning and User Stories - they are useful for the exam and they're worth reading regardless. If you’re not working in an XP shop you should also read Kent Beck’s Extreme Programming Explained to make sure that you understand XP, and you must read up on the basics of Lean and Kanban. And of course you need to memorize the Agile Manifesto and the Twelve Principles of Agile Software Development front to back.

But I know from writing the PMP several years ago that experience and general reading aren’t enough to prepare for a PMI certification exam. PMI wants everyone who holds a certification to know the same things, and to share the same values and to think and act the same way. There’s an emphasis on orthodoxy – you’re tested not on what you would do (based on your experience and common practical knowledge), but what you should do according to PMI's definition of what “the right way" is to do something. And PMI’s exams are as much a test of your ability to read and write an exam as they are of the subject matter, with trick questions and trip-up answers and questions which are purposefully hard to understand, and even some extra questions thrown in which don’t make sense at all. Writing a test like this is not fun, although the PMI-ACP exam is certainly not as hard as the PMP exam - you shouldn’t need the 3+ hours that you’re given to complete this test.

So like others, I decided to use an exam prep guide to finish my studying.

The PMI-ACP Exam: How to Pass on Your First Try by Andy Crowe is a quick overview of the material that you should know for the exam. Easy to read and easy to follow, it defines key terms and “doing Agile right”, roles and responsibilities and rituals and tools, and covers communication and collaboration issues, and includes some sample questions (and access to a sample online exam). This is not an especially insightful book, but I found it useful for last minute review and cramming.

I did most of my studying with Mike Griffiths’ PMI-ACP Exam Prep: A Course in a Book for Passing the PMI Agile Certified Practitioner (PMI-ACP) Exam, a much more complete study guide, and a good overview of Agile development that is worth keeping and reading on its own. This book builds on materials that Griffiths published earlier on his blog and it is especially good on Agile reporting tools.

Griffiths is one of the experts who created the PMI-ACP program and so he understands what you need to know in depth, and he is a good writer. However, his book is harder to study from than Crowe’s, because it contains a lot more details and because it is structured around the artificial domains that PMI uses to describe Agile development. This results in several discontinuities, where an idea or practice is introduced under “Value Driven Delivery” and then continues later under “Adaptive Planning” or “Continuous Improvement” or one of the other domains (it is not necessary by the way to learn the domains for the exam).

If you have solid experience with Agile development (which you need to in order to meet the qualifying bar) especially Scrum and XP, you should be able to pass the exam with the help of Griffiths’ guide and some general reading to fill in gaps.

Studying for the PMI-ACP has made me examine Agile development ideas and practices in more detail (which is why I decided to apply for the certification). But it hasn't changed how I think about Agile practices and methods or how I think you should follow them. I am just as convinced today as I was before that the key is not following some method in a pure way, but instead to build your own toolkit, to borrow what works from different methods and adapt them to your specific requirements, constraints and situation. And the more that you know and understand about Agile methods and practices, the more tools you have for your toolkit.

Why Scrum Won

In the 1990s and early 2000s a number of different lightweight "agile" development methods sprung up.

Today a few shops use Extreme Programming, including most notably ThoughtWorks and Industrial Logic. But if you ask around, especially in enterprise shops, almost everybody who is “doing Agile” today is following Scrum or something based on Scrum.

What happened? Why did Scrum win out over XP, FDD, DSDM, Crystal, Adaptive Software Development, Lean, and all of the other approaches that have come and gone? Why are most organizations following Scrum or planning to adopt Scrum and not the Agile Unified Process or Crystal Clear (or Crystal Yellow, or Orange, Red, Violet, Magenta or Blue, Diamond or Sapphire for that matter)?

Is Scrum that much better than every other idea that came out of the Agile development movement?

Simplicity wins out

Scrum’s principal strength is that it is simpler to understand and follow than most other methods – almost naively simple. There isn't much to it: short incremental sprints, daily standup meetings, a couple of other regular and short planning and review meetings around the start and end of each sprint, some work to prioritize (or order) the backlog and keep it up-to-date, simple progress reporting, and a flat, simple team structure. You can explain Scrum in detail in a few pages and understand it less than an hour.

This means that Scrum is easy for managers to get their heads around and easy to implement, at a team-level at least (how to successfully scale to enterprise-level Scrum in large integrated programs with distributed teams using Scrum of Scrums or Communities of Practice or however you are supposed to do it, is still fuzzy as hell).

Scrum is easy for developers to understand too and easy for them to follow. Unlike XP or some of the other more well-defined Agile methods, Scrum is not prescriptive and doesn't demand a lot of technical discipline. It lets the team decide what they should do and how they should do it. They can get up to speed and start “doing Agile” quickly and cheaply.

But simplicity isn't the whole answer

But there’s more to Scrum’s success than simplicity. The real trick that put Scrum out front is certification. There’s no such thing as a Certified Extreme Programmer but there are thousands of certified ScrumMasters and certified product owners and advanced certified developers and even more advanced certified professionals and the certified trainers and coaches and officially registered training providers that certified them.

And now the PMI has got involved with its PMI-ACP Certified Agile Practitioner designation which basically ensures that people understand Scrum, with a bit of XP, Lean and Kanban thrown in to be comprehensive.

Whether Scrum certification is good or bad or useful at all is beside the point.

Certification helped Scrum succeed for several reasons. First, certification lead to early codification and standardization of what Scrum is all about. Consultants still have their own ideas and continue to fight between themselves over the only right way to do Scrum and the future direction of Scrum and what should be in Scrum and what shouldn't, but the people who are implementing Scrum don’t need to worry about the differences or get caught up in politics and religious wars.

Certification is a win win win…

Managers like standardization and certification – especially busy, risk-adverse managers in large mainstream organizations. If they are going to “do Agile”, they want to make sure that they do it right. By paying for standardized certified training and coaching on a standardized method, they can be reassured that they should get the same results as everyone else. Because of standardization and certification, getting started with Scrum is low risk: it’s easy to find registered certified trainers and coaches offering good quality professional training programs and implementation assistance. Scrum has become a product – everyone knows what it looks like and what to expect.

Certification also makes it easier for managers to hire new people (demand a certification upfront and you know that new hires will understand the fundamentals of Scrum and be able to fit in right away) and it’s easier to move people between teams and projects that are all following the same standardized approach.

Developers like this too, because certification (even the modest CSM) helps to make them more employable, and it doesn't take a lot of time, money or work to get certified.

But most importantly, certification has created a small army of consultants and trainers who are constantly busy training and coaching a bigger army of certified Scrum practitioners. There is serious competition between these providers, pushing each other to do something to get noticed in the crowd, saturating the Internet with books and articles and videos and webinars and blogs on Scrum and Scrumness, effectively drowning out everything else about Agile development.

And the standardization of Scrum has also helped create an industry of companies selling software tools to help manage Scrum projects, another thing that managers in large organizations like, because these tools help them to get some control over what teams are doing and give them even more confidence that Scrum is real. The tool vendors are happy to sponsor studies and presentations and conferences about Agile (er, Scrum), adding to the noise and momentum behind Scrum.

Scrum certification is a win win win: for managers, developers, authors, consultants and vendors.

It looks like David Anderson may be trying to do a similar thing with Kanban certification. It’s hard to see Kanban taking over the world of software development – while it’s great for managing support and maintenance teams, and helps to control work flow at a micro-level, Kanban doesn't fit for larger project work. But then neither does Scrum. And who would have picked Scrum as the winner 10 years ago?

Contracting in Agile – You try it

One of the key principles in Agile development is

“Customer collaboration over contract negotiation”

Unfortunately, that means that if you’re trying to follow Agile methods, you’re left without useful guidelines to follow when it comes to contracting and coming up with contracts that fit the way that Agile teams work.

Time-and-materials of course is a no-brainer, regardless of how the team works – do the work, track the time and other costs, and charge the customer as you do it. But it’s especially challenging for people who have to work within contract structures such as fixed price / fixed scope, which is the way that many government contracts are awarded and the way that a number of large businesses still contract development work.

The advice for Agile teams usually runs something like: it’s up to you to convince the purchaser to change the rules and accept a fuzzier, more balanced way of contracting, with more give-and-take. Something that fits the basic assumptions of Agile development: that costs (mostly the people on the team) and schedule can be fixed, but the scope needs to be flexible and worked out as the project goes on.

But in many business situations the people paying for the work aren’t interested in changing how they think and plan – it’s their money and they want what they want when they want it. They are calling the shots. If you don’t comply with the terms of the bidding process, you don’t get the opportunity to work with the customer at all. And the people paying you (your management) also need to know how much it is going to cost and when it is going to be done and what the risks are so they know if they can afford to take the project on. This puts the developers in a difficult (maybe impossible) situation.

Money for Nothing and Change for Free

Jeff Sutherland, one of the creators of Scrum, proposes a contract structure called “Money for Nothing and your Change for Free”. The development team delivers software incrementally – if they are following Scrum properly, they should start with the work that is most important to the customer first, and deliver what the customer needs the most as early as possible. The customer can terminate the contract at any point (because they’ve already got what they really need), and pay some percentage of the remainder of the contract to compensate the developer for losing the revenue that they planned to get for completing the entire project. So obviously, the payment schedule for the contract can’t be weighted towards the end of the project (no large payments on “final acceptance” since it may never happen). That’s the “money for nothing” part.

“Change for free” means that the customer can’t add scope to the project, but can make changes as long as they substitute work still to be done in the backlog with work that is the same size or smaller. So new work can come up, the customer can change their mind, but the overall size of the project remains the same, which means that the team should still be able to deliver the project by the scheduled end date.

To do this you have to define, understand and size all of the work that needs to be done upfront – which doesn’t fit well with the iterative, incremental way that Agile teams work. And it ignores the fact that changes still carry a price: the developers have to throw away the time that they spent upfront understanding what needed to be done enough to estimate it and the work that they went in to planning it, and they have to do more work to review and understand the change, estimate it and replan. Change is cheap in Agile development, but it’s not free. If the customer needs to make a handful of changes, the cost isn’t great. But it can become a real drag to delivery and add significant cost if a customer does this dozens or hundreds of times over a project.

Fixed Price and Fixed Everything Contracts

Fixed Price contracts, and especially what Alistair Cockburn calls Fixed-Everything contracts (fixed-price, fixed-scope and fixed-time too) are a nasty fact of business. Cockburn says that these contracts are usually created out of lack of trust – the people paying for the system to be developed don’t trust the people building the software to do what they need, and try to push the risk onto the development team. Even if people started out trusting each other, these contracts often create an environment where trust breaks down – the customer doesn’t trust the developers, the developers hide things from the customer, and the people who are paying the developers don’t trust anybody.

But it’s still a common way to contract work because for many customers it is easier for them to plan around and it makes sense for organizations that think of software development projects as engineering projects and that want to treat software engineering projects the same way as they do building a road or a bridge. This is what we told you we want, this is when we need it, that’s how much you said it was going to cost (including your risk and profit margin), we agree that’s what we’re willing to pay, now go build it and we’ll pay you when you get it done.

Cockburn does talk about a case where a team was successful in changing a fixed-everything contract into a time-and-materials contract over time, by working closely with the customer and proving that they could give the customer what they needed. After each delivery, the team would meet with the customer and discuss whether to continue with the contract as written or work on something that customer really needed instead, renegotiating the contract as they went on. I’ve seen this happen, but it’s rare, unless both companies do a lot of work together and the stakes of failure on a project are low.

Ken Schwaber admits that fixed price contracting can’t be done with Scrum projects (read the book). Again, the solution is to convince the customer to accept and pay for work in an incremental, iterative way.

Martin Fowler says that you can’t deliver a fixed price, fixed time and fixed scope contract without detailed, stable and accurate requirements – which he believes can’t be done. His solution is to fix the price and time, and then work with the customer to deliver what you can by the agreed end date, and hope that this will be enough.

The most useful reference I’ve found on contracting in Agile projects is the freely-available Agile Contracts Primer from Practices for Scaling Lean and Agile Development, by Arbogast, Larman And Vodde.

Their advice: avoid fixed-priced, fixed-scope (FPFS) contracts, because they are a lose-lose for both customer and supplier. The customer is less likely to get what they need because the supplier will at some point panic over delivery and be forced to cut quality; and if the supplier is able to deliver, the customer has to pay more than they should because of the risk premium that the supplier has to add. And working this way leads to a lack of transparency and to game playing on both sides.

But, if you have to do it:

• Obviously it’s going to require up-front planning and design work to understand and estimate everything that has to get done – which means you have to bend Agile methods a lot.
• You don’t have to allow changes – you can just work incrementally from the backlog that is defined upfront. Or you can restrict the customer to only changing their mind on priority of work to be done (which gives them transparency and some control), or allow them to substitute a new requirement for an existing requirement of the same size (Sutherland’s “Change for Free”).

To succeed in this kind of contract you have to:

• Invest a lot to do detailed, upfront requirements analysis, some design work, thorough acceptance test definition and estimation – by experienced people who are going to do the work
• Don’t allow changes in requirements or scope – just replacement / substitution
• Increase the margin of the contract price
• Make sure that you understand the problem you are working on – the domain and technology
• Deliver important things early and hope that the customer will be flexible with you towards the end if you still can’t deliver everything.

PMI-ACP on Agile Contracting?

For all of the projects that have been delivered using Agile methods, contracting seems to be still a work in progress. There are lots of good ideas and suggestions, but no solid answers.

I’ve gone through the study guide materials for the PMI-ACP certification to see what PMI has to say about contracting in Agile projects. There is the same stuff about Sutherland’s “Money for Nothing and your Change for Free” and a few other options. It’s clear that the PMI didn’t take contracting in Agile projects on as a serious problem. This means that they missed another opportunity to help large organizations and people working with large organizations (the kind of people who are going to care about the PMI-ACP certification) to understand how to work with Agile methods in real-life situations.

Does the PMI-ACP set the bar high enough on Risk Management?

I’m trying to understand the PMI’s new certification for Agile Certified Practitioners, and what value the PMI brings to managing software development projects using Agile methods. So I bought RMC’s PMI-ACP Exam Prep Guide which is written by Mike Griffiths, a guy who understands a lot about project management and Agile methods, and who has been heavily involved in the PMI-ACP program.

How PMI-ACP looks at Risk

I started with how the PMI says risk management should be done in Agile projects. Unlike the PMBOK, the PMI-ACP does not treat risk management as a knowledge area. Instead, it integrates risk into the different practice domains and activities in Agile projects, from prioritization to delivery and problem management.

The first mention of risk management is in “Value-Driven Delivery”, treating risks as “anti-value” when considering what is important to the customer and the business. Fair enough.

Later in the same section there is a discussion of how risks need to be considered when managing the backlog – that you should schedule risk avoidance and risk mitigation activities early in the project, and explaining how to rank work by business value and risk. They suggest leveling the playing field by ranking all work (new features and changes and risks) by financial value, expressing everything in monetary terms. Risks have a negative financial return: risk impact in $ x risk probability in %. This only applies to risks that have avoidance / mitigation activities that can be scheduled and costed in the project – not for risks that are accepted or transferred.

I like the approach of managing risks the same as any other work, using the same costing and prioritization approach. It’s more consistent and more actionable than managing risks from separate lists.

Risk Management comes up one more time in Value-Driven Delivery, under a discussion of reporting tools and techniques, in this case how to create and use Risk Burn Down reports.

Then risk comes up again in Adaptive Planning – which makes sense. Risk assessment, like everything else in planning an Agile project, needs to be done incrementally and iteratively. But unfortunately there’s not a lot on how teams are supposed to identify risks in planning.

Later Griffiths suggests a collaborative game called Speedboat or Sailboat, to help the team come up with a list of risks and opportunities. This is Agile, so everything including risk management needs to be fun, and we don’t want to get people bummed out, so it’s important to spend time identifying opportunities too upfront. Team members post anchor (risk) and wind (opportunity) sticky notes around the picture of a boat on the water. Isn’t that nice…

Griffith does say that

“For any project, we should engage the development team, sponsors, customers, and other relevant stakeholders in the process of risk identification. Their ideas, along with reviews of previous projects’ lessons learned lists, risk logs and industry risk profiles, should be used to identify the known and likely risks for the project.”

But you can only use “lessons learned lists” and “risk logs” from previous projects if somebody on the previous project created them – and there are no actions in the PMI-ACP description of risk management to make sure that this gets done. As part of Continuous Improvement, Agile teams do conduct lessons learned reviews in each iteration, rather than waiting until the end of the project (a step that is often skipped because time and money have run out). The point is to act on lessons learned information immediately – not maybe on some other project in the future. This is good, but if people don’t save information for future use, then you can’t talk about using it in the future.

The last reference to risk management is under Problem Detection and Resolution – recommending running risk-based spikes early in the project to assess technical risks. Emphasizing that it is better to find out and fail early if you run into technical problems or limitations.

Is integrated and implicit risk management enough?

The PMI-ACP emphasizes integrated and active risk management as part of incremental planning and delivery.

“Risk management should serve as a driver for work scheduling, moving high-risk activities into earlier iterations of the project and incorporating risk mitigation activities into the backlog.”

Because risk-management activities are treated the same as other backlog items, work is always being done on reducing or containing risk based on negative value. But because risk management is built-in to different practice domains and into different tools and techniques, there’s no one place to understand how risk management should be done in an Agile project, and to assess whether it is being done well or not. You need to look at each practice area and how risk applies in each context. The way that it is organized makes it difficult to get your head around how risk management should be done in an Agile way – which is a source of risk in itself.

My criticisms aren’t of the study guide, which is well-written. They are of the PMI and the PMI-ACP framework. The PMI-ACP does put more emphasis on risk management than other descriptions of Agile development that I have seen so far. But it’s disappointing that the PMI did not take the opportunity to shore up a fundamental weakness in the Agile approach to development and recommend making risk management explicit, adding risk management activities to planning and reviews as a standard practice.

Some of these ideas are described in the Software Project Manager’s Bridge to Agility, a book that maps Agile development to PMI’s project management framework, and one of the books referenced in the PMI-ACP. But in the PMI-ACP as it is described, like most Agile development today, there’s too much reliance on the kind of risk management that comes for free in iterative and incremental development. This is probably enough for small teams working on simple application development projects, but that’s not the audience for PMI certification. Anyone using Agile methods on a larger scale or in high-risk development will need to look someplace else for help.