Thursday, December 12, 2013

Stop Telling Stories

There are beautiful, simple ideas in today’s Agile development methods that work really well. And some that don’t. Like defining all of your requirements as User Stories.

I don’t like the name. Stories are what you tell children before putting them to bed, not valuable information that you use to build complex systems. I don’t like the format that most teams use to write stories. And I don’t like how they use them.

Sometimes you need Stories, Sometimes you need Requirements

One of the “rules” of Agile is that stories have to be small – small enough to fit on an index card or a sticky note. They are too short on purpose, because they are supposed to be place holders, reminders to have conversations with the customer when you are ready to work on them:

They're not requirements. They're not use cases. They're not even narratives. They're much simpler than that.

Stories are for planning. They're simple, one-or-two line descriptions of work the team should produce.

This isn't enough detail for the team to implement and release working software, nor is that the intent of stories. A story is a placeholder for a detailed discussion about requirements. Customers are responsible for having the requirements details available when the rest of the team needs them.
James Shore, The Art of Agile - Stories

According to Mike Cohn in his book Succeeding With Agile, making stories short forces the team to shift their focus from writing about features to talking about them. Teams want to do this because these discussions are more important than what what gets written down.

But this idea can be – and often is – taken too far. Sure, most people have learned that it’s not possible to write correct, complete, comprehensive requirements specs for everything upfront. But there are lots of times when it doesn't make sense to limit yourself to 1- or 2-line placeholders for something that you hope to fill in later.

Some requirements aren't high-level expressions of customer intent that can be fleshed out in a conversation and demoed to verify that you got it right. They are specs which need to be followed line by line, or rules or tolerances that constrain your design and implementation in ways that are important and necessary for you to understand as early as possible.

Some requirements, especially in technical or scientific domains, are fundamentally difficult to understand and expensive to get wrong. You want to get as much information as you can upfront, so developers – and customers – have the chance to study the problem and think things through, share ideas, ask questions and get answers, explore options and come up with experiments and scenarios. You want and need to write these things down and get things as straight as you can before you start trying to solve the wrong problem.

And there are other times when you've already had the conversation – you were granted a brief window with people who understood very well what they needed and why. You might not get the same chance with the same people again. So you better write it down while you still remember what they said.

Short summary stories to be detailed later or detailed requirements worked out early – different problems and different situations need different approaches.

The Connextra Template: As a {type of user} I want {something}…

Stories started off as simple, free-form descriptions of work that the development team needed to do, like “Warehouse Inventory Report”. But now, the Role-Feature-Reason template also known as the Connextra template/format (because somebody working there came up with it back in 2001) is the way that we are told we should all write user requirements.

as a {type of user}, I want to {do something}, so that {reason}

How did this happen? And why?

According to Mike Cohn (who helped to popularize this template is his books and courses) there are a few reasons that stories should be written this way:

Reason 1

Something significant and I'm tempted to say magical happens when requirements are put in the first person…

Reason 2

Having a structure to the stories actually helps the product owner prioritize. If the product backlog is a jumble of things like:
  • Fix exception handing
  • Let users make reservations
  • Users want to see photos
  • Show room size options
… and so on, the Product Owner has to work harder to understand what the feature is, who benefits from it, and what the value of it is.

Reason 3

I've heard an argument that writing stories with this template actually suppresses the information content of the story because there is so much boilerplate in the text. If you find that true, then correct it in how you present the story… [which is not a Reason to use this template, but a workaround if you do use it}.

Trying to fit every requirement into this template comes with its own set of problems:

User story format is awkward and heavy-handed. “As a ___, I want ___, so I can ____.” The concept is good – there should always be an explanation of “why” the task is desired, to make sure the end result fulfills the actual need. But the amount of verbal gymnastics I've seen people go through to try to make a simple and obvious requirement into a “User Story” proves that despite what Agile says, it’s not always the best way to go.
Talia Fukuroe, 6 Reasons Why Agile Doesn't Work

Lots of other people have seen similar problems:

Steve Ropa at VersionOne (“Why I Don’t Like User Story Templates”) has worked with teams that don’t understand or properly follow core Agile ideas and practices…

But where they excel is making sure every story is expressed in the format As a ____ I can____So That____. No matter what the story is, they find a way to shoe horn it into that template. And this is where things start to fall apart. The need to fit the story into the template becomes more important than the content of the actual story. …

The Template has become a formalized gate. My understanding of stories when I first learned about them was that they were to bring natural language back to the conversation around what the software is intended to do. How are we to move away from formal “shall lists” and requirements documents if we are just replacing them with Story Templates?

Gojko Adzic says that robotically following a standardized story template leads to stories that “are grammatically correct but completely false”:

Stories like this are fake, misleading, and only hurt. They don’t provide a context for a good discussion or prioritisation. They are nothing more than functional task breakdowns, wrapped into a different form to pass the scrutiny of someone who spent two days on some silly certification course, and to provide the organisation some fake comfort that they are now, in fact, agile.

Everything is about the Customer

We are also told that User Stories must be customer-centric, written from the customer’s point of view and always describe something that customers care about.

This argument is built around two central ideas:

  1. The development team must always be clearly delivering Customer Value. From the start of the project, the team is supposed to deliver working features that customers can see, touch, explore and respond to.
  2. The Customer/Product Owner has to be able to understand every requirement, which means that every requirement must be in their language and something that they care about. (This is another example of what’s wrong with the Customer/Product Owner idea in Agile development – that a single person can be responsible for defining everything that is done in a project.)

Focusing exclusively on delivering Customer Value leaves little room for non-functional requirements and constraints, which are critically important in building any real system, and de-emphasizes important design problems and technical work that developer teams need to do in order to deliver a high-quality system and to minimize technical risks.

It has led to lots of unnecessary confusion and disagreement over how developers should take care of non-functional requirements and constraints, developers hiding technical requirements inside customer stories, or trying to warp their own technical and architectural requirements into customer-style stories that end up not making sense to customers or to developers.

It is especially a problem with under-the-covers technical requirements like security, maintainability and supportability – cross-cutting concerns that don't make sense to a customer, but are fundamental constraints that apply across all of the work that the team does and how they do it, not just work done in one Sprint.

Like the idea of a using common template, putting the customer first in requirements was well-intentioned: a way to bring the development team and customers closer together, and a way to make sure that the people paying for the work actually got what they asked for. But insisting that this is the only way that every requirement has to be framed creates unnecessary problems and risks and makes the team’s work harder than it has to be.

Don’t Get Hung Up on User Stories – Do Whatever Works

Insisting that all of the work that every development team needs to do has to be defined in the same way is arbitrary, unnecessary, and wrong. It's like back in the UML days when we learned that requirements had to be captured through Use Cases. Anyone remember how silly - and pointless - it was trying to model functional specifications using stick-men and bubbles?

Teams should use whatever format makes sense for their situation, with as little or as much detail as the problem and situation requires. There are times when I would much rather work with well-defined, detailed documented rules and scenarios that have been carefully thought through and reviewed, or a prototype that has been shown to customers and improved iteratively, than to be limited to a long list of 2-line User Stories and hope that I will be able to get someone who understands each requirement to fill the details in when the team needs them.

At Agile 2013, Jeff Patton in his talk on Agile Requirements and Product Management made it clear that the Story Template is a tool that should be used for beginners in order to learn how to ask questions. Like the “snow plow” technique for beginning skiers – something to be thrown aside once you know what you’re doing. His recommendation is to “use whatever you need to capture requirements: pictures, slides, notes, acceptance tests”. At the same conference, Scott Ambler reiterated that “stories are not enough. They are just one tool, a usage view. There are so many more options”.

Don’t worry about stories or epics and themes – or themes and epics (the Agile community can’t all agree on what’s a theme, what’s an epic, and it doesn't matter anyway). Add details when you need to. Get rid of details that you don’t need.

Don't get caught up in what Roman Pichler calls “Story Mania”:

Some product owners and teams are so fond of user stories that everything is expressed as a story. This either results in some rather odd stories – stories that capture the user interface design, complex user interactions, and technical requirements; or these aspects are simply overlooked.

Like any technique, user story writing has its strengths and limitations. I find stories particularly well suited to capture product functionality, and when applied properly, nonfunctional requirements. But user interface design and complex user interactions are better described by other techniques including design sketches, mock-ups, scenarios, and storyboards. Complement your user stories therefore with other techniques, and don’t feel obliged to only use stories.

Even some of the people who developed the original idea of User Stories agree that not everything can or should be done as a story.

Capture requirements in whatever format works best. Keep requirements as light and clear and natural as possible. If you have important technical requirements that need to be worked on, write them down and don’t worry about trying to warp them into some kind of arbitrary customer stories because you've been told that’s what you have to do in order to “be Agile”.

Let’s all stop telling stories and get some work done.

Thursday, December 5, 2013

Appsec’s Agile Problem

Agile development has a serious Appsec problem. Most Agile development teams suck at building secure software.

But one of the reasons for this is that Appsec has a serious Agile problem. Most security experts don’t understand Agile development and haven’t come to terms with the way the way that Agile teams design and build software; with the way that Agile teams think and work; and especially with the speed at which Agile teams deliver software and make decisions.

The CSSLP and Agile = Epic Fail

You can see this problem in (ISC)2’s Certified Secure Software Lifecycle Professional (CSSLP), which is supposed to help bridge between security and software development. The Official Guide to the CSSLP is 572 pages long. Of this, only 2 pages are spent on Agile development: ½ page each on Scrum and XP, and a couple of pictures. Otherwise, ISC2 pretends that software development is done in big formal Waterfall steps (requirements, design, coding, testing, deployment) with lots of documents to review and clear hand-offs at each of these steps where somebody from Security can step in and insert a big formal review/test before the next step can start. Most developers don’t work this way anymore, if they ever did.

Appsec’s Agile Challenges

It’s not clear how and when security should engage with Agile teams that are following Lean, lightweight Agile methods.

How can Security keep up with projects with such short-term planning horizons, plans and priorities that change for every 1- or 2-week sprint? What about teams following Kanban and Just in Time planning and “automagical” prioritization, and Continuous Deployment in Devops, pushing each change out to customers as soon as it is developed?

Where does Security fit in Scrum, or a Scrum of Scrums? What meetings do security engineers need to attend, and what roles are they supposed to play in these meetings? How much input can they / should they have on decisions? Is Security a Chicken or a Pig?

How can Security know when they need to do a security review, if requirements are all captured in 1-sentence User Stories which are “too short on purpose”?

How do you get security activities and requirements included in the backlog?

How can Security catch and correct design and implementation decisions before it is too late if they aren't in the same room as the development team, when developers are learning and deciding on the fly what work needs to be done and how it needs to be done?

When do you schedule security reviews and tests if the design and the code are always changing? When the team is continuously experimenting and trying out new ideas, new programming models, new languages and frameworks and libraries and toolchains?

How do you do threat modeling on a design that is never finished? And how can you assess the design of a system for security risks if “the design is the code” and “the code is the documentation” without having to go through all of the code by hand after it has already been written?

Security and compliance requires a security review for every major software release. But what if there is never a “major release”, what if the development team is releasing small changes to production 20 or 50 or 500 or 5000 times a year?

It Has Already Been Decided

Appsec isn’t prepared for the rapid pace that Agile teams deliver working software, often from the start of a project. Or for the fierce autonomy and independence of self-managing Whole Teams in which developers are free to decide who will do the work and how it will get done. Or for the speed at which these decisions are made.

This is a different way of thinking and working from top-down, plan-driven projects.

Responsibility and accountability for decisions are pushed down to the team and from there to individuals. Lots of people making lots of small decisions, quickly and often – and changing or unmaking these decisions just as quickly and just as often. The ground is always shifting, as people continuously seek out and respond to feedback and new ideas and information, adjusting and backtracking and making course corrections. Constantly changing and tuning how they work through frequent retrospection. A culture and working approach where people are encouraged to fire first and then aim, to make mistakes and embrace failure, to fail early, fail fast and fail often, as long as they keep learning.

The software – and the process that the team follows to design and build and test it – is never done, never stable and therefore “never secure”.

Agile Appsec: Case Studies

Microsoft has taken on the problem of how to do secure Agile development with its SDL-Agile process framework. Unfortunately, it only works for Microsoft: the SDL-Agile is expensive, heavyweight, and draws extensively on the scale and capabilities of Microsoft’s massive internal organization.

Two “From the Trenches” case studies at this year’s OWASP Appsec USA conference in NYC showed how other organizations are taking on the same challenges.

The first case study by Chris Eng and Ryan Boyle at Veracode, a software security as a service provider (couldn't find the link at OWASP) proves how difficult it can be for Appsec to keep up with Agile development teams, even in an organization that does Appsec for a living and has deep security engineering capabilities.

Veracode’s internal Appsec engineering program has continued to learn and adapt as their development organization grew to more than 100 application developers working in a dozen Scrum teams. In the early pre-Agile days, their program relied on static analysis checking (essentially eating their own dog food as they used the same platform technology that the development team was building for customers), staged manual pen testing and ad hoc consultation from the security engineering team.

As the development organization grew and adopted Scrum, Security had to find new ways to work closer with development without slowing the developers down or stretching their security engineering resources too thin. Security engineers got involved in Sprint planning meetings to discover risks, identify which stories needed security reviews, and do some threat modeling. But they found that planning meetings were not the best place for technical security reviews – the security engineers had already missed a lot of design and implementation decisions that developers had already made, which forced the teams to back track or add work after the Sprint had already started, making them miss their commitments. Now security engineers work earlier with the Product Owner to look for risks and to proactively review the team’s backlog and identify candidate stories that Security will need to review and sign-off on or help the team with.

In the second case study, Yair Rovek explained how at LivePerson, 200+ developers in more than 20 Scrum teams build secure software using a common set of technologies, tools and practices. Security engineering works with a central architecture team to build security into the technology platform that all of the development teams share, including custom-built developer-friendly wrappers around ESAPI and other security libraries.

Security reviews and other controls are added at different points in the development cycle: Release planning (identify risks, high-level design, compliance issues), Sprint planning, coding, testing, release. LivePerson uses static analysis tools with custom rules to check that architecture conventions are followed and to alert when a developer integrates new Open Source code so that this code can be reviewed for vulnerabilities. They schedule pen tests for every major release of their software and open up their service to customer pen testing – as a result their systems are almost continuously pen tested throughout the year.

The Future is going to be Faster – and Appsec will have to be too

In his presentation “Application Security at DevOps Speed and Portfolio Scale” at the same OWASP Appsec conference, Jeff Williams asserted that

“Our traditional techniques for doing Appsec are failing, they’re crumbling at the edges”.
Appsec has to speed up, become more flexible and Agile in itself. Because the future is going to keep getting faster.

Software development projects are getting smaller and simpler and more organizations are adopting Agile methods because smaller, Agile projects are less likely to fail and they get to market much faster. Devops, Continuous Delivery and Continuous Deployment, Kanban, the Lean Startup approach of building a Minimum Viable Product quickly and getting it out for feedback, and other ideas about how to deliver more working software faster and cheaper are becoming mainstream.

In order for Appsec to “push left” into the SDLC, Appsec has to change its role from assurance/auditing and compliance to proactively enabling self-service secure development.

We have to stop pretending that big security reviews and stage gates at major project milestones still work (if they ever did). They need to be replaced by lightweight, in-phase, iterative and incremental preventative controls – simple cheap things that make sense to developers and that they can do as part of designing and building software.

There’s still a role for pen testing and other security reviews. But not as a once-a-year annual release certification/assurance step to “prove that the system is secure” or some other fantasy. Pen tests and other reviews are just another source of feedback to the team, information that they can use to learn and adapt and improve. Security reviews need to be cheaper and scaled down, so that they fit into time boxes and so that they can be done earlier and more often.

Security has to be fit into unit testing and Continuous Integration and Continuous Delivery and the other tight, continuous feedback loops that Agile teams rely on, using tools that don’t need to be understood and run by security experts and that fit with how developers think and work.

There are a handful of organizations that are pushing Appsec further into the rapidly blurring lines between development and operations: Etsy, Netflix, and Twitter are already doing Appsec at “DevOps Speed” today, inventing new tools and ideas.

The rest of Appsec has to catch up, or be left behind.

BTW: If you are involved in security for your organization’s software, the SANS Institute would appreciate your insight. Please participate in the SANS Application Security Survey. The survey closes December 20.

Site Meter