Adopting and Adapting Agile Practices: It's more than just Cherry-Picking

I'm deeply interested in anything that will help me and help my team do a better job of building good software. But I’ve been around long enough to learn that there isn’t one way to build good software – that XP, or Scrum, or XBreed, or RUP, or Crystal, or CMMI, or TSP/PSP, or Scrumban, or any of yesterday’s or today's or tomorrow’s new better ways of making software, that none of these methods will give me all of the answers, solve all of my problems: or yours either. I’m a pragmatist. I’ve learned from experience and from thoughtful people like Steve McConnell a simple truth: that there are smart ideas and smart ways of doing things; and there are stupid ideas and stupid ways of doing things. That if you do more smart things and less stupid things, you will have a better chance of success. And that the hard part is recognizing that you are doing stupid things, and then having the discipline to stop doing these stupid things and do smart things instead.

I like to work with other pragmatists, people who are open-minded, who want to understand, who keep looking for what works. I long ago ran out of patience for orthodoxy and for people without the experience, creativity, patience and courage to question and learn, to find their own answers, and who believe that all you need to do is follow the One Right Way. And that if you do anything less, you will fail: that any house you build will be a house of straw (…that article still burns me up).

The people who put together Scrum, or XP, RUP, TSP/PSP, Software Kanban or these other ways to manage software development are smart, and they spent a lot of time thinking through the problems of how to build software, experimenting and learning. I look closely into how other people build software, I try to understand the decisions that they made and why they made them. But that doesn’t mean that what worked for them will be a perfect fit for my organization or your organization, my circumstances or yours.

And it’s clear that I’m not alone in thinking this way. A study published earlier this year by Forrester Research on Agile Development: Mainstream Adoption Has Changed Agility reported some interesting statistics on the state of the practice for software development methods, what development methods companies are following and how they are doing adopting new ideas:

• Roughly 1/3 of developers and development managers are following Waterfall or Iterative methods such as RUP or Spiral
  
• Another 1/3 are following Agile methods (Scrum, XP, Lean, FDD, …)
  
• And 1/3 aren’t following any method at all, or if they are, they don’t know what it is.

Of the organizations adopting Agile methods, less than 1/3 stick to a particular Agile methodology as closely as possible. More than 2/3 deliberately mix different Agile practices; or incorporate Agile practices and ideas into “non-Agile” approaches. The report explains that:

“Perhaps the clearest sign of the mainstreaming of Agile is the abandonment of orthodoxy: Teams are puzzling out the mix of methodologies and combining them to fit within their organizational realities…”

The reasons for this are clear: most organizations find that it is better to learn from these methods and adapt them to their specific circumstances, their culture, their needs, their constraints. The report calls this “cherry-picking”, a “mix-and-match” approach to take the best bits from different methods and create a customized solution.

But I think that “cherry picking” is the wrong metaphor here. It’s more than just taking the “best bits”: it’s understanding why these practices work, and how they work together, how they support each other. Then focusing in on where you are experiencing problems, where you need to save costs, where you need to be faster or where you need to create some other advantage: where you will get the highest return. It’s taking what makes the most sense to you, what fits the best for your organization, what’s important. Understanding what you need and which practices (alone or together) can be applied to your needs; seeing what works, and then building on this to improve.

Some practices make plain good sense no matter where you work and how you work. Frequent builds, or better continuous build and continuous integration, is a no-brainer. Shortening development and delivery cycles, with more frequent checkpoints. Automated developer testing. But it doesn’t matter if you don’t do pair programming, if the developers in your organization don’t like it, won’t do it – as long as you understand why mainline XPers think that pair programming is useful and important, and that you find your own way to write good code, maybe through peer code reviews and static analysis (note I said “and”, not “or”: static analysis is not a substitute for code reviews). It doesn’t matter if what you are doing is XP or Scrum or Scrumban or that you can find a label for it at all. What matters is that it works, and that you are committed to making it work, and to learning, and to making it better.

Risk Management in Software Security

Software security is a problem of managing risks. But most of the information on software risk management is about how to manage generic business, project and technical risks - not software security risks. Even the section on risk analysis in Gary McGraw’s book Software Security: Building Security In describes a generic risk management framework that can be used in a software security program. This is good, but most people involved in software development, and hopefully anyone who is responsible for software security, already understands the fundamentals of risk management. What people need to understand better is how to manage risks from a software security perspective, and how this is different from managing project risks or business risks.

This post is based on feedback that I shared with some of the team working on the next OWASP Development Guide, in my review of an early draft of the introductory section on risk management. I’m not sure if or how my feedback will be reflected in the new guide, since the project has recently undergone a change in leadership and there has been a lot of rethinking and resetting of expectations. So I have taken some time to put my thoughts together here and think through the issues some more.

Risk Management in Software Development

As a software development manager or project manager, our focus (and our training) is on how to identify and manage risks that could impact the delivery of a project. We learn to avoid project failure through project management structure and controls, by following good practices, and by actively and continuously managing risks inside and outside of the project team, including:

• Schedule and estimation risks
  
• Requirements and scope risks – the problem of managing change
  
• Cost and budget and financial/funding risks
  
• Staffing and personnel risks: team skills and availability, turnover, subcontractors, dependencies on partners
  
• Business strategy risks, portfolio risks, ROI/business case
  
• Stakeholder and sponsorship risks – the basic politics of business
  
• Program risks – interfaces and dependencies with other projects
  
• Legal and contracting risks
  
• Technical risks: architecture, platforms, tooling: how well do we understand them, are we on the bleeding edge (will it work).
  

Risk Management in Software Security

Discovering and managing risks from a security perspective is different: the perspective that you need to take is different, and the issues that you need to manage are different.

To find software security risks, you need to think beyond the risks of delivery and consider the operational and business context of the system. You need to look at the design of the system and its infrastructure, IT and business operations, your organization and its security posture, and your organization’s business assets.

Assets: What does the business have that is worth stealing or damaging?

Think like an attacker. Put yourself in the position of a motivated and skilled attacker: most of us can ignore the possibility of being attacked by script kiddies or amateurs who want to show off how smart they are. The bigger, more serious threat now comes from either disgruntled insiders or former employees, or from motivated professional criminals or even nation states.

Every business has something worth stealing or damaging. You need to understand what somebody would want, and how badly they might want it:

Confidential and personal data – are you holding private, personally identifiable or confidential data on customers, on your partners, your personnel, or anyone else?

Are you handling financial payments, or other high-value transactions?

Information about buying patterns or other business activities that would be valuable to competitors or other outside parties.

Information about your financials, investments and spending, or your business plans and strategy.

Intellectual Property: research data, design work, or information not about what you are doing, but how you are doing it – your operational systems, supply chain management. Or the designs and algorithms that drive the technology platform – it may not be the data behind the system that is valuable, the target could be the system itself, your technical knowledge.

Start with data and other assets, then look at the systems that you are building, that you support.

Is the system a critical online service? In rare cases, the system could be part of critical infrastructure (electrical power transmission, or a core financial system such as the NYSE, or maybe an emergency notification system). Or you may be running a completely online business – if the system is down, your business stops. Such systems may be vulnerable to Denial of Service attacks or other attacks that could affect service over an extended period of time. Although DDOS attacks are so 2006, it’s worth remembering what happened to those Internet offshore betting systems held to ransom by distributed DOS attacks a few years ago…

Or attackers could use your system as a launch platform to attack other more valuable systems – by compromising and exploiting your connectivity and trust relationships with other, more strategic or valuable systems or organizations.

Starting early with the idea of identifying assets under threat naturally supports threat modeling later.

Attack Surface: Finding the open doors and windows

Once you know what’s valuable to bad guys, you need to consider how they could get it. Look at your systems, identify and understand the paths into and out of the system, and what opportunities are offered to an attacker. Do this through high-level attack surface analysis: walk around the house, check to see how many doors and windows there are, and how easy they are to force open. The idea behind attack surface analysis is straightforward: the fewer entry points, and the harder they are to access, the safer you are.

Most enterprise systems, especially e-commerce systems, have a remarkable number of interfaces to clients and to other systems, including shared services and shared data. Focus on public and remote interfaces. What is reachable or potentially reachable by outside parties, especially unauthenticated / anonymous access?

What other systems are we sharing information with and how? What do we know about these other systems, can we trust them, rely on them?

Is the application client-facing across a private network, or public-facing across the Internet? Do you offer an application API to partners or customers to build their own interfaces? A desktop client? A browser-based web client? How much dynamic content? Online ordering, queries? How rich an experience does your client provide, using what technologies: Ajax, Flash, Java, Silverlight, …? What about mobile clients?

How much personalization and customization do you offer to your customers: more options and more combinations to design, test and review, more chances to miss something or get something wrong.

Then you look behind the attack surface to the implementation details for each interface, at the technology stack, the trust boundaries, look at the authorization and authentication controls, trace the flow of control and flow of data. And do this each time you make changes or create a new type of interface. This is the basis of threat modeling.

Capabilities and Countermeasures: How hard / expensive is it for them to get in?

What is the state of your defenses, your protective controls?

How secure is the network? Is the server OS, database, and the rest of the technology stack hardened? Are your patches up to date – are you sure?

Was the application built in a secure way? Has it been maintained in a secure way? Is it deployed and operated in a secure way? How do you know? When was your last pen test? Your last security design review? How many outstanding security vulnerabilities or bugs do you have in your development backlog?

Are there any regulatory or legal requirements that need to be satisfied? PCI, HIPAA, GLBA, … Do you understand your obligations, are you satisfying them?

Do you know enough to know how much trouble you might be in? What is the security posture/capability of the organization, of your team? Do you have someone (or some group) in the company responsible for setting security policies and helping with security-based decisions? Has the development team been trained in security awareness, and in defensive coding and security testing? Is the team following a secure SDLC? Are you prepared to deal with security incidents – do you have an incident response team, do they know what to do?

It is important for the stakeholders to understand upfront what we know and what we don’t know. How confident they should be in the team’s ability to deliver a secure solution – and the team’s ability to understand and anticipate security risks in the first place. There are a couple of good, freely-available frameworks for assessing your organization’s software security capabilities. OWASP’s SAMM framework is one that I have used before with some success. Another comprehensive organizational assessment framework is Cigital’s BSIMM which has been built up using data from 30 different software security programs, generally at larger companies.

Back to Managing the Risks

Now you can assess your risk exposure: the likelihood of a successful attack, and the impact, the cost to your company if an attack was successful. With this information you can decide how much more to spend on defenses, and put into place a defensive action plan.

Risk management is done at the strategic level first: working with business stakeholders, managers, the people who make business decisions, who decide where and how to spend money. You need to describe risks in business terms, spend some time selling and educating. The point here is to secure a mandate for security: an agreement with the people who run the business on security priorities and goals and budgets.

Then you move to tactical implementation of your security mandate: figuring out where and how much to focus, what technical and project-based trade-off decisions to make within your mandate.

Start at a high level, with a rough H/M/L rule-of-thumb rating of the probability and impact of each risk or risk area. Use this initial assessment, and your understanding of the team’s security capabilities, to determine where to dig deeper and how deep to dig, where you will need to focus more analysis, reviews, testing.

As you find more, and more specific risks and weaknesses through threat modeling or architectural risk analysis, through code reviews, or pen testing or fuzzing or whatever, consider using Microsoft’s DREAD risk assessment model. I like this model because it asks you to evaluate each risk from different perspectives, forces you to answer more questions:

D: Damage potential, what would happen to the business?
R: Reproducibility – how often does the problem come up, does it happen every time?
E: Exploitability – how easy is to take advantage of, how expensive, how skilled does the bad guy need to be, what kind of tools does he need?
A: Affected users – who (how important) and how many?
D: Discoverability: how easy is it to find?

As you identify risks, you go back and apply generic risk management methods to determine the cost trade-offs and your response:

• accept (plan for the cost, or choose to ignore it because the likelihood is small and/or the cost to fix is high)
  
• avoid (do something else, if you can)
• prevent (plug the hole, fix the problem)
  
• reduce (take steps to reduce the likelihood or impact: put in early warning tools to detect, like an IDS, so that you can react quickly; or contain the risk through fire walls, isolation and partitioning, layered defenses, ...).

In the end, risk management comes down to the the same delicate balance of the same decision-making factors:

• informed judgment
  
• legal requirements and contractual commitments
  
• the business’ general tolerance for any kind of risk, and the company’s willingness to face risks – a startup for example has a much higher tolerance for risk than an established brand
  
• politics: some issues aren’t fundamentally important to the business, but they are important to somebody important – and some issues are important to the business (or should be), but unfortunately aren’t important to anybody important
  
• and economic trade-offs – the cost today vs the potential cost tomorrow.

And then do this over and over again as your circumstances change, as you get new information, as the company's risk profile changes, as the threat landscape changes. The decisions are the same: you just take a different path to get there.