Top 10 Lists for Designing and Writing Secure and Safe Software

If you care about writing secure code, should know all about these Top 10 lists:

OWASP Top 10

The OWASP Top 10 is a community-built list of the 10 most common and most dangerous security problems in online (especially web) applications. Injection flaws, broken authentication and session management, XSS and other nasty security bugs.

These are problems that you need to be aware of and look for, and that you need to prevent in your design and coding. The Top 10 explains how to test for each kind of problem to see if your app is vulnerable (including common attack scenarios), and basic steps you can take to prevent each problem.

If you’re working on mobile apps, take time to understand the OWASP Top 10 Mobile list.

IEEE Top Design Flaws

The OWASP Top 10 is written more for security testers and auditors than for developers. It’s commonly used to classify vulnerabilities found in security testing and audits, and is referenced in regulations like PCI-DSS.

The IEEE Center for Secure Design, a group of application security experts from industry and university researchers, has taken a different approach. They have come up with a Top 10 list that focuses on identifying and preventing common security mistakes in architecture and design.

This list includes good design practices such as: earn or give, but never assume trust; identify sensitive data and how they should be handled; understand how integrating external components changes your attack surface. The IEEE’s list should be incorporated into design patterns and used in design reviews to try and deal with security issues early.

OWASP Proactive Controls

IEEE’s approach is principle-based – a list of things that you need to think about in design, in the same way that you think about things like simplicity and encapsulation and modularity.

The OWASP Proactive Controls, originally created by security expert Jim Manico, is written at the developer level. It is a list of practical, concrete things that you can do as a developer to prevent security problems in coding and design. How to parameterize queries, and encode or validate data safely and correctly. How to properly store passwords and to implement a forgot password feature. How to implement access control – and how not to do it.

It points you to Cheat Sheets and other resources for more information, and explains how to leverage the security features of common languages and frameworks, and how and when to use popular, proven security libraries like Apache Shiro and the OWASP Java Encoder.

Katy Anton and Jason Coleman have mapped all of these controls together (the OWASP Top 10, OWASP Proactive Controls and the IEEE Security Flaws), showing how the OWASP Proactive Controls implement safe design practices from the IEEE list and how they prevent or mitigate OWASP Top 10 risks.

You can use these maps to look for gaps in your application security practices, in your testing and coding, and in your knowledge, to identify areas where you can learn and improve.

Can DevOps(Sec) make Software more Secure?

There was a lot of talk at RSA this year about DevOps and security: DevOpsSec or DevSecOps or Rugged DevOps or whatever people want to call it. This included a full-day seminar on DevOps before the conference opened and several talks and workshops throughout the conference which tried to make the case that DevOps isn’t just about delivering software faster, but making software better and more secure; and that DevOps isn't just for the Cloud, but that it can work in the enterprise.

Rugged DevOps

The Rugged DevOps story is based on a few core ideas:

Delivering smaller changes, more often, reduces complexity. Smaller, less complex changes are easier to code and test and review, and easier to troubleshoot when something goes wrong. And this should result in safer and more secure code: less complex code has fewer bugs, and code that has fewer bugs also has fewer vulnerabilities.

If you’re going to deliver code more often, you need to automate and streamline the work of testing and deployment. A standardized, repeatable and automated build and deployment pipeline, with built-in testing and checks, enables you to push changes out much faster and with much more confidence, which is important when you are trying to patch a critical vulnerability.

And using an automated deployment pipeline for all changes – changes to application code and configuration and changes to infrastructure – provides better change control. You know what was changed, by who and when, on every system, and you can track all changes back to your version control system.

But this means that you need to re-tool and re-think how you do deployment and configuration management, which is why so many vendors – not just Opscode and Puppet Labs, but classic enterprise vendors like IBM – are so excited about DevOps.

The DevOps Security Testing Problem

And you also need to re-tool and re-think how you do testing, especially system testing and security testing.

In DevOps, with Continuous Delivery or especially Continuous Deployment to production, you don’t have a “hardening sprint” where you can schedule a pen test or in-depth scans or an audit or operational reviews before the code gets deployed. Instead, you have to do your security testing and checks in-phase, as changes are checked-in. Static analysis engines that support incremental checking can work here, but most other security scanning and testing tools that we rely on today won’t keep up.

Which means that you’ll need to write your own security tests. But this raises a serious question. Who’s going to write these tests?

Infosec? There’s already a global shortage of people who understand application security today. And most of these people – the ones who aren’t working at consultancies or for tool vendors – are busy doing risk assessments and running scans and shepherding the results through development to get vulnerabilities fixed, or maybe doing secure code reviews or helping with threat modeling in a small number of more advanced shops. They don’t have the time or often the skills to write automated security tests in Ruby or whatever automated testing framework that you select.

QA? In more and more shops today, especially where Agile or DevOps methods are followed, there isn't anybody in QA, because manual testers who walk through testing checklists can’t keep up, so developers are responsible for doing their own testing.

When it comes to security testing, this is a problem. Most developers still don’t have the application security knowledge to understand how to write secure code, which means that they also don’t understand enough about security to know what security tests need to be written. And writing an automated attack in Gauntlt (and from what I can tell, more people are talking about Gauntlt than writing tests with it) is a lot different than writing happy path automated unit tests in JUnit or UI-driven functional tests in Selenium or Watir.

So we shouldn’t expect too much from automated security testing in DevOps. There’s not enough time in a Continuous Delivery pipeline to do deep scanning or comprehensive fuzzing especially if you want to deploy each day or multiple times per day, and we won’t get real coverage from some automated security tests written in Gauntlt or Mittn.

But maybe that’s ok, because DevOps could force us to change the way that we think about and the way that we do application security, just as Agile development changed the way that most of us design and build applications.

DevOpsSec – a Forcing Factor for Change

Agile development pushed developers to work more closely with each other and with the Customer, to understand real requirements and priorities and to respond to changes in requirements and priorities. And it also pushed developers to take more responsibility for code quality and for making sure that their code actually did what it was supposed to, through practices like TDD and relentless automated testing.

DevOps is pushing developers again, this time to work more closely with operations and infosec, to understand what’s required to make their code safe and resilient and performant. And it is pushing developers to take responsibility for making their code run properly in production:

“You build it, you run it”
Werner Vogels, CTO Amazon

When it comes to security, DevOps can force a fundamental change in how application security is done today, from "check-then-fix" to something that will actually work: building security in from the beginning, where it makes the most difference. But a lot of things have to change for this to succeed:

Developers need better appsec skills, and they need to work more closely with ops and with infosec, so that they can understand security and operational risks and understand how to deal with them proactively. Thinking more about security and reliability in requirements and design, understanding the security capabilities of their languages and frameworks and using them properly, writing more careful code and reviewing code more carefully.

Managers and Product Owners need to give developers the time to learn and build these skills, and the time to think through design and to do proper code reviews.

Infosec needs to become more iterative and more agile, to move out front, so that they can understand changing risks and threats as developers adopt new platforms and new technologies (the Cloud, Mobile, IoT, …). So that they can help developers design and write tools and tests and templates instead of preparing checklists – to do what Intuit calls “Security as Code”.

DevOps isn’t making software more secure – not yet. But it could, if it changes the way that developers design and build software and the way that most of us think about security.

Trust instead of Threats

According to Dr. Gary McGraw’s ground breaking work on software security, up to half of security mistakes are made in design rather than in coding. So it’s critical to prevent – or at least try to find and fix – security problems in design.

For the last 10 years we’ve been told that we are supposed to do this through threat modeling aka architectural risk analysis – a structured review of the design or architecture of a system from a threat perspective to identify security weaknesses and come up with ways to resolve them.

But outside of a few organizations like Microsoft threat modeling isn’t being done at all, or at best only on an inconsistent basis.

Cigital’s work on the Build Security In Maturity Model (BSIMM), which looks in detail at application security programs in different organizations, has found that threat modeling doesn't scale. Threat modeling is still too heavyweight, too expensive, too waterfally, and requires special knowledge and skills.

The SANS Institute’s latest survey on application security practices and tools asked organizations to rank the application security tools and practices they used the most and found most effective. Threat modeling was second last.

And at the 2014 RSA Conference, Jim Routh at Aetna, who has implemented large-scale secure development programs in 4 different major organizations, admitted that he has not yet succeeded in injecting threat modeling into design anywhere “because designers don’t understand how to make the necessary tradeoff decisions”.

Most developers don’t know what threat modeling is, or how do to it, never mind practice it on a regular basis. With the push to accelerate software delivery, from Agile to One-Piece Continuous Flow and Continuous Deployment to production in Devops, the opportunities to inject threat modeling into software development are disappearing.

What else can we do to include security in application design?

If threat modeling isn’t working, what else can we try?

There are much better ways to deal with security than threat modelling... like not being a tool.
JeffCurless, comment on a blog post about threat modeling

Security people think in terms of threats and risks – at least the good ones do. They are good at exploring negative scenarios and what-ifs, discovering and assessing risks.

Developers don’t think this way. For most of them, walking through possibilities, things that will probably never happen, is a waste of time. They have problems that need to be solved, requirements to understand, features to deliver. They think like engineers, and sometimes they can think like customers, but not like hackers or attackers.

In his new book on Threat Modeling Adam Shostack says that telling developers to “think like an attacker” is like telling someone to think like a professional chef. Most people know something about cooking, but cooking at home and being a professional chef are very different things. The only way to know what it’s like to be a chef and to think like a chef is to work for some time as a chef. Talking to a chef or reading a book about being a chef or sitting in meetings with a chef won’t cut it.

Developrs aren’t good at thinking like attackers, but they constantly make assertions in design, including important assertions about dependencies and trust. This is where security should be injected into design.

Trust instead of Threats

Threats don’t seem real when you are designing a system, and they are hard to quantify, even if you are an expert. But trust assertions and dependencies are real and clear and concrete. Easy to see, easy to understand, easy to verify. You can read the code, or write some tests, or add a run-time check.

Reviewing a design this way starts off the same as a threat modeling exercise, but it is much simpler and less expensive. Look at the design at a system or subsystem-level. Draw trust boundaries between systems or subsystems or layers in the architecture, to see what’s inside and what’s outside of your code, your network, your datacenter:

Trust boundaries are like software firewalls in the system. Data inside a trust boundary is assumed to be valid, commands inside the trust boundary are assumed to have been authorized, users are assumed to be authenticated. Make sure that these assumptions are valid. And make sure to review dependencies on outside code. A lot of security vulnerabilities occur at the boundaries with other systems, or with outside libraries because of misunderstandings or assumptions in contracts.
OWASP Application Threat Modeling

Then, instead of walking through STRIDE or CAPEC or attack trees or some other way of enumerating threats and risks, ask some simple questions about trust:

Are the trust boundaries actually where you think they are, or think they should be?

Can you trust the system or subsystem or service on the other side of the boundary? How can you be sure? Do you know how it works, what controls and limits it enforces? Have you reviewed the code? Is there a well-defined API contract or protocol? Do you have tests that validate the interface semantics and syntax?

What data is being passed to your code? Can you trust this data – has it been validated and safely encoded, or do you need to take care of this in your code? Could the data have been tampered with or altered by someone else or some other system along the way?

Can you trust the code on the other side to protect the integrity and confidentiality of data that you pass to it? How can you be sure? Should you enforce this through a hash or an HMAC or a digital signature or by encrypting the data?

Can you trust the user’s identity? Have they been properly authenticated? Is the session protected?

What happens if an exception or error occurs, or if a remote call hangs or times out – could you lose data or data integrity, or leak data, does the code fail open or fail closed?

Are you relying on protections in the run-time infrastructure or application framework or language to enforce any of your assertions? Are you sure that you are using these functions correctly?

These are all simple, easy-to-answer questions about fundamental security controls: authentication, access control, auditing, encryption and hashing, and especially input data validation and input trust, which Michael Howard at Microsoft has found to be the cause of half of all security bugs.

Secure Design that can actually be done

Looking at dependencies and trust will find – and prevent – important problems in application design.

Developers don’t need to learn security jargon, try to come up with attacker personas or build catalogs of known attacks and risk weighting matrices, or figure out how to use threat modeling tools or know what a cyber kill chain is or understand the relative advantages of asset-centric threat modeling over attacker-centric modeling or software-centric modeling.

They don’t need to build separate models or hold separate formal review meetings. Just look at the existing design, and ask some questions about trust and dependencies. This can be done by developers and architects in-phase as they are working out the design or changes to the design – when it is easiest and cheapest to fix mistakes and oversights.

And like threat modeling, questioning trust doesn’t need to be done all of the time. It’s important when you are in the early stages of defining the architecture or when making a major design change, especially a change that makes the application’s attack surface much bigger (like introducing a new API or transitioning part of the system to the Cloud). Any time that you are doing a “first of”, including working on a part of the system for the first time. The rest of the time, the risks of getting trust assumptions wrong should be much lower.

Just focusing on trust won’t be enough if you are building a proprietary secure protocol. And it won’t be enough for high-risk security features – although you should be trying to leverage the security capabilities of your application framework or a special-purpose security library to do this anyways. There are still cases where threat modeling should be done – and code reviews and pen testing too. But for most application design, making sure that you aren’t misplacing trust should be enough to catch important security problems before it is too late.

10 things you can do to as a developer to make your app secure: #10 Design Security In

There’s more to secure design and architecture besides properly implementing Authentication, Access Control and Logging strategies, and choosing (and properly using) a good framework.

You need to consider and deal with security threats and risks at many different points in your design.

Adam Shostack’s new book on Threat Modeling explores how to do this in detail, with lots of exercises and examples on how to look for and plug security holes in software design, and how to think about design risks.

But some important basic ideas in secure design will take you far:

Know your Tools

When deciding on the language(s) and technology stack for the system, make sure that you understand the security constraints and risks that your choices will dictate. If you’re using a new language, take time to learn about how to write code properly and safely in that language. If you’re programming in Java, C or C++ or Perl check out CERT’s secure coding guidelines for those languages. If you're writing code on iOS, read Apple's Secure Coding Guide. For .NET, review OWASP's .NET Security project.

Look for static analysis tools like Findbugs and PMD for Java, JSHint for Javascript, OCLint for C/C++ and Objective-C, Brakeman for Ruby, RIPS for PHP, Microsoft's static analysis tools for .NET, or commercial tools that will help catch common security bugs and logic bugs in coding or Continuous Integration.

And make sure that you (or ops) understand how to lock down or harden the O/S and to safely configure your container and database (or NoSQL data) manager.

Tiering and Trust

Tiering or layering, and trust in design are closely tied together. You must understand and verify trust assumptions at the boundaries of each layer in the architecture and between systems and between components in design, in order to decide what security controls need to be enforced at these boundaries: authentication, access control, data validation and encoding, encryption, logging.

Understand when data or control crosses a trust boundary: to/from code that is outside of your direct control. This could be an outside system, or a browser or mobile client or other type of client, or another layer of the architecture or another component or service.

Thinking about trust is much simpler and more concrete than thinking about threats. And easier to test and verify. Just ask some simple questions:

Where is the data coming from? How can you be sure? Can you trust this data – has it been validated and safely encoded? Can you trust the code on the other side to protect the integrity and confidentiality of data that you pass to it? Do you know what happens if an exception or error occurs – could you lose data or data integrity, or leak data, does the code fail open or fail closed?

Before you make changes to the design, make sure that you understand these assumptions and make sure that the assumptions are correct.

The Application Attack Surface

Finally, it’s important to understand and manage the system’s Attack Surface: all of the ways that attackers can get in, or get data out, of the system, all of the resources that are exposed to attackers. APIs, files, sockets, forms, fields, URLs, parameters, cookies. And the security plumbing that protects these parts of the system.

Your goal should be to try to keep the Attack Surface as small as possible. But this is much easier said than done: each new feature and new integration point expands the Attack Surface. Try to understand the risks that you are introducing, and how serious they are. Are you creating a brand new network-facing API or designing a new business workflow that deals with money or confidential data, or changing your access control model, or swapping out an important part of your platform architecture? Or are you just adding yet another CRUD admin form, or just one more field to an existing form or file. In each case you are changing the Attack Surface, but the risks will be much different, and so will the way that you need to manage these risks.

For small, well-understood changes the risks are usually negligible – just keep coding. If the risks are high enough you’ll need to do some abuse case analysis or threat modeling, or make time for a code review or pen testing.

And of course, once a feature or option or interface is no longer needed, remove it and delete the code. This will reduce the system’s Attack Surface, as well as simplifying your maintenance and testing work.

That’s it. We’re done.

The 10 things you can do as a developer to make your app secure: from thinking about security in architectural layering and technology choices, including security in requirements, taking advantage of other people’s code by using frameworks and libraries carefully, making sure that you implement basic security controls and features like Authentication and Access Control properly, protecting data privacy, logging with security in mind, and dealing with input data and stopping injection attacks, especially SQL injection.

This isn’t an exhaustive list. But understanding and dealing with these important issues in application security – including security when you think about requirements and design and coding and testing, knowing more about your tools and using them properly – is work that all developers can do, and will take you a long, long way towards making your system secure and reliable.

Appsec’s Agile Problem

Agile development has a serious Appsec problem. Most Agile development teams suck at building secure software.

But one of the reasons for this is that Appsec has a serious Agile problem. Most security experts don’t understand Agile development and haven’t come to terms with the way the way that Agile teams design and build software; with the way that Agile teams think and work; and especially with the speed at which Agile teams deliver software and make decisions.

The CSSLP and Agile = Epic Fail

You can see this problem in (ISC)²’s Certified Secure Software Lifecycle Professional (CSSLP), which is supposed to help bridge between security and software development. The Official Guide to the CSSLP is 572 pages long. Of this, only 2 pages are spent on Agile development: ½ page each on Scrum and XP, and a couple of pictures. Otherwise, ISC² pretends that software development is done in big formal Waterfall steps (requirements, design, coding, testing, deployment) with lots of documents to review and clear hand-offs at each of these steps where somebody from Security can step in and insert a big formal review/test before the next step can start. Most developers don’t work this way anymore, if they ever did.

Appsec’s Agile Challenges

It’s not clear how and when security should engage with Agile teams that are following Lean, lightweight Agile methods.

How can Security keep up with projects with such short-term planning horizons, plans and priorities that change for every 1- or 2-week sprint? What about teams following Kanban and Just in Time planning and “automagical” prioritization, and Continuous Deployment in Devops, pushing each change out to customers as soon as it is developed?

Where does Security fit in Scrum, or a Scrum of Scrums? What meetings do security engineers need to attend, and what roles are they supposed to play in these meetings? How much input can they / should they have on decisions? Is Security a Chicken or a Pig?

How can Security know when they need to do a security review, if requirements are all captured in 1-sentence User Stories which are “too short on purpose”?

How do you get security activities and requirements included in the backlog?

How can Security catch and correct design and implementation decisions before it is too late if they aren't in the same room as the development team, when developers are learning and deciding on the fly what work needs to be done and how it needs to be done?

When do you schedule security reviews and tests if the design and the code are always changing? When the team is continuously experimenting and trying out new ideas, new programming models, new languages and frameworks and libraries and toolchains?

How do you do threat modeling on a design that is never finished? And how can you assess the design of a system for security risks if “the design is the code” and “the code is the documentation” without having to go through all of the code by hand after it has already been written?

Security and compliance requires a security review for every major software release. But what if there is never a “major release”, what if the development team is releasing small changes to production 20 or 50 or 500 or 5000 times a year?

It Has Already Been Decided

Appsec isn’t prepared for the rapid pace that Agile teams deliver working software, often from the start of a project. Or for the fierce autonomy and independence of self-managing Whole Teams in which developers are free to decide who will do the work and how it will get done. Or for the speed at which these decisions are made.

This is a different way of thinking and working from top-down, plan-driven projects.

Responsibility and accountability for decisions are pushed down to the team and from there to individuals. Lots of people making lots of small decisions, quickly and often – and changing or unmaking these decisions just as quickly and just as often. The ground is always shifting, as people continuously seek out and respond to feedback and new ideas and information, adjusting and backtracking and making course corrections. Constantly changing and tuning how they work through frequent retrospection. A culture and working approach where people are encouraged to fire first and then aim, to make mistakes and embrace failure, to fail early, fail fast and fail often, as long as they keep learning.

The software – and the process that the team follows to design and build and test it – is never done, never stable and therefore “never secure”.

Agile Appsec: Case Studies

Microsoft has taken on the problem of how to do secure Agile development with its SDL-Agile process framework. Unfortunately, it only works for Microsoft: the SDL-Agile is expensive, heavyweight, and draws extensively on the scale and capabilities of Microsoft’s massive internal organization.

Two “From the Trenches” case studies at this year’s OWASP Appsec USA conference in NYC showed how other organizations are taking on the same challenges.

The first case study by Chris Eng and Ryan Boyle at Veracode, a software security as a service provider (couldn't find the link at OWASP) proves how difficult it can be for Appsec to keep up with Agile development teams, even in an organization that does Appsec for a living and has deep security engineering capabilities.

Veracode’s internal Appsec engineering program has continued to learn and adapt as their development organization grew to more than 100 application developers working in a dozen Scrum teams. In the early pre-Agile days, their program relied on static analysis checking (essentially eating their own dog food as they used the same platform technology that the development team was building for customers), staged manual pen testing and ad hoc consultation from the security engineering team.

As the development organization grew and adopted Scrum, Security had to find new ways to work closer with development without slowing the developers down or stretching their security engineering resources too thin. Security engineers got involved in Sprint planning meetings to discover risks, identify which stories needed security reviews, and do some threat modeling. But they found that planning meetings were not the best place for technical security reviews – the security engineers had already missed a lot of design and implementation decisions that developers had already made, which forced the teams to back track or add work after the Sprint had already started, making them miss their commitments. Now security engineers work earlier with the Product Owner to look for risks and to proactively review the team’s backlog and identify candidate stories that Security will need to review and sign-off on or help the team with.

In the second case study, Yair Rovek explained how at LivePerson, 200+ developers in more than 20 Scrum teams build secure software using a common set of technologies, tools and practices. Security engineering works with a central architecture team to build security into the technology platform that all of the development teams share, including custom-built developer-friendly wrappers around ESAPI and other security libraries.

Security reviews and other controls are added at different points in the development cycle: Release planning (identify risks, high-level design, compliance issues), Sprint planning, coding, testing, release. LivePerson uses static analysis tools with custom rules to check that architecture conventions are followed and to alert when a developer integrates new Open Source code so that this code can be reviewed for vulnerabilities. They schedule pen tests for every major release of their software and open up their service to customer pen testing – as a result their systems are almost continuously pen tested throughout the year.

The Future is going to be Faster – and Appsec will have to be too

In his presentation “Application Security at DevOps Speed and Portfolio Scale” at the same OWASP Appsec conference, Jeff Williams asserted that

“Our traditional techniques for doing Appsec are failing, they’re crumbling at the edges”.

Appsec has to speed up, become more flexible and Agile in itself. Because the future is going to keep getting faster.

Software development projects are getting smaller and simpler and more organizations are adopting Agile methods because smaller, Agile projects are less likely to fail and they get to market much faster. Devops, Continuous Delivery and Continuous Deployment, Kanban, the Lean Startup approach of building a Minimum Viable Product quickly and getting it out for feedback, and other ideas about how to deliver more working software faster and cheaper are becoming mainstream.

In order for Appsec to “push left” into the SDLC, Appsec has to change its role from assurance/auditing and compliance to proactively enabling self-service secure development.

We have to stop pretending that big security reviews and stage gates at major project milestones still work (if they ever did). They need to be replaced by lightweight, in-phase, iterative and incremental preventative controls – simple cheap things that make sense to developers and that they can do as part of designing and building software.

There’s still a role for pen testing and other security reviews. But not as a once-a-year annual release certification/assurance step to “prove that the system is secure” or some other fantasy. Pen tests and other reviews are just another source of feedback to the team, information that they can use to learn and adapt and improve. Security reviews need to be cheaper and scaled down, so that they fit into time boxes and so that they can be done earlier and more often.

Security has to be fit into unit testing and Continuous Integration and Continuous Delivery and the other tight, continuous feedback loops that Agile teams rely on, using tools that don’t need to be understood and run by security experts and that fit with how developers think and work.

There are a handful of organizations that are pushing Appsec further into the rapidly blurring lines between development and operations: Etsy, Netflix, and Twitter are already doing Appsec at “DevOps Speed” today, inventing new tools and ideas.

The rest of Appsec has to catch up, or be left behind.

BTW: If you are involved in security for your organization’s software, the SANS Institute would appreciate your insight. Please participate in the SANS Application Security Survey. The survey closes December 20.

What is Important in Secure Software Design?

There are many basic architectural and design mistakes that can compromise the security of a system:

1. Missing something important in security features like access control or auditing, privacy and compliance requirements;
2. Technical mistakes in understanding and implementing defence-against-the-dark-arts security stuff like crypto, managing secrets and session management (you didn’t know enough to do something or to do it right);
3. Misunderstanding architectural responsibilities and trust zones, like relying on client-side validation, or “I thought that the data was already sanitized”;
4. Leaving the attack surface bigger than it has to be – because most developers don’t understand what a system’s attack surface is, or know that they need to watch out when they change it;
5. Allowing access by default, so when an error happens or somebody forgets to add the right check in the right place, the doors and windows are left open and the bad guys can walk right in;
6. Choosing an insecure development platform or technology stack or framework or API and inheriting somebody else’s design and coding mistakes;
7. Making stupid mistakes in business workflows that allow attackers to bypass checks and limits and steal money or steal information.

Learning about Secure Software Design

If you want to build a secure system, you need to understand secure design. Hopefully you won’t start by reading Secure Software Design by Richardson and Thies. While it does describe many of the major issues in application security and IT security in general, and some common threats and vulnerabilities, it (ironically, given the title) doesn't explain how to do secure software design. And too much of the “practical information” in the book is dangerously almost but not quite right: the section on XSS for example, which does mention output escaping, but doesn't explain how to do it properly or that it is much more important than “Scrubbing input for unnecessary characters and altering necessary but possibly dangerous characters” (however you would go about doing that safely). Or mostly wrong: the section on secure database design – no, “One of the simplest ways to protect a web application from an [sic] SQL injection attack is to validate all input parameters” is not correct, and “You should also avoid dynamic SQL and use parameterized stored procedures” is not close enough to being correct to be understood or followed properly. The book does raise awareness of application security issues, and early on the authors do point readers to CERT, SANS and OWASP, so there is hope that students will find and use those resources instead of relying on this book.

Principles – Motherhood and Apple Pie, or Goodness and Rightness and So What?

Every book that takes on secure software design, even a good book like Secure and Resilient Software Development by Merkow and Raghavan, spends time going through basic secure design principles: The importance of C and I and maybe A. Modularity and compartmentalization, separation of responsibilities, economy of mechanism (an unsimple way to say simplicity), least privilege, defence in depth certainly but not security through obscurity, complete mediation (uh huh), and psychological acceptability, and whatever else Saltzer and Schroeder wrote up 40 years ago.

All good and true and wise and right ideas to live by, but you can read this stuff all day (if you can stay awake through it) and it won’t help you to design a more secure system. There’s nothing clear or actionable here – it’s preaching and high-level hand waving. You can’t tell if you have done enough of it, you’ll never know if you got it right or what you missed, or what really important and what isn't.

Threats and Attacks and Risks – Learning to be Afraid of …something…

The rest of secure design is mostly about threats and attacks and exploits – risk-focused threat modeling exercises. Developers design something nice, and then a security expert comes in and attacks their design, looks for weaknesses and oversights, enumerates threats and walks through attack trees and vulnerabilities and tells the developers what they missed and what some theoretical attacker might be able to take advantage of to compromise the system.

This is difficult stuff for developers to understand, and difficult for them to get excited about: you’re asking developers – concrete problem solvers – to think about problems that will "probably never" happen. And to do it properly requires that you not only understand how the system works (and the technology that it works on), but also what kind of attacks are possible in what contexts (and how likely they really are), which means you need specialized experience and knowledge that most developers don’t have and can’t get easily.

But even if you know this stuff and follow a structured approach like STRIDE or maybe Trike there’s no way to know if you’ve done a good job of threat modeling, if you've done enough of it and if you've identified all the important problems, or if you’ve missed some important attack vector or critical vulnerability and your gonna be pwned anyway.

Threat modeling, at least the way it is commonly understood, with expensive meetings where architects and developers and testers and security experts and project managers get together to methodically walk through design documents, and then write up CYA paperwork afterwards, doesn’t fit with the way that most developers actually work – especially developers on Agile teams who do most of their design work incrementally and iteratively, constantly refining and filling the design in as they go. Or developers maintaining legacy systems under constant pressure to fix or change something that is already there as fast and cheaply as they can. There isn’t time or space to fit in threat modeling meetings or all that documentation and paperwork, and it’s probably not the best use of time if they could find some.

Even lighter weight threat modeling hasn't made much of an inroad in development shops and I am not convinced that it will.

Secure Design Checklists, Cheat Sheets and Patterns

When developers are designing and building a system, they want to look forward: towards understanding the problem they are trying to solve and what they need to build and how they can get it built quickly. Rather than looking back at what people missed or did wrong, it’s more valuable and practical and cost-effective to focus on what they should and can do upfront as part of the design – the practices and patterns and tools that they should use and what they shouldn’t, the problems that they have to look out for when they are making design decisions and trade-offs.

I've talked before about how important and useful checklists can be in software security: simple steps and things to think about when working on different design problems, to make sure that you aren't missing something important or doing something stupid.

Microsoft’s Patterns and Practices site includes an (unfortunately “retired”) secure architecture and design review checklist which covers most of the things that you need to think about when designing a secure app. In case this checklist disappears some day, a full copy of it is included in Merkow and Raghavan's book on Secure and Resilient Software Development.

OWASP has a secure design checklist, but it is not targeted to developers – it’s a tool to help an auditor run security design reviews in a document-heavy waterfall environment. There is an OWASP Application Architecture Cheat Sheet (currently in draft), which includes some good questions to ask in initial architecture and high-level design. The rest of the OWASP Cheat Sheets can be used to help designers and coders with specific application security problems – as long as you know what problems you need to solve.

There’s also been some work on secure patterns, which could be useful for developers who take a pattern-based approach to software design. The SEI’s Secure Design Pattern catalog is an attempt to include security in some common software design patterns (secure versions of Factory, Strategy, Builder, Chain of Responsibility…), or to apply patterns to some common software security problems. And there are a couple of books like Core Security Patterns (an intimidating 1000+ page list of security patterns for big standards-based J2EE apps) and Security Patterns in Practice (which has just been published). However, these patterns have not made it to the mainstream – I don’t know many real-life developers who are even aware of these patterns, never mind tried to apply them.

One of the most useful tools I've come across in the secure design space is SD Elements, an online software service which helps development teams make application security decisions. You start by describing your project and its security and compliance requirements and the language(s)/platform that you’re using, and SD Elements guides you through a set of questions and options on how to deal with important security aspects of the design, implementation and testing of the system. It helps you to understand the decisions that you need to make, and holds your hand as you make them.

Security in Design, not Secure Design

Secure design shouldn't be about things that you don’t understand or can’t do anything about. Secure design should be about understanding the problems that you can and should take care of on your own and the problems you shouldn't.

Understanding what your system’s attack surface looks like and what to look out for when you change it.

How trust zones work.

Where and why and how you should use proven application security frameworks and libraries like Shiro or ESAPI, or how to properly leverage the security capabilities of your application framework (Rails or Play or Spring or whatever…).

The first step – and the most important step – is to get software designers and architects to think about security when they think about design, in the same way that they think about time-to-market and developer convenience, or performance or reliability or future proofing or technical elegance. Not just the security features that they should a have stories for, but security as a continuous thread in architecture and design.

When they select tools and languages and frameworks and platforms.

When they think about architectural responsibilities and layering and patterns.

And when they work with data: identifying and tracing and protecting confidential and private information and secrets, taking care of data validation properly, and thinking about safe data access and data storage.

Secure design has to fit into design and how design it is done. It has to be part of decisions as design decisions are being made, not bolted on afterwards in audits and reviews.