OWASP has just announced their 2010 US Appsec conference. It looks like an interesting opportunity to explore the state of the art in software security. Last year, some of the leaders of this conference were concerned about how few software developers showed up for the sessions. I expect the same will happen this year: the audience will be made up of a self-referencing group of security specialists and consultants, and a handful of developers and managers who are looking to understand more about software security. And that, I think, is as it should be.
Travel and education budgets are tight. Developers and managers need to choose carefully where to spend their company’s money and time – or their own. Where can they get the most information for their own work, where can they meet people who will help them solve problems or move their careers forward?
Security experts like Jeremiah Grossman are right: developers don’t understand security, they aren’t taking ownership for building secure software, it’s not important to them.
What’s important to software developers? Delivery: if we don’t deliver, we fail. Check out the major software development conferences, the events that attract senior developers, architects, development managers, test managers, project managers. They are all about delivery, how to deliver faster, better, cheaper: Agile methods, understanding Lean/Kanban principles applied to software development, leadership and collaboration and communication and effecting organizational change, managing distributed teams and global development, getting requirements right, getting design right, tracking whether the project is on target, metrics, continuous integration, continuous delivery, continuous deployment, TDD and BDD, refactoring and improving code quality, improving the user experience, newer and better development platforms and languages and tooling.
With the notable exception of the recent NFJS UberConf, there isn’t any serious coverage of secure software development, secure SDLCs, software security problems at software development conferences. The question shouldn’t be why there are so few developers attending a software security conference. The question should be why there is so little coverage of software security at software development conferences and in the other places where developers and managers get their information: in the development-focused books and blogs and seminars.
Building Bridges
I’ve posted before about my concern about the gap between the software development and software security communities. But there is a way forward. Around 10 years ago, the development and testing community were far apart in values and goals; testing was inefficient and was seen as “somebody else’s problem”. But Agile development made it important for developers to test early and often, made it important for developers to understand testing and code quality, to find better and more efficient ways to test. Testing is cool now – and more than that, it’s expected. Developers look to professional testers for help in improving their own testing, and to find bugs that they don’t understand how to find, through exploratory testing and integration testing and more advanced testing techniques. The development community is taking responsibility for testing their own work, for the quality of their work. And I believe that software development, and software, are both better for it.
But software security is still “somebody else’s problem”. This needs to change. The solution isn’t to try to entice developers to attend security conferences. It's not to force certification in secure development through SANS or ISC. It’s not passive attempts to “infect” development managers with vague ideas about being "Rugged" that are supposed to somehow change how developers build software. And holding software producers liable for their mistakes, while clearly showing the frustration of security specialists, and while making for a provocative sound bite, is not likely to happen either.
The solution is to make secure software development a problem for software developers, a problem that we need to solve ourselves. Engage leaders in the wider software development community: the people who spend their lives thinking about and writing about better ways to build better software; the people who help shape the values and priorities of the development community; the people who help developers and managers decide where to spend time and effort and money. And help them to understand the problems and how serious these problems are, convince them that they need to be involved, convince them that we need to include security in software development, and ask them to help convince the rest of us.
Engage people like Steve McConnell, and Martin Fowler, and Kent Beck, Uncle Bob Martin, Michael Feathers, the NFJS uber geeks, Joel Spolsky and Mike Atwood, Scrum advocates like Mike Cohn and Ken Schwaber, Lean evangelists like Tom and Mary Poppendieck, David Anderson on Kanban, agile project managers like Johanna Rothman, and leaders from the software testing community like James Bach and Jonathan Kohl. And ask them who else they think should be engaged because they’ll know better who can make a difference.
Invite them to come in and work with the best of the software security community, to understand the challenges and issues in building secure software, ask them to consider how to “Build Security In” to software development. And with their help, maybe we will see software security problems owned by the people responsible for building software, and those problems solved with the help and guidance of experts in the security community in a supportive and collaborative way. Otherwise I am afraid that we will continue to see the communities drift apart, the gap between our priorities growing ever wider. And software, and software development, will not be the better for it.
13 comments:
Gunnar Peter and Jeff Williams made an attempt at the QCon conference (InfoQ) a few years ago. I occasionally see appsec-savvy Microsoft developers teaching software security at MSDN conferences or via outlets like Pluralsight. Sometimes you'll find people like me trying to talk about the dev-test side of software security at hacker conferences.
A bridge has been built in many organizations between development and quality-assurance. Many waterfall SQEs left the typical QA groups to join development as "dev-testers". The role of the SQE, at least in a lot of Agile environments, has been reborn.
Security testers and penetration-testers need to be reborn as well. Instead of identifying issues that affect quality, we must rely on the work already done by dev-testers and interface directly with them, their methods, their test harnesses, their automation methods, their manual methods and tools. Then, we must simply "fix the security bugs" and usually do so while "nobody is looking". I envision this role as the security-bugfixer and have been discussing it for quite some time now. The point is that someone is paid to fix the security bugs as they come along (because nobody else is going to do it).
Until we can get to "security-bugfixers", we will have the current role: security advisors (http://mikeboberski.blogspot.com/2010/07/what-im-up-to-these-days.html) aka the "security buddy". This person works on `building security in' to the entire lifecycle. In my opinion, avoiding classic SDLC models and even the Microsoft SDL is preferred. I think that security advisors should work with the new ALM development paradigm -- but we've discussed application lifecycle management in the past.
I tend to think that we'll have our own conferences, which may look a lot like Cigital's verifyconference.com. A lot of these efforts to combine quality and security testers have failed. Perhaps OWASP AppSec is the right place for them for now -- but I dislike that people adverse to this cause, such as Jeremiah Grossman and RSnake -- get invited.
Even at the OWASP conferences, penetration-testing's popularity and the rockstar personas surrounding them permeate and pervert the con. I would assume that this has something to do with fame and fortune: two things that the hacker community cannot seem to let go.
If these Software Developer gurus haven't gotten it by now, I doubt they ever will. Most of them formed their outlook back when the network was trusted (if not before the network).
Most security conferences are open and the knowledge is generally available to whomever wants to learn it. Asking them for "help" is unlikely to be productive.
Until such time as their "ship the prototype in 21 days" methodologies are recognized for what they are, we will have researchers presenting at security conferences informing the marketplace about which development shops can and can't do security.
But I have a lot of sympathy for the individual developers who would love to be more engaged but have employers that don't see the cost/benefit.
If these Software Developer gurus haven't gotten it by now, I doubt they ever will. Most of them formed their outlook back when the network was trusted (if not before the network)
Fortunately, the network is going away, or at least the concept of infrastructure containing both net and apps is going away with the rise of cloud computing.
There Jericho Forum predicted the end to the DMZ concept, and I am going to have to agree with much of their premises.
What we have now is metastructure (BGP, DNS, routing, switching, virtual networks -- shit that I don't care about anymore and know a lot about) vs. infostructure (apps and data). Most people no longer need to care about metastructure.
Most security conferences are open and the knowledge is generally available to whomever wants to learn it. Asking them for "help" is unlikely to be productive
I disagree. I think that developers go to security conferences, but these conferences are still full of network-centric and WiFi-centric attack information. Application security has not yet hit it's prime in terms of maturity nor popularity.
I also think that we need to raise awareness to software security issues everywhere -- all of the time.
Until such time as their "ship the prototype in 21 days" methodologies are recognized for what they are, we will have researchers presenting at security conferences informing the marketplace about which development shops can and can't do security
I don't feel that 2-4 week sprints counter the need or capability of application security. It is the awareness, understanding, co-operation, agreement, and integration of application security into the application lifecycle management processes that must be accomplished. How about a hardening sprint before the prototype goes live?
Did you know that prototyping (according to McConnell 740) is the best way to find and fix bugs? Well, I think it is also the best way to find and fix security bugs. It just must be integrated.
But I have a lot of sympathy for the individual developers who would love to be more engaged but have employers that don't see the cost-benefit
Almost all of the app developers that I have met over the past 7 years who have an interest in appsec are now working in appsec. Why? Because it's exciting. Because they love it. A lot of them said that because of the problems/drama in the information security industry overall that they would leave (or take breaks from) appsec and return to doing software engineering/architecture full-time. Most of the appdevs who said that are now full entrenched in appsec again and promised to never return back.
However, I understand the issues with the drama in infosec. Breach notification laws are a joke. PCI DSS makes Visa more money and does absolutely nothing to protect cardholders or make their lives safer from credential or payment card data theft. HIPAA is now officially the largest running joke the regulatory world has ever known.
Jim Bird bolded a Computerworld article on "Hold vendors liable for buggy software, group says" in this blog post, specifically mentioning that this strategy would never work. I am not as convinced that it will not. Without legislation, companies will profit massively in the short-term (much to the expense and chagrin of consumers). However, in the long-term, legislation will favor many small companies and consumers and the eventual trend will be that software products mentioned in popular lawsuits will tend to get more application security rigor around them. Wait and see; I bet I'm right here.
It's either legislation or litigation, folks.
I agree completely with dre. Good concept "bug-fixer". Currently bug hunters are happy to show how stupid developers are, then let the same developers fix the bugs to then laugh again when they fix them incorrectly, they don't take any responsibility they just enjoy breaking the software and showing it in the face of developers and feeling smarter than them. But at the same time they never program, they could but they are afraid of making the same mistakes and loosing their reputation of cool smarter than you hackers. Anyway we have to accept that many bug hunters are brilliant, 10x more creative than an average developer. But what the industry needs is a merge between both. OWASP wont solve this problem exactly for the reasons that dre pointed out. Look at the software produced by OWASP is as buggy as any other software or even more, they are good at organizing events and promoting their organization but I haven't seen any good result from them. A secure implementation of common libraries? no, just mediocre broken code. Good guidelines? no, very incomplete. For me OWASP is good just for a few guys behind it and consulting companies trying to sell services. Hyping xxjacking attacks that a 12 year old can find and exploit better.
It’s not passive attempts to “infect” development managers with vague ideas about being "Rugged" that are supposed to somehow change how developers build software.
The solution is to make secure software development a problem for software developers, a problem that we need to solve ourselves. Engage leaders in the wider software development community
All the energy spent being critical of the fledgling Rugged Software project would be better spent participating and making it better. You make a point that the development team and its leaders need to be the target of the influence. This is in enthusiastic agreement with the Rugged philosophy. Can't we all just get along?
The problem I frequently see and hear is that security testing is a phase that is outsourced to "experts", is expensive, and is thus left to the end of the delivery process. I often see emails from people bemoaning the fact that designing and testing for security can't be built in to the delivery process in the same way as capacity and availability can (although even this is a relatively recent phenomenon).
Until there are tools and organizations that can make security testing a mainly automated process that can be run on demand (with experts creating and managing the strategy and monitoring its effectiveness, of course), it's not going to become integrated into the delivery process.
Yes, that means developers working to create the tools and practices that allow this. But it also means security experts working with developers to create these tools and processes.
My perception - which is admittedly not based on a lot of experience - is that many security experts treat their approach as magic, special secret sauce that only they can provide, at great expense.
I would love to be proved wrong - and I would love to learn more about how security can be baked in to the delivery process.
Thanks,
Jez.
Some software developers do "get it". Most do not. Those that do understand the need and the principles involved usually do not have the time to spend on it that is required.
I taught input validation to everyone on the development team a couple years ago. Yet, every single time I do a code review, I find missing input validation.
The same developers (who are fairly competent) make similar mistakes over and over again. The security mindset just does not seem to be present - no matter how many times they end up rewriting something.
I am not sure how to overcome that. It largely seems like many developers just don't care. They will bolt security on after they are done.
I have reviewed code from other organizations where they incorporate security in the software development life cycle and found similar issues.
I think the security aspects of software development need to be more integrated into the VERY early stages of learning to program. Seems almost like you can't teach an old dog new tricks. So, once the developers have learned to develop, they can't seem to change how they develop.
Hey Jim,
I'm not sure it's the developers. In my experience anyway, it's the business that doesn't get it. And if they don't 'get it', then they're naturally not willing to invest in it because they don't see the value.
Hey Andy,
It's good to hear from you. I don't see the lack of focus on software security as a failure of the business or the customer so much. The customer asks us as professionals to build them a system: they expect us to know how to do our jobs, to write software that works and is reliable, and they expect us to follow good practices. And this includes understanding and following secure software development. I haven't worked for a customer who has told me not to build a system properly: they expect me to. Of course, they may not be willing to pay enough for it, and as long as it is clear to everyone involved what compromises will need to be made to save costs, what the risks are, then that is a business issue that can be managed. Not every system needs to be secure.
The way I see it, the bigger problem is with the people who manage software development: CTOs, development managers, project managers, Scrum Masters, lead developers and architects, and the consultants who help us understand how to build software. Most of these people don't understand what is involved in building secure and reliable software, and it is not a priority for them. They are the ones who decide or influence where to spend time and money, what tools to use, what training people are going to get.
Building software to be secure needs to be as important and as second nature as building it faster and with less risk. And I don't think that this will happen without the involvement of the leaders of the development community, without them understanding the problems and including software security in the way that we build software.
Hey Jim,
I think that's fair. A couple of points to add if I may.
Firstly, the security requirements in certain domains is clearly more critical than others. Obviously in your domain, it is as critical if not more than any functional requirement. I've spend some time in the upstream O&G space, and I'll tell you, it's far from that. I a consumer, I don't want to pay for the Caddy if the Pento will do. I just need to understand what I'm buying.
Secondly, I distinguish the 'developer' role as someone who doesn't necessarily educate the customer. I do agree with your statement 'professionals who know how to do their jobs', I just don't think it's necessarily their job. If you invest in a system, you need to understand what you're investing in. Take some initiative as a customer to understand what security means to you. It's not different than any other kind of investment. Remember what Warren Buffet say's - "I don't invest in anything I don't understand". It's done well for him. :-)
In the end, security is about risk mitigation. I agree that we need to do a better job educating.
Quality Assurance makes sure the project will be completed based on the previously agreed specifications, standards and functionality required without defects and possible problems.
And for that the software developers have to take a help from the software testing tool.
-quiz software
Hi Jim,
I talk to a lot of developers about security. A few 'get it' but most know that they dont know enough.
They see penetration testing as a 'black art' that is only practised by l33t haxors.
I dont think theres any one solution to this, although decent training would be a good start!
However I also think theres a case for teaching basic pen test techniques to developers and functional testers. It wont be for everyone, but the devs I've taught have all said that it really opened their eyes.
Now for the blatant self promotion :)
I've just released a pen test tool explicitly aimed at developers - the Zed Attack Proxy (ZAP): http://code.google.com/p/zaproxy/ (its a fork of Paros) and also started a blog: http://pentest4devs.blogspot.com/
Not a great deal to see on there yet but any feedback on ZAP or the blog will be appreciated.
Psiinon
I think that we'll have our own conferences, which may look a lot like Cigital's.
Post a Comment