Software Security Status Quo?

Veracode has released the company’s State of Software Security Report for 2012, the 5th in a series of annual reports that analyzes data collected from customers using Veracode’s cloud-based application security scanning services.

The Important Numbers

As Veracode’s data set continues to get bigger, with more customers and more apps getting scanned, the results get more interesting.

For Web apps, the state of vulnerabilities remains unchanged over the past 18 months:

• 1/3 of apps remain vulnerable to SQL Injection
• 2/3 of apps remain vulnerable to XSS, and at least half of all vulnerabilities found in scanning are XSS vulnerabilities

Over the last 3 years, there are no significant changes in the occurrence of different vulnerabilities that cannot be accounted for by changes and improvements in Veracode's technology or testing methods.

For mobile platforms (Android, iOS and Java ME), the most common vulnerabilities found are related to crypto: 64% of Android apps, 58% of iOS apps, and 47% of Java ME apps have crypto vulnerabilities. Outside of crypto, the vulnerability distributions for the different mobile platforms are quite different. It’s possible that these differences are due to fundamental strengths and weaknesses of each platform (different architectures, different APIs and default capabilities provided), but I think that it is still too early to draw meaningful conclusions from this data, as the size of the data set is still very small (although it continues to increase in size, from 1% of the total sample to 3% over the last 18 months).

But Security Vulnerabilities are Getting Fixed, Right?

Some interesting data on remediation, based on Veracode customers resubmitting the same code base for subsequent scans. Almost half of their customers resubmit all or almost all of their apps for re-scanning, regardless of how critical the app is considered to the customer’s business. What’s interesting is which vulnerabilities people chose to fix - bugs that are found in the first scan, but don’t show up later.

For Java, the bugs that are most often fixed are:

1. Untrusted search path
2. CRLF injection
3. Untrusted initialization
4. Session fixation
5. Dangerous function

So the first bugs to be fixed seem to be the easiest ones for developers to understand and take care of – low hanging fruit. Remediation decisions don’t seem to be based on risk, but on “let’s see what we can fix now and get the security guys off of our backs”. Security bugs are getting fixed, but it’s clear that SQL Injection and XSS bugs aren’t getting fixed fast enough, because there too many of these vulnerabilities to fix, and because many developers still don’t understand these problems well enough to fix them or prevent them in the first place. PHP developers are much more likely to remediate SQL injection vulnerabilities than Java or .NET developers, but it’s not clear why.

The Art and Science of Predictions

The report results were presented today in a webinar titled “We See the Future … and it’s Not Pretty”, which walked through the data and the predictions that Veracode drew from the data. While the findings seem sound, the predictions are less so: for example, that there will be higher turnover in security jobs (including CISO positions) because appsec programs are not proving effective, and security staff will give up – or get fired – as a result. I can’t see the thread that leads from the data to these conclusions. The authors should read (or re-read) The Signal and the Noise to understand what should go into a high-quality prediction, and what people should try to predict and what they shouldn't.