10 things you can do as a developer to make your app secure: #7 Logging and Intrusion Detection

This is part 7 of a series of posts on the OWASP Top 10 Proactive Development Controls: 10 things you can do as a developer to make your app secure.

Logging and Intrusion Detection

Logging is a straightforward part of any system. But logging is important for more than troubleshooting and debugging. It is also critical for activity auditing, intrusion detection (telling ops when the system is being hacked) and forensics (figuring out what happened after the system was hacked). You should take all of this into account in your logging strategy.

What to log, what to log… and what not to log

Make sure that you are always logging when, who, where and what: timestamps (you will need to take care of syncing timestamps across systems and devices or be prepared to account for differences in time zones, accuracy and resolution), user id, source IP and other address information, and event details.

To make correlation and analysis easier, follow a common logging approach throughout the application and across systems where possible. Use an extensible logging framework like SLF4J with Logback or Apache Log4j/Log4j2.

Compliance regulations like PCI DSS may dictate what information you need to record and when and how, who gets access to the logs, and how long you need to keep this information. You may also need to prove that audit logs and other security logs are complete and have not been tampered with (using a HMAC for example), and ensure that these logs are always archived. For these reasons, it may be better to separate out operations and debugging logs from transaction audit trails and security event logs.

There is data that you must log (complete sequential history of specific events to meet compliance or legal requirements). Data that you must not log (PII or credit card data or opt-out/do-not-track data or intercepted communications). And other data you should not log (authentication information and other personal data).

And watch out for Log Forging attacks where bad guys inject delimiters like extra CRLF sequences into text fields which they know will be logged in order to try to cover their tracks, or inject Javascript into data which will trigger an XSS attack when the log entry is displayed in a browser-based log viewer. Like other injection attacks, protect the system by encoding user data before writing it to the log.

Review code for correct logging practices and test the logging code to make sure that it works. OWASP’s Logging Cheat Sheet provides more guidelines on how to do logging right, and what to watch out for.

AppSensor – Intrusion Detection

Another OWASP project, the OWASP AppSensor explains how to build on application logging to implement application-level intrusion detection. AppSensor outlines common detection points in an application, places that you should add checks to alert you that your system is being attacked. For example, if a server-side edit catches bad data that should already have been edited at the client, or catches a change to a non-editable field, then you either have some kind of coding bug or (more likely) somebody has bypassed client-side validation and is attacking your app. Don’t just log this case and return an error: throw an alert or take some kind of action to protect the system from being attacked like disconnecting the session.

You could also check for known attack signatures:.Nick Galbreath (formerly at Etsy and now at startup Signal Sciences) has done some innovative work on detecting SQL injection and HTML injection attacks by mining logs to find common fingerprints and feeding this back into filters to detect when attacks are in progress and potentially block them.

In the next 3 posts, we’ll step back from specific problems, and look at the larger issues of secure architecture, design and requirements.

10 things you can do as a developer to make your app secure: #6 Protect Data and Privacy

This is part 6 of a series of posts on the OWASP Top 10 Proactive Development Controls.

Regulations – and good business practices – demand that you protect private and confidential customer and employee information such as PII and financial data, as well as critical information about the system itself: system configuration data and especially secrets. Exposing sensitive information is a serious and common problem, 6th place on the OWASP Top 10 risk list.

Protecting data and privacy is mostly about encryption: encrypting data in transit, at rest, and during processing.

Encrypting Data in Transit – Using TLS properly

For web apps and mobile apps, encrypting data in transit means using SSL/TLS.

Using SSL isn't hard. Making sure that it is setup and used correctly takes more work. OWASP’s Transport Layer Protection Cheat Sheet explains how SSL and TLS work and rules that you should follow when using them.

Ivan Ristic at Qualys SSL Labs provides a lot of useful information and free tools to explain how to setup SSL correctly and to test the strength of your website’s SSL setup.

And OWASP has another Cheat Sheet on Certificate Pinning that focuses on how you can prevent man-in-the-middle attacks when using SSL/TLS.

Encrypting Data at Rest

The first rule of crypto is: Never try to write your own encryption algorithm. The other rules and guidelines for encrypting data correctly are explained in another Cheat Sheet from OWASP which covers the different crypto algorithms that you should use and when, and the steps that you need to follow to use them.

Even if you use a standard crypto algorithm, properly setting up and managing keys and other steps can still be hard to do right. Libraries like Google KeyCzar or Jasypt will take care of these details for you.

Extra care needs to be taken with safely storing (salting and hashing) passwords – something that I already touched on an earlier post in this series on Authentication. The OWASP Password Storage Cheat Sheet walks you through how to do this.

Implement Protection in Process

The last problem to look out for is exposing sensitive data during processing. Be careful not to store this data unencrypted in temporary files and don’t include it in logs. You may even have to watch out when storing it in memory.

In the next post we'll go through security and logging.

10 things you can do as a developer to make your app secure: #5 Authentication Controls

This is part #5 of a series of posts on the OWASP Top 10 Proactive Development Controls:

In the previous post, we covered how important it is to think about Access Control and Authorization early in design, and common mistakes to avoid in implementing access controls. Now, on to Authentication, which is about proving the identity of a user, and then managing and protecting the user’s identity as they use the system.

Don't try to build this all yourself

Building your own bullet-proof authentication and session management scheme is not easy. There are lots of places to make mistakes, which is why “Broken Authentication and Session Management” is #2 on the OWASP Top 10 list of serious application security problems. If your application framework doesn't take care of this properly for you, then look at a library like Apache Shiro or OWASP’s ESAPI which provide functions for authentication and secure session management.

Users and Passwords

One of the most common authentication methods is to request the user to enter an id and password. Like other fundamentals in application security this is simple in concept, but there are still places where you can make mistakes which the bad guys can and will take advantage of.

First, carefully consider rules for user-ids and passwords, including password length and complexity.

If you are using an email address as the user–id, be careful to keep this safe: bad guys may be interested in harvesting email addresses for other purposes.

OWASP’s ESAPI has methods to generateStrongPassword and verifyPaqsswordStrength which appliy a set of checks that you could use to come up with secure passwords. Commonly-followed rules for password complexity are proving to be less useful in protecting a user’s identity than letting users take advantage of a password manager or entering long password strings. A long password phrase or random data from a password manager is much harder to crack than something like

Password1!

which is likely to pass most application password strength checks.

Password recovery is another important function that you need to be careful with – otherwise attackers can easily and quickly steal user information and hijack a user’s identity. OWASP’s Forgot Password Cheat Sheet can help you design a secure recovery function, covering things like choosing and using good User Security Questions, properly verifying the answer to these questions, and using a side channel to send a reset password token.

Storing passwords securely is another critically important step. It’s not enough just to salt and hash passwords any more. OWASP’s Password Storage Cheat Sheet explains what you need to do and what algorithms to use.

Session Management

Once you establish a user’s identity, you need to associate it with a unique session id to all of the actions that a user performs after they log-in. The user’s session id has to be carefully managed and protected: attackers can impersonate the user by guessing or stealing the session id. OWASP’s Session Management Cheat Sheet explains how to properly set up a session, how to manage the different stages in a session id life cycle, and common attacks and defences on session management.

For more information on Authentication and how to do it right, go to OWASP's Authentication Cheat Sheet.

We’re half way through the list. On to #6: Protecting Data and Privacy.

10 things you can do to make your app secure: #4 Access Control

This is #4 in a series on the OWASP Top 10 Proactive Controls: 10 things that developers can do to make sure that their app is secure.

Access Control aka Authorization, deciding who needs what access to which data and to which features, and how these rules will be enforced, needs to be carefully thought through up front in design. It’s difficult to retrofit access control later without making mistakes. Come up with a pattern early, and make sure that it is applied consistently. And make sure to follow these simple rules:

Deny by Default

In many apps, the default behaviour is to allow access to features and to data or other resources unless an access control check is added, and the check fails. Take a few seconds and think about what could go wrong with this approach. If it’s not obvious, go to OWASP’s Top 10 list of the most serious application vulnerabilities #7: Missing Function-Level Access Control. Then make sure to only permit access to a function if an authorization check passes.

What’s your Access Control Policy anyway?

Access checks – even checks that are done properly, using a positive access approach, are often sprinkled throughout application code, looking something like this:

if (user.isManager() || user.isAdministrator() || user.isEditor() || user.isUser()) { //execute action }

The problem with this approach is that it’s really hard to review your access control rules and make sure that they are correct, and it’s hard to make changes because you can’t be sure that you found all of the checks and changed them correctly.

Instead of embedding access control rules based on the user-id or role inside application logic, centralize access control rules in a data-driven authorization service which maps users against roles or other authorization schemes, and provide a simple API to this service that the application code can call. Much easier to audit, much more extensible and maintainable.

If this isn’t already available in the application framework that you are using, look for a good security library to do the job. Apache Shiro offers an easy and flexible access control framework which you can use to implement these ideas. OWASP’s ESAPI also has a framework to enforce fine-grained access control rules at function, service, URL, data, and file levels.

Don’t trust - verify

Back again to the issue of trusting data. Never use client-side data or other untrusted data in access control decisions. Only use trusted server-side data.

For more on Access Control patterns and anti-patterns and common problems in implementing Access Controls properly, please read OWASP’s Access Control Cheat Sheet.

Access Control is closely tied to Authentication – in fact, some people mix these ideas up entirely. So let’s look at key issues in implementing Authentication next.

10 things you can do to make your app secure: #3 Validate Input

This is part #3 of a series of posts on the OWASP Top 10 Proactive Development Controls.

Your first line of defence against attacks should always be to check all data from untrusted sources. Input validation is fundamental to application security, and a basic part of good defensive programming.

This is simple, and obvious – and often done wrong.

Don't Rely on Client-Side Validation

One common mistake is relying on client-side validation to catch problems. Client-side validation is useful in providing immediate feedback at the UI. But it won’t protect your system, especially in web apps. If an attacker can find a way to insert themselves in between the browser and your app, which can be done using a proxy, they can play with any of the data, including header fields and other hidden data, after local client-side editing has already been done. Data from a client, especially a client outside of your network, should never be trusted.

Another common mistake is relying on negative checking or “black list” validation to try to catch known bad data. This is a weak technique: you can’t be sure that you will catch all of the bad things or combinations, and negative checks can be subverted through (double) encoding and other evasion tricks.

White List Validation

OWASP recommends that you always use positive restrictions – white list validation – where you define the acceptable type and size, and all allowable values using regular expressions (regex). An example from the Proactive Controls: the regex for a password field could be:

^(?=.*[a-z])(?=.*[A-Z]) (?=.*\d) (?=.*[@#$%]).{10,64}$

This regular expression ensures that a password is 10 to 64 characters in length and includes a uppercase letter, a lowercase letter, a number and a special character (one or more uses of @, #, $, or %).

White list validation is trivial for enumerated data: days of the week, months of the year … anything that you can fit into a defined list. For other common data types (such as dates and time zones, numeric types, currencies, email addresses, IP addresses, URLs and domain names, and credit card numbers), you could use the Apache Commons Validator library.

Validation of free-format text fields (names, comments) is more difficult, and especially text fields that can – or need to – contain special characters, especially markup:

If your application handles markup -- untrusted input that is supposed to contain HTML -- it can be very difficult to validate. Encoding is also difficult, since it would break all the tags that are supposed to be in the input. Therefore, you need a library that can parse and clean HTML formatted text such as the OWASP Java HTML Sanitizer. A regular expression is not the right tool to parse and sanitize untrusted HTML.

You can test how solid your input validation is with different tools, such as fuzzers or static analysis taint checking (using tools that run through execution paths in the code and identify when you are referencing data that has not been validated) and through pen testing and manual exploratory testing.

Next: Access Control. How to do it, and how not to do it.

10 things you can do to make your app secure: #2 Encoding Data

This is part #2 of a series on the OWASP Top 10 Proactive Controls, the 10 things you can do as a developer to make your application secure. In the previous post, I explained why Parameterized Database Queries are so important in protecting applications from SQL injection, one of the most common and dangerous attacks.

SQL injection is only one type of injection attack. Stopping SQL injection is easy. Stopping other kinds of injection attacks – LDAP injection, XML injection or XPath injection, OS Command injection, and especially Javascript injection (aka Cross Site Scripting} – takes a lot more work. And stopping NoSQL injection – SSJS (Server-Side Javascript) injection and Schema Injection attacks against NoSQL databases – is something that we’re still learning how to do.

Stopping Injection Attacks

The solution to injection attacks is simple in concept: if you can’t clearly separate code from data (which is what you do to prevent SQL injection using a parameterized API), you have to make the data safe before handing it off to an external interpreter (such as an XML parser or an OS command shell or a browser).

You can – and should – try to do this by editing the data on input: rejecting any data that isn't considered safe. But there are limits to how many problems you can catch in input validation, especially if you need to accept and allow free-format text.

So to be safe you have to output encode or escape data before handing it to the interpreter, so that the interpreter will not recognize any executable statements in the data.

The devil is in the details: you need to understand the encoding or escaping rules for each interpreter, and you need to apply the encoding rules correctly in specific contexts (and make sure that you don’t encode data more than once). Browsers make this especially difficult, forcing you to know how and when to encode data properly in different HTML, Javascript, XML and CSS contexts. It’s not enough just to HtmlEncode data:

HTML entity encoding is okay for untrusted data that you put in the body of the HTML document, such as inside a <div> tag. It even sort of works for untrusted data that goes into attributes, particularly if you're religious about using quotes around your attributes. But HTML entity encoding doesn't work if you're putting untrusted data inside a <script> tag anywhere, or an event handler attribute like onmouseover, or inside CSS, or in a URL. So even if you use an HTML entity encoding method everywhere, you are still most likely vulnerable to XSS. You MUST use the escape syntax for the part of the HTML document you're putting untrusted data into.

OWASP XSS (Cross Site Scripting) Prevention Cheat Sheet

There are tools to help you do this: OWASP’s ESAPI encoders (for CSS escaping, HTMLEntity encoding, URL encoding and Javascript escaping, as well as Unix escaping, Windows encoding, VBScript escaping, LDAP encoding, and XML and XMLAttribute and XPath encoding), the OWASP Java Encoder for XSS protection, and Microsoft’s open source Anti-XSS Library for .NET (encoder functions for XSS protection have been taken from this library, improved on and implemented in the .NET 4.5 AntiXssEncoder class).

But even with these tools, it can be difficult to get everything right, which is why injection, especially XSS, is one of the most common major security vulnerabilities in web applications. (To learn more about how XSS works and how to find it in an app, try playing the Google’s XSS game).

CSP – a different approach to stop XSS

A completely different – and simpler – approach to protecting your web app against XSS, especially if you are building a new web app from scratch, is by establishing strong Content Security Policy rules to restrict valid sources for scripts and other resources (connections, images, media, frames…), and to block inline scripting (you will need to structure your Javascript accordingly).

Content-Security-Policy: script-src ‘self’

This HTTP header is all that you need. Of course, this comes with caveats: there are a few edge cases that may not be handled by Content Security Policy restrictions, the Content-Security-Policy header is only implemented in newer browsers (although it is backwards compatible), and you’re depending on the browsers to implement the rules correctly so your app will still be vulnerable to browser bugs.

Watch Jim Manico, a true appsec rock star, explain the Top 10 Proactive Controls. If there is anything that you disagree with, or think is missing in this Top 10 list, please take the time to comment.

Next: input validation. Simple. Obvious. And often done wrong.

Unsung Hero post at WhiteHat Security

I was invited to write something up on appsec from a development manager's perspective as part of WhiteHat Security's "Unsung Hero Program". Here is my post "Easy things are often the hardest to get right".

10 things you can do to make your app secure: #1 Parameterize Database Queries

OWASP’s Top 10 Risk list for web applications is a widely recognized tool for understanding, describing and assessing major application security risks. It is used to categorize problems found by security testing tools, to explain appsec issues in secure software development training, and it is burned into compliance frameworks like PCI DSS.

The OWASP Top 10 for web apps, and the Top 10 risk list for mobile apps, are written by security specialists for other security specialists, pen testers and compliance auditors. They are useful in understanding what is wrong or what could be wrong with an app, but they don’t help developers understand what they need to do to build secure software.

Now OWASP has a Top 10 list written for developers: 10 things that developers can and should do to build secure online apps. This list of “Proactive Controls” covers security issues in requirements, architecture and design, as well as code-level concerns. It provides a checklist to follow when developing a system, pointing to detailed guidance in each area. All available free online.

Let’s start with #1 on the list, the simplest, but one of the most important things that you can do to secure your application: Parameterize Database Queries.

#1 Parameterize Database Queries

One of the most dangerous and most common attacks on online applications is SQL Injection: attackers inserting malicious SQL into a dynamic SQL statement. SQL injection vulnerabilities are easy for an attacker to find using free tools like SQL Map or SQL Ninja or one of the many other hacking tools or even through simple manual testing: try inserting a value like

1' or '1' = '1

into the user name and password or other text fields and see what happens. Once a SQL injection vulnerability is found, it is easy to exploit.

SQL injection is also one of the easiest problems to solve. You do this by making it clear to the SQL interpreter what parts of a SQL statement make up the command, and what parts are data, by parameterizing your database statements.

OWASP has a cheat sheet that explains how to parameterize queries in Java (using prepared statements or with Hibernate), and in .NET/C#, ASP.NET, Ruby, PHP, Coldfusion and Perl. None of this is hard to understand or hard to do properly. It’s not exciting. But it will stop some of the worst security attacks.

SQL injection is only one type of injection attack. Next we’ll look at how to protect against other kinds of injection attacks by Encoding Data – or you can watch Jim Manico explain encoding and the rest of the Top 10 Proactive Controls on YouTube.