This is a long post. TL;DR: do you approach secure software from the threat perspective or the hardening perspective, and is it really that different?
In the Framework Secure Software, one of the central activities is maintaining a list of threats, mitigations and verifications. Threats can come from different sources:
- forgotten or incorrect security requirements
- insecure use of technologies, i.e. architectural threats
- third party components
- coding mistakes
I would like to focus on the second source of threats: insecure use of technologies. Although all the above sources are important and even somewhat related, I think that mitigating architectural threats is where we can win the most.
Because it is technical in nature, developers (and architects) can best tackle this. Unfortunately, most of them have never learned about specific technical security threats or learned how to apply a security mindset to their work.
That does not mean that they are not good developers, it simply means that they have not learned enough about security. How this happened and what we should do about it, could be the topic of another post and I will not focus on that. Besides, because the rapid changes in technology, even if you had thorough security education, you would not know everything unless you constantly keep up-to-date.
Therefore, we have to assume that, in a typical situation, technologies will be used insecurely. For example, the technology itself may not be able to satisfy the system’s security requirements. Or, a technology used in the wrong way could lead to insecure situations. Knowing what can go wrong and such specific threats can be mitigated is key to countering such insecure usage. In the Framework Secure Software, the use of a threat library is advised to identify architectural threats.
In this post, I would like to describe some of my experiences with using such a threat library.
Threat libraries
In the first release of the text of the Framework Secure Software, the specifics of the threat libraries were not discussed, because this was not only still a work in progress, but also because the content of such threat libraries would be highly technology-dependent, and therefore more suited to be described in another document.
I was asked recently to check a software component for security, using the approach of the Framework Secure Software. Because this was an almost finished system, its design was already clear: a HTTP server sending and receiving SAML authentication messages. This choice of technologies defines what threats we can expect: web application threats and threats specific to SAML and its underlying technology: XML signatures and XML.
For the HTTP threats, I chose a subset of the ASVS verification requirements as part of the threat library. For the other technologies, I had to compile a custom threat library.
ASVS
The OWASP ASVS project is a list of “verification requirements”, but I would like to avoid using that term because what is in the list are neither what I would like to call “requirements”, nor “verifications”. I prefer to see it as a list of standard mitigations to counter certain threats. Quoting from the ASVS 3.0.1, it has two main goals:
- to help organizations develop and maintain secure applications
- to allow security service, security tools vendors, and consumers to align their requirements and offerings
That more or less fit my goals, but the relative completeness of the ASVS standard makes it an excellent basis for a threat library.
Threats for SAML, XML signatures, XML
For SAML, XML signatures and XML threats, there was no authoritative checklist. Therefore I had to compile my own list of threats, based on every known security problem that I could find about these technologies. The result was a list of about 20 threats.
Two approaches to implementing security
There are two ways to approach security (defined in the dictionary as “protection against threats”): look at the threats and implement protections (let’s call this the threat approach); or implement the protections without worrying whether a threat may be applicable (let’s call it the hardening approach).
The reasoning behind the hardening approach is that assessing whether a certain threat is applicable may take some time, while implementing the protection is relatively simple. Therefore, it makes things easier. We see this approach a lot in platform security, in so-called hardening guides. For example, instead of assessing whether the security vulnerability in libnacho.4.1.2 really is applicable, you simply apply the security patch, because that is easier. This approach makes a lot of sense.
That does not mean however that we should not use the threat approach. In cases where there is no standard universal mitigation, which is quite often the case in software, the hardening approach is not really useful. In software, the assumption that the mitigation costs less effort than the threat analysis may also not hold.
The hardening approach in ASVS
ASVS chooses the hardening approach. In my SAML threat library, I chose the threat approach. Let’s compare them. This is information from ASVS:
V2.9: Verify that the changing password functionality includes the old password, the new password, and a password confirmation.
The threat approach in the SAML threat library
In contrast, this is an example threat from my SAML threat library:
XMLSIG: Signature exclusion
Threat: A signature check can be bypassed by omitting the signature completely.
Situation: XML signatures
Suggested Mitigation: Check if the correct signature is present and give an error message if it is not.
Suggested Verification: code review/unit test
Remarks: It should be checked whether the signature is present and if not, processing should be rejected.
References:
[1] Somorovsky, J., Mayer, A., Schwenk, J., Kampmann, M., and Jensen, M. On Breaking SAML: Be Whoever You Want to Be. https://www.usenix.org/system/files/conference/usenixsecurity12/sec12-final91.pdf, retrieved 2016-12-07.
[2] WS-Attacks.org, XML Signature Exclusion. http://ws-attacks.org/XML_Signature_Exclusion, retrieved 2016-12-08.
Which approach is best?
So when should you use which approach? The hardening approach is more efficient in cases where the mitigation is standard and easy. It also takes away the need for thinking about and understanding the threat. The threat approach forces you to think about the threat and to fully understand it, which is needed in cases where the mitigation is not so straightforward. Using a mixture of both would be the ideal approach.
The efficiency of the hardening approach comes at a price. If the underlying assumption about the ease or applicability of the mitigation is not correct, you will need to switch to the threat aproach and need to understand why this mitigation is needed, i.e. against what threat it tries to protect. If it is difficult to get this information, your security suddenly becomes a lot harder. In the ASVS example, if you do not know that the underlying threat is that an attacker can modify your password because he can get control over your browser (via an unlocked screen, or via CSRF techniques), it is difficult to think of alternative mitigations. It is also more difficult to decide whether the underlying risk is acceptable or not.
In the example of the SAML threat, we see much more information. The suggested mitigation is still there, but it is explained why not implementing the mitigation can be a problem. The situation in which this threat is applicable is also given, so that you can determine whether or not you have to mitigate this threat (which is obvious here). More detailed information is given in the references, such as example attacks. In the remarks additional information is given that can point out important implementation details or less ideal, but perhaps acceptable, approaches. The suggested verification gives some advice on how you could best verify if your mitigation is implemented correctly.
You may argue that this does not look too different from some hardening guides out there. In fact, if you look at some of the hardening guides that are made available by the Center for Internet Security (the so-called benchmarks), you will see a structure that is not unlike the structure in my SAML threat library: a description that explains the mitigation, a rationale that explains what the underlying threat is, a section called audit that can be used to verify if the mitigation is correctly implemented, a remediation section explaining in detail how to execute the mitigation, and references containing a more detailed explanation.
In the Apache web server benchmarks, we see that the guides started using this structure in 2010. Earlier guides gave you the mitigation, but without explaining why this was needed. Back then, a security expert would simply know this. But today, security has grown so big that it is impossible for a security expert to remember all details. The rationale section is a great help.
What ASVS can learn from platform security hardening
This is where ASVS can be improved. Some of the “verification requirements” in ASVS puzzle me. Let’s take two examples:
5.18 Verify that client side validation is used as a second line of defense, in addition to server side validation.
Client side validation should never be relied on if the data is to be processed by the server, and because it is so easy to bypass, how could this ever be a realistic line of defense? This is where a rationale could really help, because perhaps there is a situation where client side validation can be used as a second line of defense and where it makes sense to implement it.
19.7 Verify that all application components are signed.
Even when read in its context of application configuration, it is not clear what the underlying threat is. Do components need to be signed so that they can be verified at run-time, to prevent against tampering of binary modules on a system (but in that case, how would you protect against tampering with the tampering detection?), or is it that the component signatures are verified after downloading when installing the software? Without a rationale, this is difficult to say.
Evolution of threat libraries
When technologies are relatively new, no hardening guides are available. The threat based approach makes more sense in that case. As standard mitigations are developed, either within an organization or as common knowledge, we can save a lot of effort by moving to a hardening approach. But moving to a hardening approach makes it more difficult to implement alternative, more efficient approaches, if the underlying threat is not clear.
This is where many organizations are missing the point. Simply giving a hardening guide to developers without them understanding what the underlying threats are will not work. Training can help here, but it would be more effective to include the rationale with each mitigation. If you start from a threat library, it can naturally evolve into a useful hardening guide, because it is effectively the same thing.
The motivation behind the use of a threat library has a great influence here. If compliance is deemed more important than security, it is more likely that the hardening approach is chosen over the threat approach. Be aware that this may lead to a false sense of security, for the reasons described earlier.
Using the threat libraries in practice
With the development team, I had the opportunity to use both ASVS and my SAML library and compare the approaches.
Many of the developers had little secure programming experience, so it was interesting to compare the two threat libraries. In both approaches, developers misinterpreted certain mitigations. For example, if the mitigation said “make sure that the message is signed”, I would get a response that the messages that the application generates are signed, while instead the threat applied to incoming messages for which the signature should be verified. It is very important to phrase your threats and mitigations correctly and not to assume that developers interpret the information in the way that was intended. In the description of the threat, it is better to be very explicit about what actions are taken by whom in the attack scenario.
As expected, for the ASVS threats it was sometimes needed to explain the underlying threat. For the SAML threats, this was also needed, despite the threat description that was included. I blame both the complexity of the threats and unclear threat descriptions for this.
The developers were given the threat libraries without any additional training, which I think would have helped to get better results. Despite that, we managed to show that all threats from the threat libraries were either mitigated, not applicable or accepted. All in all this took only a few short sessions.
The result: the developers can show that they have thought about security, I was able to effectively verify their mitigations in a source code review but perhaps the biggest win by far was that the developers now have a better understanding of security in their specific area of software development.