Evaluating an Open Source Project's Security

April 10, 2012 By Tim OBrien

4 minute read time

Last week I wrote about how important it is to pay attention to the security of the OSS projects you depend on. This isn't just a one-time responsibility when you are trying to choose which component to depend on, this is an ongoing requirement. Even if you use the most secure OSS projects out there, if you don't pay attention to security updates, it is all for nothing. Staying secure requires constant vigilance.

In this post, I'm going to talk about OSS project security. Since we've been paying a lot of attention to OSS security, I wanted to lay out some guidelines for evaluating an OSS project's security. There's a wide range of approaches to security from OSS projects: on one end of the spectrum, a one-person OSS project on Github won't have a formal approach to security; on the other end of the spectrum, a project that is at the center of a billion dollar commercial ecosystem (like Apache httpd or Tomcat) will have a dedicated security team.

This post focuses on the secure end of the spectrum. Projects like Tomcat and Apache httpd that have dedicated security teams. Here are some of the baseline requirements for an OSS security teams. If you are maintain an open source project and you want to let the end-user know you take security seriously, you should consider starting a security team and following the guidelines in this post. If you are consuming open source, you should look for the following signs that this project has a mature approach to security in place:

  • A low-volume general, public announcements list - Every OSS project should have a "announcements" list which only contains release announcements or critical security announcements, no more, no less. Having a low-volume announcement list and being disciplined about what you send to this list increases the value of having an announcement list. The best case is a project that has a separate security announcement list. Users can define filters and flag these messages as important guides for security. The worst case is a project that has a noisy list that is a mixture of discussions and announcements. Keep the noise out of security announcements.
  • A private security list - If you run an open source project and someone notifies you of a vulnerability, you'll want a private place to discuss the potential impact and any proposed fixes. If your project is especially large (Linux, httpd, Tomcat) you want to limit this list to a few trusted members of the project.
  • One or more PGP Keys - This is a critical requirement, if someone identifies a security vulnerability in your software they need some assurance that the vulnerability report is being delivered to the right people. This is critical because (as a hypothetical) if I were going to compromise Tomcat, I might also attempt to compromise the accounts of the people who maintain Tomcat. Email is, in general, unencrypted over the public internet, you shouldn't put anything sensitive into a plaintext email that you wouldn't want broadcast to the entire world.
  • How to Report a Vulnerability - Every project has different requirements, but if someone is reporting a vulnerability in a project like Tomcat, the security team will likely want to know some basic common details: what JVM was being used? What version of Tomcat was vulnerable? Is there any exploit code that can test the vulnerability? Also important, who else is aware fo the vulnerability? How long have you known about the vulnerability? Are you aware of any successful attacks using this vulnerability?
  • A Description of the Security Process - This is especially important because Security teams are one part of an OSS project that is very opaque. While the public has visibility into almost all other aspects of a collaborative open source project, the security team is often working in secret to address identified vulnerabilities (possibly for months before they are generally known). To reduce friction between the transparency and the need for secrecy make sure that the public is aware of the security process. Consider retroactive transparency for discussions once the exploit has been published.

Here are a few examples of projects with mature security teams:

These projects have enough developers to have created a critical mass of both end-users and developers. All of these projects also have a strong commercial interest that can sustain continuous investment in a security team. As I've been surveying open source security, I've been impressed at the speed with which most open source projects react to security vulnerabilities. In general, projects that are attached to a respected forge (like Apache and Eclipse) are associated with a process and procedure for making sure that end-users have an interface to a security team. On the other hand, I see a very large list of projects that don't present any interface for security other than a public developer's list.

If we're going to start taking application security seriously, every open source project should take the time to satisfy these minimum standards for presenting a secure interface to end-users.

Tags: Nexus Repo Reel, Tomcat, Open Source, eclipse, Linux, Apache httpd

Written by Tim OBrien

Tim is a Software Architect with experience in all aspects of software development from project inception to developing scaleable production architectures for large-scale systems during critical, high-risk events such as Black Friday. He has helped many organizations ranging from small startups to Fortune 100 companies take a more strategic approach to adopting and evaluating technology and managing the risks associated with change.