How Do I Reduce the [Security] Defects in my Software?

I’ve spent a little time here and there trying to find the “best” tool to do static analysis of some software written in a language other than C/C++/Java.  Foolishly, I figured it would be an easy task – find an authoritative site/wiki on such tools, skim for the one(s) referencing the language of interest, and browse to the download page.

Learn something new every day…

I started with my boss, who pointed out that our group (and most of Intel – at least those who’ve made their opinions known) has settled on one commercial tool, and that’s the answer we’re giving everyone who inquires about static analysis for security.  [I won’t name the product here – you don’t think I’d be that stupid do you?  That’d just be a huge invitation to the hacker community…]

Here at Intel, there’s an internal wiki where much of this “tribal knowledge” has been consolidated.  However, Intel’s development community is heavily invested in C, C++ and Java, so other languages don’t get a whole lot of attention (for good or ill, I can’t say…yet).  There’s a few pointers to public web pages, including one to the List of tools for static code analysis.  OK, so scanning that page should yield the results I’m after, right?  Wrong.

The deeper I look into this, the more complex the question becomes.  Am I interested in just security defect identification, or in identifying defects overall?  Am I only interested in static analysis approaches, or should I also consider dynamic analysis tools and (whatever else I’m inferring is beyond my comprehension, based on the wealth of ways this information can be categorized)?

And on a philosophical level, should I focus my customers’ attention on a single-tool approach, or give them a chinese menu from which to make their own selection?

I know that my team has Security deeply embedded in its genes, but I’m of the continuing philosophy that security isn’t an end unto itself; security is just one means to a greater set of ends.  Why make something more secure?  To make it more

  1. (a) available
  2. (b) reliable
  3. (c) trustworthy
  4. (d) all of the above
  5. (e) none of the above


Personally, my bias leads me to believe that (c) begets (b) begets (a).  However, despite six years of Microsoft indoctrination, “Trustworthy” still feels too much like a buzzword to me – so I’m inclined to choose (b).  If my software is more secure, I’m likely to rely on it more readily (and advocate that others rely on it more heavily) for my critical activities.

Now, if it’s really secure [whatever that means] but horribly unreliable for non-security reasons, that still means I’m unlikely to bother with that software for any great length of time (e.g. Google Desktop crashes rather frequently on my current PC, and while I’m fairly convinced it’s not hackers causing it to fail, but rather the “security software suite” I’m forced to run that’s intercepting some low-level driver or filter, I’ve still abandoned it – and am looking for the next-sexiest desktop search software).

Back to my original train of thought: All other things equal, I’d prefer to have developers using static analysis tools for overall defect reduction, rather than recommend security defect reduction tools.  This would make the developers happier too: instead of having to remember to “run that security tool” too, they’d just get the benefit of overall quality improvements (of which security should just be one component).

As for the static/dynamic/other analysis angle, I feel like I’m just learning to doggie-paddle in this area.  I’m not even qualified to discuss the difference between these realms yet.

However, on the “single tool” vs. “chinese menu” question, I have a very clear opinion: neither and both.  Really, what my customers are asking from me is to reduce the burden of research and analysis.  Ideally they’d like me to give them the answer they’d have come to themselves (given enough time), but it’s usually acceptable to provide a shorter, more organized list than they’d get out of Google.  I can usually be a big hero if I can:

  • do the research with their inquiry in mind (e.g. “What tool or approach should I use to identify and eliminate the greatest number of significant security issues in my code with the least amount of effort?”)
  • eliminate the tools that obviously aren’t intended for their language/job role
  • write up a prioritized list of tools for them to review/try, ordered by which is most likely to meet their needs

Am I the only one out here that thinks this way?  If not, I certainly haven’t found such a list from anyone who shares my point of view.  I’ve got the “single approved tool” on the one hand, and the “canonical lists of tools” on the other.  NIST has made a good start by categorizing tools based on semi-abstract goals for their use (e.g. “safer languages”, “dynamic analysis”), but I have to wade through multiple lists and descriptions to figure out if each tool analyses code in my sought-after language.

Is there anywhere I can go to find a list of all tools that address a specific development language, and sort them at least by age, number of versions, number of users or number of features?