Software bugs

Dave Goldsmith has some experience trying to report a security vulnerability to a company that does not have a security-specific process:

I reply:

Can you give me some guidance on your response guidelines to security vulnerabilities? Is there a timeframe that you try and have vulnerabilities fixed by?

They reply:

Hi David,

We’re always looking for new ideas and fixes to roll out in future updates but as as rule we don’t comment on possibilities or timeframes.

His commenters discuss the lack of reporting guidelines for security flaws in websites. In a more recent post, he reveals that 37signals was the previously unnamed vendor, and writes:

Vulnerability reporting should not be handled in the same way that you manage feature requests.

[Via ZDNet, which discusses whether treating security flaws as defects is a more effective way to communicate the problems they cause.]

Issues to consider here are the usual with security bugs: vendor communication versus disclosure. In the case of security problems in end-user software the problem with disclosure is the race for users to patch their systems against attacks being launched on those systems, with consequences that vary per attacked system. When it’s a website with a security problem, disclosure has a different weight, as rather than leaving some users in the lurch while a fix is rushed out (or leaving them to fight for one), the vulnerability can largely be fully exploited immediately. There’s only one target. You may be writing off this vulnerability’s consequences all together in the hope of future responsiveness from the vendor.

Popularity: 64% [?]

The bMighty Blog describes how a bug in Firefox which was present in the 2.x series and was in the 3.0 released version has created some controversy: did the security researcher know about the bug earlier and choose the timing of the announcement purely for the publicity?

In other words, this is an issue that could have surfaced at any point in the past two or three years — but did not. Instead, news of the bug surfaced just five hours into biggest, and perhaps most important, software launch in Mozilla’s history.

The article goes on to describe various possible motivations for timing a security announcement with a highly publicised major release: it might benefit the researcher, but it also hurts Mozilla a lot. This is hostile bug reporting at its finest, if it wasn’t a coincidence.

Popularity: 51% [?]

Marie Hagman writes a Bug Reporting Best Practices guide with a focus on reporting bugs that are likely to be fixed:

Bugs that less likely to be fixed:

It would be great to fix every bug in the product, but it’s also great to ship J. In prioritizing which issues to fix, here are the some factors that cause certain bugs to miss the triage bar. Bugs that are least likely to be addressed:

• Are not reproducible or very hard to reproduce. This may be because the problem occurs intermittently or there are not enough details in the bug report
• Have strange and complex steps to introduce failure
• Have no perceived customer impact
• Are edge cases
• Risk introducing greater instability through the bug fix then the bug itself causes. Especially late in the product cycle, the bar for these types of bugs is very high because the bugs that may be introduced by the fix we will likely not have time to fix.

Popularity: 54% [?]

Ubuntu is having a bug jam between 8th and 10th August 2008. As The Fridge says:

So, what is the Ubuntu Global Bug Jam? Put simply, it is a world-wide online and face-to-face event to get people together to fix Ubuntu bugs – we want to get as many people online fixing bugs, having a great time doing so, and putting their brick in the wall for free software. This is not only a great opportunity to really help Ubuntu, but to also get together with other Ubuntu fans to make a difference together, either via your LoCo team, your LUG, other free software group, or just getting people together in your house/apartment to fix bugs and have a great time.

Popularity: 54% [?]

Joey Hess suggests that using the word “stupid” or using imperatives like “fix it” in bug reports is not a good way to get your bugs looked at. However Gunnar Wolf gives a recent example of an aggressive bug report that made him care.

Popularity: 82% [?]

Jeff Waugh wants to report bad search results to Google, but where? His commenters have some good suggestions:

Scott Says:
June 17th, 2007 at 18:49

Perhaps you have to be logged in to get this form?

["Dissatisfied with your search results?" form]

Willem Says:
June 17th, 2007 at 20:48

Or bether yet (with some googling )

["Inappropriate or irrelevant search results" form]

Popularity: 95% [?]

Jacub Steiner has some tips for bug reporters who are making a feature request (ie asking for new functionality, rather than reporting broken existing functionality):

If you take the time to write a functional specification, you are very likely going to motivate someone to get it implemented. You take the burden of designing the behaviour and let the developer worry about implementation details, data structures, etc. You invest your time and effort just like the hacker would have. Here’s a few suggestions.

• Define the functionality by creating a few test cases on what problems you are trying to solve.
• Be very specific and go through the process step by step. It will make you find problems with your design. If you were to just suggest functionality and don’t work out the details, you will, in the best scenario, waste developer’s time discussing the flaws you would have found yourself.
• It is better to find a bad design while writing the spec than after it has been implemented. Convincing a developer to redo something is close to impossible when he already invested a lot of time in it.
• Be visual. Create mockups of the interface as you go step by step through the process of solving the task. Many times I have thought my descriptions are clear, when they weren’t. Images tend to be less vague (Well this may simply be because my writting sucks, too).

Popularity: 67% [?]

Havoc Pennington describes the coordinated release process for security bugs:

The simplest thing is to quietly notify any of the major Linux or BSD distributions and let them take it from there… Once you notify someone, wait to hear back. The upstream maintainer would normally announce the vulnerability and commit patches to CVS at the same coordinated time that vendors post packages. If you patch in CVS before anyone is ready with packages, your users are vulnerable during the gap (and generally unhappy about it). Worse, by committing a patch to CVS you’re doing something that a black hat could notice, but most sysadmins will not notice.

Popularity: 100% [?]

Bug reporting is one way into the Free Software world: it’s one step up the pyramid of project contribution. Posts in this category are aimed at helping you make good bug reports, or finding bug projects in need of help.

Popularity: 35% [?]

The Open Source Vulnerability Database is an independent vulnerability database created by the security community. It is available to individuals and corporations without charge.

They are currently looking for security professionals to help maintain the database.

(Found through Raven.)

Popularity: 83% [?]

Reporting a program’s vulnerability to an attack of some kind is a bit different to reporting normal bugs, because public exposure of the vulnerability may cause it to be exploited before the developers can develop a fix. Hence, normal bug reporting procedures, such as making an entry in a project’s public bug database, may not be the best approach. In this case, you may want to approach the vendors or developers privately, even if it goes against their bug reporting guidelines.

CERT’s policy (via SLUG) gives vendors at least 45 days to address a security issue before advising the public. Their Vulnerabilities, Incidents, & Fixes FAQ gives some guidelines for people who have discovered vulnerabilities.

The Organisation for Internet Safety has also issued a document called "Guidelines for Security Vulnerability Reporting and Response Process – V1.0" [780kb PDF file].

Popularity: 58% [?]

Isaac Jones has an article explaining how bug fixing is a good path into free software (and a software development career).

Via Debian weekly news.

Popularity: 50% [?]

The Eclipse project has a set of bug writing guidelines, including a great deal of detail about what should go in bug reports. It also gives some motivation for writing good bug reports.

Popularity: 44% [?]

Eric S Raymond and Rick Moen have a HOWTO on asking questions of technical groups the smart way. See in particular the section about not assuming a particular behaviour is a bug.

Popularity: 46% [?]

Good technical questions have many similarities with good bug reports: see in particular The Cardinal Rule of Reporting Technical Problems:

Never, never, never, never, never say `doesn’t work’.

Never.

Proper Prophylaxis:

Just say “I wanted X, but it does Y. How do I get X?”

Popularity: 51% [?]

Joel Spolsky illustrates painless bug tracking by example, advocating using a bug database and advising how to make people use it. He has a second article on the need for a good QA team.

Popularity: 59% [?]

Callum McKenzie notes that bug opening rates rates equal bug fixing rates in gnome-games, but Luis Villa’s experience is that that’s how bug statistics work in Free Software.

Popularity: 45% [?]

The GNOME project has a Desktop Integration Bounty Hunt.

Popularity: 57% [?]

Netscape have a security bug bounty page.

Popularity: 57% [?]

The Debian QA team hunt down and fix bugs in Debian packages, among other things. Most activity seems to be coordinated via their mailing list.

Popularity: 53% [?]