The asumptions that “bugs of a certain kind to be equally hard to find”, for example, is doubtful.
The text also (wrongly) assumes that:
- Testing only involves finding bugs
- There is a finite number of bugs
- There’s value in finding all bugs
- There is a clear definition of what a bug is
- The testing task is done when “all bugs are found”
- Testers are able (or try) to estimate the number of bugs when they test or finish testing
- Knowing all these numbers (if they were real) would have any benefit.

I can’t see any benefit of using the index a real project. If anything, it looks dangerous.
Do you have a real-life experience of using it in a project? What kind of conclusions were drawn?

—Michael B.

Finding the same bug under an assumption that each bug would be found randomly with something along the same probability gives an estimator for the total number of bugs by conditional probability.

When the index yields infinity it is because no reasonable estimate of the upper bound exists. When two testers find a disjoint set of bugs, you have no relative scale to fall back on. The the absence of overlap, you need some other form of information to estimate overall bug count.

For a relatively bug free strong system, two testers finding a unique bug each would mean the index is infinity.

I think the index can be a suggestion if the testers need to work together or not.

If the index is a low number for a couple of developers, they probably test the same way and don’t need to work together. If the index is high (or infinite) for two developers, they must sit and test together.

Ask the testers to work independently for a short time at the beginning of a project. Then compare bug lists and estimate the number of bugs left to find. Then work together as usual to find the rest of the bugs.

There’s also a lot of relevant literature in epidemiology (where they need to estimate population prevalences of diseases with imperfect tests and population samples) and in educational testing (where you want to evaluate both students and exam questions). I (and others) have been applying similar models to Mechanical Turk and other crowdsourced data annotation problems.

With enough testers, you can start estimating the difficulty of the items using something like an item-response model. You can also tease apart accuracy versus bias, though presumably with bugs the idea is that it’s all-or-nothing and there’s an absolute notion of what constitutes a bug.

EastwoodDC is right about it being the simple form of the Petersen capture-recapture estimator for population size. The form you give is a biased estimator, you can “unbias” it by using N = (E1+1)(E2+1)/(S+1) -1. Lots more on this type of estimator in Krebs, Ecological Methodology

I’ll add this to my lecture on capture-recapture so my bio students can see they have something in common with the comp sci kids.

This can apply to more than just programming; In doing statistical analysis, I’m always on the lookout for possible problems in the data. Given that I occasionally find problems that others have missed, I often worry about the ones that got away, and how many studies are skewed by bad data.

simulate=@(eCount,p1,p2,reps) …
lincoln(rand(eCount,reps)<p1,rand(eCount,reps)<p2);

simulate(100,0.4,0.3,100)

See what happens when you vary p1 and p2:
f=@(p,q) mean(simulate(100,p,q,1e3));
x=linspace(0,1,20);
[XX,YY]=meshgrid(x,x);
PP = arrayfun(f,XX,YY);
surf(XX,YY,PP)

The derivative with respect to each of the E’s is constant, so a deviation in E from the expected value will impose an corresponding percentage change in the value n from the true value. You can always hope that the two deviations in E will have opposite signs and still give a good n.

It looks like the variation in S will be more critical, just because it is a smaller value to start with. It’s still proportional for small changes.

It would be interesting to work up what the correction factor is as a function of the assumed correlation between the E’s. You could then determine how robust your n was under varying assumptions.