In the past few weeks, I've been working fast and furious at getting web2project to our v1.0 milestone.  As part of that effort, I track open issues, problematic modules, community feedback via the forums, death threats via all methods, and other related aspects.  And somewhere along the way, it led me to ask a seemingly simple question:

How do you measure an individual's contribution?

So you answer:

There's the obvious, lines of code (LoC). We all know the numerous problems with lines of code as a metric... to put it simply, more code doesn't always mean good work.  Duplication of code is the easiest way to get more of it but it breaks the DRY Principle.  In fact, in the past weeks, I've been tracking our LoC and watching it slowly drop as refactorings occur.

What about commit count? At first glance, this seems like a reasonable measure.  After all, a commit is supposed to be atomic and an entirely implemented feature.  The problem is that it can quickly fall down if you don't have a way/process to look at the quality of individual commits.

Even more interestingly, on Open Source projects, you often end up with a small set of people who serve as liasons to the community for evaluating and merging patches.  The commits they make might not even be code they wrote.

Good point, what about issues reported? While this doesn't get a codebase closer to launch, it does lend itself to assisting others in tracking down bugs.  Unfortunately, we get back to the quality issue.  If a bug doesn't have useful information on reproducing the condition, it's not useful and could actually distract from useful development.  Even worse, if the issues are a result of their own sloppy code, they could pad their own numbers.*

* Disclosure: While writing this post, I had to track down and resolve an issue I created in r299 of web2project.  Oops.

Ok, fine... how about issues closed? At first glance, this appears to work better but it leaves out a few considerations:  Unfortunately, issues can be closed for a variety of reasons.  They could be duplicates of others, incomplete/not useful in reproducing the conditions, or even just fixed.  And that's not even considering the complexity or severity of an issue. Let me put it this way... cleaning up the formatting of a file is nowhere near the complexity or the value of fixing a bug that causes data loss.

Alright then, you're starting to annoy me... how about community involvement? While this one generally doesn't result in more/better code like the ones above, it does some great things... and they can be done by team members or the general community alike.

• If more junior team members or the community handles the Tier-1 Tech Support, the more advanced team and community members can focus on more complex issues.  Forum post count, mailing list count, etc can skew perspectives here because "Frequently Asked Questions" are normally "Frequently Answered Questions" too.
• If team members can elicit more informative and useful bug reports or feature requests, the entire community benefits.
• When the team is actively engaged in the community providing encouragement, guidance, and feedback, the community responds in positive ways by sharing more code, ideas, and feedback.
• This is marketing 101 and with careful and deliberate focus, it is possible to groom friends, allies, and evangelists.  And every project - whether Open Source or proprietary - can benefit from this.

So... I'm stumped.  How do you measure and evaluate contributions of an individual?

Yeah... that's why I asked you.