Tuesday, December 06, 2005

First consumer of Cheesecake spotted in the wild

That would be Will Guaraldi, the main developer of PyBlosxom. In a recent blog post, he talks about using the Cheesecake metrics as an indicator of the 'kwalitee' of his software. I'm glad to see that somebody is actually using this stuff, it motivates me to keep working on it. I also welcome comments such as the ones made by PJE:

I wouldn't take the Cheesecake metrics too seriously at this point; there are too many oddly-biased measurements in there. For example, to score all the points for documentation, you'd have to include both a News and a Changelog, as well as a FAQ, Announce, and a Thanks - to name just a few. Including that many files for anything but a huge project seems counterproductive. PyLint is also incredibly picky by default about things that really don't matter, and you're also being penalized for not including ez_setup.py even though you're not using setuptools. (On the flip side, you're given 10 points free for having PKG-INFO, which the distutils automatically generates...)

The Cheesecake metrics would be more meaningful if they were split out into at least 3 separate ratings for installability, documentation, and code quality, and the documentation rating(s) didn't implicitly encourage having lots of redundant files. In effect, you should probably consider the metrics more as a "list of things that some people have in their packages", and then pay attention to only the ones you actually care to have.

It's true that currently Cheesecake puts somewhat too much emphasis on the different files it expects. I tried to find a set of files that I've seen commonly used in Python projects and other open source projects. Also, I tried to include directories that I think should be included in any Python package. But there should be more emphasis on things such as unit test coverage and other metrics (see this page for the list of things I'm currently contemplating, and BTW it's a Wiki, so feel free to add more stuff).

The 3 separate ratings that PJE mentions make a lot of sense. If people have more such ideas/suggestions/comments, please send them to me directly (grig at gheorghiu dot net) or leave a comment here. I can only hope that more people will follow Will's example and start feeding back some of the Cheesecake metrics into their projects.


ogrisel said...

Yet another consumer: the cheesecake index for funkload 1.4.0 is 460/560 ~82% as mentioned in the release notes.

Funkload is a functional and load testing utility for your web app.

dhan said...

I love cheese cake very much