The 3rd milestone for the Cheesecake/SoC project has been completed -- code name devon. This iteration had 3 stories:

1. Create functional tests that actually execute cheesecake_index script. Check that Cheesecake is:
  • properly cleaning up
  • leaving log file when package is broken and is removing it otherwise
  • computing score properly
  • handling its command line options properly

2. Write script that will automatically download and score all packages from PyPI.
  • Each package should have its score and complete Cheesecake output logged.
  • Gather time statistics for each package.
  • Make a summary after scoring all packages:
    • number of packages for which Cheesecake raised an exception
    • manually check first/last 10 packages and think about improving scoring techniques
3. Add support for egg packages
  • Refactor supported packages interface
  • Add support for installing eggs via setuptools easy_install
As far as story #2 is concerned, Michał and I discussed some modifications and tweaks we need to do to the scoring algorithms so that more packages get higher scores. Here are some ideas that we already implemented:
  • don't decrease installability score if a package is not hosted on PyPI (the package still needs to have a valid download link on its PyPI page);
  • split required files and directories into 3 categories: high, medium, and low importance, each category getting a score of 30, 20, and 10 points respectively;
  • here is the current classification, where Doc means the file can also have a 'txt' or 'html' extension, and OneOf means the score is given if any one of the files/directories in the specified list is found:
cheese_files = {
Doc('readme'): 30,
OneOf(Doc('license'), Doc('copying')): 30,
OneOf(Doc('announce'), Doc('changelog')): 20,
Doc('install'): 20,
Doc('authors'): 10,
Doc('faq'): 10,
Doc('news'): 10,
Doc('thanks'): 10,
Doc('todo'): 10,
cheese_dirs = {
OneOf('doc', 'docs'): 30,
OneOf('test', 'tests'): 30,
'demo': 10,
OneOf('example', 'examples'): 10,

We're getting ready to release cheesecake out in the wild pretty soon, I'd say in a couple of weeks -- so stay tuned!

We've also seen some activity on the cheesecake-users and cheesecake-dev mailing lists, and as always we encourage people interested in this project to send us feedback/suggestions/criticisms. We've been known to always take constructive criticism into account :-)


Read also Michał's post on devon.


Popular posts from this blog

Performance vs. load vs. stress testing

Dynamic DNS updates with nsupdate and BIND 9

Running Gatling load tests in Docker containers via Jenkins