Yesterday, Monty and I were fussing around with lcov and genhtml trying to generate code coverage analysis for Drizzle. After a few hours, I was finally able to get some good output, and I’ve published the results temporarily on my website.
We’re currently at 70.4% code coverage which is less-than-ideal, but at least we now have a baseline from which to improve. We’re all about making incremental improvements, and having statistics to tell us whether we’re going in the right direction is important. This is a good first step.
So, what exactly do these code coverage numbers mean?
OK, so for those readers not familiar with gcov or lcov, here is what these code coverage numbers actually mean… They represent the percentage of executable source lines which are executed during a run of Drizzle’s test suite. Basically, the percent gives us a rough idea of the percent of our code paths which are being verified by the test suite.
What we do is have GCov produce coverage data during compile for each source code file. We then run our test suite, and then have LCov collect (capture) the source code lines in each file was executed during the run of the test suite. We then call on the genhtml utility to produce pretty HTML pages from the “info files” generated from GCov.
Automating LCov Reports
The goal is to have LCov reports run for each revision in trunk, and have the output automatically rsynced up to the Drizzle.org server. I’m working with Mike Shadle from Intel and Monty on getting this done, along with Doxygen output, and hope to get the automated test coverage reports done by the end of the week.
Part of the automation is producing a text-based report which I can then store in a database and keep historical data about our progress in improving test coverage. Of course, those historical graphs will also be published. 🙂
How Can You Help Improve Drizzle’s Code Coverage?
Actually, this is a great starter task for new contributors to get involved in Drizzle development. The way to improve code coverage percentages is to analyze the blocks of code which are not covered in the following way:
- Determine if the code block is dead code
- If not dead code, come up with an addition to an existing or new test case which will ensure the code block is executed
- If it is dead code, remove the code from the tree after determining why the code is dead
Some more details on the above steps will come in another blog post later today… OK, gotta get back to work. Feel free to peruse the LCov HTML reports. There are some interesting things buried in there. 🙂