Prelude…
My day job is squarely in the Java world, and trust me, in many respects it is a wonderful world. The amount of high quality, free, open source tools surrounding the Java platform is enormous. For someone interested not only in development but in “software engineering”, there is a risk that you spend all your time exploring all the great tools instead of doing some productive work…
Now, in the past professionally, and recently as a hobby, I have been using Borland/Embarcadero tools ranging from Turbo Pascal for Windows back in 1993 (who remembers TPW 1.0, hands up?) up to Delphi 2010. The Delphi community and the range of free libraries and tools has always been good, if not great. Delphi has been a very “complete” environment for the traditional Windows client and I’d say many Delphi developers probably feel uncomfortable leaving their IDE to use another tool. This has been different for other languages like C++ or Java where weaker development environments have had to be compensated by a range of IDE external tools. As far as a development IDE, Delphi is still very competitive.
As software engineering practices have advanced, a lot of the tooling focus has moved out from the developers IDE to the Continuous Integration system. With the advent of good tooling to support CI, the discipline of software engineering has taken a huge step from the closets of University CS departments out to the world of enterprise software development. Many are the development teams who embrace agile practices (code word:scrum), TDD and metrics driven development. One (in my view) welcome aspect of this is that the responsibility for quality has to a very large degree moved to the development team from the (usually understaffed) QA function. The whole point of Continuous Integration is to stay focused on the goal of having a quality product. And today, honestly, I’d think that a development shop who doesn’t use a CI tool like Hudson is seriously behind the times.
Now as people start using Hudson to build Delphi projects I know that people (at least I do) desire the same level of quality introspection and visibility in to their Delphi projects as they have with their Java, .Net and C++ projects. If that doesn’t materialize, the viability of Delphi as a development tool will diminish.
So – what’s missing you might ask?
Here’s my list:
- An open source code coverage tool for Delphi
- Static code analysis tools to find bugs and enforce coding guidelines (yes I love CTRL-D in Delphi 2010!)
- Advanced mocking framework to facilitate unit testing
- Hudson plugin to parse and visualize Delphi compiler output properly
- Hudson plugin to parse and visualize DUnit output (or alternatively adopt DUnit to what JUnit outputs)
- Hudson plugin to keep track of code coverage for tests
I understand that Embarcadero has embraced Hudson as a CI tool themselves – why not create and release some Delphi specific Hudson plugins, Embarcadero?
Introducing Delphi Code Coverage
So with that way too long prelude (still reading?) – to the point of this article. I have just submitted a new open source project to Google code called Delphi Code Coverage and it is to my knowledge the only free code coverage tool for Delphi. I’ve had this tool lingering on my hard drive in an unfinished state for a year and decided to spend a few hours to polish it up well enough to release it to the world. The inspiration comes again from the Java world where I have been using Emma as my code coverage tool of choice for quite some time.
Right now the tool only does line coverage, i.e. it will keep track of what lines were executed and visualize that in an html report. So you could say that it is an embryo of a Code coverage tool compared to mature code coverage tools out there. With some luck, this is a tool that more people would be willing to invest some time in and help make it as awesome as the awesome Delphi community deserves. There are plenty of features to implement to match the state of the art in Code Coverage tooling…
How it works and what does it do
Delphi Code Coverage is a command line tool that will run your executable and using the accompanying detailed MAP file keep track of what source lines were executed.
Here is a sample ( I hope self-explanatory):
CodeCoverage -e TestApp.exe -m TestApp.map -u TestUnit AnotherUnit AThirdUnit
This command line will generate 4 html files, three for the units you supplied and one with a summary for all the supplied units.
More details on the usage is available on the Google code site.
That’s all folks for now. I hope you give the tool a spin and let me know what you think!
Chister
Whatever you just changed had it actually run with my application, so good work. Also good work on the XML output, I was about to raise an issue to suggest this.
By a complex project, I mean something like this, with code taken from several different locations, where I want to do coverage on both DataOverride and FetchAPerson
uses
…
TestConfig in ‘TestConfig.pas’,
TestFetchAPerson in ‘TestFetchAPerson.pas’,
FetchAPerson in ‘..\..\..\Transactions\FetchAPerson.pas’,
DataOverride in ‘..\..\..\Transactions\DataOverride.pas’,
…
Although obviouslu they are in the same directory so that is a bad example!
uses
…
TestConfig in ‘TestConfig.pas’,
TestFetchAPerson in ‘TestFetchAPerson.pas’,
FetchAPerson in ‘..\..\..\Transactions\FetchAPerson.pas’,
DataOverride in ‘..\..\..\Overrides\DataOverride.pas’,
…
Would be a better example
Christer,
Great idea, and just what I need to grow my plan for increased DUnit testing
How is this job at picking up complex projects, i.e. lots of units in different folders ?
Also, I know this is very early, but when I run this over one of my projects, the command line just freezes and there is no output, etc. Any ideas ?
Thanks
Mark H
Mark,
Thanks – I just published a new version with quite a few bug fixes that I encourage you to try.
As for picking up complex projects there is basically no support for that, but I would be more than happy to add support for what you need. Can you send me an example of your structure?
If you run in to a problem, please send me the debuglog.txt that is generated as that can give me some pointers to what went wrong.
/Christer
Interesting project! I’d love to see this integrated into DUnit’s test runner.
FYI: There are two third party static code analysis tools:
Pascal Analyzer:
http://www.peganza.com/products_pal.htm
Code Healer
http://www.socksoftware.com/codehealer.php
Delphi itself comes with support for Metrics and Audits – you need to enable “Model support” for that.
It is worth noting for those unfamiliar with the subject that Code Coverage is really a part of Unit Tests, although it can be used otherwise. You run the code coverage of the units while the unit tests are running. You want to see how much of your code is “exercised” by the unit tests.
In an ideal world 100% of your code should be exercised by your unit tests, but that usually doesn’t happen. Any code that is not exercised by a unit test may be dead code that represents a state your application never reaches:
Example:
if True = False then
WriteLn(‘This is dead code!’);
or it is code that you have not accounted for in your tests, and it may have a bug in it.
Without code coverage analysis you really don’t know if your tests are effective. Anyone can write a bunch of tests that always pass, but if they only exercise 1% of the code then they really aren’t doing much good (although that could be the 1% of your code that is the most likely to cause problems, so common sense on the part of the architect is worth more then some meaningless metric of 100% coverage).
So knowing what code is covered and what code is not makes your unit testing more effective. I actually had a scenario where some code was being tested with a property in one state, and then that property got changed and all of a sudden all the tests that passed were failing. Turned out it was just one line of code that previously had not been tested, and it just happened that it was either always one way, or the other in the testing.