Once you've installed the Hackystat sensor for Eclipse, you can start (or continue) to perform your Java-based development. After collecting some data, you can log in to your Hackystat account to run Zorro analyses in order to learn more about your TDD practices. This section describes the currently available Zorro analyses.
The Zorro analysis "TDD Stream Episode" provides a report illustrating the results of the TDD recognition process on an episode-by-episode basis. Figure 7.1, “ TDD Stream Episode Analysis ” illustrates one invocation of the TDD Stream Episode analysis.
First, each row in the table corresponds to one "episode" according to the Zorro episode classification rules. (In general, episode boundaries in Zorro correspond to the running of unit tests.)
Each row contains a left and a right cell. The right cell provides a sequence of abstract development events (Add Method, Remove Field, etc.) inferred from the low-level data collected by the IDE sensor. The left cell lists the classification of the episode based upon the analysis of the events in this episode listed in the right column. Zorro classifies an episode into one of the following categories: TDD, Refactor, TLD, or Unclassified. There are actually four subcategories of TDD: (TDD, 1), (TDD, 2), (TDD, 3) and (TDD, 4), which are described in Section 7.7.3, “TDD Recognition Rules”. In the example figure, both rows were classified as (TDD, 2).
You can use the TDD Stream Episode analysis to obtain feedback on your development process, and to understand more about how the Zorro recognition system works. This analysis shows you whether or not you are using TDD according to the rules for TDD recognition implemented in Zorro. If you are just learning about TDD, this can help you understand whether you are conforming to the Stop Light pattern of development. If you are an experienced TDD practitioner, this analysis can help you understand if your usage of TDD conforms to the "canonical" TDD process, or if you are using a "hybrid" approach to software development in which you use TDD only part of the time.
The "TDD Episode Demography" analysis provides a summary of the distribution of episode types over a specific interval of time. Unlike the TDD Stream Episode analysis, which shows you detailed feedback on specific episodes, this command provides a higher level view on the distribution of episode types over a longer period of time. Figure 7.2, “ TDD Episode Demography ” illustrates one invocation of this analysis.
You can use this command to obtain more abstract feedback on your development process. For example, if you are aiming for 100% compliance with the "canonical" definition of TDD, this analysis can help you determine how close you are that goal.
Telemetry-based analysis uses Zorro recognition results to derive percentage of test-driven development over total development process, and furthermore it allows us to correlate ratio of software development in TDD with other process and product measures including productivity proxy metric line of code, and project quality proxy metric test coverage. With test-driven development recognition results by Zorro, we developed three reduction functions that report episode count by type, TDD development percentage of project, and project member's TDD development percentage respectively.
ZorroEpisode: returns the number of episodes belong to a type specified by parameter "type", which is a value out of "tdd", "refactor", "tld", "validation", or "Total" that denotes the sum of all episode types.
ProjectTestDriven: computes percentage of TDD development of a project. It takes two parameters — type and withOutlier. Depending on the given parameters, this redunction function can report percentage of TDD episode either by number or by time. Outlier episodes are the episodes with long duration which do not seem like TDD episode. They can be excluded by giving a false value to withOutlier parameter.
MemberTestDriven: computes percentage of TDD development over a project member specified by parameter "memberEmail". Other two arguments are exactly the same as redunction function ProjectTestDriven.
Using these three reduction functions, interesting telemetry stream/chart/report can be defined to improve our understandings of test-driven development and its impacts on other product and process measurements. In the rest of this section, I am going to present three Zorro telemetry analyses.Figure 7.3, “ Number of TDD episodes by type ” illustrates day by day TDD episode count with telemetry chart definition named TDD-Episode.
From this chart we can see that development of this project is TDD at most of the time and developers seldomly refactored the code.
Simply knowing number of episodes and their distributions is great if we only want to know whether the discipline of TDD is sustained in software project development. Supported by software project telemetry, we can also related the recognition of TDD by Zorro to other metrics to improve our understanding of test-driven development. A good telemetry analysis Tdd-Chart, which juxtapose percentage of TDD, code churn and active time which representing productivity, and test coverage which representing software quality in Figure 7.4, “ Project test-driven development analysis. ”.
On average test-driven development percentage is satisfying in 55-75% range and code coverage is stable at 90-95%, but there are three dips that happened at day Jul 2, Jul 4 and Jul 7. Coincidently, there is a pattern that there was high productivity (bigger code churn value) and low percentage of TDD development in previous day of the three dip days. A hypothesis of this could be that test-driven development helps code coverage stays at high level.
While Tdd-Chart telemetry chart tell how percentage of TDD can be correlated with productivity and qualify metrics, it will be useful as well to observe how individual developer do TDD development. For this purpose, we defined a telemetry report called Member-TDD that relates project member's TDD development percentage and productivity. Figure 7.5, “ Member test-driven development analysis (member A) ” and Figure 7.6, “ Member test-driven development analysis (member B) ” are two charts from telemetry report Member-TDD.
A pity of this report is that project member B did not do much programming, but we still can see the potentials what it can bring us. For each project member, this report illustrates percentage of TDD, active time and code churn, which gives us insights about individual programmer's devotion on TDD.
The three TDD redunction functions, two telemetry charts and a telemetry report are the explorative telemetry-based analyses of TDD we have at this moment. With the increasing practices and experiences on TDD in project development, we will be developing more telemetry streams, charts and reports to find ways using TDD wisely in our software project development.