Coco TCL Tutorial

Analysis of Tcl script coverage

Demo application build

Copy the sample to your working directory:

$ cp -r /opt/SquishCoco/doc/demoapp_tcl ./
$ cd demoapp_tcl

Edit Makefile and adjust the following variables to match your system's library names and include paths:

TCLLIBS=-ltk8.5 -litcl3.4 -ltcl8.5
TCLINCLUDES=-I/usr/include/tcl8.5

Build the C++ executable with:

$ make

Test the simple Tk GUI with an invocation of the launcher script:

$ ./proto

Ad-hoc Tcl coverage run

Launch the application under test via the Tcl coverage scanning tool:

$ coveragescannertcl --cs-output=tcl ./proto

The GUI will show up as usually while code coverage is monitored. At this point the application might already be shut down via the File > Exit menu entry or the X button in the window's title bar. Observe the two files tcl.csmes and tcl.csexe being generated. The latter contains data about the code covered in the last execution. The former is a measurement database containing information about the instrumented code and possibly imported previous executions. In this case the base name of the output files was set via the --cs-output switch. If omitted, it will be deduced from the application name.

Result analysis

The results can be viewed interactively in the GUI provided by Coco. To launch it, run:

$ coveragebrowser

Select File > Open to open the tcl.csmes file. You'll see all Tcl files and procedures listed in the Sources and Methods tabs, respectively. When selecting either of them, a source code viewer will open that shows instrumented code with a gray background color. Note that the coloring is still limited to instrumented, i.e. seen code only. To check which code got actually executed (or not) select File > Load Execution Report and open tcl.csexe. Now red and green colors denoting positive and negative coverage will show up for files, procedures and the code itself.

Filtered Tcl coverage run

As you might have noticed, the ad-hoc run produced results for several unnamed and internal files. These stem from internal Tcl/Tk files or e.g. mouse event handlers that result in on-the-fly eval calls. Compared to the application-specific Tcl files the coverage information for these internal pieces of code is of little interest.

The easiest way to hide the non-relevant pieces of code is through usage of the various --cs-exclude-code options. A recommended list of switches will be shown in the repetition of the previous run with filtering options applied:

$ coveragescannertcl --cs-output=tcl  --cs-exclude-code-regex="^\s*[^\n]*\s*$"
   --cs-exclude-code-regex="(tcl|tk)Init" --cs-exclude-code-regex="namespace eval ::itcl"
   --cs-exclude-code-regex="^# This file is generated"
   --cs-exclude-code-regex="tk::Menu(Motion|Leave)" ./proto

When viewing the new tcl.csmes in CoverageBrowser you'll notice that the list of files is reduced to the relevant canvas.tclx, input.tclx, mainwindow.tclx, point.itcl and rect.itcl scripts.

TCL coverage report generation

Text, XML and HTML reports can be generated from the CoverageBrowser GUI, or from the command line. For this example, we'll pretend that the execution data has not yet been imported into the measurement database. This step can be accomplished from the command line:

$ cmcsexeimport --title="First run" -m tcl.csmes -e tcl.csexe

The resulting proto.csmes file now contains what we need to generate a report:

$ cmreport --title="Coverage Data" -m tcl.csmes --html=report.html

The resulting report.html file (including a report_html/ directory) can now be visited with a Web browser. It includes a listing of executions, source files, functions and metrics on a global level, per file and per function.

Analysis of C/C++ code coverage

Instrumentation

While Tcl code can be instrumented for coverage analysis on-the-fly, this step requires a rebuild for C or C++ applications. However, this can be accomplished with almost no change to your regular application build. Here are the two possible approaches:

• Modify your build system to make use of the code scanning tools csgcc, csg++, etc. that will both instrument the code and later use the original compiler to compile the code as usual.
• Sneak Coco's compiler wrappers named gcc, g++, etc. into the executable search PATH. They will first instrument the code and then let the real compiler do its job.

We'll make use of approach number 2. First, clean the existing object files:

$ make clean

Next, recompile the application with a changed PATH and a switch that enables the compiler wrappers. For brevity, we'll do this all in a single line:

$ PATH=~/SquishCoco/wrapper/bin:$PATH COVERAGESCANNER_ARGS="--cs-on"  make

As a result we'll not only get the main executable (proto.exe) but also a corresponding instrumentation file proto.exe.csmes. Information about the instrumented code (gray background color) can optionally be reviewed at this point:

$ coveragebrowser -m proto.exe.csmes

Execution

Let's run the application:

$ ./proto

Then quit it via its GUI right after. The current working directory will now contain a proto.exe.csexe containing information about the completed execution.

Result analysis

Same as with Tcl the results can now be reviewed in CoverageBrowser. A shortcut invocation that opens both the instrumentation and execution files is shown here:

$ coveragebrowser -m proto.exe.csmes -e proto.exe.csexe

TCL/C++ coverage report generation

The process of report generation works analogous to the approach shown for Tcl code above.

Analysis of mixed Tcl and C/C++ code coverage

Coverage data for the Tcl and C/C++ parts of a hybrid application can be gathered independently as shown above, from a single or multiple executions. The only extra step needed is the merging of the data into a single measurement database. This can be used to generate a single, combined report taking both languages into account.

Combined coverage run

Assuming the instrumentation of the C/C++ code is still in place, we can gather coverage data for both languages with a single run:

$ coveragescannertcl --cs-output=tcl ./proto

With the proto.exe.csmes file still in place from the application rebuild we'll now have two sets of files:

1. tcl.csexe
2. tcl.csmes
3. proto.exe.csexe
4. proto.exe.csmes

At this point, the coverage data for Tcl and C/C++ could be analyzed independently using cmcsexeimport. But we want to gain an overall picture and will therefore merge the results in the next section.

Merging measurement data

Instead of using the Coverage Browser GUI, we'll use the command line tools. In the first step, we'll merge the instrumentation files stemming from the two languages into one:

cmmerge -o all.csmes tcl.csmes proto.exe.csmes

To save some disk space, the --append option can be used rather than creating the new all.csmes file.

In the second step, the two execution files are merged using the merge policy for the 2nd language:

$ cmcsexeimport -m all.csmes -e tcl.csexe --title="Hybrid" --passed
$ cmcsexeimport -m all.csmes -e proto.exe.csexe -p merge --title="Hybrid" --passed

For the merge to actually happen, the title has to be identical.

Report generation for combined coverage

An HTML report encompassing coverage for both application parts can be generated with a single command line call:

$ cmreport --title="Hybrid Tcl/C++ Coverage" -m all.csmes --html=report.html

Select Source Files to see both Tcl (various files) and C++ code (main.cpp) being listed in a combined report.