Skip to content

How to publish Code Coverage summary reports Github actions

In How to use Cake with Github Actions I discussed how to use my preferred build script for my dotnet based projects and using it with GitHub actions. In this post I will expand on that post a little and discuss how you can generate unit test code coverage reports and even upload summary reports to view on your GitHub actions.

What is code coverage

Code coverage is a metric that can help you understand how much of your source is tested. It's a very useful metric that can help you assess the quality of your test suite.

Code coverage tools will use one or more criteria to determine how your code was exercised or not during the execution of your test suite. The common metrics that you might see mentioned in your coverage reports include:

  • Function coverage: how many of the functions defined have been called.
  • Statement coverage: how many of the statements in the program have been executed.
  • Branches coverage: how many of the branches of the control structures (if statements for instance) have been executed.
  • Condition coverage: how many of the boolean sub-expressions have been tested for a true and a false value.
  • Line coverage: how many of lines of source code have been tested

These metrics are usually represented as the number of items actually tested, the items found in your code, and a coverage percentage (items tested / items found). Code coverage is important because it’s an objective way to determine software quality before a project is sent off to end users. Any tests executed through a continuous delivery pipeline are essentially useless without a code coverage tool because without code coverage, there’s no way to know just how much source code is actually being tested.

Continuous Architecture in Practice

Software Architecture In the age of Agility and Devops

in-depth guidance for addressing today's key quality attributes and cross-cutting concerns such as security, performance, scalability, resilience, data, and emerging technologies. Each key technique is demonstrated through a start-to-finish case study reflecting the authors’ deep experience with complex software environments.

Buy Now
Read Review

Using ReportGenerator

For my projects I like to use ReportGenerator to generate my code coverage reports. ReportGenerator is a command line tool which only requires a few parameters. Integration into your build pipeline will only take a couple of minutes. Integration ReportGenerator works on Windows, Linux and mac OS. Support for GitHub actions and Azure DevOps is available.

If you haven't previously checked out this awesome library then I highly recommend you do so, and if you like what you see there and start using it yourself then why not hit the Sponsor button, to help encourage Daniel to keep on developing this great product!

Generating Code Code Coverage reports with Cake

In order to generate Unit Test code coverage reports with your existing unit test project you will need to add a couple of additional libraries to your unit test projects. In my case, I tend to make use of Directory.Build.props file to manage all my common package references for my unit test project. If you don't know what a Directory.Build.props file is you may want to check out What is this Directory.Build.props file all about? where I dig in a little deeper into what it is and how to use it.

The two libraries you will need to add are Coverlet.Collector and Coverlet.msbuild 

 <PackageReference Include="coverlet.collector" Version="6.0.0" />
 <PackageReference Include="coverlet.msbuild" Version="6.0.0"/>

With this in place we can update our Cake Build script to generate Code Coverage reports when executing our Unit tests.

Task("Test")
    .IsDependentOn("Build")
    .Does(() => {
       
       var testSettings = new DotNetTestSettings  {
                 Configuration = configuration,
                 NoBuild = true,
       };
        var coverageOutput = Directory(TEST_COVERAGE_OUTPUT_DIR);             
     
       GetFiles("./tests/**/*.csproj").ToList().ForEach(project => {
          Information($"Testing Project : {project.ToString()}");
            
          var codeCoverageOutputName = $"{project.GetFilenameWithoutExtension()}.cobertura.xml";
          var coverletSettings = new CoverletSettings {
              CollectCoverage = true,
               CoverletOutputFormat = CoverletOutputFormat.cobertura,
               CoverletOutputDirectory =  coverageOutput,
               CoverletOutputName =codeCoverageOutputName,
               ArgumentCustomization = args => args.Append($"--logger trx")
          };
                  
          Information($"Running Tests : { project.ToString()}");
          DotNetTest(project.ToString(), testSettings, coverletSettings );        
        });
     
      Information($"Directory Path : { coverageOutput.ToString()}");
          
      var glob = new GlobPattern($"./{ coverageOutput}/*.cobertura.xml");
         
      Information($"Glob Pattern : { glob.ToString()}");
      var outputDirectory = Directory("./coverage/reports");
     
      var reportSettings = new ReportGeneratorSettings
      {
         ArgumentCustomization = args => args.Append($"-reportTypes:Html;Html_Dark")
      };
         
      ReportGenerator(glob, outputDirectory, reportSettings);
          
});

The full source code and complete example for this code is available on my PayBolt project

C# Bitcoin Lightning Network Client
https://github.com/threenine/paybolt
1 forks.
4 stars.
0 open issues.

Recent commits:

If we execute our build script locally this will generate a great report for us to review locally either by opening the report Directly in Rider or browsing it in your preferred browser.

I find it really useful to browse these reports to code coverage etc.

Now this is great for when you're running locally and also when you run your Cake Script locally you'll get some handy feedback in the terminal window as the script is executing.

This is great, but what is really useful is if we can see this summary information also displayed on Github when our action has executed.

Well this is relatively easily achieved by making some tweaks to our Build Script and adding an additional step to our Github Action.

First we'll add the additional piece to our test script running check if our script is running on Github and if it is generate Summary Report in Markdown.

  if (BuildSystem.GitHubActions.IsRunningOnGitHubActions)
  {
    var summaryDirectory = Directory("./coverage");
    var summarySettings = new ReportGeneratorSettings
    {
      ArgumentCustomization = args => args.Append($"-reportTypes:Html;MarkdownSummaryGithub")
    };
        ReportGenerator(glob, summaryDirectory, summarySettings);
  }

This will generate a file in our Coverage directory which will be name SummaryGithub.md which as it name suggest will generate a summary report we can view on Github

With this now in place can add the following to our github action

      - name: Create check run with code coverage Markdown report
        uses: LouisBrunner/checks-action@v1.2.0
        if: always()
        with:
          token: ${{ github.token }}
          name: Code coverage
          conclusion: ${{ job.status }}
          output: '{"summary":"Code coverage"}'
          output_text_description_file: coverage/SummaryGithub.md

Once we commit our changes and our action is run we'll see our summary report on Github.

Gary Woodfine
Latest posts by Gary Woodfine (see all)
Tags: