Reports

Table of contents

1. Overview
2. Configuration Structure
   1. Basic Example
3. Report Types
   1. API Coverage Report
4. Report Formats
   1. Text Format
   2. HTML Format
   3. CTRF Format
      1. Why Use CTRF?
5. Complete Configuration Example
6. Quick Reference
   1. Formatter Types
   2. Common Properties

---

Overview

Specmatic can generate various types of reports to help you understand your API testing coverage and results. Report configuration is specified under the report key in your Specmatic configuration file.

Configuration Structure

The report configuration has two optional properties:

• types - Configure behavior for specific report types
• formatters - Control how reports are generated and formatted

Basic Example

• version: 2
  report:
    formatters:
      - type: text
        layout: table
    types:
      APICoverage:
        OpenAPI:
          successCriteria:
            minThresholdPercentage: 100
            maxMissedEndpointsInSpec: 0
            enforce: true
  

• {
    "version": 2,
    "report": {
      "formatters": [
        {
          "type": "text",
          "layout": "table"
        }
      ],
      "types": {
        "APICoverage": {
          "OpenAPI": {
            "successCriteria": {
              "minThresholdPercentage": 100,
              "maxMissedEndpointsInSpec": 0,
              "enforce": true
            }
          }
        }
      }
    }
  }
  

---

Report Types

API Coverage Report

The API Coverage report provides a comprehensive analysis of any mismatches between your API specification and implementation. It helps you identify:

• Endpoints defined in your spec but not implemented
• Implemented endpoints not documented in your spec
• Coverage percentage across your API surface

Currently, Specmatic supports API Coverage configuration for OpenAPI specifications.

For detailed information about API Coverage configuration keys and their functions, see our in-depth article.

---

Report Formats

Specmatic can generate reports in various formats.

Text Format

The text format outputs reports directly to your console or terminal, making it ideal for CI/CD pipelines and quick feedback during development.

HTML Format

The HTML format generates rich, interactive reports with customizable branding and styling. These reports are ideal for sharing with stakeholders and archiving test results.

CTRF Format

The Common Test Report Format (CTRF) is a standardized JSON schema for test results that enables seamless integration across different testing tools and CI/CD platforms.

---

Why Use CTRF?

• Universal Compatibility: Works with popular CI/CD platforms (GitHub Actions, Jenkins, GitLab CI, etc.)
• Tool Agnostic: Standardized format means reports can be consumed by various visualization and analysis tools
• Better Insights: Leverage CTRF-compatible tools for advanced analytics, trend analysis, and reporting dashboards
• Easy Integration: JSON format makes it simple to parse and integrate with custom tooling

Commercial Feature: CTRF report generation is only available in the commercial version of Specmatic. Please visit the pricing page for more information.

---

Complete Configuration Example

Here’s a comprehensive example showing all report configuration options:

• version: 2
  report:
    formatters:
      - type: text
        layout: table
      - type: html
        layout: table
        lite: false
        title: "Specmatic API Tests - QA Environment"
        logo: "assets/qa-logo.svg"
        logoAltText: "QA Environment"
        heading: "Contract Test Results - QA"
        outputDirectory: "./build/reports/specmatic/html"
      - type: ctrf
    types:
      APICoverage:
        OpenAPI:
          successCriteria:
            minThresholdPercentage: 100
            maxMissedEndpointsInSpec: 0
            enforce: true
  

• {
    "version": 2,
    "report": {
      "formatters": [
        {
          "type": "text",
          "layout": "table"
        },
        {
          "type": "html",
          "layout": "table",
          "lite": false,
          "title": "Specmatic API Tests - QA Environment",
          "logo": "assets/qa-logo.svg",
          "logoAltText": "QA Environment",
          "heading": "Contract Test Results - QA",
          "outputDirectory": "./build/reports/specmatic/html"
        },
        {
          "type": "ctrf"
        }
      ],
      "types": {
        "APICoverage": {
          "OpenAPI": {
            "successCriteria": {
              "minThresholdPercentage": 100,
              "maxMissedEndpointsInSpec": 0,
              "enforce": true
            }
          }
        }
      }
    }
  }
  

---

Quick Reference

Formatter Types

• text - Console output (default)
• html - Interactive HTML reports (default)
• ctrf - Common Test Report Format (commercial only)

Common Properties

Property	Applies To	Required
type	All formatters	Yes
layout	text	No
lite	html	No
title	html	No
logo	html	No
logoAltText	html	No
heading	html	No
outputDirectory	html	No