Reports

Table of contents

  1. Overview
  2. Configuration Structure
    1. Basic Example
  3. Report Types
    1. API Coverage Report
  4. Report Formatters
    1. Default Behavior
    2. Available Formatter Types
  5. Text Formatter
    1. Configuration Options
    2. Example
  6. HTML Formatter
    1. Configuration Options
    2. Basic Example
    3. Customized Example
  7. CTRF Formatter
    1. Why Use CTRF?
    2. Configuration
  8. Multiple Formatters
  9. Complete Configuration Example
  10. Quick Reference
    1. Formatter Types
    2. Common Properties

Overview

Specmatic can generate various types of reports to help you understand your API testing coverage and results. Report configuration is specified under the report key in your Specmatic configuration file.

Configuration Structure

The report configuration has two optional properties:

  • types - Configure behavior for specific report types
  • formatters - Control how reports are generated and formatted

Basic Example 

  • version: 2
report:
  formatters:
    - type: text
      layout: table
  types:
    APICoverage:
      OpenAPI:
        successCriteria:
          minThresholdPercentage: 100
          maxMissedEndpointsInSpec: 0
          enforce: true

  • {
  "version": 2,
  "report": {
    "formatters": [
      {
        "type": "text",
        "layout": "table"
      }
    ],
    "types": {
      "APICoverage": {
        "OpenAPI": {
          "successCriteria": {
            "minThresholdPercentage": 100,
            "maxMissedEndpointsInSpec": 0,
            "enforce": true
          }
        }
      }
    }
  }
}

Report Types

API Coverage Report

The API Coverage report provides a comprehensive analysis of any mismatches between your API specification and implementation. It helps you identify:

  • Endpoints defined in your spec but not implemented
  • Implemented endpoints not documented in your spec
  • Coverage percentage across your API surface

Currently, Specmatic supports API Coverage configuration for OpenAPI specifications.

For detailed information about API Coverage configuration keys and their functions, see our in-depth article.

Report Formatters

Formatters control how Specmatic generates and presents reports. You can configure multiple formatters to generate reports in different formats simultaneously.

Default Behavior

If no formatters are specified, Specmatic generates both text and html reports by default.

Available Formatter Types

Specmatic supports three formatter types:

Type Description
text Console/terminal output with optional table layout
html Rich, interactive HTML reports with customizable branding
ctrf Common Test Report Format for CI/CD integration

Text Formatter

The text formatter outputs reports directly to your console or terminal, making it ideal for CI/CD pipelines and quick feedback during development.

Configuration Options

Property Type Default Description
type string text Formatter type
layout string table Output layout format

Example 

  • report:
  formatters:
    - type: text
      layout: table

  • {
  "report": {
    "formatters": [
      {
        "type": "text",
        "layout": "table"
      }
    ]
  }
}

HTML Formatter

The HTML formatter generates rich, interactive reports with customizable branding and styling. These reports are ideal for sharing with stakeholders and archiving test results.

Configuration Options

Property Type Default Description
type string html Formatter type
lite boolean false Generate a lightweight version of the report
title string "Specmatic Report" Browser tab title
logo string "assets/specmatic-logo.svg" Path to custom logo image
logoAltText string "Specmatic" Alt text for the logo
heading string "Contract Test Results" Main heading displayed in the report
outputDirectory string "./build/reports/specmatic/html" Directory where HTML reports are saved

Basic Example 

  • report:
  formatters:
    - type: html

  • {
  "report": {
    "formatters": [
      {
        "type": "html"
      }
    ]
  }
}

Customized Example 

  • report:
  formatters:
    - type: html
      title: "API Contract Test Report - Production"
      logo: "assets/company-logo.png"
      logoAltText: "Company Name"
      heading: "Production API Test Results"
      outputDirectory: "./reports/api-tests"
      lite: false

  • {
  "report": {
    "formatters": [
      {
        "type": "html",
        "title": "API Contract Test Report - Production",
        "logo": "assets/company-logo.png",
        "logoAltText": "Company Name",
        "heading": "Production API Test Results",
        "outputDirectory": "./reports/api-tests",
        "lite": false
      }
    ]
  }
}

CTRF Formatter

The Common Test Report Format (CTRF) is a standardized JSON schema for test results that enables seamless integration across different testing tools and CI/CD platforms.

Why Use CTRF?

  • Universal Compatibility: Works with popular CI/CD platforms (GitHub Actions, Jenkins, GitLab CI, etc.)
  • Tool Agnostic: Standardized format means reports can be consumed by various visualization and analysis tools
  • Better Insights: Leverage CTRF-compatible tools for advanced analytics, trend analysis, and reporting dashboards
  • Easy Integration: JSON format makes it simple to parse and integrate with custom tooling

Commercial Feature: CTRF report generation is only available in the commercial version of Specmatic. Please visit the pricing page for more information.

Configuration

CTRF reports require minimal configuration - just specify the type:

  • report:
  formatters:
    - type: ctrf

  • {
  "report": {
    "formatters": [
      {
        "type": "ctrf"
      }
    ]
  }
}

The CTRF report for contract tests will be generated in the build/reports/specmatic/ctrf directory.

Multiple Formatters

You can configure multiple formatters to generate reports in different formats simultaneously:

  • report:
  formatters:
    - type: text
      layout: table
    - type: html
      title: "API Contract Tests"
      heading: "Test Results Dashboard"
      outputDirectory: "./reports/html"
    - type: ctrf

  • {
  "report": {
    "formatters": [
      {
        "type": "text",
        "layout": "table"
      },
      {
        "type": "html",
        "title": "API Contract Tests",
        "heading": "Test Results Dashboard",
        "outputDirectory": "./reports/html"
      },
      {
        "type": "ctrf"
      }
    ]
  }
}

Complete Configuration Example

Here’s a comprehensive example showing all report configuration options:

  • version: 2
report:
  formatters:
    - type: text
      layout: table
    - type: html
      layout: table
      lite: false
      title: "Specmatic API Tests - QA Environment"
      logo: "assets/qa-logo.svg"
      logoAltText: "QA Environment"
      heading: "Contract Test Results - QA"
      outputDirectory: "./build/reports/specmatic/html"
    - type: ctrf
  types:
    APICoverage:
      OpenAPI:
        successCriteria:
          minThresholdPercentage: 100
          maxMissedEndpointsInSpec: 0
          enforce: true

  • {
  "version": 2,
  "report": {
    "formatters": [
      {
        "type": "text",
        "layout": "table"
      },
      {
        "type": "html",
        "layout": "table",
        "lite": false,
        "title": "Specmatic API Tests - QA Environment",
        "logo": "assets/qa-logo.svg",
        "logoAltText": "QA Environment",
        "heading": "Contract Test Results - QA",
        "outputDirectory": "./build/reports/specmatic/html"
      },
      {
        "type": "ctrf"
      }
    ],
    "types": {
      "APICoverage": {
        "OpenAPI": {
          "successCriteria": {
            "minThresholdPercentage": 100,
            "maxMissedEndpointsInSpec": 0,
            "enforce": true
          }
        }
      }
    }
  }
}

Quick Reference

Formatter Types

  • text - Console output (default)
  • html - Interactive HTML reports (default)
  • ctrf - Common Test Report Format (commercial only)

Common Properties

Property Applies To Required
type All formatters Yes
layout text No
lite html No
title html No
logo html No
logoAltText html No
heading html No
outputDirectory html No