Reports

Overview

Specmatic can generate various types of reports to help you understand your API testing coverage and results. Report configuration is specified under the report key in your Specmatic configuration file.

Configuration Structure

The report configuration has two optional properties:

• types - Configure behavior for specific report types
• formatters - Control how reports are generated and formatted

Basic Example

Version 3 (recommended)

YAML

specmatic.yaml

version: 3
specmatic:
  governance:
    report:
      formats:
        - ctrf
        - html
      outputDirectory: build/reports/specmatic
    successCriteria:
      minCoveragePercentage: 100
      maxMissedOperationsInSpec: 0
      enforce: true

specmatic.json

specmatic.json

{
  "version": 3,
  "specmatic": {
    "governance": {
      "report": {
        "formats": [
          "ctrf",
          "html"
        ],
        "outputDirectory": "build/reports/specmatic"
      },
      "successCriteria": {
        "minCoveragePercentage": 100,
        "maxMissedOperationsInSpec": 0,
        "enforce": true
      }
    }
  }
}

Version 2

YAML

specmatic.yaml

version: 2
report:
  formatters:
    - type: ctrf
    - type: html
  types:
    APICoverage:
      OpenAPI:
        successCriteria:
          minThresholdPercentage: 100
          maxMissedEndpointsInSpec: 0
          enforce: true

JSON

specmatic.json

{
  "version": 2,
  "report": {
    "formatters": [
      {
        "type": "ctrf"
      },
      {
        "type": "html"
      }
    ],
    "types": {
      "APICoverage": {
        "OpenAPI": {
          "successCriteria": {
            "minThresholdPercentage": 100,
            "maxMissedEndpointsInSpec": 0,
            "enforce": true
          }
        }
      }
    }
  }
}

Report Types

API Coverage Report

The API Coverage report provides a comprehensive analysis of any mismatches between your API specification and implementation. It helps you identify:

• Endpoints defined in your spec but not implemented
• Implemented endpoints not documented in your spec
• Coverage percentage across your API surface

Currently, Specmatic supports API Coverage configuration for OpenAPI specifications.

important

For detailed information about API Coverage configuration keys and their functions, see our in-depth article.

---

Report Formats

Specmatic can generate reports in various formats.

Text Format

The text format (enabled by default) outputs reports directly to your console or terminal, making it ideal for CI/CD pipelines and quick feedback during development.

HTML Format

The HTML format generates rich, interactive reports with customizable branding and styling. These reports are ideal for sharing with stakeholders and archiving test results.

CTRF Format Commercial

The Common Test Report Format (CTRF) is a standardized JSON schema for test results that enables seamless integration across different testing tools and CI/CD platforms.

---

Why Use CTRF?

• Universal Compatibility: Works with popular CI/CD platforms (GitHub Actions, Jenkins, GitLab CI, etc.)
• Tool Agnostic: Standardized format means reports can be consumed by various visualization and analysis tools
• Better Insights: Leverage CTRF-compatible tools for advanced analytics, trend analysis, and reporting dashboards
• Easy Integration: JSON format makes it simple to parse and integrate with custom tooling