Skip to main content

Reports Sent to Insights

Run the command that generates the report with --ci, --build-id, and repository metadata to submit it directly to Insights. test submits its test report, mock submits its mock-usage report, and run-suite submits both reports separately.

Build visibility

After a build is submitted, it can take up to 30 minutes to appear in Insights.

Request composition

Each Insights request combines:

  • CLI arguments: CI flag, build ID, repository name, ID, URL, branch name, metadata, and an optional Specmatic configuration path
  • the CTRF or spec-metadata report generated by that command
  • the Specmatic configuration, when present
  • product and tool version information in publisher
  • authentication headers: the license ID and, when configured, the submission token from the command's --token option or SPECMATIC_INSIGHTS_TOKEN

Authentication information is sent in headers, not in the JSON request body shown below.

What is shared

The API specification files are not sent to Insights. Reports contain operation metadata derived from those files, such as paths, methods, response codes, protocol, specification path, and coverage or compatibility status.

For operations that Specmatic executes using generated test data or examples, the corresponding CTRF test entry includes the generated or example-derived request and response data.

Report sources

CommandReport sent to InsightsArtifact
spec-metadata-report --ciSpec metadata reportspec_metadata_report.json
test --ciContract test coverage reporttest/ctrf/ctrf-report.json
mock --ciMock usage reportstub/ctrf/ctrf-report.json
run-suite --ciThe contract test coverage and mock usage reports it generatesThe same test/ctrf/ctrf-report.json and stub/ctrf/ctrf-report.json artifacts
backward-compatibility-check --ciBackward compatibility reportbackward_compatibility/ctrf/ctrf-report.json

For AsyncAPI, GraphQL, and gRPC test and mock runs, the CTRF artifact is stored under the corresponding protocol directory, for example async/test/ctrf/ctrf-report.json or grpc/stub/ctrf/ctrf-report.json.

Pass --ci --build-id <id> --repo-id <id> --repo-name <name> --repo-url <url> --branch-name <branch> to the command that generates the report. run-suite uses one build ID for its separately submitted test and mock reports.

Build identity and default metadata

--build-id is the stable identifier that correlates all reports from one CI build in Insights. A run-suite invocation therefore uses one build ID for its separately submitted test and mock reports. Use the build or workflow identifier supplied by your CI provider; on GitHub Actions, this is ${{ github.run_id }}.

--run-attempt identifies which execution of that build produced a report. It is 1 for the original execution and increases when the CI provider retries or reruns it. Insights can therefore distinguish a report from an initial run from one produced by a later attempt without splitting them into unrelated builds.

Resolution order

Each value is resolved in this order, with the first available value winning:

  1. CLI argument, such as --build-id or --repo-id
  2. System property, such as -Dspecmatic.build.id
  3. Specmatic environment variable, such as SPECMATIC_BUILD_ID
  4. Detected CI-provider environment variable
  5. Local Git metadata, for repository URL, repository name, and branch only

CI publishing must be enabled explicitly with --ci, SPECMATIC_CI=true, or -Dspecmatic.ci=true, because publishing is a sensitive operation. CLI arguments always override system properties and environment variables.

For CI publishing, --build-id and --repo-id are required. A provider can supply both automatically, but Git cannot supply a trustworthy repository ID: if the active provider has no repository-ID field, provide --repo-id or SPECMATIC_REPO_ID yourself.

OptionSystem propertySpecmatic environment variable
--repo-idspecmatic.repo.idSPECMATIC_REPO_ID
--repo-namespecmatic.repo.nameSPECMATIC_REPO_NAME
--repo-urlspecmatic.repo.urlSPECMATIC_REPO_URL
--branch-namespecmatic.branch.nameSPECMATIC_BRANCH_NAME
--build-idspecmatic.build.idSPECMATIC_BUILD_ID
--run-attemptspecmatic.run.attemptSPECMATIC_RUN_ATTEMPT
--tokenspecmatic.tokenSPECMATIC_INSIGHTS_TOKEN
--cispecmatic.ciSPECMATIC_CI

CI-provider defaults

Select your CI provider to see the values Specmatic reads. The example in each tab passes them explicitly as CLI options, which is useful when running Specmatic in a container or other isolated execution context. On a normal CI runner, use specmatic test --ci: Reporter reads the native variables automatically. A dash means the provider does not supply a suitable value, so provide --repo-id yourself when publishing from CI.

ParameterRuntime environment variable
Build IDGITHUB_RUN_ID
Run attemptGITHUB_RUN_ATTEMPT
Repository IDGITHUB_REPOSITORY_ID
Repository nameGITHUB_REPOSITORY
Repository URLGITHUB_SERVER_URL + GITHUB_REPOSITORY
BranchGITHUB_HEAD_REF, then GITHUB_REF_NAME, then GITHUB_REF
- name: Run Specmatic in Docker
run: |
docker run --rm \
<image> test --ci \
--build-id '${{ github.run_id }}' \
--run-attempt '${{ github.run_attempt }}' \
--repo-id '${{ github.repository_id }}' \
--repo-name '${{ github.repository }}' \
--repo-url '${{ github.server_url }}/${{ github.repository }}' \
--branch-name '${{ github.head_ref || github.ref_name }}'

--run-attempt is 1 for the first execution. GitLab and Buildkite expose zero-based retry counts, so Specmatic adds one. Azure's SYSTEM_JOBATTEMPT is a job-level attempt; the GitHub value is a workflow-run attempt.

Running inside Docker

CI variables belong to the runner process. Docker containers do not inherit them automatically, so pass the provider variables through when launching the container, or pass the normalized SPECMATIC_* overrides instead. For example, a GitHub Actions job can forward GITHUB_RUN_ID, GITHUB_RUN_ATTEMPT, and GITHUB_REPOSITORY_ID with docker run -e ...; Docker Compose can use equivalent environment entries or an env_file.

If no CI variables are available inside the container, specify the required values explicitly with CLI options or SPECMATIC_* variables. Specmatic can still infer repository URL, name, and branch from a Git checkout, but it cannot infer a CI build ID or a stable repository ID from Git alone.

--metadata and --config are CLI-only options.

Contract test report payload

test --ci submits its CTRF contract-test report. This example retains every field from the captured report, but shows one operation, one test ID, and one test entry. The aggregate counters retain their original values.

Request headers:

Authorization: license 9dcf94d1-c731-4b85-8a10-0fad31d87bcd
Content-Type: application/json
User-Agent: Specmatic-Reporter/io.specmatic.enterprise:executable:1.23.0 running on 17.0.19
X-Auth-Token: Bearer <token>

X-Auth-Token is present when build-submission two-factor authentication is configured with the command's --token <token> option or SPECMATIC_INSIGHTS_TOKEN.

{
"repo": "specmatic-order-api-java",
"repoId": "641599527",
"repoUrl": "https://github.com/specmatic/specmatic-order-api-java",
"branchName": "main",
"orgId": "c6e33dd8-af67-4658-8783-f83ad9eab021",
"createdAt": "2026-08-05T16:00:34.771279Z",
"projectDir": null,
"specmaticConfigPath": "./specmatic.yaml",
"specmaticCoverage": {
"results": {
"tool": {
"name": "Specmatic v2.51.1"
},
"summary": {
"tests": 293,
"passed": 228,
"failed": 65,
"skipped": 0,
"pending": 0,
"other": 0,
"start": 1785945575736,
"stop": 1785945578067,
"suites": 1,
"extra": {
"executionDetails": [
{
"protocol": "http",
"specType": "openapi",
"specification": "io/specmatic/examples/store/openapi/api_order_v5.yaml",
"type": "git",
"repository": "https://github.com/specmatic/specmatic-order-contracts",
"branch": "main",
"coverageMetrics": {
"apiCoverage": 44,
"absoluteCoverage": 44,
"coveredOperations": 16,
"totalOperationsWithFilters": 36,
"totalOperations": 36
},
"operations": [
{
"path": "/orders/{id}",
"method": "PATCH",
"contentType": "application/json",
"responseCode": 400,
"responseContentType": "application/json",
"coverageStatus": "covered",
"eligibleForCoverage": true,
"omittedStatus": "NONE",
"testIds": [
"1766bd7e-f542-4b00-8b35-8a802a396399"
],
"metrics": {
"attempts": 24,
"matches": 24
},
"reasons": [],
"qualifiers": [],
"status": "covered"
}
]
}
]
}
},
"tests": [
{
"id": "1766bd7e-f542-4b00-8b35-8a802a396399",
"name": "-ve Scenario: PATCH /orders/(id:number) -> 4xx with the request from the example 'UPDATE_ORDER' where REQUEST.PARAMETERS.PATH.id is mutated from number to boolean",
"status": "passed",
"duration": 4,
"suite": "io/specmatic/examples/store/openapi/api_order_v5.yaml",
"message": "",
"rawStatus": "Success",
"tags": [
"content-type:application/json",
"method:patch",
"status:400"
],
"type": "ContractTest",
"extra": {
"valid": true,
"wip": false,
"input": "PATCH /orders/true\nAuthenticate: <redacted>\nContent-Type: application/json\nSpecmatic-Response-Code: 400\n\n{\n \"productid\": 10,\n \"count\": 1,\n \"status\": \"pending\"\n}",
"inputTime": 1785945577908,
"outputs": [
{
"title": "Response",
"content": "400 Bad Request\nSet-Cookie: <redacted>\nContent-Type: application/json\n\n{\n \"timestamp\": \"2026-08-05T21:29:37.913574\",\n \"status\": 400,\n \"error\": \"Bad Request\",\n \"message\": \"Method parameter 'id': Failed to convert value of type 'java.lang.String' to required type 'int'; For input string: \\\"true\\\"\"\n}",
"time": 1785945577913
}
],
"reasons": [
{
"id": "T10004",
"title": "Executed Using Negative Generation",
"documentationUrl": "https://docs.specmatic.io/rules#t10004",
"summary": "This operation was executed by generating -ve payloads, due to negative generation being enabled"
}
],
"qualifiers": []
}
}
],
"environment": {
"repositoryName": "specmatic-order-contracts",
"repositoryUrl": "https://github.com/specmatic/specmatic-order-contracts.git"
},
"extra": {
"apiCoverage": "44%",
"absoluteCoverage": "44%",
"specmaticConfigPath": "./specmatic.yaml"
}
},
"reportFormat": "CTRF",
"specVersion": "1.0.0",
"reportId": "017f5ea6-61bf-42d7-98a9-37019c58bbd0",
"timestamp": "2026-08-05T15:59:35.736+00:00",
"generatedBy": "Specmatic v2.51.1"
},
"specmaticStubUsage": null,
"specmaticCentralRepoReport": null,
"specmaticConfig": {
"version": 3,
"systemUnderTest": {
"service": {
"definitions": [
{
"definition": {
"source": {
"git": {
"url": "https://github.com/specmatic/specmatic-order-contracts.git"
}
},
"specs": [
"io/specmatic/examples/store/openapi/api_order_v5.yaml"
]
}
}
]
}
}
},
"metadata": {
"some-key": "some-value"
},
"publisher": {
"name": "io.specmatic.enterprise:executable",
"version": "1.23.0"
}
}

testIds link a covered operation to its CTRF test entries. An operation can have many test IDs in a real report; this example shows only one. When an operation runs using generated data or an example, its extra.input and extra.outputs send that request and response data. Authentication and session values in the example are redacted.

Mock report payload

mock --ci submits its CTRF mock-usage report. This example retains every field from the captured report, but shows one operation, one test ID, and one mock interaction. The aggregate counters retain their original values.

Request headers:

Authorization: license 9dcf94d1-c731-4b85-8a10-0fad31d87bcd
Content-Type: application/json
User-Agent: Specmatic-Reporter/io.specmatic.enterprise:executable:1.23.0 running on 17.0.19
X-Auth-Token: Bearer <token>

X-Auth-Token is present when build-submission two-factor authentication is configured with the command's --token <token> option or SPECMATIC_INSIGHTS_TOKEN.

{
"repo": "specmatic-order-bff-java",
"repoId": "636154288",
"repoUrl": "https://github.com/specmatic/specmatic-order-bff-java",
"branchName": "main",
"orgId": "c6e33dd8-af67-4658-8783-f83ad9eab021",
"createdAt": "2026-08-05T16:21:58.321184Z",
"projectDir": null,
"specmaticConfigPath": "./specmatic.yaml",
"specmaticCoverage": null,
"specmaticStubUsage": {
"results": {
"tool": {
"name": "Specmatic v2.51.1"
},
"summary": {
"tests": 36,
"passed": 36,
"failed": 0,
"skipped": 0,
"pending": 0,
"other": 0,
"start": 0,
"stop": 1785946913212,
"suites": 1,
"extra": {
"executionDetails": [
{
"protocol": "http",
"specType": "openapi",
"specification": "io/specmatic/examples/store/openapi/api_order_v5.yaml",
"type": "git",
"repository": "https://github.com/specmatic/specmatic-order-contracts",
"branch": "main",
"coverageMetrics": {
"apiCoverage": 6,
"absoluteCoverage": 6,
"coveredOperations": 2,
"totalOperationsWithFilters": 36,
"totalOperations": 36
},
"operations": [
{
"path": "/products/{id}",
"method": "DELETE",
"responseCode": 200,
"responseContentType": "text/plain",
"coverageStatus": "covered",
"eligibleForCoverage": true,
"omittedStatus": "NONE",
"testIds": [
"e394fe8c-e42e-440d-a8bf-ba2d93b4557a"
],
"metrics": {
"attempts": 1,
"matches": 1
},
"reasons": [],
"qualifiers": [],
"status": "covered"
}
]
}
]
}
},
"tests": [
{
"id": "e394fe8c-e42e-440d-a8bf-ba2d93b4557a",
"name": "Scenario: DELETE /products/(id:number) -> 200",
"status": "passed",
"duration": 0,
"suite": "io/specmatic/examples/store/openapi/api_order_v5.yaml",
"message": "",
"rawStatus": "Success",
"tags": [
"method:delete",
"status:200"
],
"type": "Mock",
"extra": {
"valid": true,
"wip": false,
"input": "DELETE /products/1234\nAccept: application/json, application/yaml, application/*+json\nContent-Type: application/json\nAuthenticate: <redacted>\nUser-Agent: Java/17.0.19\nHost: localhost:8090\nConnection: keep-alive\nContent-Length: 0\n\n",
"inputTime": 0,
"outputs": [
{
"title": "Response",
"content": "200 OK\nContent-Type: text/plain\nX-Specmatic-Result: success\nX-Specmatic-Type: random\n\nAWFDU",
"time": 0
}
],
"reasons": [],
"qualifiers": []
}
}
],
"environment": {
"repositoryName": "specmatic-order-contracts",
"repositoryUrl": "https://github.com/specmatic/specmatic-order-contracts.git"
},
"extra": {
"apiCoverage": "5%",
"absoluteCoverage": "5%",
"specmaticConfigPath": "./specmatic.yaml"
}
},
"reportFormat": "CTRF",
"specVersion": "1.0.0",
"reportId": "56dd5943-ba7d-4140-acca-d37dd571c0e4",
"timestamp": "2026-08-05T16:21:04.364+00:00",
"generatedBy": "Specmatic v2.51.1"
},
"specmaticCentralRepoReport": null,
"specmaticConfig": {
"version": 3,
"systemUnderTest": {
"service": {
"$ref": "#/components/services/productSearchBffService",
"runOptions": {
"$ref": "#/components/runOptions/productSearchBffServiceTest"
}
}
}
},
"metadata": {
"some-key": "some-value"
},
"publisher": {
"name": "io.specmatic.enterprise:executable",
"version": "1.23.0"
}
}

testIds link a covered operation to its CTRF mock entries. When mock interactions use generated data or examples, their extra.input and extra.outputs send that request and response data. Authentication values in the mocked request are redacted.

Backward compatibility report payload

After backward-compatibility-check, the compatibility result is sent in specmaticBccReport. This example retains every field from the captured report, but shows one operation, one test ID, and one incompatible test entry. The aggregate counters retain their original values.

Request headers:

Authorization: license 9dcf94d1-c731-4b85-8a10-0fad31d87bcd
Content-Type: application/json
User-Agent: Specmatic-Reporter/io.specmatic.enterprise:executable:1.23.0 running on 17.0.19
X-Auth-Token: Bearer <token>

X-Auth-Token is present when build-submission two-factor authentication is configured with the command's --token <token> option or SPECMATIC_INSIGHTS_TOKEN.

{
"repo": "specmatic-order-contracts",
"repoId": "378410509",
"repoUrl": "https://github.com/specmatic/specmatic-order-contracts",
"branchName": "main",
"orgId": "c6e33dd8-af67-4658-8783-f83ad9eab021",
"createdAt": "2026-08-05T16:39:18.838+00:00",
"projectDir": null,
"specmaticBccReport": {
"results": {
"tool": {
"name": "Specmatic v2.46.5"
},
"summary": {
"tests": 134,
"passed": 72,
"failed": 6,
"skipped": 0,
"pending": 0,
"other": 56,
"start": 1785947958838,
"stop": 1785947959389,
"suites": 2,
"extra": {
"executionDetails": [
{
"protocol": "http",
"specType": "openapi",
"specification": "io/specmatic/examples/store/openapi/api_products_v1.yaml",
"branch": "main",
"operations": [
{
"path": "/products/{id}",
"method": "PUT",
"contentType": "application/json",
"responseCode": 200,
"testIds": [
"bbe51550-b4f5-490e-923a-30ce336ea18b"
],
"responseContentType": "application/json",
"qualifiers": [
"changed"
],
"status": "incompatible"
}
]
}
]
}
},
"tests": [
{
"id": "bbe51550-b4f5-490e-923a-30ce336ea18b",
"name": "PUT /products/(id:number) -> 200 (requestContentType application/json, responseContentType application/json)",
"status": "failed",
"duration": 0,
"suite": "io/specmatic/examples/store/openapi/api_products_v1.yaml",
"message": "In scenario \"Update a product. Response: OK\"\nAPI: PUT /products/(id:number) -> 200\n\n >> REQUEST.BODY.name (io/specmatic/examples/store/openapi/api_products_v1.yaml:100:9)\n \n R1001: Type mismatch\n Documentation: https://docs.specmatic.io/rules#r1001\n Summary: The value type does not match the expected type defined in the specification\n \n This is type string in the new specification, but type null in the old specification",
"rawStatus": "incompatible",
"tags": [
"status:200",
"method:put",
"path:/products/{id}",
"content-type:application/json",
"response-content-type:application/json"
],
"type": "Backward Compatibility",
"extra": {
"wip": false,
"breakingChanges": [
{
"breadcrumb": "REQUEST.BODY.name",
"sourceLocations": [
{
"filePath": "io/specmatic/examples/store/openapi/api_products_v1.yaml",
"line": 100,
"column": 9
}
],
"rule": {
"id": "R1001",
"title": "Type mismatch",
"documentationUrl": "https://docs.specmatic.io/rules#r1001",
"summary": "The value type does not match the expected type defined in the specification"
},
"description": "This is type string in the new specification, but type null in the old specification",
"severity": "error"
}
]
}
}
],
"environment": {},
"extra": {
"specmaticConfigPath": "./specmatic.yaml",
"reportType": "BackwardCompatibility"
}
},
"reportFormat": "CTRF",
"specVersion": "1.0.0",
"reportId": "c447845e-9ad4-411e-bb3e-a393fc8b79c3",
"timestamp": "2026-08-05T16:39:18.838+00:00",
"generatedBy": "Specmatic v2.46.5"
},
"metadata": {
"some-key": "some-value"
},
"publisher": {
"name": "io.specmatic.enterprise:executable",
"version": "1.23.0"
}
}

testIds link an operation to its compatibility-test entries. breakingChanges identifies the contract differences found for a failing test.

Spec Metadata Report payload

For a central contract repository containing OpenAPI, AsyncAPI, GraphQL SDL, and Protobuf specifications, the JSON request body has the following shape:

Request headers:

Authorization: license 9dcf94d1-c731-4b85-8a10-0fad31d87bcd
Content-Type: application/json
User-Agent: Specmatic-Reporter/io.specmatic.enterprise:executable:1.23.0 running on 17.0.19
X-Auth-Token: Bearer <token>

X-Auth-Token is present when build-submission two-factor authentication is configured with the command's --token <token> option or SPECMATIC_INSIGHTS_TOKEN.

{
"repo": "specmatic-order-contracts",
"repoId": "378410509",
"repoUrl": "https://github.com/specmatic/specmatic-order-contracts",
"branchName": "main",
"orgId": "c6e33dd8-af67-4658-8783-f83ad9eab021",
"createdAt": "2026-08-05T15:40:04.194777Z",
"projectDir": null,
"specmaticConfigPath": null,
"specmaticCoverage": null,
"specmaticStubUsage": null,
"specmaticCentralRepoReport": [
{
"specification": "io/specmatic/examples/store/openapi/api_order_v1.yaml",
"operations": [
{
"path": "/products/{id}",
"method": "GET",
"responseCode": 200,
"responseContentType": "application/json"
}
],
"specType": "openapi"
},
{
"specification": "io/specmatic/examples/store/asyncapi/order_service_async_v1.yaml",
"operations": [
{
"operation": "placeOrder",
"channel": "place-order",
"action": "receive"
},
{
"operation": "processOrder",
"channel": "process-order",
"action": "send"
},
{
"operation": "notify",
"channel": "notification",
"action": "send"
}
],
"specType": "asyncapi"
},
{
"specification": "io/specmatic/examples/store/graphql/products_bff.graphqls",
"operations": [
{
"operationType": "query",
"operationName": "findAvailableProducts"
},
{
"operationType": "query",
"operationName": "findOffersForDate"
},
{
"operationType": "mutation",
"operationName": "createProduct"
}
],
"specType": "graphqlsdl"
},
{
"specification": "io/specmatic/examples/store/grpc/order_bff/order_bff.proto",
"operations": [
{
"packageName": "com.store.order.bff",
"service": "OrderService",
"rpc": "findAvailableProducts"
},
{
"packageName": "com.store.order.bff",
"service": "OrderService",
"rpc": "createOrder"
},
{
"packageName": "com.store.order.bff",
"service": "OrderService",
"rpc": "createProduct"
}
],
"specType": "protobuf"
}
],
"specmaticConfig": null,
"metadata": {
"some-key": "some-value"
},
"publisher": {
"name": "io.specmatic.enterprise:executable",
"version": "1.23.0"
}
}

specmaticCentralRepoReport contains one item per specification. The fields in each operations item depend on the specification type:

specTypeOperation fields
openapipath, method, responseCode, responseContentType
asyncapioperation, channel, action
graphqlsdloperationType, operationName
protobufpackageName, service, rpc

The --metadata key=value option adds the entry to metadata. Fields unrelated to a Spec Metadata Report, such as specmaticCoverage and specmaticStubUsage, are null.