Skip to main content

Service Virtualization

Mocking

Pre-requisites​

  • Create a directory named specmatic in your home directory.
  • Make sure you have installed Specmatic from Download page.

Inline Examples​

  • Create a file named employees.yaml in the specmatic directory with the following contents.

    employees.yaml
    openapi: 3.0.0
    info:
      title: Employees
      version: "1.0"
    servers: []
    paths:
      /employees:
        post:
          summary: ""
          requestBody:
            content:
              application/json:
                schema:
                  $ref: "#/components/schemas/Employee"
                examples:
                  CREATE_EMPLOYEE_SUCCESS:
                    value:
                      name: Jill Doe
                      department: Engineering
                      designation: Director
          responses:
            "201":
              description: Employee Created Response
              content:
                application/json:
                  schema:
                    type: object
                    properties:
                      id:
                        type: integer
                    required:
                      - id
                  examples:
                    CREATE_EMPLOYEE_SUCCESS:
                      value:
                        id: 10
    components:
      schemas:
        Employee:
          type: object
          required:
            - name
            - department
            - designation
          properties:
            name:
              type: string
            department:
              type: string
            designation:
              type: string

  • In the same directory, create a file named specmatic.yaml (or specmatic.json) with the following contents:

    specmatic.yaml
    version: 3
    dependencies:
      services:
        - service:
            description: Employee Service
            definitions:
              - definition:
                  source:
                    filesystem:
                      directory: .
                  specs:
                    - spec:
                        path: ./employees.yaml

  • Change into the specmatic directory in your terminal, and start the Specmatic mock server with the following command:

    java -jar specmatic.jar mock

  • In a new tab, run the following curl command:

    curl -X POST -H 'Content-Type: application/json' -d '{"name": "Jill Doe", "department": "Engineering", "designation": "Director"}' http://localhost:9000/employees

  • You should get the following response back:

    {
      "id": 10
    }

  • Specmatic ties the named example CREATE_EMPLOYEE_SUCCESS listed under the request parameters and the response sections of the OpenAPI spec to create a test.

  • Here's a detailed breakdown of the contract test:

    • Request: Specmatic uses the request body in the request example named CREATE_EMPLOYEE_SUCCESS to match the incoming HTTP request.
    • Response: Once matched, Specmatic looks for an example with same name (CREATE_EMPLOYEE_SUCCESS) under responses. In this case the response code happens to be 200, so that is the response that Specmatic will return.

Inline Examples for Responses without a body​

In addition to handling responses with a defined body, Specmatic also supports configuring inline examples for responses with no body. This is particularly useful in scenarios where an HTTP response is expected to return only a status code without any associated payload.

To configure a request that expects a response without a body, you can define an inline example within the requestBody of your OpenAPI specification without associating it with any specific response. Specmatic automatically ties such examples to responses with no body.

Example Usage​

Consider the following addition to the employees.yaml file, where we define an example for a request that triggers a response with no body:

employees.yaml
openapi: 3.0.0
info:
  title: Employees
  version: "1.0"
servers: []
paths:
  /employees:
    delete:
      summary: ""
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                id:
                  type: integer
              required:
                - id
            examples:
              DELETE_EMPLOYEE_SUCCESS:
                value:
                  id: 10
      responses:
        "204":
          description: No Content
components:
  schemas:
    Employee:
      type: object
      required:
        - name
        - department
        - designation
      properties:
        name:
          type: string
        department:
          type: string
        designation:
          type: string

In this example:

  • Request Example: We have defined an inline example named DELETE_EMPLOYEE_SUCCESS under the requestBody section for the DELETE /employees operation.
  • Response: Since the response for this operation has no body (indicated by the 204 No Content status code) and the inline example DELETE_EMPLOYEE_SUCCESS is not associated with any response, Specmatic will automatically tie this inline example to the response with no body.

If there are multiple such examples defined for a particular path, all of them will be tied to the corresponding response with no body. This allows you to cover various test scenarios, even for responses that do not return any payload.

Key Points:​

  • No Response Body Association: If an inline example is defined without associating it with a specific response, Specmatic assumes it to be relevant for responses with no body.
  • Automatic Binding: Specmatic will bind all such inline examples to the response with no body for the respective request.
  • Multiple Examples: When multiple such examples are defined, each will be considered for the response with no body, allowing comprehensive testing.

This capability enhances the flexibility and coverage of contract testing by ensuring that even scenarios involving responses with no body are thoroughly validated.

Externalizing Example Data​

info

This feature is deprecated, it is recommended that you use Specmatic Studio to manage and generate examples instead.

It may not always be possible to add examples inline in the OpenAPI specifications. And sometimes certain examples may not belong in the API specification. In such cases, we can add examples outside the spec in the form of JSON files.

Let's see how this is done.

  • Run the examples command:

    docker run -v "$(pwd):/usr/src/app" specmatic/enterprise examples generate employees.yaml

  • It generates a request-response mapping JSON file in the employees_examples directory containing an example of the API in the spec.

    • The directory name follows the format <spec file name without extension>_examples.
    • The example file name as generated indicates the operation it was generated from. In fact, the name is not limited to any format. You can choose a name that is meaningful to you.

  • Open the file, and you'll see this:

    {
        "http-request": {
            "path": "/employees",
            "method": "POST",
            "headers": {
                "Content-Type": "application/json"
            },
            "body": {
                "name": "Jill Doe",
                "department": "Engineering",
                "designation": "Director"
            }
        },
        "http-response": {
            "status": 201,
            "body": {
                "id": 479
            },
            "status-text": "Created",
            "headers": {
                "Content-Type": "application/json"
            }
        }
    }

  • Specmatic mock recognizes the _examples directory naming convention, and loads files in this directory automatically on startup.

Note that when the mock starts, it will log the count of examples loaded for a spec and any errors found. If you want to see the actual paths of the example files that were loaded, use the --debug flag.

  • Now let's update the request section to suit our needs, by modifying the value of name to "Jack Sprat", and the value of department to "Sales". And in the response, let us modify the value of id to 20, like so:

    {
        "http-request": {
            "path": "/employees",
            "method": "POST",

            "headers": {
                "Content-Type": "application/json"
            },

            "body": {
                "name": "Jack Sprat",
                "department": "Sales",
                "designation": "Director"
            }
        },
        "http-response": {
            "status": 201,

            "body": {
                "id": 20
            },

            "headers": {
                "Content-Type": "application/json"
            },

            "status-text": "Created"
        }
    }

  • Kill the mock (Ctrl+C) if it is running, and start it again.

    • You will see that file name is mentioned in the startup logs.

  • In a new tab, run the following curl command:

    curl -X POST -H 'Content-Type: application/json' -d '{"name": "Jack Sprat", "department": "Sales", "designation": "Director"}' http://localhost:9000/employees

  • Specmatic mock will response with this payload:

    {
      "id": 20
    }

Handling No Response Body APIs​

Specmatic also supports externalizing examples for APIs that return no response body. To handle this:

  • Create an example file with the http-request section, but omit the http-response body.
  • Specmatic will automatically associate this example with the corresponding response that has no body.

For example:

{
    "http-request": {
        "path": "/employees",
        "method": "DELETE",
        "headers": {
            "Content-Type": "application/json"
        },
        "body": {
            "id": 10
        }
    },
    "http-response": {
        "status": 204,
        "status-text": "No Content",
        "headers": {
            "Content-Type": "application/json"
        }
    }
}

When this file is placed in the employees_examples directory and the mock is restarted, the Specmatic mock will respond appropriately to the DELETE request with a 204 No Content status, confirming that the externalized example is correctly tied to a response with no body.

Note: You may add more example files into the employees_examples directory. There is no limit to the number of example files that can be added.

Intelligent Service Virtualization - Example cannot go out of sync​

An important and unique feature of Specmatic is that it constantly validates both inline and external examples against the specification to make sure they are always aligned.

Let's try this feature out.

Create a file named out-of-sync.json with the following contents:

{
  "http-request": {
    "path": "/employees",
    "method": "POST",

    "headers": {
      "Content-Type": "application/json"
    },

    "body": {
      "name": "Janet",
      "department": "Sales",
      "designation": "Director"
    }
  },
  "http-response": {
    "status": 201,

    "body": {
      "id": "abc123"
    },

    "headers": {
      "Content-Type": "application/json"
    },

    "status-text": "Created"
  }
}

  • Note: the value of id in the response is a string in the example, but an integer in the specification.

  • Now when you start the mock server, you'll see the following error message in the log:

    In scenario ". Response: Employee Created Response"
    API: POST /employees -> 201

      >> REQUEST.BODY.department

          Contract expected string but mock contained 10 (number)

  • Specmatic's mock rejected the example because of the mismatch. Let us ascertain the same by running the following curl command:

    curl -X POST -H 'Content-Type: application/json' -d '{"name": "Janet", "department": "Sales", "designation": "Director"}' http://localhost:9000/employees

  • You'll get this response:

    {
        "id": 783
    }

  • As you can see the response contains a generated value for the id, rather than "abc123", which was in the invalid example.

  • Since the invalid example was rejected by Specmatic mock, the response is a value generated based on the response in the specification.

Strict Mode​

Suppose you do not wish Specmatic to return an auto-generated response when there are no matching examples. You can run Specmatic mock server in strict mode.

Let's try this out.

  • Start Specmatic mock with the --strict flag, using the following command:

    java -jar specmatic.jar mock --strict

  • Now run the following curl command:

    curl -X POST -H 'Content-Type: application/json' -d '{"name": "Janet", "department": "Sales", "designation": "Director"}' http://localhost:9000/employees

  • Specmatic will return a 400, with a detailed error message.

To recap, in strict mode, Specmatic will only respond to requests that have matching examples, inline or external.

You can enable strict mode using configuration as described here.

Data Type-Based Examples​

Sometimes the exact value may not be that important in an example.

For example, suppose that in the request we expect, the important values to be matched against department and designation. name must be present, but we don't particularly care about the exact value.

Let's see how we can formulate an example that meets these requirements.

  • Create a new file in the employees_examples directory named any_name.json with the following contents:

    employees_example/any_name.json
    {
      "http-request": {
        "path": "/employees",
        "method": "POST",

        "headers": {
          "Content-Type": "application/json"
        },

        "body": {
          "name": "(string)",
          "department": "Sales",
          "designation": "Director"
        }
      },
      "http-response": {
        "status": 201,

        "body": {
          "id": 30
        },

        "headers": {
          "Content-Type": "application/json"
        },

        "status-text": "Created"
      }
    }

  • Start the Specmatic mock and execute the following command, which has the name "Janet":

    curl -X POST -H 'Content-Type: application/json' -d '{"name": "Janet", "department": "Engineering", "designation": "Director"}' http://localhost:9000/employees

  • You'll get this response:

    {
        "id": 30
    }

  • Modify the name to "Justin", or any other name, and you'll get the same response.

    • This is because Specmatic just checks that the name in your curl request is a string. It doesn't care about the exact value.

  • But modify the value of department in the curl command to "Facilities".

    • This time round, the request does not match the example, because the values of department in the example and the request are different ("Sales" and "Engineering" respectively).
    • Since no example available matches the incoming request, a response is generated based on the specification and returned.

Plain Text Request Bodies - Examples With Regular Expressions​

For specifications where the request body is a string, you may find it helpful to use a regular expression in the example.

Let's try this out.

  1. Create a new specification file named phonebook.yaml. This defines a PhoneBook API with a single POST /search endpoint that accepts a plain-text name string in the request body and returns a 200 OK JSON array of results, each containing required name and number fields.

    phonebook.yaml
    openapi: 3.0.0
    info:
      title: PhoneBook API
      version: "1.0"
    paths:
      /search:
        post:
          summary: Search By Name
          requestBody:
            required: true
            content:
              text/plain:
                schema:
                  type: string
                  description: The name to be searched for
          responses:
            "200":
              description: OK
              content:
                application/json:
                  schema:
                    type: array
                    items:
                      type: object
                      required:
                        - name
                        - number
                      properties:
                        name:
                          type: string
                          description: The name of the person
                        number:
                          type: string
                          description: The phone number of the person

  2. Now, create a file named phonebook_examples/search_john.json with the following contents:

    phonebook_examples/search_john.json
    {
      "http-request": {
        "path": "/search",
        "method": "POST",
        "headers": {
          "Content-Type": "text/plain"
        },
        "body": "(string)",
        "bodyRegex": "^[Jj][Oo][Hh][Nn].*$"
      },
      "http-response": {
        "status": 200,
        "body": [
          {
            "name": "John Doe",
            "number": "123-456-7890"
          },
          {
            "name": "John Smith",
            "number": "987-654-3210"
          },
          {
            "name": "John Johnson",
            "number": "555-555-5555"
          }
        ],
        "headers": {
          "Content-Type": "application/json"
        },
        "status-text": "OK"
      }
    }

  3. Add phonebook.yaml as a dependency in specmatic.yaml, like so:

    specmatic.yaml
    version: 3
    components:
      sources:
        contractRepository:
          filesystem:
            directory: .
    dependencies:
      services:
        - service:
            description: Phone book
            definitions:
              - definition:
                  source:
                    $ref: "#/components/sources/contractRepository"
                  specs:
                    - spec:
                        path: phonebook.yaml
                    - spec:
                        path: employees.yaml

  4. Run the following curl command:

    curl -X POST -H "Content-Type: text/plain" -d "John" http://localhost:9000/search

  • You'll get this response:

    [
      {
        "name": "John Doe",
        "number": "123-456-7890"
      },
      {
        "name": "John Smith",
        "number": "987-654-3210"
      },
      {
        "name": "John Johnson",
        "number": "555-555-5555"
      }
    ]

  • If the bodyRegex matches the incoming request body, that example is utilized. Otherwise, a random response is generated.

  • Try modifying the last name in the curl request. As long as the request begins with John case-insensitive, you will receive the same response as defined by the regex in the bodyRegex in the example.

Correlated Request And Response Values​

Sometimes, your application may require the request and response to be coordinated coherently. This coordination may take two different forms which we shall explore below.

In preparation, let's update our employees.yaml specification with the following content:

openapi: 3.0.0
info:
  title: Employees
  version: '1.0'
servers: []
paths:
  /employees:
    patch:
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Employee'
      responses:
        '200':
          description: Employee Updated Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EmployeeResponse'
components:
  schemas:
    Employee:
      type: object
      required:
        - employeeCode
      properties:
        employeeCode:
          type: string
        name:
          type: string
        department:
          type: string
        designation:
          type: string
    EmployeeResponse:
      allOf:
        - $ref: '#/components/schemas/Employee'
        - type: object
          required:
            - id
            - employeeCode
          properties:
            id:
              type: integer
            employeeCode:
              type: string

Since we have changed the spec, remove all the examples in employees_examples, as they will now be invalid.

Direct Substitution - Copying Values From Request To Response​

  • In the employees_examples directory, add a file named direct_substitution.json with the following content:

    {
        "http-request": {
            "method": "PATCH",
            "path": "/employees",
            "body": {
                "name": "Jake",
                "employeeCode": "(EMPLOYEE_CODE:string)"
            }
        },
        "http-response": {
            "status": 200,
            "body": {
                "id": 10,
                "name": "Jake",
                "employeeCode": "$(EMPLOYEE_CODE)",
                "department": "Sales",
                "designation": "Associate"
            }
        }
    }

  • Execute the following curl command:

    curl -X PATCH -H 'Content-Type: application/json' -d '{"name": "Jake", "employeeCode": "abc123"}' http://localhost:9000/employees

  • You'll get this response:

    {
        "id": 10,
        "name": "Jake",
        "employeeCode": "abc123",
        "department": "Sales",
        "designation": "Associate"
    }

  • Here's what's happening.

    • The value of employeeCode in the incoming curl request is stored to a variable named EMPLOYEE_CODE, due to "employeeCode": "(EMPLOYEE_CODE:string)".
    • Then the value stored in EMPLOYEE_CODE is used as the value of employeeCode in the response, due to "employeeCode": "$(EMPLOYEE_CODE)".

  • Try the same curl command again, but change the employeeCode. You'll find that any employeeCode sent in the request will be reflected in the response.

Data Lookup​

Sometimes, the values of multiple response keys which have very different values need to be coordinated with a single request value, and for this the data look feature is quite useful.

Let's try this out.

  • Add a new file into the employees_examples directory named data_lookup.json with the following contents:

    {
        "http-request": {
            "method": "PATCH",
            "path": "/employees",
            "body": {
                "employeeCode": "(EMPLOYEE_CODE:string)",
                "department": "(DEPARTMENT:string)"
            }
        },
        "http-response": {
            "status": 200,
            "body": {
                "id": 10,
                "employeeCode": "$(EMPLOYEE_CODE)",
                "name": "Jack",
                "department": "$(DEPARTMENT)",
                "designation": "$(dataLookup.departments[DEPARTMENT].desig)"
            }
        },
        "dataLookup": {
            "departments": {
                "Sales": {
                    "desig": "Associate"
                },
                "Engineering": {
                    "desig": "Manager"
                }
            }
        }
    }

  • Observe the value of designation in the response. Specmatic will resolve this expression at runtime on receiving a request that matches the request in this example.

  • First let's see it in action. Execute this curl command:

    curl -X PATCH -H 'Content-Type: application/json' -d '{"department": "Sales", "employeeCode": "abc123"}' http://localhost:9000/employees

  • You'll get the following response, in which the value of designation is "Associate", as per the lookup.

    {
        "id": 10,
        "name": "Jack",
        "employeeCode": "abc123",
        "department": "Sales",
        "designation": "Associate"
    }

  • Note: The curl request matched the example request, so Specmatic loaded the corresponding response in the example, evaluated the expression $(dataLookup.departments[DEPARTMENT].desig) to get value of designation (which turned out to be "Associate"), evaluated the direct substitution expressions to get the values of employeeCode and department, and returned this response to the caller.

  • Specmatic evaluates the expression $(dataLookup.departments[DEPARTMENT].desig) from the example response using the following steps:

    • Access the dataLookup object (you'll find it in the example JSON).
    • Retrieve the departments collection from the dataLookup object.
    • Select the department corresponding to the value of the variable DEPARTMENT from the departments collection.
      • In this case, the value of DEPARTMENT is set to "Sales" (see the section on Direct Substitution above if you don't understand how that came about).
    • Access the desig property of the selected department.

  • It's the value of the desig property that gets used as the value of designation in the response.

  • The format of these expressions is fixed: top-level-object.collection[LOOKUP_VARIABLE_NAME].property

    • The names themselves, such as dataLookup, departments, desig are not restricted to any format. You may use names that are meaningful to you.

  • Next execute this curl command:

    curl -X PATCH -H 'Content-Type: application/json' -d '{"department": "Engineering"}' http://localhost:9000/employees

  • You will get the following response, in which the value of designation is "Manager", as per the lookup.

    {
      "id": 10,
      "name": "Jack",
      "employeeCode": "abc123",
      "department": "Engineering",
      "designation": "Manager"
    }

  • You can use as many collections and properties as you like in the dataLookup data object.

Partial Examples​

This feature lets you focus on the keys that need, allowing Specmatic to fill in the missing details.

Partial Request Examples​

For example, suppose I want to set up a response for all requests with name set to "George". I don't care about the specific values of name, employeeCode (which is mandatory), department or designation in the request.

  • Create a file in employees_examples named partial.json, with the following contents:

    {
        "partial": {
            "http-request": {
                "method": "PATCH",
                "path": "/employees",
                "body": {
                    "name": "George"
                }
            },
            "http-response": {
                "status": 200,
                "body": {
                    "id": 10,
                    "employeeCode": "abc123",
                    "name": "George",
                    "department": "Sales",
                    "designation": "Manager"
                }
            }
        }
    }

  • Note: In the request, the example only contains name. But because this is a partial example, Specmatic knows that employeeCode is mandatory and will validate the incoming request accordingly, despite it being missing in the example.

  • Start the mock, and execute the following curl command:

    curl -X PATCH -H 'Content-Type: application/json' -d '{"name": "George", "employeeCode": "abc12dd3"}' http://localhost:9000/employees

  • You'll get this response:

    {
        "id": 10,
        "employeeCode": "abc123",
        "name": "George",
        "department": "Sales",
        "designation": "Manager"
    }

  • Try the same but vary status code. As long as the name is "George", you will get the above response.

Partial Response Examples​

The same idea extends to the response.

  • Replace the content of employees_examples/partial.json with the following contents:

    {
        "partial": {
            "http-request": {
                "method": "PATCH",
                "path": "/employees",
                "body": {
                    "name": "George"
                }
            },
            "http-response": {
                "status": 200,
                "body": {
                    "name": "George",
                    "department": "Sales",
                    "designation": "Manager"
                }
            }
        }
    }

  • Note that the two mandatory keys in the response named id and employeeCode are omitted from the example.

  • But because it's a partial example, when you execute the previous curl command, Specmatic will autogenerate values for id and employeeCode from their schemas in the specification.

Use Meaningful Response Values From An External Dictionary​

When Specmatic's mock receives a request and finds no matching examples for it, Specmatic returns a response generated from the response schema in the API specification. While the generated response is schema-valid, it will not have meaningful values drawn from the context of your business domain. So in order to get domain-relevant responses when there examples, you can provide a dictionary of sample values to Specmatic. Specmatic will look up this dictionary when it needs to generate domain-relevant examples.

Refer to Dictionary with service virtualization for more details.

Delay Simulation​

Specmatic allows granular control over simulating a slow response for certain requests.

Example Specific Delay​

Let us create an expectation file in the products-api_examples folder and call it expectation-with-delay.json with below content.

{
  "http-request": {
    "method": "GET",
    "path": "/products/11"
  },
  "http-response": {
    "status": 200,
    "body": {
      "name": "Slow Soap",
      "sku": "slow1234"
    }
  },
  "delay-in-milliseconds": 3000
}

We have set the delay to 3 seconds here. Once the Specmatic mock server has loaded this expectation, time the request for product id 11, and you should see a 3-second delay.

% time curl http://localhost:9000/products/11
{
    "name": "Slow Soap",
    "sku": "slow1234"
}

curl http://localhost:9000/products/11  0.01s user 0.01s system 0% cpu 3.082 total

All other requests, other than the specific request (product id 11) where a delay has been set up, will have either no delay or fallback to global delay.

Global Delay​

A Global delay can be applied to all requests handled by service virtualization. By configuring the delayInMilliseconds parameter in Specmatic Config, you can simulate response times with the specified delay in milliseconds.

specmatic.yaml
version: 3
dependencies:
  services:
    - service:
        $ref: "#/components/services/apiOrderV3"
  settings:
    delayInMilliseconds: 10000
components:
  sources:
    specmaticOrderContracts:
      git:
        url: https://github.com/specmatic/specmatic-order-contracts.git
  services:
    apiOrderV3:
      description: API Order V3
      definitions:
        - definition:
            source:
              $ref: "#/components/sources/specmaticOrderContracts"
            specs:
              - io/specmatic/examples/store/openapi/api_order_v3.yaml

Note: If the delay is specified in the example file, it will be used to simulate response times for that specific example.

Otherwise, the global delay will be applied.

SSL / HTTPS Mocking​

There are multiple ways to run the Specmatic Mock with SSL.

Auto-Generated Cert Store​

This is the quickest approach.

java -jar specmatic.jar mock --httpsKeyStoreDir=path/to/keystore-dir --port=443 product-api.yaml

This will create a specmatic.jks file in the dir that you mentioned above, and you can now access the mock over https.

Bring Your Own Key Store​

If you already have a keystore and self-signed certificate you can pass it to Specmatic through below command options.

% specmatic mock --help
...
      --httpsKeyStore=<keyStoreFile>
                             Run the proxy on https using a key in this store
      --httpsKeyStorePassword=<keyStorePassword>
                             Run the proxy on https, password for pre-existing
                               key store
      --httpsPassword=<keyPassword>
                             Key password if any

Incoming mTLS (client certificate required by mock)​

Use this when your consumer should call the Specmatic mock with a client certificate.

How it works​

  • Configure HTTPS key material for the mock and set mtlsEnabled: true in the same certificate block.
  • Requests without a client certificate are rejected.
  • Requests with a client certificate are accepted and logged as (mTLS).
  • mtlsEnabled is applied per mock host/port configuration.

Configuration example​

specmatic.yaml
version: 3
dependencies:
  services:
    - service:
        definitions:
          - definition:
              source:
                filesystem:
                  directory: ./specs
              specs:
                - <path-to-openapi-or-wsdl-file>
        runOptions:
          openapi:
            baseUrl: https://<mock-host>:<mock-port>
            cert:
              mtlsEnabled: true
              keyStore:
                file: <path-to-jks-file>
                alias: <keystore-alias>
                password: <key-password>
              keyStorePassword: <keystore-password>

Note: Incoming mTLS requires HTTPS key material. If mtlsEnabled: true is set without a keystore, mock startup fails.

Dynamic Expectations​

(a.k.a. Dynamic Mocking) - Setting Expectations Over Specmatic Http API

Context​

It is not always possible to know ahead of time what expectation data needs to be setup. Example: Consider below scenario

  • Let us say our system under test needs to look up the SKU value from another service (over which we do not have control or cannot mock) before creating products
  • In this scenario we will be unable to create the expectation json file with any specific value of SKU since we will only know this at test runtime / dynamically

Dynamic Mocks are helpful in such scenarios which involve a work flow with multiple steps where the input of a step depends on output of its previous step.

Expectations Http Endpoint​

Specmatic mock server can accept expectation json through below http endpoint.

http://localhost:9000/_specmatic/expectations

Please see postman collection for reference.

Specmatic will verify these expectations against the OpenAPI Specifications and will only return a 2xx response if they are as per API Specifications. Specmatic returns 4xx response if the expectation json is not as per the OpenAPI Specifications.

Anatomy of a Component / API Test​

Anatomy of a Component / API Test

Please see this video for reference.

The above image shows how Specmatic Smart Mocking fits into your Component Test. A good component test isolates the system / component under test from its dependencies. Here Specmatic is emulating the dependencies of the mobile application thereby isolating it.

API Tests are just Component Tests where the System Under Test is a Service / API. Here is an example of how you can leverage Specmatic dynamic mocking in an API Test. Below are the pieces involved.

Let us analyse each phase of this API test.

  • Arrange - In this step, we set up Specmatic mock server with expectation json through Specmatic http endpoint to emulate the Products API to return expected response. We also verify that Specmatic has accepted this expectation data by asserting that the response code is 2xx. This confirms that are our expectation data is in line with the OpenAPI Specification of Products OpenAPI Specification.
  • Act - Here the test invokes System Under Test to exercise the functionality we need to test. This inturn results in the System Under Test invoking its dependency (Products Service) which is being emulated by Specmatic. Specmatic returns the response we have set up in the previous step to the System Under Test. System Under Test processes this data and responds to API Test.
  • Assert - We now verify the response from System Under Test to ascertain if it has returned the correct response.

Programmatically Starting Mock Server Within Tests​

If your tests are written in a JVM based language, you can start and stop the mock server within your tests programmatically.

Add specmatic-core jar dependency with scope set to test since this need not be shipped as part of your production deliverable.

<dependency>
  <groupId>io.specmatic</groupId>
  <artifactId>specmatic-core</artifactId>
  <version>2.42.2</version>
  <scope>test</scope>
</dependency>

Now you can import the utility to create the mock server. Below code snippets are in Kotlin. However, the overall concept is the same across all JVM languages such as Clojure, Scala or plain Java.

This utility can now be used in your test setup / beforeAll method to start the mock server. Specmatic automatically looks for your Specmatic configuration file in project root directory / classpath to locate your API Specification files that need to run as part of the mock server.

com/example/YourTest.kt
import io.specmatic.stub.ContractStub
import io.specmatic.test.SpecmaticContractTest
import org.junit.jupiter.api.BeforeAll

class YourTest: SpecmaticContractTest
  companion object {
    private val mock: ContractStub
    @BeforeAll
    @JvmStatic
    fun setUp() {
        mock = createStub()
    }
  }
}

We can now programmatically set dynamic expectations on the mock with the setExpectation(expectedJson) method where expecationJson is in the same format as seen here

mock.setExpectation(expectationJson);

If you have several such JSON expectation files that you would like to set up at once, you can pass a list of files or dir containing these expectation JSON files while creating the mock.

httpMock = createStub(listOf("./src/test/resources"))

The above createStub() function creates your Specmatic HTTP mock with default host, port, etc. Below is an example with all values being passed in

private val mock: ContractStub
@BeforeAll
@JvmStatic
fun setUp() {
    mock = createStub(listOf("./src/test/resources"), "localhost", 8090, strict = true)
}

The last parameter (strict = true), enables strict mode where Specmatic HTTP mock will only respond to requests where expectations have been set. For any other requests, 400 Bad Request is returned.

And subsequently once your tests are done, you can shut down the mock server as part of your teardown / afterAll method.

@AfterAll
@JvmStatic
fun tearDown() {
    service?.stop()
    mock.close()
}

Here is a complete example of Specmatic Contract Tests that leverages the above technique.

Kotlin Example

Please note that this is only a utility for the purpose of convenience in Java projects. Other programming languages can simply run the Specmatic standalone executable just as easily. If you do happen to write a thin wrapper and would like to contribute the same to the project, please refer to our contribution guidelines.

Transient Expectations (a.k.a. Transient Mocks)​

A transient mock becomes unavailable immediately after it has been exercised.

Setting transient expectations​

For example, create this mock file for products-api.yaml contract:

{
  "http-stub-id": "123",
  "http-request": {
    "method": "GET",
    "path": "/storestatus",
  },
  "http-response": {
    "status": 200,
    "body": "open"
  }
}

Send a GET request to http://localhost:9000/storestatus. The response will be "open".

Now try the same request again. The response is a randomized string.

This is useful particularly when there are no distinguishing features of the request like in the above example, and we need to simulate a succession of calls to that API giving different responses.

Clearing Transient Expectations​

If the test fails and you need to start a new run of the test, you may need to clear all the transient mocks so that the two tests do not step on each other's toes.

To do that, send a DELETE request to /_specmatic/http-stub/${http-stub-token}.

To clear the transient mock in the above example, you would call http://localhost:9000/_specmatic/http-stub/123 with the DELETE verb.

Request Matchers​

Use matchers in request examples when you want the mock to accept dynamic values instead of a single fixed example value.

In service virtualization, the request is validated against the specification first, and then matcher evaluation decides whether an example matches the incoming request.

The full matcher reference, including transient example behavior, times, matcher exhaustion, and the complete list of supported matchers, is documented in Matchers.

Externalised Response Generation​

There may be circumstances where we need to compute the response or part of it based on the request in the expectation. Here is an example.

openapi: 3.0.1
info:
  title: New Feature
  version: "1"
paths:
  /triple/{value}:
    get:
      summary: three times
      parameters:
        - name: value
          in: path
          required: true
          schema:
            type: number
      responses:
        "200":
          description: three times
          content:
            application/json:
              schema:
                type: number

This OpenAPI specification expects given input to be multiplied by three. It may not be possible to create expectation for each individual number. In this can we can create an expectation that can call an external command to which it can pass the incoming request and then return the value generated by that external command.

{
    "http-request": {
        "method": "GET",
        "path": "/triple/(value:number)"
    },
    "http-response": {
        "status": 200,
        "body": "10",
        "externalisedResponseCommand": "./path/to/response.sh"
    }
}

In the above expectation file since we are providing the externalisedResponseCommand, Specmatic will ignore the data inside http-reponse body. Instead, it calls the command (response.sh) that is mentioned in externalisedResponseCommand and pass the incoming request as an environment variable SPECMATIC_REQUEST.

#!/bin/bash

value=${SPECMATIC_REQUEST:20:1}

cat << EOF
{
    "status": 200,
    "body": $((value * 3)),
    "status-text": "OK",
    "headers": {
        "X-Specmatic-Result": "success",
        "Content-Type": "text/plain"
    }
}
EOF

The above shell script is just an example, the external command can be any executable / script / program which can read an environment variable. The example shell script here is reading the path parameter and multiplying it by three. The response of this script / command is returned as the response to the request.

Adapters​

Sometimes the consumer sends a few parameters extra that the API doesn't handle directly, for example due to an API gateway. In such cases, the consumer's view of the API is slightly but meaningfully different from the actual API implementation. Hence the consumer needs to apply minor adaptations to the API specification at runtime, to be able to mock it out. This can be done using Overlays, an OpenAPI standard format for declaring modifications to other specifications. Read more about Specmatic's support for Overlays here.

Specmatic also supports adapters for runtime request/response transformation. Read more about them here.​

Precedence Across Types Of Examples​

There are now several ways in which to provide expectations.

  1. Transient expectations
  2. Dynamic expectations
  3. Expectations from externalized examples
  4. Expectations from examples in the specification

They are resolved in the order in which they appear above.

This means, if a request matches an example in the specification, but also matches a dynamic expectation, the response will be served from the dynamic expectations. Put differently, dynamic expectations override expectations from examples.

Checking Health Status Of Mock Server​

You can use the /actuator/health endpoint to verify if the mock server is operational. To do this, send a GET request to this endpoint using Postman or a curl command.

The response will provide the current health status of the mock server, indicating whether it is ready to handle requests. This allows you to confirm that the mock server is up before routing any traffic through it.

Example curl Request:​

curl -X GET http://<stub_server_url>/actuator/health
# Example successful response:
# {
#   "status": "UP"
# }

Here's the OpenAPI specification describing the /actuator/health endpoint.

specmatic.yaml
openapi: 3.0.3
info:
  title: Health Check API
  description: API for checking the health status of the mock server.
  version: 1.0.0
paths:
  /actuator/health:
    get:
      description: Returns the health status of the mock server.
      responses:
        "200":
          description: Health status of the mock server.
          content:
            application/json:
              schema:
                type: object
                properties:
                  status:
                    type: string
                    enum:
                      - UP
                    example: UP

Retrieving Messages Served By The Mock Server​

You can use the /_specmatic/messages endpoint to retrieve the list of all messages served by the mock server. This is useful for debugging, verifying interactions, and asserting that specific traffic actually hit the mock.

What can be filtered​

If you call the endpoint without any query parameters, it will return everything that the mock server has served so far.

You can also apply one or more filters by passing query parameters in the form key=value, Each query parameter acts as an equality check, A filter key is a path-like selector that points to a value inside the each message.

REQUEST​

  • REQUEST.PATH (URL path)
  • REQUEST.METHOD
  • REQUEST.PARAMETERS.PATH.<paramName>
  • REQUEST.PARAMETERS.QUERY.<paramName>
  • REQUEST.PARAMETERS.HEADER.<paramName>
  • REQUEST.BODY.<pointerToField>

RESPONSE​

  • RESPONSE.STATUS
  • RESPONSE.HEADER.<paramName>
  • RESPONSE.BODY.<pointerToField>

Note: When the target is inside an array, use [index] Example: RESPONSE.BODY[1].details[0][1].name

Example curl Requests​

Fetch ALL served messages:

curl -X GET http://<stub_server_url>/_specmatic/messages

Fetch only messages where the request method is GET:

curl -G -X GET "http://<stub_server_url>/_specmatic/messages" --data-urlencode "REQUEST.METHOD=GET"

Fetch only messages where response status is 200:

curl -G -X GET "http://<stub_server_url>/_specmatic/messages" --data-urlencode "RESPONSE.STATUS=200"

Fetch only messages where a request header matches:

curl -G -X GET "http://<stub_server_url>/_specmatic/messages" --data-urlencode "REQUEST.HEADER.Accept=application/json"

Fetch only messages where a nested response body field matches:

curl -G -X GET "http://<stub_server_url>/_specmatic/messages" --data-urlencode "RESPONSE.BODY[1].details[0][1].name=John"

Provide multiple filters (N query params). All filters apply together:

curl -G -X GET "http://<stub_server_url>/_specmatic/messages" \
  --data-urlencode "REQUEST.METHOD=POST" \
  --data-urlencode "RESPONSE.STATUS=201"

Specmatic Configuration with Base URL, Host, Port, and Path​

By default, Specmatic mock servers run on http://0.0.0.0:9000

However, you can customize the endpoint by specifying the host, port or baseUrl, according to your requirements in the Specmatic Config.

  • Specifying only host or port will autofill missing parts from the default base URL.
  • If baseUrl is specified, it will be used directly.

Specifying Host​

With the following configuration, the mock server will run on 127.0.0.1 rather than the default host 0.0.0.0

specmatic.yaml
version: 3
components:
  sources:
    contractRepository:
      git:
        url: https://github.com/your-username/your-repo.git
  runOptions:
    productsMock:
      openapi:
        type: mock
        host: 127.0.0.1
dependencies:
  services:
    - service:
        description: Employee Service
        definitions:
          - definition:
              source:
                $ref: "#/components/sources/contractRepository"
              specs:
                - spec:
                    path: products.yaml

Specifying Port​

With the following configuration, the mock server will run on port 5000 (http://0.0.0.0:5000) rather than the default port 9000

specmatic.yaml
version: 3
components:
  sources:
    contractRepository:
      git:
        url: https://github.com/your-username/your-repo.git
  runOptions:
    productsMock:
      openapi:
        type: mock
        port: 5000
dependencies:
  services:
    - service:
        description: Employee Service
        definitions:
          - definition:
              source:
                $ref: "#/components/sources/contractRepository"
              specs:
                - spec:
                    path: products.yaml

Specifying Base URL​

With the following configuration, the mock server will run on http://localhost:1234/api/v2 rather than the default url http://0.0.0.0:9000

specmatic.yaml
version: 3
components:
  sources:
    contractRepository:
      git:
        url: https://github.com/your-username/your-repo.git
  runOptions:
    productsMock:
      openapi:
        type: mock
        baseUrl: http://localhost:1234/api/v2
dependencies:
  services:
    - service:
        description: Employee Service
        definitions:
          - definition:
              source:
                $ref: "#/components/sources/contractRepository"
              specs:
                - spec:
                    path: products.yaml

Note: You can either provide a baseUrl or specify one of or a combination of host, port, and basePath. They are mutually exclusive and cannot be used together for the same mock configuration.

Customizing the Default Base URL​

The default base URL can also be customized:

  • Through command-line arguments using --host and --port arguments

Running multiple Specmatic Mocks on different host/port Combinations​

Overview​

This setup demonstrates how to run Specmatic stubs on different host/port combinations This allows serving different APIs on their respective baseURLs while keeping their examples specific to each specification.

Note: This is not limited to baseUrl but can be extended to host, port, and basePath as well.

specmatic.yaml
version: 3
systemUnderTest:
  service:
    $ref: "#/components/services/employeeService"
    runOptions:
      $ref: "#/components/runOptions/providerApis"
components:
  sources:
    centralContractRepo:
      git:
        url: https://github.com/your-username-or-org/your-repo.git
  services:
    employeeService:
      description: Employee Service
      definitions:
        - definition:
            source:
              $ref: "#/components/sources/centralContractRepo"
            specs:
              - spec:
                  id: employeeSpec
                  path: path/to/employees.yaml
              - spec:
                  id: payrollSpec
                  path: path/to/payroll.yaml
  runOptions:
    providerApis:
      openapi:
        type: mock
        specs:
          - spec:
              id: employeeSpec
              port: 5000
          - spec:
              id: payrollSpec
              baseUrl: http://localhost:8081/payroll

The above configuration sets up two stubs:

  1. The first stub serves the employeeSpec specification on port 5000 (final endpoint: http://0.0.0.0:5000)
  2. The second stub serves the payrollSpec specification with the complete baseUrl http://localhost:8081/payroll

Benefits​

  • BaseURL-based segregation: Each spec runs on a dedicated baseURL, ensuring clear separation.
  • Spec-specific examples: Requests return expected responses per specification.
  • Flexibility: Allows hosting multiple versions or separate APIs without conflict.

This setup enables serving and testing multiple specifications efficiently using Specmatic.

Disable hot-reload​

Specmatic mock server monitors your specification files and examples for changes. When a file is modified, the mock server automatically restarts to reflect the changes without requiring manual intervention.

However, you may need to switch off this at times. For example, in some higher environment where the mock runs, the number of files being watched may exceed the value of /proc/sys/fs/inotify/max_user_watches, and you may not have access to the environment's config/setup to change this value. In such a case, the ability to disable hot-reload provides a pragmatic path forward.

You can use a command-line switch: java -jar specmatic.jar mock --hot-reload=disabled <your-spec.yaml>.

You can also disable it via the following configuration in specmatic.yaml:

specmatic.yaml
version: 3
components:
  sources:
    contractRepository:
      git:
        url: https://github.com/your-username/your-repo.git
  runOptions:
    productsMock:
      openapi:
        type: mock
        baseUrl: null
dependencies:
  services:
    - service:
        description: Employee Service
        definitions:
          - definition:
              source:
                $ref: "#/components/sources/contractRepository"
              specs:
                - spec:
                    path: products.yaml
  settings:
    hotReload: false

Using matching branches in the central contract repo​

Specmatic can be asked to use a branch of the central repo with the same name as the branch that is locally checked out.

This feature may be switched on in the following ways:

  • On the commandline, use --match-branch, e.g. specmatic test --match-branch, or specmatic mock --match-branch.
  • Set an environment variable or system property named MATCH_BRANCH to true.
  • Set matchBranch: true in specmatic.yaml
specmatic.yaml
version: 3
components:
  sources:
    contractRepository:
      git:
        url: https://github.com/your-username/your-repo.git
        matchBranch: true
  runOptions:
    productsMock:
      openapi:
        type: mock
        baseUrl: null
dependencies:
  services:
    - service:
        description: Employee Service
        definitions:
          - definition:
              source:
                $ref: "#/components/sources/contractRepository"
              specs:
                - spec:
                    path: products.yaml
  settings:
    hotReload: false

To understand where this may be useful, consider the following scenario. Let's say there's a branch in the central contract repo named product-discount with changes to the product API specification. Then to develop the feature, the API developer creates a branch named product-discount in the API repo and makes changes to the API implementation to support the new feature.

With this setup, the API developer can run specmatic test --match-branch with the product-discount branch checked out in the API repository, and since there is a branch on the central repo with the same name, specifications from the product-discount branch on the central repo will be used to run tests.

The same is true for the consumer of the API, who can run specmatic mock --match-branch.

Not all features though require specification changes. So if there is no matching branch in the central repository, specmatic test --match-branch will simply create a new local branch off the default branch and run tests from it.

Sample Java Project​

https://github.com/specmatic/specmatic-order-bff-java