EARLY ACCESS

We highly recommend fully setting up SDK tests in all of your SDK repositories before moving on to exploring this.

Custom End-to-End API Contract Tests against live APIs is in an early feature set stage.

Custom End-to-End API Contract Tests with Arazzo

Speakeasy creates custom end-to-end contract tests that run against a real API.

This document explains how to write complex tests via the Arazzo specification. It also covers key configuration features for these tests such as:

Server URLs
Security credentials
Environment Variable Provided Values

Arazzo is a simple, human-readable, and extensible specification for defining API workflows. Arazzo powers test generation, enabling custom tests for any use case and rich tests capable of:

Testing multiple operations.
Testing different inputs.
Validating the correct response is returned.
Running against a real API or mock server.
Configuring setup and teardown routines for complex E2E tests.

The Arazzo Specification defines sequences of API operations and their dependencies for contract testing, validating API behavior across multiple interconnected endpoints and complex workflows.

When a .speakeasy/tests.arazzo.yaml file exists in the SDK repo, the Arazzo workflow generates tests for each workflow defined in the file.

Prerequisites

The following are requirements for generating tests:

Testing feature prerequisites are met.

Writing custom End-to-End tests

The following example Arazzo document defines a simple E2E test for the life cycle of a user resource in the example API.

arazzo: 1.0.0
info:
  title: Test Suite
  summary: E2E tests for the SDK and API.
  version: 0.0.1
sourceDescriptions:
  - name: The API
    url: https://example.com/openapi.yaml
    type: openapi
workflows:
  - workflowId: user-lifecycle
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: {
            "email": "Trystan_Crooks@hotmail.com",
            "first_name": "Trystan",
            "last_name": "Crooks",
            "age": 32,
            "postal_code": 94110,
            "metadata": {
              "allergies": "none",
              "color": "red",
              "height": 182,
              "weight": 77,
              "is_smoking": true
            }
          }
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/postal_code == 94110
        outputs:
          id: $response.body#/id
      - stepId: get
        operationId: getUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/first_name == Trystan
          - condition: $response.body#/last_name == Crooks
          - condition: $response.body#/age == 32
          - condition: $response.body#/postal_code == 94110
        outputs:
          user: $response.body
          age: $response.body#/age
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        requestBody:
          contentType: application/json
          payload: $steps.get.outputs.user
          replacements:
            - target: /postal_code
              value: 94107
            - target: /age
              value: $steps.get.outputs.age
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/first_name == Trystan
          - condition: $response.body#/last_name == Crooks
          - condition: $response.body#/age == 32
          - condition: $response.body#/postal_code == 94107
        outputs:
          email: $response.body#/email
          first_name: $response.body#/first_name
          last_name: $response.body#/last_name
          metadata: $response.body#/metadata
      - stepId: updateAgain
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        requestBody:
          contentType: application/json
          payload: {
            "id": "$steps.create.outputs.id",
            "email": "$steps.update.email",
            "first_name": "$steps.update.first_name",
            "last_name": "$steps.update.last_name",
            "age": 33,
            "postal_code": 94110,
            "metadata": "$steps.update.metadata"
          }
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/first_name == Trystan
          - condition: $response.body#/last_name == Crooks
          - condition: $response.body#/age == 33
          - condition: $response.body#/postal_code == 94110
      - stepId: delete
        operationId: deleteUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        successCriteria:
          - condition: $statusCode == 200

The above workflow defines 4 steps that feed into each other, representing the creation of a user, retrieving that user via its new ID, updating the user, and finally deleting the user. Outputs are defined for certain steps and then used as inputs for the following steps.

This generates the test shown below:

// src/__tests__/sdk.test.ts

import { assert, expect, it, test } from "vitest";
import { SDK } from "../index.js";
import { assertDefined } from "./assertions.js";
import { createTestHTTPClient } from "./testclient.js";

test("Sdk User Lifecycle", async () => {
  const sdk = new SDK({
    serverURL: process.env["TEST_SERVER_URL"] ?? "http://localhost:18080",
    httpClient: createTestHTTPClient("user-lifecycle"),
  });

  const createResult = await sdk.createUser({
    email: "Trystan_Crooks@hotmail.com",
    firstName: "Trystan",
    lastName: "Crooks",
    age: 32,
    postalCode: "94110",
    metadata: {
      allergies: "none",
      additionalProperties: {
        "color": "red",
        "height": "182",
        "weight": "77",
        "is_smoking": "true",
      },
    },
  });
  expect(createResult.httpMeta.response.status).toBe(200);
  expect(createResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(createResult.user?.postalCode).toBeDefined();
  expect(createResult.user?.postalCode).toEqual("94110");

  const getResult = await sdk.getUser(assertDefined(createResult.user?.id));
  expect(getResult.httpMeta.response.status).toBe(200);
  expect(getResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(getResult.user?.firstName).toBeDefined();
  expect(getResult.user?.firstName).toEqual("Trystan");
  expect(getResult.user?.lastName).toBeDefined();
  expect(getResult.user?.lastName).toEqual("Crooks");
  expect(getResult.user?.age).toBeDefined();
  expect(getResult.user?.age).toEqual(32);
  expect(getResult.user?.postalCode).toBeDefined();
  expect(getResult.user?.postalCode).toEqual("94110");

  const user = assertDefined(getResult.user);
  user.postalCode = "94107";
  user.age = getResult.user?.age;
  const updateResult = await sdk.updateUser(
    assertDefined(createResult.user?.id),
    assertDefined(getResult.user),
  );
  expect(updateResult.httpMeta.response.status).toBe(200);
  expect(updateResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(updateResult.user?.firstName).toBeDefined();
  expect(updateResult.user?.firstName).toEqual("Trystan");
  expect(updateResult.user?.lastName).toBeDefined();
  expect(updateResult.user?.lastName).toEqual("Crooks");
  expect(updateResult.user?.age).toBeDefined();
  expect(updateResult.user?.age).toEqual(32);
  expect(updateResult.user?.postalCode).toBeDefined();
  expect(updateResult.user?.postalCode).toEqual("94107");

  const updateAgainResult = await sdk.updateUser(
    assertDefined(createResult.user?.id),
    {
      id: assertDefined(createResult.user?.id),
      email: assertDefined(updateResult.user?.email),
      firstName: updateResult.user?.firstName,
      lastName: updateResult.user?.lastName,
      age: 33,
      postalCode: "94110",
      metadata: updateResult.user?.metadata,
    },
  );
  expect(updateAgainResult.httpMeta.response.status).toBe(200);
  expect(updateAgainResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(updateAgainResult.user?.firstName).toBeDefined();
  expect(updateAgainResult.user?.firstName).toEqual("Trystan");
  expect(updateAgainResult.user?.lastName).toBeDefined();
  expect(updateAgainResult.user?.lastName).toEqual("Crooks");
  expect(updateAgainResult.user?.age).toBeDefined();
  expect(updateAgainResult.user?.age).toEqual(33);
  expect(updateAgainResult.user?.postalCode).toBeDefined();
  expect(updateAgainResult.user?.postalCode).toEqual("94110");

  const deleteResult = await sdk.deleteUser(
    assertDefined(createResult.user?.id),
  );
  expect(deleteResult.httpMeta.response.status).toBe(200);
});

# tests/test_sdk.py

import io
import openapi
from openapi import SDK
import os
import pytest
from tests.test_client import create_test_http_client


def test_sdk_user_lifecycle():
    with SDK(
        server_url=os.getenv("TEST_SERVER_URL", "http://localhost:18080"),
        client=create_test_http_client("user-lifecycle"),
    ) as sdk:
        assert sdk is not None

        create_res = sdk.create_user(
            request=openapi.BaseUser(
                email="Trystan_Crooks@hotmail.com",
                first_name="Trystan",
                last_name="Crooks",
                age=32,
                postal_code="94110",
                metadata=openapi.Metadata(
                    allergies="none",
                    **{
                        "color": "red",
                        "height": "182",
                        "weight": "77",
                        "is_smoking": "true",
                    },
                ),
            )
        )
        assert create_res is not None
        assert create_res.email == "Trystan_Crooks@hotmail.com"
        assert create_res.postal_code is not None
        assert create_res.postal_code == "94110"

        get_res = sdk.get_user(id=create_res.id)
        assert get_res is not None
        assert get_res.email == "Trystan_Crooks@hotmail.com"
        assert get_res.first_name is not None
        assert get_res.first_name == "Trystan"
        assert get_res.last_name is not None
        assert get_res.last_name == "Crooks"
        assert get_res.age is not None
        assert get_res.age == 32
        assert get_res.postal_code is not None
        assert get_res.postal_code == "94110"

        user = get_res
        user.postal_code = "94107"
        user.age = get_res.age

        update_res = sdk.update_user(id=create_res.id, user=user)
        assert update_res is not None
        assert update_res.email == "Trystan_Crooks@hotmail.com"
        assert update_res.first_name is not None
        assert update_res.first_name == "Trystan"
        assert update_res.last_name is not None
        assert update_res.last_name == "Crooks"
        assert update_res.age is not None
        assert update_res.age == 32
        assert update_res.postal_code is not None
        assert update_res.postal_code == "94107"

        update_again_res = sdk.update_user(
            id=create_res.id,
            user=openapi.User(
                id=create_res.id,
                email=update_res.email,
                first_name=update_res.first_name,
                last_name=update_res.last_name,
                age=33,
                postal_code="94110",
                metadata=update_res.metadata,
            ),
        )
        assert update_again_res is not None
        assert update_again_res.email == "Trystan_Crooks@hotmail.com"
        assert update_again_res.first_name is not None
        assert update_again_res.first_name == "Trystan"
        assert update_again_res.last_name is not None
        assert update_again_res.last_name == "Crooks"
        assert update_again_res.age is not None
        assert update_again_res.age == 33
        assert update_again_res.postal_code is not None
        assert update_again_res.postal_code == "94110"

        sdk.delete_user(id=create_res.id)

// tests/sdk_test.go

package sdk_test

import (
	"context"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
	"openapi"
	"openapi/internal/utils"
	"os"
	"testing"
)

func TestSDK_UserLifecycle(t *testing.T) {
	ctx := context.Background()

	s := openapi.New(
		openapi.WithServerURL(utils.GetEnv("TEST_SERVER_URL", "http://localhost:18080")),
		openapi.WithClient(createTestHTTPClient("user-lifecycle")),
	)

	createRes, err := s.CreateUser(ctx, openapi.BaseUser{
		Email:      "Trystan_Crooks@hotmail.com",
		FirstName:  openapi.String("Trystan"),
		LastName:   openapi.String("Crooks"),
		Age:        openapi.Float64(32),
		PostalCode: openapi.String("94110"),
		Metadata: &openapi.Metadata{
			Allergies: openapi.String("none"),
			AdditionalProperties: map[string]string{
				"color":      "red",
				"height":     "182",
				"weight":     "77",
				"is_smoking": "true",
			},
		},
	})
	require.NoError(t, err)
	assert.Equal(t, 200, createRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", createRes.User.Email)
	assert.NotNil(t, createRes.User.PostalCode)
	assert.Equal(t, openapi.String("94110"), createRes.User.PostalCode)

	getRes, err := s.GetUser(ctx, createRes.User.ID)
	require.NoError(t, err)
	assert.Equal(t, 200, getRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", getRes.User.Email)
	assert.NotNil(t, getRes.User.FirstName)
	assert.Equal(t, openapi.String("Trystan"), getRes.User.FirstName)
	assert.NotNil(t, getRes.User.LastName)
	assert.Equal(t, openapi.String("Crooks"), getRes.User.LastName)
	assert.NotNil(t, getRes.User.Age)
	assert.Equal(t, openapi.Float64(32), getRes.User.Age)
	assert.NotNil(t, getRes.User.PostalCode)
	assert.Equal(t, openapi.String("94110"), getRes.User.PostalCode)

	user := *getRes.User
	user.PostalCode = openapi.String("94107")
	user.Age = getRes.User.Age

	updateRes, err := s.UpdateUser(ctx, createRes.User.ID, user)
	require.NoError(t, err)
	assert.Equal(t, 200, updateRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", updateRes.User.Email)
	assert.NotNil(t, updateRes.User.FirstName)
	assert.Equal(t, openapi.String("Trystan"), updateRes.User.FirstName)
	assert.NotNil(t, updateRes.User.LastName)
	assert.Equal(t, openapi.String("Crooks"), updateRes.User.LastName)
	assert.NotNil(t, updateRes.User.Age)
	assert.Equal(t, openapi.Float64(32), updateRes.User.Age)
	assert.NotNil(t, updateRes.User.PostalCode)
	assert.Equal(t, openapi.String("94107"), updateRes.User.PostalCode)

	updateAgainRes, err := s.UpdateUser(ctx, createRes.User.ID, openapi.User{
		ID:         createRes.User.ID,
		Email:      updateRes.User.Email,
		FirstName:  updateRes.User.FirstName,
		LastName:   updateRes.User.LastName,
		Age:        openapi.Float64(33),
		PostalCode: openapi.String("94110"),
		Metadata:   updateRes.User.Metadata,
	})
	require.NoError(t, err)
	assert.Equal(t, 200, updateAgainRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", updateAgainRes.User.Email)
	assert.NotNil(t, updateAgainRes.User.FirstName)
	assert.Equal(t, openapi.String("Trystan"), updateAgainRes.User.FirstName)
	assert.NotNil(t, updateAgainRes.User.LastName)
	assert.Equal(t, openapi.String("Crooks"), updateAgainRes.User.LastName)
	assert.NotNil(t, updateAgainRes.User.Age)
	assert.Equal(t, openapi.Float64(33), updateAgainRes.User.Age)
	assert.NotNil(t, updateAgainRes.User.PostalCode)
	assert.Equal(t, openapi.String("94110"), updateAgainRes.User.PostalCode)

	deleteRes, err := s.DeleteUser(ctx, createRes.User.ID)
	require.NoError(t, err)
	assert.Equal(t, 200, deleteRes.HTTPMeta.Response.StatusCode)
}

Input and Outputs

Inputs

Inputs can be provided to steps in several ways: via inputs defined in the workflow, references from previous steps, or via values defined inline.

Workflow Inputs

Workflow inputs provide input parameters to the workflow that can be used by any step defined in the workflow. The inputs field of a workflow is a JSON Schema object that defines a property for each input the workflow exposes.

Test Generation uses any examples defined for a property in the inputs json schemas as literal values for the test. Since tests are non-interactive, if no examples are defined, the test generation randomly generates values for the inputs.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    inputs: # JSON Schema for the inputs where each property represents a workflow input
      type: object
      properties:
        email:
          type: string
          examples:
            - Trystan_Crooks@hotmail.com # Examples are used as literal values for the test
        firstName:
          type: string
          examples:
            - Trystan
        lastName:
          type: string
          examples:
            - Crooks
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: {
            "email": "$inputs.email", # Payload populated with the literal value from inputs
            "first_name": "$inputs.firstName",
            "last_name": "$inputs.lastName",
          }
        successCriteria:
          - condition: $statusCode == 200

Step References

Parameters and request body payloads can reference values via Runtime Expressions from previous steps in the workflow. This enables generation of tests that are more complex than a simple sequence of operations. Speakeasy’s implementation currently only allows referencing a previous step’s outputs, requiring definition of values to expose to future steps.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: #....
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
        outputs:
          id: $response.body#/id # The id field of the response body exposed as an output for the next step
      - stepId: get
        operationId: getUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id # The id output from the previous step used as the value for the id parameter
        successCriteria:
          - condition: $statusCode == 200

Inline Values

For any parameters or request body payloads a step defines, literal values can be provided inline to populate the tests when static values are suitable for the test.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: "some-test-id" # A literal value for parameters that matches the json schema of the parameter as defined in the associated operation
        requestBody:
          contentType: application/json
          payload: { # Literal values that match the content type of the request body
            "email": "Trystan_Crooks@hotmail.com",
            "first_name": "Trystan",
            "last_name": "Crooks",
            "age": 32,
            "postal_code": 94110,
            "metadata": {
              "allergies": "none",
              "color": "red",
              "height": 182,
              "weight": 77,
              "is_smoking": true
            }
          }
        successCriteria:
          - condition: $statusCode == 200

Payload Values

When using the payload field of a request body input, the value can be a static value, a value with interpolated Runtime Expressions or a Runtime Expression by itself.

The payload value can be overlayed using the replacements field which represents a list of targets within the payload to replace with the value of the replacements, which can be a static value or a Runtime Expression.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: get
        # ...
        outputs:
          user: $response.body
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: "some-test-id"
        requestBody:
          contentType: application/json
          payload: $steps.get.outputs.user # Response body of the previous step used as the payload
          replacements: # Overlay the payload with replacements
            - target: /postal_code # Overlays the postal_code field with a static value
              value: 94107
            - target: /age # Overlays the age field with the value of the age output from a previous step
              value: $steps.some-other-step.outputs.age
        successCriteria:
          - condition: $statusCode == 200

Outputs

As shown above, outputs can be defined for each step in a workflow allowing values from the response body to be used as values in following steps.

Current Speakeasy implementation supports only referencing values from a response body, using the Runtime Expressions syntax and json-pointers.

Any number of outputs can be defined for a step.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: #....
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
        outputs: # Map of output id to runtime expression that populates the output
          id: $response.body#/id # Json-pointers reference fields within the response body
          email: $response.body#/email
          age: $response.body#/age
          allergies: $response.body#/metadata/allergies

Success Criteria

The successCriteria field of a step is a list of Criterion Objects that validate the success of the step. For test generation these form the basis of the test assertions.

successCriteria can be as simple as a single condition testing the status code of the response, or as complex as testing multiple individual fields within the response body.

Speakeasy’s implementation currently only supports simple criteria and the equality operator == for comparing values, and testing status codes, response headers and response bodies.

For testing values within the response body, criteria for testing the status code and content type of the response are also required to help the generator determine which response schema to validate against due to the typed nature of the SDKs.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: #....
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          # or
          - context: $response.body
            type: simple
            condition: |
              {
                "email": "Trystan_Crooks@hotmail.com",
                "first_name": "Trystan",
                "last_name": "Crooks",
                "age": 32,
                "postal_code": 94110,
                "metadata": {
                  "allergies": "none",
                  "color": "red",
                  "height": 182,
                  "weight": 77,
                  "is_smoking": true
                }
              }

Testing operations requiring binary data

Some operations require providing binary data to test uploading or downloading files. In these cases test files can be provided using the x-file directive in the example for that field.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: postFile
    steps:
      - stepId: test
        operationId: postFile
        requestBody:
          contentType: multipart/form-data
          payload:
            file: "x-file: some-test-file.txt"
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/octet-stream
          - context: $response.body
            condition: "x-file: some-other-test-file.dat"
            type: simple

The files are sourced from the .speakeasy/testfiles directory in the root of the SDK repo, where the path provided in the x-file directive is relative to the testfiles directory.

The contents of the sourced file are used as the value for the field being tested.

Configuring an API to Test Against

By default, tests are generated to run against Speakeasy’s mock server (URL of http://localhost:18080) which validates the SDKs are functioning correctly but does not guarantee the correctness of the API.

The generator can be configured to run all tests against another URL or just individual tests. This is done through the x-speakeasy-test-server extensions in the .speakeasy/tests.arazzo.yaml file.

If the extension is found at the top level of the Arazzo file then all workflows/tests are configured to run against the specified server URL. If the extension is found within a workflow then only that workflow is configured to run against the specified server URL.

The server URL can be either a static URL or a x-env: EXAMPLE_ENV_VAR value that pulls the value from the environment variable EXAMPLE_ENV_VAR where the name of the environment variable can be any specified name.

arazzo: 1.0.0
# ...
x-speakeasy-test-server: 
  baseUrl: "https://api.example.com" # If specified at the top level of the Arazzo file, all workflows will be configured to run against the specified server URL
workflows:
  - workflowId: some-test
    x-speakeasy-test-server: 
      baseUrl: "x-env: TEST_SERVER_URL" # If specified within a workflow, only that workflow will be configured to run against the specified server URL. This will override any top level configuration.
    # ...

A default value can be provided in the x-env directive if the environment variable is not set. This can be useful for local development or non-production environments.

x-speakeasy-test-server: 
  baseUrl: "x-env: TEST_SERVER_URL; http://localhost:18080" # Run against the local mock server if the environment variable is not set

If all tests are configured to run against other server URLs, mock server generation can be disabled within the .speakeasy/gen.yaml file.

# ...
generation:
  # ...
  mockServer:
    disabled: true # Setting this to true will disable mock server generation

Configuring security credentials for Contract tests

When running tests against a real API, the SDK may need to be configured with security credentials to authenticate with the API. This can be done by adding the x-speakeasy-test-security extension to the document, a workflow or a individual step.

The x-speakeasy-test-security extension allows static values or values pulled from the environment to be used when instantiating an SDK instance and making requests to the API.

arazzo: 1.0.0
# ...
x-speakeasy-test-security: # Defined at the top level of the Arazzo file, all workflows will be configured to use the specified security credentials
  value:
    # The keys in the value map are the names of the security schemes defined in the OpenAPI document.
    # For simple schemes, the values is for example the API key value required.
    # For schemes like basic auth or OAuth2 that require multiple values, the value is a map of the required values.
    apiKey: x-env: TEST_API_KEY # Values can be pulled from the environment
    basicAuth:
      username: "test-user" # Or defined as static values
      password: x-env: TEST_PASSWORD
workflows:
  - workflowId: some-test
    x-speakeasy-test-security: # Security can be defined/overridden for a specific workflow
      value:
        apiKey: "test-key"
    # ...
    steps:
      - stepId: step1
        x-speakeasy-test-security: # Or security can be defined/overridden for a specific step
          value:
            authToken: x-env: TEST_AUTH_TOKEN
        # ...
      - stepId: step2
        # ...

Configuring environment variable provided values for Contract tests

When running tests against a real API, certain input values can be filled from dynamic environment variables using the Speakeasy environment variable extension.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: my-env-var-test
    steps:
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: "x-env: TEST_ID; default" # Environment variable with optional default value if the env variable is not present
        requestBody:
          contentType: application/json
          payload: {
            "email": "x-env: TEST_EMAIL; default, # Environment variable with optional default value if the env variable is not present
            "first_name": "Trystan",
            "last_name": "Crooks",
            "age": 32,
            "postal_code": 94110,
            "metadata": {
              "allergies": "none",
              "color": "red",
              "height": 182,
              "weight": 77,
              "is_smoking": true
            }
          }
        successCriteria:
          - condition: $statusCode == 200