EARLY ACCESS

We highly recommend fully setting up SDK tests in all of your SDK repositories before moving on to exploring this.

Custom End-to-End API Contract Tests against live APIs is in an early feature set stage.

Custom End-to-End API Contract Tests with Arazzo

Speakeasy can also be used to create custom end-to-end contract tests that run against a real API.

This document will walk through how to write those more complex tests via the Arazzo specification. It will also walk through key configuration features for these tests such as:

Server URLs
Security credentials
Environment Variable Provided Values

Arazzo is a simple, human-readable, and extensible specification for defining API workflows. Arazzo powers test generation, allowing you to define custom tests for any use case and define rich tests capable of:

Testing multiple operations.
Testing different inputs.
Validating the correct response is returned.
Run against a real API or mock server.
Configure setup and teardown routines for complex E2E tests.

The Arazzo Specification allows you to define sequences of API operations and their dependencies for contract testing, enabling you to validate that your API behaves correctly across multiple interconnected endpoints and complex workflows.

When a .speakeasy/tests.arazzo.yaml file is found in your SDK repo, the Arazzo workflow will be used to generate tests for each of the workflows defined in the file.

Prerequisites

The following are requirements for generating tests:

Testing feature prerequisites are met.

Writing custom End-to-End tests

The following is an example Arazzo document defining a simple E2E test for the life cycle of a user resource in the example API.

arazzo: 1.0.0
info:
  title: Test Suite
  summary: E2E tests for the SDK and API.
  version: 0.0.1
sourceDescriptions:
  - name: The API
    url: https://example.com/openapi.yaml
    type: openapi
workflows:
  - workflowId: user-lifecycle
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: {
            "email": "Trystan_Crooks@hotmail.com",
            "first_name": "Trystan",
            "last_name": "Crooks",
            "age": 32,
            "postal_code": 94110,
            "metadata": {
              "allergies": "none",
              "color": "red",
              "height": 182,
              "weight": 77,
              "is_smoking": true
            }
          }
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/postal_code == 94110
        outputs:
          id: $response.body#/id
      - stepId: get
        operationId: getUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/first_name == Trystan
          - condition: $response.body#/last_name == Crooks
          - condition: $response.body#/age == 32
          - condition: $response.body#/postal_code == 94110
        outputs:
          user: $response.body
          age: $response.body#/age
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        requestBody:
          contentType: application/json
          payload: $steps.get.outputs.user
          replacements:
            - target: /postal_code
              value: 94107
            - target: /age
              value: $steps.get.outputs.age
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/first_name == Trystan
          - condition: $response.body#/last_name == Crooks
          - condition: $response.body#/age == 32
          - condition: $response.body#/postal_code == 94107
        outputs:
          email: $response.body#/email
          first_name: $response.body#/first_name
          last_name: $response.body#/last_name
          metadata: $response.body#/metadata
      - stepId: updateAgain
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        requestBody:
          contentType: application/json
          payload: {
            "id": "$steps.create.outputs.id",
            "email": "$steps.update.email",
            "first_name": "$steps.update.first_name",
            "last_name": "$steps.update.last_name",
            "age": 33,
            "postal_code": 94110,
            "metadata": "$steps.update.metadata"
          }
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          - condition: $response.body#/first_name == Trystan
          - condition: $response.body#/last_name == Crooks
          - condition: $response.body#/age == 33
          - condition: $response.body#/postal_code == 94110
      - stepId: delete
        operationId: deleteUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id
        successCriteria:
          - condition: $statusCode == 200

The above workflow defined 4 steps that each feed into the next, representing the creation of a user, retrieving that user via its new ID, updating the user, and finally deleting the user. Outputs have been defined for certain steps that are then used as inputs for the following steps.

It will generate the test shown below:

// src/__tests__/sdk.test.ts

import { assert, expect, it, test } from "vitest";
import { SDK } from "../index.js";
import { assertDefined } from "./assertions.js";
import { createTestHTTPClient } from "./testclient.js";

test("Sdk User Lifecycle", async () => {
  const sdk = new SDK({
    serverURL: process.env["TEST_SERVER_URL"] ?? "http://localhost:18080",
    httpClient: createTestHTTPClient("user-lifecycle"),
  });

  const createResult = await sdk.createUser({
    email: "Trystan_Crooks@hotmail.com",
    firstName: "Trystan",
    lastName: "Crooks",
    age: 32,
    postalCode: "94110",
    metadata: {
      allergies: "none",
      additionalProperties: {
        "color": "red",
        "height": "182",
        "weight": "77",
        "is_smoking": "true",
      },
    },
  });
  expect(createResult.httpMeta.response.status).toBe(200);
  expect(createResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(createResult.user?.postalCode).toBeDefined();
  expect(createResult.user?.postalCode).toEqual("94110");

  const getResult = await sdk.getUser(assertDefined(createResult.user?.id));
  expect(getResult.httpMeta.response.status).toBe(200);
  expect(getResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(getResult.user?.firstName).toBeDefined();
  expect(getResult.user?.firstName).toEqual("Trystan");
  expect(getResult.user?.lastName).toBeDefined();
  expect(getResult.user?.lastName).toEqual("Crooks");
  expect(getResult.user?.age).toBeDefined();
  expect(getResult.user?.age).toEqual(32);
  expect(getResult.user?.postalCode).toBeDefined();
  expect(getResult.user?.postalCode).toEqual("94110");

  const user = assertDefined(getResult.user);
  user.postalCode = "94107";
  user.age = getResult.user?.age;
  const updateResult = await sdk.updateUser(
    assertDefined(createResult.user?.id),
    assertDefined(getResult.user),
  );
  expect(updateResult.httpMeta.response.status).toBe(200);
  expect(updateResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(updateResult.user?.firstName).toBeDefined();
  expect(updateResult.user?.firstName).toEqual("Trystan");
  expect(updateResult.user?.lastName).toBeDefined();
  expect(updateResult.user?.lastName).toEqual("Crooks");
  expect(updateResult.user?.age).toBeDefined();
  expect(updateResult.user?.age).toEqual(32);
  expect(updateResult.user?.postalCode).toBeDefined();
  expect(updateResult.user?.postalCode).toEqual("94107");

  const updateAgainResult = await sdk.updateUser(
    assertDefined(createResult.user?.id),
    {
      id: assertDefined(createResult.user?.id),
      email: assertDefined(updateResult.user?.email),
      firstName: updateResult.user?.firstName,
      lastName: updateResult.user?.lastName,
      age: 33,
      postalCode: "94110",
      metadata: updateResult.user?.metadata,
    },
  );
  expect(updateAgainResult.httpMeta.response.status).toBe(200);
  expect(updateAgainResult.user?.email).toEqual("Trystan_Crooks@hotmail.com");
  expect(updateAgainResult.user?.firstName).toBeDefined();
  expect(updateAgainResult.user?.firstName).toEqual("Trystan");
  expect(updateAgainResult.user?.lastName).toBeDefined();
  expect(updateAgainResult.user?.lastName).toEqual("Crooks");
  expect(updateAgainResult.user?.age).toBeDefined();
  expect(updateAgainResult.user?.age).toEqual(33);
  expect(updateAgainResult.user?.postalCode).toBeDefined();
  expect(updateAgainResult.user?.postalCode).toEqual("94110");

  const deleteResult = await sdk.deleteUser(
    assertDefined(createResult.user?.id),
  );
  expect(deleteResult.httpMeta.response.status).toBe(200);
});

# tests/test_sdk.py

import io
import openapi
from openapi import SDK
import os
import pytest
from tests.test_client import create_test_http_client


def test_sdk_user_lifecycle():
    with SDK(
        server_url=os.getenv("TEST_SERVER_URL", "http://localhost:18080"),
        client=create_test_http_client("user-lifecycle"),
    ) as sdk:
        assert sdk is not None

        create_res = sdk.create_user(
            request=openapi.BaseUser(
                email="Trystan_Crooks@hotmail.com",
                first_name="Trystan",
                last_name="Crooks",
                age=32,
                postal_code="94110",
                metadata=openapi.Metadata(
                    allergies="none",
                    **{
                        "color": "red",
                        "height": "182",
                        "weight": "77",
                        "is_smoking": "true",
                    },
                ),
            )
        )
        assert create_res is not None
        assert create_res.email == "Trystan_Crooks@hotmail.com"
        assert create_res.postal_code is not None
        assert create_res.postal_code == "94110"

        get_res = sdk.get_user(id=create_res.id)
        assert get_res is not None
        assert get_res.email == "Trystan_Crooks@hotmail.com"
        assert get_res.first_name is not None
        assert get_res.first_name == "Trystan"
        assert get_res.last_name is not None
        assert get_res.last_name == "Crooks"
        assert get_res.age is not None
        assert get_res.age == 32
        assert get_res.postal_code is not None
        assert get_res.postal_code == "94110"

        user = get_res
        user.postal_code = "94107"
        user.age = get_res.age

        update_res = sdk.update_user(id=create_res.id, user=user)
        assert update_res is not None
        assert update_res.email == "Trystan_Crooks@hotmail.com"
        assert update_res.first_name is not None
        assert update_res.first_name == "Trystan"
        assert update_res.last_name is not None
        assert update_res.last_name == "Crooks"
        assert update_res.age is not None
        assert update_res.age == 32
        assert update_res.postal_code is not None
        assert update_res.postal_code == "94107"

        update_again_res = sdk.update_user(
            id=create_res.id,
            user=openapi.User(
                id=create_res.id,
                email=update_res.email,
                first_name=update_res.first_name,
                last_name=update_res.last_name,
                age=33,
                postal_code="94110",
                metadata=update_res.metadata,
            ),
        )
        assert update_again_res is not None
        assert update_again_res.email == "Trystan_Crooks@hotmail.com"
        assert update_again_res.first_name is not None
        assert update_again_res.first_name == "Trystan"
        assert update_again_res.last_name is not None
        assert update_again_res.last_name == "Crooks"
        assert update_again_res.age is not None
        assert update_again_res.age == 33
        assert update_again_res.postal_code is not None
        assert update_again_res.postal_code == "94110"

        sdk.delete_user(id=create_res.id)

// tests/sdk_test.go

package sdk_test

import (
	"context"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
	"openapi"
	"openapi/internal/utils"
	"os"
	"testing"
)

func TestSDK_UserLifecycle(t *testing.T) {
	ctx := context.Background()

	s := openapi.New(
		openapi.WithServerURL(utils.GetEnv("TEST_SERVER_URL", "http://localhost:18080")),
		openapi.WithClient(createTestHTTPClient("user-lifecycle")),
	)

	createRes, err := s.CreateUser(ctx, openapi.BaseUser{
		Email:      "Trystan_Crooks@hotmail.com",
		FirstName:  openapi.String("Trystan"),
		LastName:   openapi.String("Crooks"),
		Age:        openapi.Float64(32),
		PostalCode: openapi.String("94110"),
		Metadata: &openapi.Metadata{
			Allergies: openapi.String("none"),
			AdditionalProperties: map[string]string{
				"color":      "red",
				"height":     "182",
				"weight":     "77",
				"is_smoking": "true",
			},
		},
	})
	require.NoError(t, err)
	assert.Equal(t, 200, createRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", createRes.User.Email)
	assert.NotNil(t, createRes.User.PostalCode)
	assert.Equal(t, openapi.String("94110"), createRes.User.PostalCode)

	getRes, err := s.GetUser(ctx, createRes.User.ID)
	require.NoError(t, err)
	assert.Equal(t, 200, getRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", getRes.User.Email)
	assert.NotNil(t, getRes.User.FirstName)
	assert.Equal(t, openapi.String("Trystan"), getRes.User.FirstName)
	assert.NotNil(t, getRes.User.LastName)
	assert.Equal(t, openapi.String("Crooks"), getRes.User.LastName)
	assert.NotNil(t, getRes.User.Age)
	assert.Equal(t, openapi.Float64(32), getRes.User.Age)
	assert.NotNil(t, getRes.User.PostalCode)
	assert.Equal(t, openapi.String("94110"), getRes.User.PostalCode)

	user := *getRes.User
	user.PostalCode = openapi.String("94107")
	user.Age = getRes.User.Age

	updateRes, err := s.UpdateUser(ctx, createRes.User.ID, user)
	require.NoError(t, err)
	assert.Equal(t, 200, updateRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", updateRes.User.Email)
	assert.NotNil(t, updateRes.User.FirstName)
	assert.Equal(t, openapi.String("Trystan"), updateRes.User.FirstName)
	assert.NotNil(t, updateRes.User.LastName)
	assert.Equal(t, openapi.String("Crooks"), updateRes.User.LastName)
	assert.NotNil(t, updateRes.User.Age)
	assert.Equal(t, openapi.Float64(32), updateRes.User.Age)
	assert.NotNil(t, updateRes.User.PostalCode)
	assert.Equal(t, openapi.String("94107"), updateRes.User.PostalCode)

	updateAgainRes, err := s.UpdateUser(ctx, createRes.User.ID, openapi.User{
		ID:         createRes.User.ID,
		Email:      updateRes.User.Email,
		FirstName:  updateRes.User.FirstName,
		LastName:   updateRes.User.LastName,
		Age:        openapi.Float64(33),
		PostalCode: openapi.String("94110"),
		Metadata:   updateRes.User.Metadata,
	})
	require.NoError(t, err)
	assert.Equal(t, 200, updateAgainRes.HTTPMeta.Response.StatusCode)
	assert.Equal(t, "Trystan_Crooks@hotmail.com", updateAgainRes.User.Email)
	assert.NotNil(t, updateAgainRes.User.FirstName)
	assert.Equal(t, openapi.String("Trystan"), updateAgainRes.User.FirstName)
	assert.NotNil(t, updateAgainRes.User.LastName)
	assert.Equal(t, openapi.String("Crooks"), updateAgainRes.User.LastName)
	assert.NotNil(t, updateAgainRes.User.Age)
	assert.Equal(t, openapi.Float64(33), updateAgainRes.User.Age)
	assert.NotNil(t, updateAgainRes.User.PostalCode)
	assert.Equal(t, openapi.String("94110"), updateAgainRes.User.PostalCode)

	deleteRes, err := s.DeleteUser(ctx, createRes.User.ID)
	require.NoError(t, err)
	assert.Equal(t, 200, deleteRes.HTTPMeta.Response.StatusCode)
}

Input and Outputs

Inputs

Inputs can be provided to steps in a number of ways, either via inputs defined in the workflow, references from previous steps, or via values defined inline .

Workflow Inputs

Workflow inputs are a way to provide input parameters to the workflow that can be used by any step defined in the workflow. The inputs field of a workflow is a JSON Schema object that defines a property for each input the workflow wants to expose.

Test Generation can use any examples defined for a property in the inputs json schemas as literal values to use as inputs for the test. As tests are none interactive, if no examples are defined, the test generation will just randomly generate values for the inputs, as it can’t ask the user for input.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    inputs: # This is the JSON Schema for the inputs each property in the inputs object represents a workflow input
      type: object
      properties:
        email:
          type: string
          examples:
            - Trystan_Crooks@hotmail.com # Examples defined will be used as literal values for the test
        firstName:
          type: string
          examples:
            - Trystan
        lastName:
          type: string
          examples:
            - Crooks
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: {
            "email": "$inputs.email", # The payload will be populated with the literal value defined in the inputs
            "first_name": "$inputs.firstName",
            "last_name": "$inputs.lastName",
          }
        successCriteria:
          - condition: $statusCode == 200

Step References

Parameters and request body payloads can reference values via Runtime Expressions from previous steps in the workflow. This allows for the generation of tests that are more complex than a simple sequence of operations. Speakeasy’s implementation currently only allows the referencing of a previous step’s outputs, which means you will need to define what values you want to expose to future steps.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: #....
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
        outputs:
          id: $response.body#/id # The id field of the response body will be exposed as an output for the next step
      - stepId: get
        operationId: getUser
        parameters:
          - name: id
            in: path
            value: $steps.create.outputs.id # The id output from the previous step will be used as the value for the id parameter
        successCriteria:
          - condition: $statusCode == 200

Inline Values

For any parameters or request body payloads a step defines, literal values can be provided inline to populate the tests if static values are suitable for the test.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: "some-test-id" # A literal value can be provided inline for parameters that matches the json schema of the parameter as defined in the associated operation
        requestBody:
          contentType: application/json
          payload: { # literals values that match the content type of the request body can be provided inline
            "email": "Trystan_Crooks@hotmail.com",
            "first_name": "Trystan",
            "last_name": "Crooks",
            "age": 32,
            "postal_code": 94110,
            "metadata": {
              "allergies": "none",
              "color": "red",
              "height": 182,
              "weight": 77,
              "is_smoking": true
            }
          }
        successCriteria:
          - condition: $statusCode == 200

Payload Values

When using the payload field of a request body input, the value can be a static value to use, a value with interpolated Runtime Expressions or a Runtime Expression by itself.

The payload value can then be overlayed using the replacements field which represents a list of targets within the payload to replace with the value of the replacements, which themselves can be a static valuue or a Runtime Expression.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: get
        # ...
        outputs:
          user: $response.body
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: "some-test-id"
        requestBody:
          contentType: application/json
          payload: $steps.get.outputs.user # use the response body of the previous step as the payload for this step
          replacements: # overlay the payload with the below replacements
            - target: /postal_code # overlays the postal_code field with a static value
              value: 94107
            - target: /age # overlays the age field with the value of the age output of a previous step
              value: $steps.some-other-step.outputs.age
        successCriteria:
          - condition: $statusCode == 200

Outputs

As shown above, outputs can be defined for each step in a workflow allowing values from things such as the response body to be used as values in following steps.

Current Speakeasy supports only referencing values from a response body, using the Runtime Expressions syntax and json-pointers.

Any number of outputs can be defined for a step.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: #....
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
        outputs: # Outputs are a map of an output id to a runtime expression that will be used to populate the output
          id: $response.body#/id # json-pointers are used to reference fields within the response body
          email: $response.body#/email
          age: $response.body#/age
          allergies: $response.body#/metadata/allergies

Success Criteria

The successCriteria field of a step is a list of Criterion Objects that are used to validate the success of the step. For test generation these will form the basis of the test assertions.

successCriteria can be as simple as a single condition testing the status code of the response, or as complex as testing multiple individual fields within the response body.

Speakeasy’s implementation currently only supports simple criteria and the equality operator == for comparing values, and testing status codes, response headers and response bodies.

For testing values within the response body, criteria for testing the status code and content type of the response are also required, to help the generator determine which response schema to validate against due to the typed nature of the SDKs.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: some-test
    steps:
      - stepId: create
        operationId: createUser
        requestBody:
          contentType: application/json
          payload: #....
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/json
          - condition: $response.body#/email == Trystan_Crooks@hotmail.com
          # or
          - context: $response.body
            type: simple
            condition: |
              {
                "email": "Trystan_Crooks@hotmail.com",
                "first_name": "Trystan",
                "last_name": "Crooks",
                "age": 32,
                "postal_code": 94110,
                "metadata": {
                  "allergies": "none",
                  "color": "red",
                  "height": 182,
                  "weight": 77,
                  "is_smoking": true
                }
              }

Testing operations requiring binary data

Some operations will required providing binary data to test uploading or downloading files etc. In these cases test files can be provided to the test using the x-file directive in the example for that field.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: postFile
    steps:
      - stepId: test
        operationId: postFile
        requestBody:
          contentType: multipart/form-data
          payload:
            file: "x-file: some-test-file.txt"
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.header.Content-Type == application/octet-stream
          - context: $response.body
            condition: "x-file: some-other-test-file.dat"
            type: simple

The files will be sourced from the .speakeasy/testfiles directory in the root of your SDK repo, where the path provided in the x-file directive is relative to the testfiles directory.

The contents of the sourced file will be used as the value for the field being tested.

Configuring an API to Test Against

By default, tests will be generated to run against Speakeasy’s mock server (URL of http://localhost:18080) which will validate the SDKs are functioning correctly but not guaranteeing the correctness of the API.

The generator can be configured to run all tests against another URL or just individual tests. This is done through the use x-speakeasy-test-server extensions in the .speakeasy/tests.arazzo.yaml file.

If the extension is found at the top level of the Arazzo file then all workflows/tests will be configured to run againsts the specified server URL. If the extension is found within a workflow then only that workflow will be configured to run against the specified server URL.

The server URL can either be a static URL or a x-env: EXAMPLE_ENV_VAR value that will be pull the value from the environment variable EXAMPLE_ENV_VAR where the name of the environment variable can be any specified name.

arazzo: 1.0.0
# ...
x-speakeasy-test-server: 
  baseUrl: "https://api.example.com" # If specified at the top level of the Arazzo file, all workflows will be configured to run against the specified server URL
workflows:
  - workflowId: some-test
    x-speakeasy-test-server: 
      baseUrl: "x-env: TEST_SERVER_URL" # If specified within a workflow, only that workflow will be configured to run against the specified server URL. This will override any top level configuration.
    # ...

A default value can be provided in the x-env directive if the environment variable is not set. This can be useful for local development or non-production environments.

x-speakeasy-test-server: 
  baseUrl: "x-env: TEST_SERVER_URL; http://localhost:18080" # Run against the local mock server if the environment variable is not set

If all tests are configured to run against other server URLs, mock server generation can be disabled within the .speakeasy/gen.yaml file.

# ...
generation:
  # ...
  mockServer:
    disabled: true # Setting this to true will disable mock server generation

Configuring security credentials for Contract tests

When running tests against a real API, the SDK may need to be configured with security credentials to authenticate with the API. This can be done by adding the x-speakeasy-test-security extension to the document, a workflow or a individual step.

The x-speakeasy-test-security extension allows static values or values pulled from the environment to be used when instantiating an SDK instance and making requests to the API.

arazzo: 1.0.0
# ...
x-speakeasy-test-security: # Defined at the top level of the Arazzo file, all workflows will be configured to use the specified security credentials
  value:
    # The keys in the value map are the names of the security schemes defined in the OpenAPI document.
    # For simple schemes, the values is for example the API key value required.
    # For schemes like basic auth or OAuth2 that require multiple values, the value is a map of the required values.
    apiKey: x-env: TEST_API_KEY # Values can be pulled from the environment
    basicAuth:
      username: "test-user" # Or defined as static values
      password: x-env: TEST_PASSWORD
workflows:
  - workflowId: some-test
    x-speakeasy-test-security: # Security can be defined/overridden for a specific workflow
      value:
        apiKey: "test-key"
    # ...
    steps:
      - stepId: step1
        x-speakeasy-test-security: # Or security can be defined/overridden for a specific step
          value:
            authToken: x-env: TEST_AUTH_TOKEN
        # ...
      - stepId: step2
        # ...

Configuring environment variable provided values for Contract tests

When running tests against a real API, you may need to fill in certain input values from dynamic environment variables. This can be done using the Speakeasy environment variable extension.

arazzo: 1.0.0
# ....
workflows:
  - workflowId: my-env-var-test
    steps:
      - stepId: update
        operationId: updateUser
        parameters:
          - name: id
            in: path
            value: "x-env: TEST_ID; default" # Provide an environment variable and an optional default value if that env variable is not present.
        requestBody:
          contentType: application/json
          payload: {
            "email": "x-env: TEST_EMAIL; default, # Provide an environment variable and an optional default value if that env variable is not present.
            "first_name": "Trystan",
            "last_name": "Crooks",
            "age": 32,
            "postal_code": 94110,
            "metadata": {
              "allergies": "none",
              "color": "red",
              "height": 182,
              "weight": 77,
              "is_smoking": true
            }
          }
        successCriteria:
          - condition: $statusCode == 200