# Customizing SDK Contract Tests ## Test Generation Overview When `generateNewTests` is enabled in the [`.speakeasy/gen.yaml`](/docs/speakeasy-reference/generation/gen-yaml) file, workflows for any newly discovered operations will automatically be created in the `.speakeasy/tests.arazzo.yaml` file during the next generation. Tests will then be generated based on these workflows. Note that for tests to actually be generated during [`speakeasy run`](/docs/speakeasy-reference/cli/run), the `generateTests` setting must also be enabled. Both settings can be enabled by running [`speakeasy configure tests`](/docs/sdk-testing/bootstrapping-test-generation#enabling-test-generation). ## Disable auto generation of tests for specific operations To disable auto generation of tests for specific operations, the `x-speakeasy-test` extension can be added to the operation in the OpenAPI document: ```yaml openapi: 3.1.0 # ... paths: /example1: get: x-speakeasy-test: false # Disables auto generation of tests for this operation # ... # ... ``` Alternatively, if a workflow/test already exists that references the operation in the `.speakeasy/tests.arazzo.yaml` file, then no new workflow will be created for the operation. ## Automatic Test Synchronization Speakeasy synchronizes workflows in the `.speakeasy/tests.arazzo.yaml` file based on the presence of the `x-speakeasy-test-rebuild` extension. When `generateNewTests` is enabled, Speakeasy automatically applies this extension to newly created workflows. The extension indicates that the workflows should remain synchronized with any subsequent changes to the OpenAPI specification. ### Auto-sync with OpenAPI Changes (`x-speakeasy-test-rebuild: true`) When set to `true`, the workflow will automatically be kept in sync with the state of the operation in the OpenAPI specification. The test will be rebuilt each time the spec changes, incorporating: - New parameters - Request body field changes - Response body updates - Updates to examples This is useful if you want Speakeasy to automatically maintain your tests as your API evolves. ```yaml arazzo: 1.0.0 # ... workflows: - workflowId: some-test x-speakeasy-test-rebuild: true # Test will auto-sync with OpenAPI changes # ... ``` ### Manual Test Maintenance (`x-speakeasy-test-rebuild: false` or omitted) When the extension is omitted or set to `false`, the workflow needs to be maintained manually and won't sync with the OpenAPI specification. This gives you complete control over the test content and is useful when: - Writing custom tests with specific logic - Maintaining test state independent from the OpenAPI document - Catching breaking changes or regressions in the API behavior ```yaml arazzo: 1.0.0 # ... workflows: - workflowId: some-test x-speakeasy-test-rebuild: false # Test requires manual maintenance # ... ``` ## Grouping tests By default, all tests will be generated into a single file related to the main SDK class for example `sdk.test.ts`, `test_sdk.py`, `SDKTest.java`, `test_sdk.rb`, or `SDKTests.cs`. This can be configured by adding a `x-speakeasy-test-group` extension to the workflow which will determine which tests will be grouped together in seperate test files. ```yaml arazzo: 1.0.0 # ... workflows: - workflowId: some-test x-speakeasy-test-group: user # All tests in the user group will end up in the `user.test.ts`/`test_user.py`/`user_test.go`/`test_user.rb`/`UserTests.cs` files # ... ``` ## Generate tests only for specific targets By default, all tests will be generated for all supported targets. Using the `x-speakeasy-test-targets` or `x-speakeasy-test-targets-exclude` extensions, it is possible to generate tests only for specific targets. This may be useful if tests are either not needed for certain languages or they may be failing in a certain language. ```yaml arazzo: 1.0.0 # ... workflows: - workflowId: some-test x-speakeasy-test-targets: # Only generate tests for the typescript target - typescript # ... ``` or ```yaml arazzo: 1.0.0 # ... workflows: - workflowId: some-test x-speakeasy-test-targets-exclude: # generate tests for all languages except typescript - typescript # ... ``` ## Customizing SDK Test Manifests Directly modifying SDK tests is not common, but it is possible to customize the actual content of an existing SDK test by modifying the `.speakeasy/tests.arazzo.yaml` file. To learn more about these kinds of modifications see [here](/docs/sdk-testing/api-contract-tests#writing-custom-end-to-end-tests). ## Skip response body assertions To omit all response body assertions from generated tests, set the `skipResponseBodyAssertions` option to `true` in the `generation.tests` section of the [`.speakeasy/gen.yaml`](/docs/speakeasy-reference/generation/gen-yaml) file: ```yaml generation: tests: skipResponseBodyAssertions: true ``` When enabled, generated tests still assert on status codes but do not assert on the shape or values of the response body. This is useful when: - API response bodies are non-deterministic (e.g., dynamic data, timestamps, random IDs) - A test server or mock doesn't return realistic response payloads - SDK calls need to be verified as successful without brittle assertions on exact response content This setting applies globally to all generated tests for the target. There is no per-operation override. ## Next Steps - [Setup testing in GitHub Actions](/docs/sdk-testing/github-actions) Once the original `tests.arazzo.yaml` document is bootstrapped, the Arazzo document will only be modified automatically when new operations are detected and `generateNewTests: true` is set in your `./speakeasy/gen.yaml`, new tests will be bootstrapped for these new operations. But pre-bootstrapped tests will be left alone. If you want to bootstrap all workflows in the Arazzo document from scratch again (for example to match changes to operations or examples in your spec), run the following command: ```bash speakeasy configure tests --rebuild ``` You can also selectively bootstrap individual tests by: 1. Deleting the specific workflow from the arazzo doc in `.speakeasy/tests.arazzo.yaml` 2. Removing the corresponding entry in `.speakeasy/gen.lock` under `generatedTests` For example, to bootstrap a test for the `createLink` operation again, delete its entry in the gen.lock file: ```yaml generatedTests: createLink: "2025-05-21T12:47:33+10:00" # Delete this to bootstrap test again from scratch ``` This approach preserves any other manual changes you've made to your arazzo doc.