Background image

End-to-end API testing with Arazzo, TypeScript, and Deno

Brian Flad

Brian Flad

October 30, 2024

Featured blog post image

We've previously written about the importance of building contract & integration tests to comprehensively cover your API's endpoints, but there's still a missing piece to the puzzle. Real users don't consume your API one endpoint at a time - they implement complex workflows that chain multiple API calls together.

That's why reliable end-to-end API tests are an important component of the testing puzzle. For your APIs most common workflows, you need to ensure that the entire process works as expected, not just the individual parts.

In this tutorial, we'll build a test generator that turns Arazzo specifications into executable end-to-end tests. You'll learn how to:

  • Generate tests that mirror real user interactions with your API
  • Keep tests maintainable, even as your API evolves
  • Validate complex workflows across multiple API calls
  • Catch integration issues before they reach production

We'll use a simple "Build-a-bot" API as our example, but the principles and code you'll learn apply to any REST API.

Arazzo? What & Why

Arazzo is a specification that describes how API calls should be sequenced to achieve specific outcomes. Think of it as OpenAPI for workflows - while OpenAPI describes what your API can do, Arazzo describes how to use it effectively.

Arazzo was designed to bridge the gap between API reference documentation and real-world usage patterns. Fortunately for us, it also makes a perfect fit for generating end-to-end test suites that validate complete user workflows rather than isolated endpoints.

By combining these specifications, we can generate tests that validate not just the correctness of individual endpoints, but the entire user journey.

Info Icon

Arazzo?

Arazzo roughly translates to "tapestry" in Italian. Get it? A tapestry of API calls "woven" together to create a complete user experience. We're still undecided about how to pronounce it, though. The leading candidates are "ah-RAT-so" (like fatso) and "ah-RAHT-zoh" (almost like pizza, but with a rat). There is a minor faction pushing for "ah-razzo" as in razzle-dazzle. We'll let you decide.

Let's look at a simplified (and mostly invalid) illustrative example. Imagine a typical e-commerce API workflow:

arazzo.yaml

arazzo: 1.0.0
workflowId: purchaseProduct
sourceDescriptions:
- url: ./openapi.yaml
steps:
- stepId: authenticate
operationId: loginUser
# post login details
# response contains auth token
# if successful, go to checkInventory
- stepId: checkInventory
operationId: getProductsStock
# with auth token from previous step:
# get stock levels of multiple products
# response contains product IDs and stock levels
# if stock levels are sufficient, go to createOrder
- stepId: createOrder
operationId: submitOrder
# with auth token from first step
# and product IDs and quantities from previous step
# post an order that is valid based on stock levels
# response contains order ID
# if successful, go to getOrder
# ...

Arazzo allows us to define these workflows, and specify how each step should handle success and failure conditions, as well as how to pass data between steps and even between workflows.

From specification to implementation

The example above illustrates the concept, but let's dive into a working implementation. We'll use a simplified but functional example that you can download and run yourself. Our demo implements a subset of the Arazzo specification, focusing on the most immediately valuable features for E2E testing.

We'll use the example of an API called Build-a-bot, which allows users to create and manage their own robots. You can substitute this with your own OpenAPI document, or use the Build-a-bot API to follow along.

Arazzo deep dive

Let's examine the key components of an Arazzo specification using our Build-a-bot API example. The specification describes a workflow for creating, assembling, and activating a robot - a perfect example of a complex, multi-step process that would be difficult to test with traditional approaches.

The header identifies this as an Arazzo 1.0.0 specification and references the OpenAPI document. This connection between Arazzo and OpenAPI is crucial - it allows us to validate that the workflow steps match the API's capabilities.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

Each workflow has a unique ID and can specify required inputs. In this case, we need an API key for authentication.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

Let's look at the first step in our workflow. This step demonstrates several key Arazzo features.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

Each step has a unique stepId, and a description. The stepId is particularly important, as it allows us to reference this step's outputs in subsequent steps.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

Each step also specifies an operationId, which corresponds to an operation in the OpenAPI document. This connection ensures that the workflow steps are always in sync with the API's capabilities.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

A step can define a list of parameters that should be passed to the operation. These parameters can be static values, references to outputs from previous steps, runtime expressions that reference many other variables, or reusable objects.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

In this example, the x-api-key header is required for authentication. The value for the header is gathered at runtime from the workflow's inputs, using a runtime expression: $inputs.BUILD_A_BOT_API_KEY. We'll explore other available expressions later.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

Next up, the step defines a requestBody object. This object specifies the request body for the operation, which is required for creating a new robot design session.

Because the request body doesn't specify a contentType, Arazzo will look up the content-type in the OpenAPI document for the operation.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

This brings us to the meat of the step, the successCriteria list. This list defines the conditions that must be met for the step to be considered successful.

Each success criterion in the step serves a specific purpose.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

First, we validate the HTTP status code. This is a basic but essential check - the robot-creation endpoint should return 201 Created.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

Next, we validate the robotId using a regex pattern. This ensures the ID follows the expected UUID v4 format. Notice how we use the context field to specify which part of the response to validate, and the type field to indicate we're using a regex pattern.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

The next three criteria validate specific fields in the response body using direct comparisons. These ensure the robot is created with the correct model, name, and initial status.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

Finally, we use JSONPath to validate the structure of the links array. The condition $.length == 5 checks that exactly five links are returned.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

After successful validation, the step defines its outputs and next action.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

The outputs section extracts the robotId from the response body using a JSON pointer. This ID will be available to subsequent steps through the runtime expression $steps.createRobot.outputs.robotId.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

The onSuccess action specifies that after successful validation, the workflow should proceed to the addParts step. This explicit flow control helps maintain clear step sequencing.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

The header identifies this as an Arazzo 1.0.0 specification and references the OpenAPI document. This connection between Arazzo and OpenAPI is crucial - it allows us to validate that the workflow steps match the API's capabilities.

Each workflow has a unique ID and can specify required inputs. In this case, we need an API key for authentication.

Let's look at the first step in our workflow. This step demonstrates several key Arazzo features.

Each step has a unique stepId, and a description. The stepId is particularly important, as it allows us to reference this step's outputs in subsequent steps.

Each step also specifies an operationId, which corresponds to an operation in the OpenAPI document. This connection ensures that the workflow steps are always in sync with the API's capabilities.

A step can define a list of parameters that should be passed to the operation. These parameters can be static values, references to outputs from previous steps, runtime expressions that reference many other variables, or reusable objects.

In this example, the x-api-key header is required for authentication. The value for the header is gathered at runtime from the workflow's inputs, using a runtime expression: $inputs.BUILD_A_BOT_API_KEY. We'll explore other available expressions later.

Next up, the step defines a requestBody object. This object specifies the request body for the operation, which is required for creating a new robot design session.

Because the request body doesn't specify a contentType, Arazzo will look up the content-type in the OpenAPI document for the operation.

This brings us to the meat of the step, the successCriteria list. This list defines the conditions that must be met for the step to be considered successful.

Each success criterion in the step serves a specific purpose.

First, we validate the HTTP status code. This is a basic but essential check - the robot-creation endpoint should return 201 Created.

Next, we validate the robotId using a regex pattern. This ensures the ID follows the expected UUID v4 format. Notice how we use the context field to specify which part of the response to validate, and the type field to indicate we're using a regex pattern.

The next three criteria validate specific fields in the response body using direct comparisons. These ensure the robot is created with the correct model, name, and initial status.

Finally, we use JSONPath to validate the structure of the links array. The condition $.length == 5 checks that exactly five links are returned.

After successful validation, the step defines its outputs and next action.

The outputs section extracts the robotId from the response body using a JSON pointer. This ID will be available to subsequent steps through the runtime expression $steps.createRobot.outputs.robotId.

The onSuccess action specifies that after successful validation, the workflow should proceed to the addParts step. This explicit flow control helps maintain clear step sequencing.

arazzo.yaml

arazzo: 1.0.0
info:
title: Build-a-Bot Workflow
version: 1.0.0
description: This workflow guides you through the process of creating, assembling, and activating a robot using the Build-a-Bot API.
sourceDescriptions:
- name: buildABotAPI
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: createAndActivateRobot
description: Create, assemble, and activate a new robot
inputs:
type: object
properties:
BUILD_A_BOT_API_KEY:
type: string
description: The API key for the Build-a-Bot API
steps:
- stepId: createRobot
description: Create a new robot design session
operationId: createRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
requestBody:
payload:
model: humanoid
name: MyFirstRobot
successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath
onSuccess:
- name: gotoAddParts
type: goto
stepId: addParts
outputs:
robotId: $response.body#/robotId
- stepId: addParts
description: Add parts to the robot
operationId: addRobotParts
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
parts:
- type: arm
name: Hydraulic Arm
quantity: 2
- type: sensor
name: Infrared Sensor
quantity: 1
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoAssembleRobot
type: goto
stepId: assembleRobot
- stepId: assembleRobot
description: Assemble the robot
operationId: assembleRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "assembled"
onSuccess:
- name: gotoConfigureRobotFeatures
type: goto
stepId: configureRobotFeatures
- stepId: configureRobotFeatures
description: Configure robot features
operationId: configureRobotFeatures
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
requestBody:
payload:
features:
- name: Voice Recognition
description: Enables voice command recognition.
- name: Obstacle Avoidance
description: Navigates around obstacles.
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
onSuccess:
- name: gotoActivateRobot
type: goto
stepId: activateRobot
- stepId: activateRobot
description: Activate the robot
operationId: activateRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
outputs:
activationTime: $response.body#/activationTime
capabilities: $response.body#/capabilities
onSuccess:
- name: gotoGetRobot
type: goto
stepId: getRobot
- stepId: getRobot
description: Get the robot details
operationId: getRobot
parameters:
- name: x-api-key
in: header
value: $inputs.BUILD_A_BOT_API_KEY
- name: robotId
in: path
value: $steps.createRobot.outputs.robotId
successCriteria:
- condition: $statusCode == 200
- condition: $response.body#/robotId == $steps.createRobot.outputs.robotId
- condition: $response.body#/status == "activated"
- condition: $response.body#/activationTime == $steps.activateRobot.outputs.activationTime
- condition: $response.body#/capabilities == $steps.activateRobot.outputs.capabilities
outputs:
robot: $response.body
components: {}

This combination of validation patterns, data extraction, and flow control creates the foundation for testing complex API workflows.

Each success criterion serves a specific purpose in ensuring the API behaves as expected, while the outputs and success actions enable smooth workflow progression.

We'll explore the concepts we covered in more detail in the next sections, but first, let's set up a development environment so we can see Arazzo in action.

Setting up the development environment

First, clone the demo repository:


git clone https://github.com/speakeasy-api/e2e-testing-arazzo.git
cd e2e-testing-arazzo

You'll need Deno v2 (opens in a new tab) installed. On macOS and Linux, you can install Deno using the following command:


curl -fsSL https://deno.land/install.sh | sh

The repository contains:

  • A simple API server built with @oak/acorn (opens in a new tab) that serves as the Build-a-bot API (in packages/server/server.ts)
  • An Arazzo specification file (arazzo.yaml)
  • An OpenAPI specification file (openapi.yaml)
  • The test generator implementation (packages/arazzo-test-gen/generator.ts)
  • Generated E2E tests (tests/generated.test.ts)
  • An SDK created by Speakeasy to interact with the Build-a-bot API (packages/sdk)

Running the Demo

To run the demo, start the API server:


deno task server

Deno will install the server's dependencies, then start the server on http://localhost:8080. You can test the server by visiting http://localhost:8080/v1/robots, which should return a 401 Unauthorized error:


{
"status": 401,
"error": "Unauthorized",
"message": "Header x-api-key is required"
}

Next, in a new terminal window, generate the E2E tests:


deno task dev

After installing dependencies, this command will generate the E2E tests in tests/generated.test.ts and watch for changes to the Arazzo specification file.

You can run the tests in a new terminal window:


deno task test

This command will run the generated tests against the API server:

> deno task test
Task test deno test --allow-read --allow-net --allow-env --unstable tests/
running 1 test from ./tests/generated.test.ts
Create, assemble, and activate a new robot ...
  Create a new robot design session ... ok (134ms)
  Add parts to the robot ... ok (2ms)
  Assemble the robot ... ok (1ms)
  Configure robot features ... ok (2ms)
  Activate the robot ... ok (3ms)
  Get the robot details ... ok (1ms)
Create, assemble, and activate a new robot ... ok (143ms)
 
ok | 1 passed (6 steps) | 0 failed (147ms)

Beautiful, everything works! Let's see how we got here.

Building an Arazzo test generator

Let's start with the overall structure of the test generator.

Project structure

The test generator is a Deno project that consists of several modules, each with a specific responsibility:

  • generator.ts: The main entry point that orchestrates the test generation process. It reads the Arazzo and OpenAPI specifications, validates their compatibility, and generates test cases.

  • readArazzoYaml.ts and readOpenApiYaml.ts: Handle parsing and validation of the Arazzo and OpenAPI specifications respectively. They ensure the specifications are well-formed and contain all required fields.

  • expressionParser.ts: A parser for runtime expressions like $inputs.BUILD_A_BOT_API_KEY and $steps.createRobot.outputs.robotId. These expressions are crucial for passing data between steps and accessing workflow inputs.

  • successCriteria.ts: Processes the success criteria for each step, including status code validation, regex patterns, direct comparisons, and JSONPath expressions.

  • generateTestCase.ts: Takes the parsed workflow and generates the actual test code, including setup, execution, and validation for each step.

  • security.ts: Handles security-related aspects like API key authentication and other security schemes defined in the OpenAPI specification.

  • utils.ts: Contains utility functions for common operations like JSON pointer resolution and type checking.

The project also includes a runtime-expression directory containing the grammar definition for runtime expressions:

  • runtimeExpression.peggy: A Peggy grammar file that defines the syntax for runtime expressions
  • runtimeExpression.js: The generated parser from the grammar
  • runtimeExpression.d.ts: TypeScript type definitions for the parser

Let's dive deeper into each of these components to understand how they work together to generate effective E2E tests.

Parsing until you parse out

While our project says "test generator" on the tin, the bulk of our work will go into parsing different formats. To generate tests from an Arazzo document, we need to parse:

  1. The Arazzo document
  2. The OpenAPI document
  3. Conditions in the Arazzo success criteria
  4. Runtime expressions in the success criteria, outputs, and parameters
  5. Regular expressions in the success criteria
  6. JSONPath expressions in the success criteria
  7. JSON pointers in the runtime expressions

We won't cover all of these in detail, but we'll touch on each to get a sense of the complexity involved and the tools we use to manage it.

Parsing the Arazzo specification

The first step in our test generator is parsing the Arazzo specification in readArazzoYaml.ts. This module reads the Arazzo YAML file and should ideally validate its structure against the Arazzo specification.

For our demo, we didn't implement full validation, instead parsing the YAML file into a JavaScript object. We then use TypeScript interfaces to define the expected structure of the Arazzo document:


export interface ArazzoDocument {
arazzo: string;
info: ArazzoInfo;
sourceDescriptions: Array<ArazzoSourceDescription>;
workflows: Array<ArazzoWorkflow>;
components: Record<string, unknown>;
}
export interface ArazzoWorkflow {
workflowId: string;
description: string;
inputs: {
type: string;
properties: Record<string, { type: string; description: string }>;
};
steps: Array<ArazzoStep>;
}
export interface ArazzoStep {
stepId: string;
description: string;
operationId: string;
parameters?: Array<ArazzoParameter>;
requestBody?: ArazzoRequestBody;
successCriteria: Array<ArazzoSuccessCriterion>;
outputs?: Record<string, string>;
}

These TypeScript interfaces help with autocompletion, type checking, and documentation, making it easier to work with the parsed Arazzo document in the rest of our code.

The real complexity comes in validating that the parsed document follows all the rules in the Arazzo specification. For example:

  • Each workflowId must be unique within the document
  • Each stepId must be unique within its workflow
  • An operationId must reference a valid operation in the OpenAPI document
  • Runtime expressions must follow the correct syntax
  • Success criteria must use valid JSONPath or regex patterns

We don't validate all these rules in our demo, but in production, we'd use Zod (opens in a new tab) or Valibot (opens in a new tab) to enforce these constraints at runtime and provide helpful error messages when the document is invalid.

The OpenAPI team hasn't finalized the Arazzo specification's JSON Schema yet, but once they do, we can use it to validate the Arazzo document against the schema with tools like Ajv (opens in a new tab).

Speakeasy also provides a command-line interface for linting Arazzo documents:


# Expects arazzo.yaml in the current directory
speakeasy lint arazzo

Parsing the OpenAPI specification

The OpenAPI specification's path is gathered from the Arazzo document. In our test, we simply use the first sourceDescription to find the OpenAPI document path. But in a production generator, we'd need to handle multiple sourceDescriptions and ensure the OpenAPI document is accessible.

We parse the OpenAPI document in readOpenApiYaml.ts and use TypeScript interfaces from the npm:openapi-types (opens in a new tab) package to define the expected structure of the OpenAPI document.

We won't cover the OpenAPI parsing in detail, but it's similar to the Arazzo parsing: Read the YAML file, parse it into a JavaScript object, and type check it against TypeScript interfaces.

For OpenAPI, writing a custom validator is more complex due to the specification's size and complexity. We recommend validating against the official OpenAPI 3.1.0 JSON Schema (opens in a new tab) using Ajv (opens in a new tab), or Speakeasy's own OpenAPI linter:


speakeasy lint openapi --schema openapi.yaml

Parsing success criteria

This is where things get interesting. Success criteria in Arazzo are a list of conditions that must be met for a step to be considered successful. Each criterion can be one of the following types:

  • simple: Selects a value with a runtime expression and compares it to an expected value
  • jsonpath: Selects a value using a JSONPath expression and compares it to an expected value
  • regex: Validates a value against a regular expression pattern
  • xpath: Selects a value using an XPath expression and compares it to an expected value, used for XML documents

In our demo, we don't implement the xpath type, but we do cover the other three. Here's an example of a success criterion in the Arazzo document:


successCriteria:
- condition: $statusCode == 201
- condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
context: $response.body#/robotId
type: regex
- condition: $response.body#/model == "humanoid"
- condition: $response.body#/name == "MyFirstRobot"
- condition: $response.body#/status == "designing"
- context: $response.body#/links
condition: $.length == 5
type: jsonpath

The condition field is required, and contains the expression to evaluate, while the context field specifies the part of the response to evaluate. The type field indicates the type of validation to perform.

If no type is specified, the success criterion is treated as a simple comparison, where the condition is evaluated directly.

Evaluating simple criteria

Here's an example of how we parse a simple success criterion:


condition: $statusCode == 201

We split this simple condition into:

  • Left-hand side: $statusCode - Runtime expression to evaluate
  • Operator: == - Comparison operator or assertion
  • Right-hand side: 201 - Expected value

We'll evaluate the runtime expression $statusCode and compare it to the expected value 201. If the comparison is true, the criterion passes; otherwise, it fails.

Runtime expressions can also reference other variables, like $inputs.BUILD_A_BOT_API_KEY or $steps.createRobot.outputs.robotId, or fields in the response body, like $response.body#/model.

We'll cover runtime expressions in more detail later.

Evaluating JSONPath criteria

For JSONPath criteria, we use the jsonpath type and a JSONPath expression to select a value from the response:


context: $response.body#/links
condition: $.length == 5
type: jsonpath

Let's break down the JSONPath criterion:

  • context: $response.body#/links - Runtime expression to select the links array from the response body
  • condition: $.length == 5 - JSONPath expression compared to an expected value
  • type: jsonpath - Indicates the criterion type

We further need to break down the condition into:

  • Left-hand side: $.length - JSONPath expression to evaluate
  • Operator: == - Comparison operator
  • Right-hand side: 5 - Expected value

We evaluate the JSONPath expression $.length and compare it to the expected value 5. If the comparison is true, the criterion passes.

Evaluating regex criteria

For regex criteria, we use the regex type and a regular expression pattern to validate a value:


context: $response.body#/robotId
condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i
type: regex

Let's break down the regex criterion:

  • context: $response.body#/robotId - Runtime expression to select the robotId field from the response body
  • condition: /^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i - Regular expression pattern to validate the value as a UUID v4
  • type: regex - Indicates the criterion type

We evaluate the runtime expression $response.body#/robotId against the regular expression pattern. If the value matches the pattern, the criterion passes.

In our implementation, we use TypeScript's factory.createRegularExpressionLiteral to create a regular expression literal from the pattern string. This ensures that the pattern is properly escaped and formatted as a valid JavaScript regular expression.

The generated test code looks something like this:


assertMatch(
response.body.robotId,
new RegExp(/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i),
'robotId should be a valid UUID v4'
);

This code uses Deno's built-in assertMatch function to validate that the robotId matches the UUID v4 pattern. If the value doesn't match, the test fails with a helpful error message.

Parsing runtime expressions

Runtime expressions are used throughout the Arazzo specification to reference variables, fields in the response body, or outputs from previous steps. They follow a specific syntax defined in the Arazzo specification as an ABNF (augmented Backus–Naur form) grammar.

To parse runtime expressions, we use a parser generated from the ABNF grammar. In our demo, this is a two-step process. First, we use the abnf (opens in a new tab) npm package to generate a Peggy grammar file from the ABNF grammar:


cd packages/arazzo-test-gen
abnf_gen runtime-expression/runtimeExpression.abnf

This generates a runtime-expression/runtimeExpression.peggy file that defines the syntax for runtime expressions. We then use the peggy (opens in a new tab) npm package to generate a parser from the Peggy grammar:


cd packages/arazzo-test-gen
peggy --dts --output runtime-expression/runtimeExpression.js --format es runtime-expression/runtimeExpression.peggy

This generates a runtime-expression/runtimeExpression.js file that contains the parser for runtime expressions. We also generate TypeScript type definitions in runtime-expression/runtimeExpression.d.ts.

The parser reads a runtime expression like $response.body#/robotId and breaks it down into tokens. We then evaluate the tokens to resolve the expression at runtime.

Evaluating runtime expressions

Once we've parsed a runtime expression, we need to evaluate it to get the value it references. For example, given the expression $response.body#/robotId, we need to extract the robotId field from the response body.

The evaluateRuntimeExpression function in utils.ts handles this evaluation. Here's an example of how it works:


switch (root) {
case "$statusCode": {
// Handle $statusCode expressions
result = factory.createPropertyAccessExpression(
factory.createIdentifier("response"),
factory.createIdentifier("status"),
);
break;
}
case "$response.": {
// Handle $request and $response expressions
const data = factory.createIdentifier("data");
// use parseRef to handle everything after $response.body
const pointer = parsePointer(expression.slice(15));
result = pointer.length > 0
? factory.createPropertyAccessExpression(data, pointer.join("."))
: data;
break;
}
// Handle other cases ...
}

Here, we handle two types of runtime expressions: $statusCode and $response.body. We extract the status field from the response object for $statusCode, and the body object from the response object for $response.body.

We use the TypeScript compiler API to generate an abstract syntax tree (AST) that represents the expression. This AST is then printed to a string and saved as a source file that Deno can execute.

Supported runtime expressions

In our demo, we support a limited set of runtime expressions:

  • $statusCode: The HTTP status code of the response
  • $steps.stepId.outputs.field: The output of a previous step
  • $response.body#/path/to/field: A field in the response body selected by a JSON pointer

Arazzo supports many more runtime expressions, for example:

ExpressionDescription
$urlThe full URL of the request
$methodThe HTTP method of the request
$statusCodeThe HTTP status code of the response
$request.header.{name}The value of the specified request header
$request.query.{name}The value of the specified query parameter from the request URL
$request.path.{name}The value of the specified path parameter from the request URL
$request.bodyThe entire request body
$request.body#/path/to/propertyThe value of the specified JSON pointer path from the request body
$response.header.{name}The value of the specified response header
$response.bodyThe entire response body
$response.body#/path/to/propertyThe value of the specified JSON pointer path from the response body
$inputs.{name}The value of the specified workflow input
$outputs.{name}The value of the specified workflow output
$steps.{stepId}.{outputName}The value of the specified output from the step with ID {stepId}
$workflows.{id}.{inputName}The value of the specified input from the workflow with ID {id}
$workflows.{id}.{outputName}The value of the specified output from the workflow with ID {id}

Parsing regular expressions

Regular expressions in Arazzo are used to validate string values against patterns. They're particularly useful for validating IDs, dates, and other structured strings.

In our implementation, we handle regex patterns in the parseRegexCondition function:


function parseRegexCondition(
condition: string,
usedAssertions: Set<string>,
context: string,
): Expression {
usedAssertions.add("assertMatch");
return factory.createCallExpression(
factory.createIdentifier("assertMatch"),
undefined,
[
evaluateRuntimeExpression(context),
factory.createNewExpression(
factory.createIdentifier("RegExp"),
undefined,
[factory.createRegularExpressionLiteral(condition)],
),
factory.createStringLiteral(condition),
],
);
}

This function takes three parameters:

  • condition: The regex pattern to match against
  • usedAssertions: A set to track which assertion functions we've used
  • context: The runtime expression that selects the value to validate

The function generates code that:

  1. Evaluates the context expression to get the value to validate
  2. Creates a new RegExp object from the pattern
  3. Uses Deno's assertMatch function to validate the value against the pattern

The generated code looks like this:


assertMatch(
response.body.robotId,
new RegExp(/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i),
'robotId should be a valid UUID v4'
);

This approach has several advantages:

  • It preserves the original pattern's flags (like i for case-insensitive matching).
  • It provides clear error messages when validation fails.
  • It integrates well with Deno's testing framework.

In a production implementation, we'd want to add:

  • Validation of the regex pattern syntax
  • Support for named capture groups
  • Error handling for malformed patterns
  • Performance optimizations like pattern caching

But for our demo, this simple implementation is sufficient to show how regex validation works in Arazzo.

Parsing JSONPath expressions

JSONPath expressions are a powerful way to query JSON data. In Arazzo, we use them in success criteria to select objects or values from complex response structures. While JSON Pointer (which we'll cover next) is great for accessing specific values, JSONPath shines when you need to:

  • Validate arrays (for example, checking array length)
  • Filter elements (for example, finding items matching a condition)
  • Access deeply nested data with wildcards
  • Aggregate values (for example, counting matches)

Here's how our test generator handles JSONPath expressions:


function parseJsonPathExpression(path: string, context: string): Expression {
return factory.createCallExpression(
factory.createIdentifier("JSONPath"),
undefined,
[
factory.createObjectLiteralExpression(
[
factory.createPropertyAssignment(
factory.createIdentifier("wrap"),
factory.createFalse(),
),
factory.createPropertyAssignment(
factory.createIdentifier("path"),
factory.createStringLiteral(path),
),
factory.createPropertyAssignment(
factory.createIdentifier("json"),
evaluateRuntimeExpression(context),
),
],
true,
),
],
);
}

This function generates code that evaluates a JSONPath expression against a context object (usually the response body). For example, given this success criterion:


successCriteria:
- context: $response.body#/links
condition: $.length == 5
type: jsonpath

Our generator creates a test that:

  1. Extracts the links array from the response body using a JSON pointer
  2. Evaluates the JSONPath expression $.length against this array
  3. Compares the result to the expected value 5

The generated test code looks something like this:


assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: response.body.links
}),
5,
"links array should contain exactly 5 elements"
);

JSONPath is particularly useful for validating:

  • Array operations: $.length, $[0], $[(@.length-1)]
  • Deep traversal: $..name (all name properties at any depth)
  • Filtering: $[?(@.status=="active")] (elements where status is active)
  • Wildcards: $.*.name (name property of all immediate children)

A few things to keep in mind when using JSONPath:

  1. JSONPath isn't well standardized, so different implementations vary widely. Arazzo makes provisions for this by allowing us to specify the JSONPath version in the test specification.
  2. Even though we can specify a version, we still need to be cautious when using advanced features. Some features might not be supported by the chosen JSONPath library.
  3. Check the JSONPath comparison (opens in a new tab) page to see how different libraries handle various features, and decide which features are safe to use.

Parsing JSON Pointers

While JSONPath is great for complex queries, JSON Pointer (RFC 6901) (opens in a new tab) is perfect for directly accessing specific values in a JSON document. In Arazzo, we use JSON Pointers in runtime expressions to extract values from responses and pass them to subsequent steps.

Here's how our test generator handles JSON Pointers:


function evaluateRuntimeExpression(expression: string): Expression {
// ...
case "$response.": {
const data = factory.createIdentifier("data");
// Parse everything after $response.body
const pointer = parsePointer(expression.slice(15));
result = pointer.length > 0
? factory.createPropertyAccessExpression(data, pointer.join("."))
: data;
break;
}
// ...
}

This function parses runtime expressions that use JSON Pointers. For example, given this output definition:


outputs:
robotId: $response.body#/robotId

Our generator creates code that:

  1. Takes the part after # as the JSON Pointer (/robotId)
  2. Converts the pointer segments into property access expressions
  3. Generates code to extract the value

The generated test code looks something like this:


// During test setup
const context = {};
// ...
// In the first test
const data = response.json();
// Generated because of outputs: { robotId: $response.body#/robotId } in the Arazzo document
// highlight-next-line
context["createRobot.outputs.robotId"] = data.robotId;
// ...
// In a subsequent test
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
// ...
});

Generating end-to-end tests

Now that we understand how to parse Arazzo documents, let's look at how we generate executable tests from them. Our generator creates type-safe test code using TypeScript's factory methods rather than string templates, providing better error detection and maintainability.

Test structure

The generator creates a test suite for each workflow in the Arazzo document. Each step in the workflow becomes a test case that executes sequentially.

Let's explore the structure of a generated test case.

We start by setting up a test suite for the workflow, using the Arazzo workflow description as the suite name.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

Next we define the serverUrl, apiKey, and context variables. The serverUrl points to the API server. We use the servers list in the OpenAPI document to determine the server URL.

We also set up the apiKey for authentication. In our demo, we use a hardcoded API key, but in a real-world scenario, we'd likely get this after authenticating with the API.

We'll use the context object to store values extracted from the response body for use in subsequent steps.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

For each step in the workflow, we generate a test case that executes the step and validates the success criteria.

Our first step is to create a new robot design session.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

The HTTP method and path are extracted from the OpenAPI document using the operationId from the Arazzo step.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

We set up the request headers, including the x-api-key header for authentication.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

The request body is set up using the requestBody object from the Arazzo step.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

We extract the response body as JSON.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

We assert the success criteria for the step.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

Finally, we extract the outputs from the step and store them in the context object for use in subsequent steps.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

We start by setting up a test suite for the workflow, using the Arazzo workflow description as the suite name.

Next we define the serverUrl, apiKey, and context variables. The serverUrl points to the API server. We use the servers list in the OpenAPI document to determine the server URL.

We also set up the apiKey for authentication. In our demo, we use a hardcoded API key, but in a real-world scenario, we'd likely get this after authenticating with the API.

We'll use the context object to store values extracted from the response body for use in subsequent steps.

For each step in the workflow, we generate a test case that executes the step and validates the success criteria.

Our first step is to create a new robot design session.

The HTTP method and path are extracted from the OpenAPI document using the operationId from the Arazzo step.

We set up the request headers, including the x-api-key header for authentication.

The request body is set up using the requestBody object from the Arazzo step.

We extract the response body as JSON.

We assert the success criteria for the step.

Finally, we extract the outputs from the step and store them in the context object for use in subsequent steps.

generated.test.ts

/**
* This test file was generated from an arazzo.yaml document.
* Do not edit this file manually.
*/
import "jsr:@std/dotenv/load";
import { JSONPath } from "npm:jsonpath-plus";
import { assertEquals, assertMatch } from "jsr:@std/assert";
Deno.test("Create, assemble, and activate a new robot", async (test) => {
const serverUrl = "http://localhost:8080";
const apiKey = Deno.env.get("BUILDABOT_API_KEY_AUTH");
const context: Record<string, unknown> = {};
await test.step("Create a new robot design session", async () => {
const response = await fetch(`${serverUrl}/v1/robots`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: '{"model":"humanoid","name":"MyFirstRobot"}',
});
const data = await response.json();
assertEquals(response.status, 201, "$statusCode == 201");
assertMatch(
data.robotId,
new RegExp(
/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i,
),
"/^[0-9A-F]{8}-[0-9A-F]{4}-[4][0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i",
);
assertEquals(data.model, "humanoid", '$response.body#/model == "humanoid"');
assertEquals(
data.name,
"MyFirstRobot",
'$response.body#/name == "MyFirstRobot"',
);
assertEquals(
data.status,
"designing",
'$response.body#/status == "designing"',
);
assertEquals(
JSONPath({
wrap: false,
path: "$.length",
json: data.links,
}),
5,
"$.length == 5",
);
context["createRobot.outputs.robotId"] = data.robotId;
});
await test.step("Add parts to the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/parts`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"parts":[{"type":"arm","name":"Hydraulic Arm","quantity":2},{"type":"sensor","name":"Infrared Sensor","quantity":1}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Assemble the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/assemble`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"assembled",
'$response.body#/status == "assembled"',
);
});
await test.step("Configure robot features", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/features`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body:
'{"features":[{"name":"Voice Recognition","description":"Enables voice command recognition."},{"name":"Obstacle Avoidance","description":"Navigates around obstacles."}]}',
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
});
await test.step("Activate the robot", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}/activate`, {
method: "POST",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
context["activateRobot.outputs.activationTime"] = data.activationTime;
context["activateRobot.outputs.capabilities"] = data.capabilities;
});
await test.step("Get the robot details", async () => {
const robotId = context["createRobot.outputs.robotId"];
const response = await fetch(`${serverUrl}/v1/robots/${robotId}`, {
method: "GET",
headers: {
"X-API-Key": `${apiKey}`,
"Content-Type": "application/json",
},
body: null,
});
const data = await response.json();
assertEquals(response.status, 200, "$statusCode == 200");
assertEquals(
data.robotId,
context["createRobot.outputs.robotId"],
"$response.body#/robotId == $steps.createRobot.outputs.robotId",
);
assertEquals(
data.status,
"activated",
'$response.body#/status == "activated"',
);
assertEquals(
data.activationTime,
context["activateRobot.outputs.activationTime"],
"$response.body#/activationTime == $steps.activateRobot.outputs.activationTime",
);
assertEquals(
data.capabilities,
context["activateRobot.outputs.capabilities"],
"$response.body#/capabilities == $steps.activateRobot.outputs.capabilities",
);
context["getRobot.outputs.robot"] = data;
});
});

This structure repeats for each step in the workflow, creating a series of test cases that execute the workflow sequentially. The generated tests validate the API's behavior at each step, ensuring that the workflow progresses correctly.

Future development and improvements

Our generated tests are a good start, but they might not be truly end-to-end if we don't consider the interfaces our users interact with to access the API.

Testing with SDKs

In our demo, we use the fetch API to interact with the Build-a-bot API. While this is a common approach, it's not always the most user-friendly. Developers often prefer SDKs that provide a more idiomatic interface to the API.

To make our tests more end-to-end, we could use the SDK Speakeasy created from the OpenAPI document to interact with the API.

Since the SDK is generated from the OpenAPI document, with names and methods derived from the API's tags and operation IDs, we could use Arazzo to validate the SDK's behavior against the API's capabilities.

For example, we could:

  1. Get the operationId from the Arazzo step and derive the corresponding SDK method import.
  2. Call the SDK method with the required parameters.
  3. Validate the response against the success criteria.
  4. Extract the outputs from the response and store them in the context object.
  5. Repeat for each step in the workflow.

This approach would provide a more realistic end-to-end test, validating the SDK's behavior against the API's capabilities.

Handling authentication

In our demo, we use a hard-coded API key for authentication. In a real-world scenario, we'd likely need to authenticate with the API to get a valid API key.

OpenAPI also supports more advanced authentication schemes like OAuth 2.0, JWT, and API key in headers, query parameters, or cookies. Our test generator should handle these schemes to ensure the tests are realistic and cover all authentication scenarios.

Arazzo can point to the security schemes in the OpenAPI document, allowing us to extract the required authentication parameters and set them up in the test suite.

Hardening the parsers against vulnerabilities

Our parsers are simple and work well for the demo, but they lack robust error handling and edge case coverage.

For example, JSONPath-plus, the library we use for JSONPath, recently fixed a remote code execution vulnerability. We should ensure our parser is up to date and secure against similar vulnerabilities, or limit the JSONPath features we support to reduce the attack surface.

This applies to parsers in general, and the risk is even higher when parsing user input and generating code from it.

Deno provides some protection by limiting access to the filesystem and network by default, but the nature of API testing means we need to access the network and read files.

Where to next?

The Arazzo specification, although released as v1.0.0, is in active development. The OpenAPI team is working on a JSON Schema for Arazzo, which will provide a formal definition of the specification's structure and constraints.

We found the specification slightly ambiguous in places, but the team is active on GitHub (opens in a new tab) and open to feedback and contributions. If you're interested in API testing, Arazzo is a great project to get involved with.

At Speakeasy, we're building tools to make API testing easier and more effective. Our TypeScript, Python, and Go SDK generators can already generate tests from OpenAPI documents, and we're working on integrating Arazzo support. Our CLI can already lint Arazzo documents, and we'll have more to share soon.

We're excited to see how Arazzo evolves and how it can help developers build robust, end-to-end tests for their APIs.

CTA background illustrations

Speakeasy Changelog

Subscribe to stay up-to-date on Speakeasy news and feature releases.