Terraform

Migrate any Terraform provider with data transformations

Speakeasy Team

December 7, 2025 - 6 min read

---

Moving an existing Terraform provider to code generation isn’t always straightforward. The provider you built by hand likely evolved over time with naming conventions, structural decisions, and interface choices that made sense for your users—but don’t exactly match your API.

Maybe you simplified tag management by using a map instead of the array-of-objects structure your API returns. Perhaps you concatenated multiple API fields into a single identifier. These thoughtful design decisions improve the user experience, but they create a challenge when migrating to a provider generated from an OpenAPI spec.

We’ve added support for data transformations in Speakeasy’s Terraform provider generator. Using JQ expressions, you can transform data between your provider’s interface and your API’s structure, maintaining your existing interface while moving to generated code.

The migration challenge

Existing Terraform providers often diverge from their underlying APIs in useful ways. These differences accumulate as you refine the provider based on user feedback, add convenience features, or work around API quirks.

Here are common patterns that create migration challenges:

Structural simplification

Many APIs represent tags or labels as arrays of objects:

{
  "tags": [
    { "key": "environment", "value": "production" },
    { "key": "team", "value": "platform" },
    { "key": "cost-center", "value": "engineering" }
  ]
}

But from a Terraform user’s perspective, this structure is unnecessarily complex. A simple map is more ergonomic:

resource "platform_instance" "main" {
  name = "api-server"
 
  tags = {
    environment  = "production"
    team         = "platform"
    cost-center  = "engineering"
  }
}

Your hand-coded provider handles this transformation internally. Moving to generation means either accepting the more complex API structure or finding a way to maintain the simplified interface.

Composite identifiers

Some APIs split related identifiers across multiple fields, but Terraform providers often combine them for convenience:

# API has separate project_id, region, and name fields
# Provider combines them into a single ID for simplicity
resource "platform_database" "main" {
  id = "projects/my-project/regions/us-east1/databases/prod-db"
}

The provider parses this combined identifier internally when making API calls. Generated code would naturally use separate fields unless you can specify the transformation.

Field naming improvements

API field names don’t always align with Terraform conventions. Your provider might rename fields for consistency:

# API uses "displayName" but provider uses "name"
# API uses "maxInstances" but provider uses "max_instance_count"
resource "platform_service" "api" {
  name                = "api-service"  # API: displayName
  max_instance_count  = 10             # API: maxInstances
}

These naming improvements create a better experience, but they’re custom logic that needs to persist through migration.

Data transformations with JQ

Data transformations use JQ expressions  to convert between your provider interface and your API structure. JQ is a widely-used JSON manipulation language—if you’ve worked with complex JSON transformations, you’ve likely encountered it.

The transformation layer sits between Terraform and your API, automatically converting data in both directions:

• Requests: Transform Terraform configuration values into the structure your API expects
• Responses: Transform API responses into the structure your Terraform provider exposes

Converting tags to a map

Here’s how to transform tag arrays into the simpler map structure:

requestBody:
  content:
    application/json:
      schema:
        type: object
        properties:
          tags:
            type: array
            items:
              type: object
              properties:
                key:
                  type: string
                value:
                  type: string
      x-speakeasy-data-transform:
        schema:
          type: object
          additionalProperties:
            type: string
        transform: |
          .tags |= (
            if type == "object" then
              to_entries | map({key: .key, value: .value})
            else
              .
            end
          )

Users configure tags as a simple map in their Terraform configuration. The transformation converts it to the array structure before sending it to the API. Response transformations work in reverse, converting arrays back to maps.

Building composite identifiers

Create combined identifiers from separate API fields:

responses:
  "200":
    content:
      application/json:
        schema:
          type: object
          properties:
            project_id:
              type: string
            region:
              type: string
            name:
              type: string
        x-speakeasy-data-transform:
          schema:
            type: object
            properties:
              id:
                type: string
          transform: |
            .id = "projects/\(.project_id)/regions/\(.region)/databases/\(.name)"

The API returns three separate fields. The transformation concatenates them into a single identifier that Terraform users work with.

Renaming fields

Normalize field names to match Terraform conventions:

schema:
  type: object
  properties:
    displayName:
      type: string
    maxInstances:
      type: integer
x-speakeasy-data-transform:
  schema:
    type: object
    properties:
      name:
        type: string
      max_instance_count:
        type: integer
  transform: |
    {
      displayName: .name,
      maxInstances: .max_instance_count
    }

The transformation maps between your provider’s naming conventions and the API’s field names.

The JQ playground

Writing and testing JQ expressions requires iteration. We’ve built a playground specifically for Terraform provider transformations at jq.speakeasy.com .

The playground includes:

• Syntax highlighting and validation
• Real-time transformation preview
• Example transformations for common patterns
• Error messages that explain what went wrong

Use the playground to develop transformations before adding them to your OpenAPI specification. Test with sample data to verify the transformation produces the expected structure in both directions.

Beyond migration: forward-looking transformations

While migration is the primary use case, data transformations also support forward-looking changes. If you know your API will change in the future, you can implement the new structure in your Terraform provider ahead of time.

For example, suppose your API currently uses a field name you plan to deprecate:

# API currently uses 'fubar' but you're migrating to 'configuration'
x-speakeasy-data-transform:
  schema:
    type: object
    properties:
      configuration:
        type: object
  transform: |
    {
      fubar: .configuration
    }

Your Terraform provider exposes the new configuration field name while the API still expects fubar. When the API changes, you remove the transformation rather than forcing a breaking change on Terraform users.

This technique works for any planned API evolution where you want to minimize downstream impact.

When to use transformations

Data transformations add complexity. They’re valuable when:

• Migrating existing providers: Maintain your current interface while moving to generation
• Significant UX improvements: The transformation provides a meaningfully better user experience (like tags as maps)
• Planned API changes: You’re preparing for a known future API modification

Avoid transformations when:

• The API structure is reasonable: If your API’s structure works well in Terraform, don’t add unnecessary transformation logic
• Simple renames: Minor naming differences usually aren’t worth the complexity
• You can change the API: If you control the API and it’s not yet released, consider changing the API instead

Generated code that directly reflects your API is simpler and easier to maintain. Use transformations when the value clearly outweighs the added complexity.

Next steps

Data transformations are available now for Terraform provider generation. If you’re considering migrating an existing provider or need to bridge the gap between your API structure and ideal provider interface, check out the documentation:

• Terraform provider generation
• JQ playground 

Data transformations solve a specific problem: they let you maintain the provider interface you’ve carefully designed while moving to generated code. This bridges the gap between manual implementation and generation, making migration feasible even when your provider has evolved significantly from your API’s structure.