Test your MCP server with the eval command

The eval command enables rapid testing of your MCP server directly from the command line, without firing up an MCP client like Cursor or Claude Desktop. Use it to verify tool functionality, debug issues, and iterate quickly during development.

Enable the eval command

The eval command is not generated by default. To include it, set the evalProvider flag in your gen.yaml configuration:

targets:
  mcp-typescript:
    evalProvider: anthropic

Set evalProvider to one of the following supported providers:

Provider	Value
Anthropic	anthropic
OpenAI	openai

The eval command uses the Vercel AI SDK internally, so adding support for additional providers is straightforward. If you’d like support for a provider not listed here, reach out to us.

After setting the evalProvider flag, regenerate your MCP server to include the eval command.

Usage

Run the eval command with a natural language prompt describing what you want to test:

./bin/mcp-server.js eval \
  --username "{your-username}" \
  --password "{your-password}" \
  --anthropic-api-key "{your-anthropic-api-key}" \
  "give me a list of the names of all the connections in my account"

The command sends your prompt to the configured AI provider, which determines the appropriate MCP tools to call and returns the result.

Flags

The eval command accepts the following flags depending on your API’s authentication requirements and the configured provider:

• --username: Username for API authentication (if required by your API).
• --password: Password for API authentication (if required by your API).
• --anthropic-api-key: Your Anthropic API key (required when using the anthropic provider).
• --openai-api-key: Your OpenAI API key (required when using the openai provider).

The authentication flags vary based on your API’s security scheme. The eval command inherits the same authentication options as the start command.

Example workflows

Verify tool discovery

Confirm that your MCP server exposes the expected tools:

./bin/mcp-server.js eval \
  --anthropic-api-key "{your-anthropic-api-key}" \
  "what tools are available?"

Test a specific operation

Verify that a specific API operation works correctly:

./bin/mcp-server.js eval \
  --username "{your-username}" \
  --password "{your-password}" \
  --anthropic-api-key "{your-anthropic-api-key}" \
  "create a new booking from station 1 to station 5 for tomorrow"

Debug error handling

Test how your server handles invalid inputs:

./bin/mcp-server.js eval \
  --username "{your-username}" \
  --password "{your-password}" \
  --anthropic-api-key "{your-anthropic-api-key}" \
  "delete a booking with ID that doesn't exist"