API Advice

How To Build A Best In Class Python SDK

Tristan Cartledge

April 29, 2024 - 13 min read

API Advice

---

This tutorial demonstrates one way to code and package a Python SDK.

Why Are We Building an SDK?

Since the 2000s, HTTP APIs have enabled businesses to be more connected than ever and, as a result, produce higher utility and profit — no wonder APIs have exploded in popularity.

So, your business has created an API? Great! At first, developers might use curl or create a Postman collection to experiment with your API. However, the true power of an API is only unleashed when developers can interact with it in an automated and programmatic way. To get some real work done, you’ll want to wrap the HTTP API in your language of choice, so that it can be testable and have the most important aspects abstracted for your convenience and productivity.

What Are We Building?

We’ll build a Python SDK that wraps an HTTP API.

Fictional e-commerce startup ECOM enables the selling of products through the creation of online stores and associated products on its website. The ECOM HTTP API enables developers to create stores and manage products in an automated way. We want to give Python programmers easy access to the ECOM API with a robust, secure, and user-friendly SDK that handles the underlying HTTP API service in a type-safe, programmatic way.

Example SDK Repository

You can follow along without downloading our complete example repository, but if you get stuck or want to see the final result, you can find the code on GitHub at speakeasy-api/ecom-sdk .

Initialize the Project

Before we begin coding the SDK, let’s set up our project.

Install pyenv

While not strictly necessary in all cases, we prefer to manage Python versions with pyenv. If you’re building the SDK with us, install pyenv for your operating system.

On macOS, you can install pyenv with Homebrew:

Decide on a Python Version

For this tutorial, we’ll use Python 3.8, because it is the oldest version that still receives security updates.

Install Python 3.8

Install Python 3.8 with pyenv:

Set Up a Python Virtual Environment

We’ll set up a Python virtual environment to isolate our project dependencies from the system Python installation.

Create a new Python 3.8 virtual environment:

Activate the virtual environment:

After activating the virtual environment, depending on the shell you’re using, your shell prompt might change to indicate that you are now using the virtual environment. Any Python packages you install will be installed in the virtual environment and won’t affect the system Python installation.

Upgrade pip

When you create a new virtual environment, it’s good practice to upgrade pip to the latest version:

Decide on a Build Backend

In the past, Python used a setup.py or setup.cfg configuration file to prepare Python packages for distribution, but new standards (like PEPs 517, 518, 621, and 660) and build tools are modernizing the Python packaging landscape. We want to configure a modern build backend for our SDK and we have many options to choose from:

• Hatchling 
• Poetry 
• PDM 
• Setuptools 
• Flit 
• Maturin 

Some factors to consider when selecting a build backend include:

• Python version support: Some build backends only support a subset of Python versions, so ensure your chosen build backend supports the correct versions.
• Features: Determine the features you need and choose a backend that meets your requirements. For example, do you need features like project management tools, or package uploading and installation capabilities?
• Extension support: Do you need support for extension modules in other languages?

We’ll use Hatchling  in this tutorial for its convenient defaults and ease of configurability.

Set Up a Project With Hatchling

First, install Hatch:

Now, create a new project with Hatch:

The -i flag tells Hatch to ask for project information interactively. Enter the following information when prompted:

Hatch will create a new project directory with the name ecom-sdk and initialize it with a basic pyproject.toml file.

Change into the project directory:

See the Project Structure

Run tree to see the project structure:

The project directory should now contain the following files:

The LICENSE.txt file contains the MIT license, the README.md file has a short installation hint, and the pyproject.toml file contains the project metadata.

Update the README File

If you continue to support and expand this SDK, you’ll want to keep the README.md file up to date with the latest documentation, providing installation and usage instructions, and any other information relevant to your users. Without a good README, developers won’t know where to start and won’t use the SDK.

For now, let’s add a short description of what the SDK does:

Create a Basic SDK

Add a ./src/ecom_sdk/sdk.py file containing Python code, for example:

We’ll use the ./src/ecom_sdk/sdk.py file to ensure our testing setup works.

Create a Test

As we add functionality to the SDK, we will populate the ./tests directory with tests.

For now, create a ./src/ecom_sdk/test_sdk.py file containing test code:

Later in this guide, we’ll run the test with scripts provided by Hatch.

Inspect the Build Backend Configuration

The pyproject.toml file contains the project metadata and build backend configuration.

Be sure to take a peek at the pyproject.toml guide  and specification  for details on all the possible metadata fields available for the [project] table. For example, you may want to enhance the discoverability of your SDK on PyPI by specifying keyword metadata or additional classifiers.

Test Hatch

Here’s the Hatch test script that we’ll use to run tests:

The first time you run the test script, Hatch will install the necessary dependencies and run the tests. Subsequent runs will be faster because the dependencies are already installed.

After running the test script, you should see output similar to the following:

Code the SDK

Now that we have the project set up, let’s start coding the SDK.

Add an SDK Method To Fetch a List of Stores

In the ./src/ecom_sdk/sdk.py file, define a class, constructor, and list_stores() method for fetching a list of stores from our API:

Now we can begin writing tests in the ./tests/test_sdk.py file to test that the newly added list_stores() method works as intended.

In requests.get(), we use HTTP_TIMEOUT_SECONDS to set a maximum bound for waiting for the request to finish. If we don’t set a timeout, the requests library will wait for a response forever.

Add the following to ./tests/test_sdk.py:

Here we use the responses library to mock API responses since we don’t have a test or staging version of the ECOM HTTP API. However, even with the ECOM API, we might choose to have some part of the testing strategy mock the API responses as this approach can be faster and tightly tests code without external dependencies.

The test_sdk_class() test function checks that we can instantiate the class and the correct values are set internally.

The test_sdk_list_stores() test function makes a call to sdk.list_stores() to test that it receives the expected response, which is a JSON array of stores associated with the user’s account.

Add Dependencies

We need to add the requests and responses libraries to the project dependencies. Add the following to the pyproject.toml file:

Run the Tests

Let’s run the Hatch test script we configured earlier to check that everything is working correctly:

Once Hatch has installed our new dependencies, you should see output similar to the following:

Exception Handling

To create a pleasant surface for our SDK users, we’ll hide details of HTTP implementation behind exception handling and provide helpful error messages should things go wrong.

Let’s modify the list_stores() function to catch some more common errors and print helpful information for the user:

When an exception is raised from the call to requests.get(), we catch the exception, add useful information, and then re-raise the exception for the SDK user to handle in their code.

The call to r.raise_for_status() modifies the requests library behavior to raise an HTTPError if the HTTP response status code is unsuccessful. Read more about raise_for_status() in the Requests library documentation .

Now we’ll add tests for a 403 response and a connection error to the test_sdk.py file. We’ll also import the requests library into the test_sdk.py file:

This test ensures that the helpful error message from our SDK is displayed when a connection error or 403 response occurs.

Check that the new code passes the test:

Type Safety With Enums

To make development easier and less error-prone for our SDK users, we’ll define Enums to use with the list_products endpoint.

Add the following Enums to the EComSDK class:

The ProductSort and ProductSortOrder Enums inherit from the str class and can be used with the list_products() method we’ll define next – ProductSort specifies the sort category, and ProductSortOrder determines the sort order of the product list.

Now define the list_products() function at the bottom of the EComSDK class:

The list_products() function accepts named parameters sort_by and sort_order. If the list_products() function is called with named parameters, the parameters are added to the params dictionary to be included in the request’s HTTP query parameters.

An SDK user can call the list_products() function with the string list_products(sort_by="quantity") or with the safer equivalent using Enums: list_products(sort_by=sdk.ProductSort.QUANTITY). By encouraging the user to use Enums in this way, we prevent HTTP error requests that don’t clearly identify what went wrong should a user mistype a string, for example, “prise” instead of “price”.

Now add another test at the bottom of the test_sdk.py file:

Check that the new test passes:

Output Type Safety With Pydantic

To make the SDK even more user-friendly, we can use Pydantic to create data models for the responses from the ECOM API.

First, add Pydantic to the project dependencies in the pyproject.toml file:

Now, create a Product model in the ./src/ecom_sdk/models.py file:

Next, import the Product model into the ./src/ecom_sdk/sdk.py file:

Modify the list_products() function to return a list of Product models:

The list_products() function now returns a list of Product models created from the JSON response.

Now update the test_sdk.py file to test the new Product model:

We’ll also need to update the test_sdk_list_products() test to check that the Product models are returned from the list_products() function.

Check that the new tests pass:

We’ve now added type safety to the SDK using Pydantic, ensuring that the SDK user receives a list of Product models when calling the list_products() function. The same principle can be applied to other parts of the SDK to ensure that the user receives the correct data types.

Add Type Safety to the SDK Constructor

To ensure that the SDK user provides the correct data types when instantiating the EComSDK class, we can use Pydantic to create a Config model for the SDK constructor.

First, create a Config model in the ./src/ecom_sdk/models.py file:

Next, import the Config model into the ./src/ecom_sdk/sdk.py file:

Now, modify the EComSDK class to accept a Config model in the constructor:

The EComSDK class now accepts a Config model in the constructor, ensuring that the SDK user provides the correct data types when instantiating the class.

Let’s update the test_sdk.py file to test the new Config model:

Follow the same pattern to update the other tests in the test_sdk.py file to use the new Config model. Replace sdk = EComSDK(api_url, api_key) with:

Check that the new tests pass:

Things to Consider

We’ve described some basic steps for creating a custom Python SDK, but there are many facets to an SDK project besides code that contribute to its success or failure. For any publicly available Python library, you should consider such aspects as documentation, linting, and licensing.

Documentation

To focus on the nuts and bolts of a custom Python SDK, this guide does not cover developing an SDK’s documentation. But documentation is critical to ensure your users can easily pick up your library and use it productively. Without good documentation, developers may opt not to use your SDK at all.

All SDK functions, parameters, and behaviors should be documented, and creating beautiful, functional documentation is an essential part of making your SDK usable. You can roll your own documentation and add Markdown to the project README.md file or use a popular library like Sphinx  that includes such features as automatic highlighting, themes, and HTML or PDF outputs.

Linting

Consistently formatted code improves readability and encourages contributions that are not jarring in the context of the existing codebase. While this guide doesn’t cover linting, linting your SDK code should form part of creating an SDK.

For best results, linting should be as far-reaching and opinionated as possible, with little to zero default configuration required. Some popular options for linting include Flake8 , Ruff , and Black .

Licensing

A project’s license can significantly influence the type and number of potential open-source contributors. Choosing the right license for your SDK is key. Do you want your SDK’s license to be community-orientated? Simple and permissive? To preserve a particular philosophy on enforcing the sharing of code modifications? In this example, we selected the MIT license for simplicity and ease of use. If you’re unsure which license would best suit your needs, consider using a tool like Choose a license .

Other Endpoints

This tutorial covered a basic SDK for two endpoints. Consider that real-world SDKs can have many more endpoints and features. You can use the same principles to add more endpoints to your SDK.

Authentication Methods

If APIs use OAuth 2.0 or other authentication methods, you’ll need to add authentication methods to your SDK. You can use libraries like Authlib  to handle OAuth 2.0 authentication.

Pagination

If the API returns paginated results, you’ll need to handle pagination in your SDK.

Retries and Backoff

If the API is rate-limited or has intermittent failures, you’ll need to add retry and backoff logic to your SDK.

Build the Package

Now that we have a working SDK, we can build the package for distribution.

Hatch will build the package and output the distribution files:

You should now have a ./dist directory containing your source (.tar.gz) and built (.whl) distributions:

Upload the SDK to the Distribution Archives

We now have just enough to upload the SDK to the PyPI test repo at test.pypi.org.

To upload the build to TestPyPI, you need:

1. A test PyPI account. Go to https://test.pypi.org/account/register/  to register.

2. A PyPI API token. Create one at https://test.pypi.org/manage/account/#api-tokens , setting the “Scope” to “Entire account”.

If you plan to automate SDK publishing in your CI/CD pipeline, you should create per-project tokens. For automated releasing using CI/CD, it is recommended that you create per-project API tokens.

Finally, publish the package distribution files by executing:

The -r test switch specifies that we are using the TestPyPI repository.

Hatch will ask for a username and credentials. Use __token__ as the username (to indicate that we are using a token value rather than a username) and paste your PyPI API token in the credentials field.

Your shiny new package is now available to all your new Python customers!

The ecom-sdk python package can now be installed using:

We use the --index-url switch to specify the TestPyPI repo instead of the default live repo. We use the --no-deps switch because the test PyPI repo doesn’t have all dependencies (because it’s a test repo) and the pip install command would fail otherwise.

Automatically Create Language-Idiomatic SDKs From Your API Specification

We’ve taken the first few steps in creating a Python SDK but as you can see, we’ve barely scratched the surface. It takes a good deal of work and dedication to iterate on an SDK project and get it built right. Then comes the maintenance phase, dealing with pull requests, and triaging issues from contributors.

You might want to consider automating the creation of SDKs with a managed service like Speakeasy. You can quickly get up and running with SDKs in nine languages with best practices baked in and maintain them automatically with the latest language and security updates.