Should I provide a declarative API? (You probably should)

Declarative APIs are becoming more and more popular, especially in the context of Infrastructure as Code.

At meshcloud we’ve implemented a declarative API. In this post I want to provide insights into the process and answer these questions:

• Does it make sense to provide declarative APIs for all systems?
• Which use-cases benefit from it and which don’t?

But first things first, let’s start with a look at what a declarative API actually is all about:

What is a declarative API?

At first let’s have a look at the classical way of implementing an API. That is implementing it imperatively. With imperative APIs you have to provide dedicated commands the system has to execute: Create this VM, update the memory settings of this VM, remove a certain network from this VM, etc.

A declarative API is a desired state system. You provide a certain state you want the system to create. You don’t care about all the steps needed to achieve that state. You just tell the system: "Please make sure that the state I provide will be there."

This approach is best known from Infrastructure as Code tools like Kubernetes or Terraform. You tell them that you want a certain set of resources with some given configuration. They take care of creating and updating the resources.

Why provide a declarative API?

With a declarative API you move complexity from the consumer of the system to the system itself. Creating, updating and even deleting objects is no longer a customer concern.

That means you can provide a way simpler API for your consumers by providing a declarative API to your system for some use-cases. This results in a reduced amount of errors due to misunderstandings between client and API provider.

It is for sure not the ideal solution to all use-cases, but more about that later.

Let’s have a look at an example that shows how a declarative API can simplify the consumer’s interaction with your system.

Example: Synchronizing Groups

Imagine you have a user group synchronized between a central Identity Provider (IdP) and your system (target). A group would have these properties:

Group:
    id = "123-456"
    displayName = "My Group"
    members = ["uid1", "uid2"]

Your system provides integrations for multiple IdPs and multiple clients of you API exist. These clients should focus on getting information from the IdP – and then on getting it into your target system.

Solving it with an Imperative API

Now imagine that you have an imperative API in the target system: What do you have to do to always keep all groups in sync?

Let’s at first have a look at the operations that are available:

• createGroup: Creates a group with the given attributes. If a group already exists, an error is returned.
• updateGroup: Updates an existing group with the given attributes. If the group does not exist yet, it returns an error.
• deleteGroup: Deletes a group by its id.
• getAllGroups: Returns all groups that are available in the target system. This endpoint could provide some filter options, but this is not relevant for this blog post.

What you need to do for a full sync of groups from the IdP to your system:

As you can see, creating such a synchronization process is a rather complex task. Especially in the given example. You want a lightweight solution to implement multiple clients for the different IdPs.

This complexity requires a lot of effort every time you integrate a new IdP at a customer.

Solving it with a declarative API

In cases like the group sync, moving all the complex update logic to the target system and providing a declarative API simplifies the client by a lot.

How would a declarative API look like?

• applyGroups You only need one operation to which you provide a list of groups, like this:

[
    {
        id: "123-456"
        groupName: "developers",
        members: ["dev1", "dev2"]
    },
  {
        id: "456-789"
        groupName: "managers",
        members: ["manager1"]
    },
    {
        id: "789-132"
        groupName: "operators",
        members: ["op1"]
    }
]

On the next synchronization dev2 became a manager and has been moved to the managers group. In addition, the operators group has been removed, as the company decided to go with a DevOps team. So op1 has been moved to the developers group and the developers group has been renamed to devops. All you have to do is run the exact same process as before, which is:

That means the second call will be looking like this:

[
    {
        id: "123-456"
        groupName: "devops",
        members: ["dev1", "ops1"]
    },
  {
        id: "456-789"
        groupName: "managers",
        members: ["manager1", "dev2"]
    }
]

The target system will take care of removing dev2 from the developers group and will add it to the managers group. It will rename the developers group to devops and add member ops1 to that group. It will also take care of removing the operators group.

For sure, all the logic mentioned in the imperative approach now needs to be implemented in your target system. BUT you will only have to implement it once.

You may even come up with an architecture of your system that simplifies implementation of that declarative approach further.

This example is limited to a holistic synchronization of all groups all the time. If you have multiple sources you get your input from, you need some kind of bucket for the groups coming from one system. You could e.g. simply add another input to the applyGroups function that allows submitting an additional bucket parameter. That allows your system to only take all groups related to the given bucket into consideration for the consolidation of which groups to create, update or delete.

As this is especially relevant for deleting groups, more details about this will be part of my upcoming blog post on "How to implement deletion for a declarative API?".

The as-code advantage

Another great advantage of the declarative approach is that you can store what you applied in a version control system (VCS). That approach provides you several advantages.

• You have a full history of all changes
• You have a nice overview of the expected state in a system.
• You can work cooperatively on the desired state with a whole team.

Kubernetes or Terraform are good, real-world examples of storing the desired state in a VCS. The Kubernetes YAML files and the Terraform files are usually stored in a VCS.

If in contrast, you only apply imperative commands to the target system, you would always have to ask the target system about the current state. You may also override changes or reapply changes someone else had done before.

Declarative vs Imperative API, opponents or a nice team?

You may ask yourself: Should I build a completely declarative system without providing any imperative commands?

In some rare cases that might be the right thing to do. In most cases it makes much more sense to combine an imperative and a declarative API. Actually that is what the big Infrastructure as Code tools do. In the Kubernetes documentation you can find an imperative as well as a declarative way of managing your Kubernetes objects.

Besides use-cases like the group synchronization or Infrastructure as Code that profit a lot from a declarative system, there are also use-cases that benefit from an imperative API.

So you should decide dependent on the different use-cases in your system which API fits better for which case.

Example for less effort with Imperative API

Let’s extend our previous example with the user-groups and say you need these groups to assign them to projects in your system. These projects cannot simply be created in your system, but a central workflow tool with an approval process must be used to create new projects. After the workflow completed, this tool will create the according project in your system via your API. Afterwards the project will only be maintained in your system. There will be no updates coming from the workflow tool.

An imperative approach with an operation createProject is just the right thing for that use-case. Sure, it would also work with a declarative approach. You’d just call the apply operation once to create the project. But as only a create operation will be done once, there is no need for the complex handling of updating or deleting existing projects. So you can save quite a lot of effort by not implementing the declarative handling in your system, if you won’t use it.

Example for simpler client implementation with Imperative API

Another example is only updating a certain attribute of e.g. the project. Let’s say the project has some tags that can be set on it. Tags are simple key/value pairs. If you want to update them in a declarative API, you have to provide the complete project like this:

Project:
    id: "123-456",
    displayName: "My Project",
    assignedGroups: [
        "456-789"
    ],
    tags: [
        environment: "dev"
    ]

But if a system – that only knows about the project id and the tags – provides the tags, how does it get the other information it needs to update the project? It would have to call a get endpoint on the target system first to get all data of the project first. Then it could set the new tags and update the project.

For this use-case an imperative API makes the client’s life much easier. It could just call an imperative operation like addTag or setTags.

target addTag "environment" to "prod"
target setTags ["environment" to "prod", "confidentiality" to "internal"]

When should you provide a declarative API?

In the end you have to decide per use-case which kind of API design fits best. The following comparison provides some advantages of the 2 approaches. They can help you in making that decision.

Imperative API

• finer control on client side
• fits perfectly fine when only creating new objects and not keeping these objects in sync after creation
• easier to use when only updating partial data (especially when attributes of an object are sourced from multiple different systems)

Declarative API

• way easier client code for data synchronization scenarios as the client only needs to provide the desired state and the rest is done by the target system
• version control option for desired state
• single point of implementation at server side
• hide complexity of creating a certain state in the backend. Clients don’t have to care about that complexity

In general, I think the following rule of thumb can also help with that decision:

• Use a declarative API if you want to keep objects in sync with a central source. This is especially true if you have multiple clients of that API, as the complex handling of updates and deletion only have to be implemented once centrally in your system.
• Use an imperative API for one-time operations. You just want to create, update or delete one specific thing and afterwards you don’t care about the object’s lifecycle anymore. In that case you should consider an imperative API.

---

But now I’d like to hear from you: Have you implemented a declarative API? What challenges did you come across? Did I miss anything in my blog post? Let me know!

I recommend you to visit our blog where you will find many interesting posts on engineering topics: e.g. our guides to TLS/SSL certificates or the guide to testing IaC.