Design REST Polymorphic API

Question

I have a client/server application that exposes data and commands through REST APIs. This API exposes a GET /recipes/:id endpoint that instructs the client on how to show a recipe. A recipe can be made of diverse activities, each of them carrying specific contextual data. Summing up, the API response would look like this

GET /recipes/xxxx
[
 {
 activityType: "TYPE_1",
 commonData: "this",
 contextualData: {
 foo : "bar"
 }
 }, 
 {
 activityType: "TYPE_2",
 commonData: "that",
 contextualData: {
 hello : "world"
 }
 }, 
]

of course, clients can correctly parse contextual data by inferring the data model from the "activityType" field.

Even if this apparently works fine, I still feel that it's missing the whole purpose of REST: the API is polimorphic and the correct interpretation of the result is not built in the contract but relies on a semantic rule. How the client and the server can negotiate the data type if this can be arbitrarly changed from the server: eg. what happens if a new activity type is introduced without client changes?

Last but not the least: some code generators could struggle with this type of API definition and would be difficult to generate code from API documentation

a possible solution to this would be by "splitting" the API like this

GET /recipes/xxxx
[
 {
 type: "TYPE_1",
 commonData: "this",
 contextualData: {
 href : "/recipes/xxx/activities/1"
 }
 }, 
 {
 type: "TYPE_2",
 commonData: "that",
 contextualData: {
 href: "/recipes/xxx/activities/2"
 }
 }, 
]

the call to "/recipes/xxx/activities/1" endpoint, for example, would have

Content-Type: application/vnd.myapp.type1+json

and "/recipes/xxx/activities/2" would have

Content-Type: application/vnd.myapp.type2+json

But, apart from introducing new technical considerations (do we really want to fetch this data in multiple steps?) still the "activities" endpoint would return view models of arbitrary types

Is this the right approach to tacke this design issue? is there any "restful" way that ensures client and server can agree on a contract without any "dynamic" interpretation needed nor parsing magic?

• Ewan: is it really a problem if you include the type? why is that different from expecting the clients to be able to interpret a single type per end point?
• Carmine Ingaldi: @Ewan two main problems I've detected are: "how to restfully handle the case where a TYPE_3 is added and the client doesn't know how to process it?" and "how the client can apply the correct processing logic if a breaking change is introduced in contextual data for a certain activity type?"

Answer 1

Even if this apparently works fine, I still feel that it's missing the whole purpose of REST: the API is polymorphic and the correct interpretation of the result is not built in the contract but relies on a semantic rule.

I think you misunderstand the "whole purpose of REST" - extending message semantics with new meanings happens all the time: new HTTP methods, new fields, new status codes, new media types, new link relations....

Even at the grain of a "web page", it is normal to add new links, new forms, and so on, in the expectation that old consumers will continue to be able to use the parts that they recognize, and new consumers will be able to leverage the new capabilities.

---

how to restfully handle the case where a TYPE_3 is added and the client doesn't know how to process it?

The usual answer is that the client ignores the parts of the message that it doesn't understand.

David Orchard wrote about this problem in 2003 (in the context of XML Vocabularies); Greg Young discusses similar considerations in his 2017 book on versioning event sourced systems.

---

how the client can apply the correct processing logic if a breaking change is introduced in contextual data for a certain activity type?

We call them "breaking changes" for a reason - when you do that, things break. It's part of the collection of trade offs you need to consider when introducing such a change.

A good resource to review here is Rich Hickey's Spec-ulation keynote.

An alternative to consider is to replace the contract with a new one in a controlled way: the introduction of support for the new contract and the end of life for supporting the old contract are separable events. See Hintjens 2015.

• Carmine Ingaldi: Thanks for the insights! One thing to notice, though, is that even if the protocol can change, the idea of using design principles such REST (or graphql or whatever) is to not only shape the contract but also its evolution and interoperability. API verbs can change as long as they are built in in the transport protocol and everybody is able to agree and accept its new semantic, providing means to foster backward/forward compatibilty

Answer 2

Do you have a media type for the aggregate representation? Like application/vnd.myapp.recipies?

If you don't, you should obviously. :) So this media type can then include the semantics to parse individual "types". So define type 1-20 or something. You could even define there that the client must ignore types it doesn't support, so you would not need to do anything else.

Alternatively you can update the recipies media type each time new types are available (publish new version of it, like: application/vnd.myapp.recipies-v2). This is useful if the activities themselves can change types. Because in this case the client needs to indicate which types it understands or else it will stop understanding activities it understood at some earlier point.

Also, you can play around with media type parameters maybe. So client can say Accepts: application/vnd.myapp.recipes types=1-20, or something like that. With this you can get both the above: not update the media type, but still indicating what exact types the client supports.

Answer 3

You have bought into the REST lie, ie that a client can understand the content without supporting documentation.

The truth is that if you just throw a json string out it means nothing. even if we assume the client can read json because its requested that format, it wont understand the meaning of the fields or data.

To a computer program there's no advantage to using GET /recipe/3 returning a single structure vs MADEUPVERB /{methodname}/{encodedparameters} returning some binary format response which caters for any possible type.

Both rely on the client having been pre programmed with the information on how to decode the response and use it appropriately.

Its fairly common to use a "_type" property in json to denote the type of object to deserialise to. If an unknown type is received then you just can't display that recipe.

The advantage of REST over other methods is that JavaScript code running in a browser can use the in built browser functionality to make calls. An advantage that is becoming less relevant as browsers become more advanced and javascript gets more libraries.

• Thomas Junk: I called it once JSON over the wire-Protocol. That's what is left.
• Robert Bräutigam: I think you mistunderstand. It's not that there is no documentation, but that the documentation does not rely on certain things like: where the json came from, or what happened before, even what request came before a response. The client is supposed to react to the json, not assume and hardcode what is coming. Basically anything can come at any time, so you have to tell the server what you are able to handle and then react.
• Ewan: you single out statelessness, but its not relevant to the question. both styles are stateless

Answer 4

the API is polimorphic and the correct interpretation of the result is not built in the contract but relies on a semantic rule.

Build it into the contract.

You were half right when you mentioned Content Types, but you overly complicated it by splitting up the resource. Just make a defined Content Type for the first response (the one with TYPE_1 and TYPE_2)

Given that I'm assuming that originally there was just TYPE_1 activities, you could call the new Content Type that understands TYPE_1 and TYPE_2 activities as version 2

Content-Type: application/vnd.myapp.v2+json

An old client that only understands TYPE_1 activities can request the recipes in v1 format.

I don't really understand the domain logic here, but I guess the server could serve them a representation wit just TYPE_1 activities (or if that doesn't fit how your domain works and you just want these old clients to not work anymore, return a 406 instead for any client that requests v1)

How the client and the server can negotiate the data type if this can be arbitrarly changed from the server

Don't do that is the simple answer there.

The server cannot arbitrarly change the representation of a resource.

That is going to break your clients whether you use REST or any other API paradigm (RPC etc).

If your client has to understand the contextualData field, you can't have the server introducing new formats of it without warning the developers of the clients or giving the client some way to know if it is getting a data that it won't understand (HTTP solves this with content types, as discussed).

So if you change the format of a resource representation you need to at least publish the updated spec (v3) some where so that the developers implementing your clients can update their clients to support these changes.

I mean otherwise how are clients ever going to know how to interpret TYPE_1 or TYPE_2 or TYPE_3 etc in the first place.

That has to be communicated to the developers of these clients some how, and when you do that you can introduce a new Content Type that contains the newest TYPE_X that you just introduced.