Idempotency and concurrency in a REST web service

Question

Is concurrency and idempotency mutually exclusive for a REST web service or can it be done so a service can be both concurrent and idempotent? Using a optimistic lock methodology like using Version property seem to be preventing a method to be idempotent if I'm not mistaken?

For example, let's say you have User A wanting to update Person.Name

Current Person.Version = 1

User A sets

Person.Name = "Name X"
Person.Version = 2

Now the Person.Version = 2

User A send same request, with Person.Version = 2 value. It fails because you expect the Person.Version value 3.

• AilurusFulgens: Could you explain why you think that optimistic lock and idempotency are mutually exclusive ?
• dstr: @AilurusFulgens: added sample
• Joeri Sebrechts: Why are you having the client send a version to the server? Shouldn't it be the server's job to generate a new version?
• guillaume31: Version number should be incremented by the server, not sent by the client. That said, to me concurrency management trumps idempotency in that case. They are not mutually exclusive, it's just that idempotency makes no sense here since you're trying to reapply a change on a resource version that doesn't exist any more.
• Erik Eidt: I suspect that the version in the change request is not intended to set the new version, but is intended to be treated as guard to be checked at the server.

Answer 1

Is concurrency and idempotency mutually exclusive for a REST web service or can it be done so a service can be both concurrent and idempotent?

Yes, of course

User A send same request, with Person.Version = 2 value. It fails because you expect the Person.Version value 3.

I think you are confusing a couple different ideas.

From Gregor Hohpe

The term idempotent is used in mathematics to describe a function that produces the same result if it is applied to itself, i.e. f(x) = f(f(x)).

In Messaging this concepts translates into the a message that has the same effect whether it is received once or multiple times. This means that a message can safely be resent without causing any problems even if the receiver receives duplicates of the same message.

So the first thing to notice in your system is that the effect on the receiver of sending the same message twice is the same as sending it exactly once.

The advantage being that it is safe for the sender to repeat the message as many times as necessary to receive an ack.

Now, in your example, you seem to be expecting something beyond that, which is that the repeated message generates the same response. That's an awesome property for your system to have, but it requires a bit more work.

The basic plot is that your server receives the duplicated message, and checks to see if it has already been handled. A naive approach would be to look at the version, and check if the change applied in this version has already been applied.

Specifically, the server gets the second message, sees that it's already on version 2, sees that the name is currently equal to "Person X", and returns success!

It's not such a great answer if somebody else slips in another change between the duplicate messages.

A real implementation is more complicated - the basic form of it is to keep track of the responses you send, and when you get a duplicate message, you send back the same response that you sent before. Udi Dahan covers this approach. The persistence of the responses is in the domain model, so the messaging is still stateless, which satisfies the REST constraint.

• raine: I'm late to the party, but I'd thought I'd comment instead of asking another very similar question. Scenario: HTTP Requests with an "Idempotency Token". What if a malicious client forges an existing "idempotency token" and obtains sensitive info? Most articles show "response stores" with just the request token as a key. None mention the user id. Is there a consensus on how to approach idempotency in a "safe" (from a data ownership perspective) way?

Answer 2

In your example you have correct idempotent service. The resource state after 1 call is the same as the resource state after 3, 5 or 100 calls. It doesn't matter that the second, the third etc. calls will return a 4xx error.

From HTTP RFC:

https://www.rfc-editor.org/rfc/rfc7231#section-4.2.2

For example, if a sends a PUT request and the underlying connection is closed any response is received, then the client can establish a new and retry the idempotent request. It knows that repeating request will have the same intended effect, even if the original succeeded, though the response might differ.

Same for DELETE method:

https://stackoverflow.com/questions/4088350/is-rest-delete-really-idempotent

Answer 3

In a concurrent environnment, access to data is always following the Cap theorem. That is to say, you have an implementation choice between the following :

• Global read/write lock i.e. no high availability
• Local caches i.e. no consistency between clients
• Centralised system i.e. scale limits

In a classic REST service, updates are directly sent to the DB who will arbiter what to do with your data, most likely, only keep the most recent update. If you need a versionned system you should add new rows instead of updating old ones.

Answer 4

Idempotency means that making the same request twice or 100 times doesn’t affect the outcome. Importantly: The request doesn’t change the outcome. The server state can be changed by something else, and as a result the outcome changes, but that doesn’t affect idempotency.

What is not idempotent: "Give me the next 10 records". "Delete the first item". Idempotent (kind of): "Delete all records about John Smith". Doing it once or 100 times doesn’t affect the server, just a possible notice how many records were deleted.