REST API security: HMAC/key hashing vs JWT

Question

I just read this article that is a few years old but describes a clever way of securing your REST APIs. Essentially:

• Each client has a unique public/private key pair
• Only the client and the server know the private key; it is never sent over the wire
• With each request, the client takes several inputs (the entire request itself, the current timestamp, and the private key) and runs them through an HMAC function to produce a hash of the request
• The client then sends the normal request (which contains the public key) and the hash to the server
• The server looks up the client's private key (based on the provided public key) and does some timestamp check (that admittedly I don't understand) that verifies the request is not a victim of a replay attack
• If all is well, then the server uses the private key and the same HMAC function to generate its own hash of the request
• The server then compares both hashes (the one sent by the client as well as the one it generated); if they match, the request is authenticated and allowed to proceed

I then stumbled across JWT, which sounds very similar. However the first article does not mention JWT at all, and so I am wondering if JWT is different than the above auth solution, and if so, how.

• nyxz: Lol... I just wanted to ask the same exact question and came across yours. One of the first things I found about stateless authentication was from the AWS: docs.aws.amazon.com/AmazonSimpleDB/latest/DeveloperGuide/… , and implemented something like this massimilianosciacco.com/… . Then I found JWS/JWT and it is somehow similar. But as far as I understand JWT is a standard and the other solutions described above are some custom implementations (not standardized). Someone correct me if I am wrong.
• smeeb: Good to know I'm not the only one worrying about these kinds of details! JWT certainly feels similar, and the bonus is that it's standardized. I'm just wondering how it fairs (security-wise) with this custom HMAC solution.

Answer 1

Let's get this started with a very basic answer.

JWT (as used in the context of OAuth and OpenID) does not require shared secrets between client and API. There are 3 components and pairs of 2 share a secret each: client <-> identification server, identification server <-> API.

This moves most complexity from the API to the identification server, the API just has to check that the token was issued by the identification server and was not tempered with. To verify that the API checks that the JWT-signature is valid with the known single shared secret between identification server and API. That's it!

How the identification server validates the user identity can vary widely (in many cases it's the old username+password pair over a TLS-connection), but is of no effect on your API.

Privacy and security of the message and the token itself when using JWT are handled by TLS, JWT is ignorant of such issues.

• Daniel Kaplan: I found this answer to be confusing. It handwaves details like, "JWT (as used in the context of OAuth and OpenID)" Might be worth explaining why that is instead of assuming the readers implicitly understand that.

Answer 2

disclaimer

this is based on a lots of reading and the implementation of both patterns in production systems; but, as usual with answers on the internet, this should be considered as a starting point and not a definitive conclusion

tl:dr

• jwt - great for user auth (scales well; low-cost, low-latency, low-dependency)
  • facebook
  • oauth
  • etc
• hmac+key - great for sdk auth (safer for longer living keys since the request-signature is single use)
  • aws
  • visa
  • etc

pros and cons

jwt's

• summary
  • the client-private-key is used as the request-signature and so is sent on every request, increasing vulnerability
    • however, in consumer applications, the client-private-key is only as safe as the device it is stored on anyway, so using it as the request-signature is likely not the biggest risk
  • the request-signature does not vary per request, so in case of a leak of the request-signature, the attacker can make any request they want
    • however, in consumer applications, the access granted is typically scoped to only that user, so the damage they can do is limited in scope
• pros:
  • summary
    • distributed + stateless
  • details
    • the request-signature, the jwt, can be authenticated publicly, by anyone
      • [distributed] no api calls to issuer-server required, client-public-key is published at wellknown url, cacheable, static
      • [stateless] no state to manage, access, or upkeep for authenticating requests
    • the request-signature, the jwt, identifies the requester and scope
      • [distributed] no api calls to issuer-server required, client identity is embedded and extractable from the request-signature
      • [stateless] no state to manage, access, or upkeep for identifying the requester
  • effect
    • no centralized state gatekeeper/bottleneck
    • no added latency due to queries against some database
    • no added cost for provisioning databases
    • no scaling issues as the number of tokens increases (expiration x users = gets large fast)
• cons:
  • summary
    • the client-private-key === the request-signature === the jwt
  • details
    • the client-private-key (the jwt) is transmitted over the wire (could be leaked through usage)
    • the request-signature (the jwt) does not vary per message (if leaked, can be used to make any request)
    • the client-private-key (the jwt) is stored in plaintext by the client (can be leaked from storage device)

hmac+keys's

• summary
  • the issuer must keep a database of the client-keypairs they issued and access it's data to authenticate each request-signature, increasing latency and cost
  • however, the request-signature is single use when implemented w/ nonce, so there is no risk in it being stolen through usage, making it safer to have longer living client-keypairs for backend applications
• pros
  • summary
    • eliminates vulnerabilities present in jwt based auth, making it safer to use when managing more than the data of one user
  • details
    • the client-private-key is never sent over the wire, eliminating the possibility of it being leaked through usage
    • the request-signature changes for each request (and can be made immune to replays), eliminating any risk from request-signatures being leaked
  • effect
    • keeps the client-private-key more secure over time
• cons
  • summary
    • centralized + stateful
  • details
    • you have to save the client-public-key and the client-private-key-hash in a database, along with scope and identity
    • you have to lookup the client-private-key-hash for each authentication request, along with scope and identity
    • the client-private-key is stored in plaintext by the client (can be leaked from storage device)

comparison in flow

here's a breakdown in the flow of keypairs and signatures used in both patterns.

you'll see a 🔴 wherever there is a systematic vulnerability

for jwt:

• issuer creates issuer-keypair (openssl keypair)
  • publishes issuer-public-key publicly at the openapi discovery flow url
  • securely retains issuer-private-key to sign jwts with
• issuer creates client-keypair (a jwt token)
  • effectively, client-public-key = issuer-public-key, which is already publicly published by issuer
  • the client-private-key is the jwt, sent to client, forgotten by issuer
• client retains client-keypair
  • client-public-key reference is embedded in client-private-key (as jwt.issuer)
  • client-private-key retained as plaintext 🔴 (vuln. because this depends on client secure storage implementation)
• client signs requests and sends request with signature to backend
  • request-signature = client-private-key
    • 🔴 (vuln. because the secret, the jwt, is sent in each request; increases chance of leak)
    • 🔴 (vuln. because the request-signature does not depend on the request; if leaked, can be used to make any request)
• backend authenticates request against the signature
  • looks up client-public-key data
    • from the well-known url that the issuer-public-key is published at (since effectively issuer-public-key = client-public-key)
  • confirms request-signature is authentic against client-public-key
    • openssl signature check
  • looks up identity from request-signature
  • confirms request is authorized against identity

for hmac+key:

• issuer creates client-keypair for client
  • client-public-key -> sent to client, retained in database as plaintext by issuer
  • client-private-key -> sent to client, hashed to compare against in database by issuer
• client retains client-keypair
  • client-public-key retained as plaintext
  • client-private-key retained as plaintext 🔴 (vuln. because this depends on client secure storage implementation)
• client signs requests and sends requests with signature to backend
  • request-signature = hash(hash(client-private-key), request)
• backend asks issuer to authenticate request against the signature
  • looks up client-public-key data
    • from the database that the plaintext client-public-key and hashed client-private-key-hash are persisted to
  • confirms request-signature is authentic against client-public-key data
    • request-signature === hash(client-private-key-hash, request)
  • looks up identity from client-public-key data
  • confirms request is authorized against identity

• Basilevs: The wording is unfortunate as it makes reader imagine that JWT requires a private key to be exposed somehow. Meanwhile JWT can include the nonce and all other benefits of the described custom protocol. "the request-signature does not vary per request" is obviously incorrect. How would a constant signature be useful?
• Ulad Kasach: @Basilevs can you elaborate on how "the request-signature does not vary per request" is obviously incorrect.? The JWT is sent via the authorization header. This JWT is in-effect the request-signature for the request, as it is alone used to authenticate the request. Given that the JWT is only signed + generated when the JWT is issued, then the JWT does not vary per request. Therefore, the request-signature, which === the JWT, does not vary per request.
• Basilevs: JWT as any other session token has limited lifetime.
• Ulad Kasach: Yes. Given that the JWT does not vary throughout the duration of its lifetime, the requests generated with it have a static request-signature throughout that lifetime. (A direct contrast to HMAC request-signatures, which vary for each request)