OpenAPI Generator vs. CDC Testing with Pact

Question

We're overhauling our frontend and backend service contract reliability and are investigating two tools/techniques that seem to conflict. Consumer and provider code generation from an OpenAPI Spec (OAS) with a tool like openapi generator vs. consumer driven contract (CDC) testing with a tool like pact.

OAS Code Generation

OAS works great for generating the consumer code, and we're working on integrating provider-side generation to complete the contract confidence on both sides. As long as contract alterations start w/ the OAS and providers and consumers generate their code, is this a suitable strategy?

Pact Unit Testing

Pact CDC testing doesn't seem to involve an OAS at all, but instead programmatically builds contracts between the consumer and provider via unit testing. When using a pact broker, the addition of the can-i-deploy tool seems like a nice addition to a ci/cd pipeline. One nice thing w/ pact is that it appears to support kafka event mocking, which would be something openapi-generator doesn't cover.

If every service, front and back, is using OAS code generation, is pact useful? I could see it's utility in an environment without codegen, but otherwise starts to feel redundant/conflicting.

Thanks for any insight or anecdotes you can provide!

Answer 1

Pact is a contract testing framework that uses specification by example to ensure providers actually implement what the consumer needs. This removes ambiguity, but comes at a cost (writing / maintaining tests). It should be noted that the status quo here is end-to-end tests, which are more expensive than contract tests (for this purpose).

This article talks about the differences between schemas and how they relate to contract testing, and how they might be used together.

If every service, front and back, is using OAS code generation, is pact useful?

The short answer here, is how do you ensure that the compatible versions of your consumers and providers are in sync? If there are breaking changes between versions of your provider, you now need to synchronise the release of all consumers. And if there is a problem with releasing, you then need to reverse it all out - this is a key problem that contract testing addresses.

Pact CDC testing doesn't seem to involve an OAS at all

Pactflow has a feature to combine these two approaches, but the philosophy behind it is outlined here

When using a pact broker, the addition of the can-i-deploy tool seems like a nice addition to a ci/cd pipeline. One nice thing w/ pact is that it appears to support kafka event mocking, which would be something openapi-generator doesn't cover.

Yes, that's more of a practical benefit of the Pact Broker and Pact ecosystem. If you need to expand beyond REST (and what OAS) can document, you will need a different strategy. Pact might be more general purpose for those use cases.

• Tommy Adamski: thank you for the fast and thoughtful reply! Your explanation and references have definitely helped fill in some knowledge gaps! @matthew-fellows Knowing about contract breaks without introducing the need to immediately refresh implementations everywhere w/ schema codegen feels like a substantial benefit! I will continue down the pact route as I do think there are demonstrable benefits
• Matthew Fellows: Great to hear it helped, you may also like to join our community at slack.pact.io