(V3) Verifying a contract with a missing content field is always successful when it should error #1103

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Prerequisites

• I'm using the latest version of pact-python.
• I have read the console error message carefully (if applicable).

Description

I have two pacts, one is incorrectly setting the content thus:

 "contents": {
 "Payload": "exists"
 }

The other is correctly setting the content thus:

 "contents": {
 "content": {
 "Payload": "exists"
 },
 "contentType": "application/json",
 "encoded": false
 }

When verifying with content that does not match, I expect both pacts to fail to verify. However, the first pact with the incorrect json verifies successfully and there is no error or warning that anything is wrong:

Verifying a pact between my-consumer and my-provider
 This works as expected when verifying my-provider (0s loading, 8ms verification)
 Given When reading a message from the SQS queue
 generates a message which
 includes metadata
 "contentType" with value "application/json" (OK)
 has a matching body (FAILED)
Verifying a pact between other-consumer and my-provider
 This is silently ignored while verifying my-provider (0s loading, 6ms verification)
 Given When reading a message from the SQS queue
 generates a message which
 has a matching body (OK)

I'll see if I can get approval to share a producible demo repo, but for now the pacts are:

{
 "consumer": {
 "name": "other-consumer"
 },
 "provider": {
 "name": "my-provider"
 },
 "interactions": [
 {
 "_id": "1",
 "type": "Asynchronous/Messages",
 "description": "This is silently ignored while verifying my-provider",
 "providerStates": [
 {
 "name": "When reading a message from the SQS queue"
 }
 ],
 "contents": {
 "Payload": "exists"
 }
 }
 ],
 "metadata": {
 "pactRust": {
 "ffi": "0.4.22",
 "models": "1.2.3"
 },
 "pactSpecification": {
 "version": "4.0"
 }
 }
}

{
 "consumer": {
 "name": "my-consumer"
 },
 "provider": {
 "name": "my-provider"
 },
 "interactions": [
 {
 "_id": "2",
 "type": "Asynchronous/Messages",
 "description": "This works as expected when verifying my-provider",
 "providerStates": [
 {
 "name": "When reading a message from the SQS queue"
 }
 ],
 "contents": {
 "content": {
 "Payload": "exists"
 },
 "contentType": "application/json",
 "encoded": false
 }
 }
 ],
 "metadata": {
 "pactRust": {
 "ffi": "0.4.22",
 "models": "1.2.3"
 },
 "pactSpecification": {
 "version": "4.0"
 }
 }
}

And here's an idea of running the verification. I actually use a whole pytest setup with fixtures, but for brevity this ought to be able to reproduce the issue. Again, hopefully I'll be able to share a repo soon.

def run_message_handler(name: str, metadata: dict[str, Any] | None) -> Message:
 """This should fail to match due to missing Payload field for **both** pacts"""
 return Message(contents=b"{}", content_type="application/json", metadata=None)
verifier = Verifier("my-provider")

pact_dir = Path(file).parent / "pacts"

verifier = verifier.add_source(pact_dir).set_error_on_empty_pact()
verifier.message_handler(run_message_handler).verify()

*caveat to "I'm using the latest version": I'm using the latest version available on pypi, version 2.3.1 (as opposed to 2.3.2 available via github releases)

Reproducible demo

No response

Steps to reproduce

step 1: copy the two pact jsons and place them in separate files in a pacts/ directory
step 2: verify the pacts using pact v3, see verify.py for an example

Expected behavior

it should fail with an error due to missing "content" and/or "contentType" but it does not.

You can also enable logging (before step 3) and confirm there is no warning or error logged.

Actual behavior

"other-consumer" pact (pact-missing-contents.json) is verified successfully.
"my-consumer" pact (pact-with-contents.json) fails to verify (due to the mismatched content, as expected).

Your environment

• Public source code: no
• Is this a consumer or provider issue? Do you have information about the other side?: provider
• Pact Python version used: pip list | grep pact: pact-python 2.3.1
• Information about your Pact broker (version, hosted where, pactflow, ...): N/A
• Operating system and version (e.g. Ubuntu 20.04.2 LTS, macOS Ventura): Debian GNU/Linux 12 (bookworm)

Self-service

• I'd be willing to fix this bug myself.