Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Add Keccak-256 (Ethereum-style, original 0x01 padding)#3245

Open
fabrice102 wants to merge 1 commit into
aws:main from
fabrice102:keccak-256
Open

Add Keccak-256 (Ethereum-style, original 0x01 padding) #3245
fabrice102 wants to merge 1 commit into
aws:main from
fabrice102:keccak-256

Conversation

@fabrice102

@fabrice102 fabrice102 commented May 12, 2026

Copy link
Copy Markdown
Contributor

Issues:

N/A

Description of changes:

Adds Keccak-256 with the original Keccak submission's padding byte (0x01) — the variant used by Ethereum, Solidity's keccak256(), and other blockchain ecosystems. This is not FIPS 202 SHA3-256 (which uses 0x06 padding); rate (1088 bits) and digest length (256 bits) match, but the padding byte differs, so outputs differ for every input.

Approach:

The implementation lives inside the FIPS module
(crypto/fipsmodule/sha/sha3.c) so it shares the audited Keccak-f[1600] permutation, FIPS202_Init/Update/Finalize buffering, and assembly backends with the SHA-3 family. The only delta in the module's algorithmic code path is one extra accepted padding byte:

if (pad != SHA3_PAD_CHAR &&
 pad != SHAKE_PAD_CHAR &&
 pad != KECCAK_PAD_CHAR) { // 0x01
 return 0;
}

The new public surface is exactly EVP_keccak_256() (plus the existing EVP_Digest* machinery). This matches the BoringSSL/AWS-LC convention established by SHA-3: the EVP_MD is the public API; direct C entry points (KECCAK_256_Init/Update/Final, one-shot KECCAK_256()) live in crypto/fipsmodule/sha/internal.h and are exported only for in-tree consumers, mirroring SHA3_256_*.

Call-outs:

  • NOT FIPS-approved. KECCAK_256_Final and the KECCAK_256() one-shot intentionally omit FIPS_service_indicator_update_state(), so calling Keccak-256 leaves the per-thread approved-services counter unchanged. Verified by a new row in service_indicator_test.cc::kDigestTestVectors with AWSLC_NOT_APPROVED. The negative invariant is enforced both at runtime (the parameterized EVPMDServiceIndicatorTest.EVP_Digests catches accidental _update_state calls) and structurally (a future reviewer can grep the file and see only "Intentionally no" comments).

  • No NID, no OID. The Ethereum-style 0x01-padding Keccak is not standardised and OpenSSL has not assigned an OID for it either. The EVP_MD's type is NID_undef, and the digest is registered in nid_to_digest_mapping by name only. As a consequence, EVP_keccak_256() cannot be used with EVP_DigestSign* / EVP_DigestVerify* for RSA-PKCS#1-v1_5 or ECDSA, and EVP_marshal_digest_algorithm will fail with DIGEST_R_UNKNOWN_HASH. objects.txt, obj_dat.h, and nid.h are intentionally untouched.

  • OpenSSL compatibility. The registered EVP_MD name is KECCAK-256 (uppercase, hyphenated), matching OpenSSL 3.2+'s default-provider registration in providers/implementations/include/prov/names.h. Portable code that does EVP_MD_fetch(NULL, "KECCAK-256", NULL) on OpenSSL works against AWS-LC via EVP_get_digestbyname("KECCAK-256"). A keccak-256 lowercase alias is also registered.

  • FIPS module impact. The audited Keccak-f[1600] permutation, absorb/squeeze code, and existing SHA-3 entry points are unchanged byte-for-byte. The only edit inside the module boundary is the 3-line FIPS202_Init pad-byte check above plus the new KECCAK_256_* wrappers.

Testing:

All Keccak-256 tests live in crypto/digest_extra/digest_test.cc alongside the rest of the digest tests. New coverage:

  • kTestVectors[] — 3 well-known Ethereum-ecosystem vectors (empty / "abc" / fox-pangram). These exercise the EVP one-shot, byte-at-a-time streaming, copy/move-on-context, and unaligned-input paths in TestDigest().
  • DigestTest.Keccak256DiffersFromSHA3_256 — invariant: for the same input, Keccak-256 and SHA3-256 outputs must always differ. Guards against a future maintainer "fixing" the padding byte.
  • DigestTest.Keccak256KAT — file-driven tests using 137 short-message vectors (0..1088 bits) and 100 long-message vectors (2184..110688 bits), in the same Len/Msg/MD format used by SHA-3 KATs. Mirrors the FileTestGTest pattern from sha3_test.cc::NISTTestVectors.
  • DigestTest.Getters — extended with EVP_get_digestbyname("KECCAK-256"), the lowercase alias, and an EVP_MD_type == NID_undef assertion.
  • service_indicator_test.cc::kDigestTestVectors — one new row, parameterized as EVP_Digests/13, asserts AWSLC_NOT_APPROVED through Init / Update / Final.

KAT vectors are generated by util/gen_keccak256_kat.py, which uses pycryptodome (an independent Keccak-256 implementation) as the reference. The script sanity-checks pycryptodome against the well-known Ethereum vectors before generating, so a misconfigured environment fails loudly. Reference and implementation under test come from completely different code, so the test asserts genuine cross-implementation agreement.

bssl speed -filter KECCAK-256 is wired up next to the SHA-3 benchmarks for performance regression tracking.

Verified end-to-end:

  • Debug build: 2707/2707 active crypto_test cases pass, including DigestTest.Keccak256*, SHA3Test.* (no regression), and the full digest suite.
  • FIPS build (-DFIPS=1 -DBUILD_SHARED_LIBS=1): 3719/3719 active cases pass, including all 14 rows of EVPMDServiceIndicatorTest.EVP_Digests (was 13).
  • bssl speed -filter KECCAK-256 produces sensible numbers across all default chunk sizes.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

### Description of changes:
Adds Keccak-256 with the original Keccak submission's padding byte (0x01)
— the variant used by Ethereum, Solidity's `keccak256()`, and other
blockchain ecosystems. This is **not** FIPS 202 SHA3-256 (which uses
0x06 padding); rate (1088 bits) and digest length (256 bits) match, but
the padding byte differs, so outputs differ for every input.
### Approach:
The implementation lives inside the FIPS module
(`crypto/fipsmodule/sha/sha3.c`) so it shares the audited Keccak-f[1600]
permutation, FIPS202_Init/Update/Finalize buffering, and assembly
backends with the SHA-3 family. The only delta in the module's
algorithmic code path is one extra accepted padding byte:
 if (pad != SHA3_PAD_CHAR &&
 pad != SHAKE_PAD_CHAR &&
 pad != KECCAK_PAD_CHAR) { // 0x01
 return 0;
 }
The new public surface is exactly `EVP_keccak_256()` (plus the existing
`EVP_Digest*` machinery). This matches the BoringSSL/AWS-LC convention
established by SHA-3: the `EVP_MD` is the public API; direct C entry
points (`KECCAK_256_Init/Update/Final`, one-shot `KECCAK_256()`) live in
`crypto/fipsmodule/sha/internal.h` and are exported only for in-tree
consumers, mirroring `SHA3_256_*`.
### Call-outs:
- **NOT FIPS-approved.** `KECCAK_256_Final` and the `KECCAK_256()`
 one-shot intentionally omit `FIPS_service_indicator_update_state()`,
 so calling Keccak-256 leaves the per-thread approved-services counter
 unchanged. Verified by a new row in
 `service_indicator_test.cc::kDigestTestVectors` with
 `AWSLC_NOT_APPROVED`. The negative invariant is enforced both at
 runtime (the parameterized `EVPMDServiceIndicatorTest.EVP_Digests`
 catches accidental `_update_state` calls) and structurally (a future
 reviewer can grep the file and see only "Intentionally no" comments).
- **No NID, no OID.** The Ethereum-style 0x01-padding Keccak is not
 standardised and OpenSSL has not assigned an OID for it either. The
 EVP_MD's `type` is `NID_undef`, and the digest is registered in
 `nid_to_digest_mapping` by name only. As a consequence,
 `EVP_keccak_256()` cannot be used with `EVP_DigestSign*` /
 `EVP_DigestVerify*` for RSA-PKCS#1-v1_5 or ECDSA, and
 `EVP_marshal_digest_algorithm` will fail with
 `DIGEST_R_UNKNOWN_HASH`. `objects.txt`, `obj_dat.h`, and `nid.h` are
 intentionally untouched.
- **OpenSSL compatibility.** The registered EVP_MD name is `KECCAK-256`
 (uppercase, hyphenated), matching OpenSSL 3.2+'s default-provider
 registration in `providers/implementations/include/prov/names.h`.
 Portable code that does `EVP_MD_fetch(NULL, "KECCAK-256", NULL)` on
 OpenSSL works against AWS-LC via
 `EVP_get_digestbyname("KECCAK-256")`. A `keccak-256` lowercase alias
 is also registered.
- **FIPS module impact.** The audited Keccak-f[1600] permutation,
 absorb/squeeze code, and existing SHA-3 entry points are unchanged
 byte-for-byte. The only edit inside the module boundary is the
 3-line `FIPS202_Init` pad-byte check above plus the new
 `KECCAK_256_*` wrappers.
### Testing:
All Keccak-256 tests live in `crypto/digest_extra/digest_test.cc`
alongside the rest of the digest tests. New coverage:
- `kTestVectors[]` — 3 well-known Ethereum-ecosystem vectors (empty /
 "abc" / fox-pangram). These exercise the EVP one-shot, byte-at-a-time
 streaming, copy/move-on-context, and unaligned-input paths in
 `TestDigest()`.
- `DigestTest.Keccak256DiffersFromSHA3_256` — invariant: for the same
 input, Keccak-256 and SHA3-256 outputs must always differ. Guards
 against a future maintainer "fixing" the padding byte.
- `DigestTest.Keccak256KAT` — file-driven tests using 137 short-message
 vectors (0..1088 bits) and 100 long-message vectors (2184..110688
 bits), in the same `Len/Msg/MD` format used by SHA-3 KATs. Mirrors
 the `FileTestGTest` pattern from `sha3_test.cc::NISTTestVectors`.
- `DigestTest.Getters` — extended with `EVP_get_digestbyname("KECCAK-256")`,
 the lowercase alias, and an `EVP_MD_type == NID_undef` assertion.
- `service_indicator_test.cc::kDigestTestVectors` — one new row,
 parameterized as `EVP_Digests/13`, asserts `AWSLC_NOT_APPROVED`
 through Init / Update / Final.
KAT vectors are generated by `util/gen_keccak256_kat.py`, which uses
pycryptodome (an independent Keccak-256 implementation) as the
reference. The script sanity-checks pycryptodome against the well-known
Ethereum vectors before generating, so a misconfigured environment fails
loudly. Reference and implementation under test come from completely
different code, so the test asserts genuine cross-implementation
agreement.
`bssl speed -filter KECCAK-256` is wired up next to the SHA-3
benchmarks for performance regression tracking.
Verified end-to-end:
- Debug build: 2707/2707 active `crypto_test` cases pass, including
 `DigestTest.Keccak256*`, `SHA3Test.*` (no regression), and the full
 digest suite.
- FIPS build (`-DFIPS=1 -DBUILD_SHARED_LIBS=1`): 3719/3719 active cases
 pass, including all 14 rows of `EVPMDServiceIndicatorTest.EVP_Digests`
 (was 13).
- `bssl speed -filter KECCAK-256` produces sensible numbers across all
 default chunk sizes.

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.27273% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.14%. Comparing base (443d7f7) to head (b8bce73).
⚠️ Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
crypto/fipsmodule/sha/sha3.c 78.12% 7 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@
## main #3245 +/- ##
==========================================
+ Coverage 78.10% 78.14% +0.03% 
==========================================
 Files 689 689 
 Lines 123201 123303 +102 
 Branches 17136 17173 +37 
==========================================
+ Hits 96223 96349 +126 
+ Misses 26068 26045 -23 
+ Partials 910 909 -1 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@manastasova manastasova left a comment
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really clean PR! My only major comment is around the decision to add Keccak256 into the FIPS module. Would it be possible to integrate it outside of the module instead, similar to how blake2 is handled, while reusing the FIPS 202 functions from the FIPS module? This would also mean relocating the test vectors.

0x85, 0x3c, 0x64, 0xa1, 0x56, 0x6f, 0xeb, 0x76, 0x25, 0x9a, 0x4a, 0x44,
0x23, 0xf7, 0xcf, 0x46};

// Keccak-256 of |kPlaintext|, computed by util/keccak_ref.py.

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

util/gen_keccak256_kat.py ?

}

// |EVP_MD_FLAG_DIGALGID_ABSENT| matches OpenSSL 3.2+'s
// |PROV_DIGEST_FLAG_ALGID_ABSENT| on KECCAK-{224,256,384,512}. With

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Are there other algorithms that should update the flag EVP_MD_FLAG_DIGALGID_ABSENT?

// Keccak-256 has no NID/OID (the Ethereum-style 0x01-padding variant is
// not standardised). Registered by name only; matches OpenSSL 3.2+'s
// "KECCAK-256" provider name for cross-library lookups.
{NID_undef, EVP_keccak_256, "KECCAK-256", "keccak-256"},

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the keccak-256 entry? blake2 has no entry in the table. Should we add it as well?

// permutation as the SHA-3 family but is NOT FIPS-approved: it does not
// update the FIPS service indicator. Streaming functions are exposed only
// internally; public consumers reach Keccak-256 via |EVP_keccak_256|.
//

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra comment line here

Comment thread tool/speed.cc
!SpeedHash(EVP_sha3_384(), "SHA3-384", selected) ||
!SpeedHash(EVP_sha3_512(), "SHA3-512", selected) ||
#endif
#if defined(OPENSSL_IS_AWSLC)

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it not compatible with openssl?

Comment thread util/gen_keccak256_kat.py
@@ -0,0 +1,105 @@
#!/usr/bin/env python3

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing license header.

Comment thread util/gen_keccak256_kat.py
out_dir = os.path.normpath(out_dir)
os.makedirs(out_dir, exist_ok=True)

# Sanity check pycryptodome's output against the well-known Ethereum

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice :)

Comment thread sources.cmake
crypto/fipsmodule/sha/testvectors/SHA3_384ShortMsg.txt
crypto/fipsmodule/sha/testvectors/SHA3_512LongMsg.txt
crypto/fipsmodule/sha/testvectors/SHA3_512ShortMsg.txt
crypto/keccak/testvectors/KECCAK_256ShortMsg.txt

@manastasova manastasova May 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alphabetical order would be better

@justsmth justsmth requested a review from jakemas May 27, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manastasova manastasova manastasova left review comments

@jakemas jakemas Awaiting requested review from jakemas

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /