Skip to content

Fix XMSS signature vector tests #62

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ArtiomTr merged 3 commits into from
Jan 26, 2026
Merged

Fix XMSS signature vector tests #62

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7c0de75
updated XMSS signature verification vector tests to use correct signa…
JuliusMieliauskas Jan 24, 2026
17385a3
updated test vector deserialization to correctly handle exceptions
JuliusMieliauskas Jan 24, 2026
5695519
fix formatting
JuliusMieliauskas Jan 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 37 additions & 15 deletions lean_client/containers/tests/test_vectors/runner.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -626,8 +626,32 @@ impl TestRunner {
) -> Result<(), Box<dyn std::error::Error>> {
let json_content = fs::read_to_string(path.as_ref())?;

// Parse using the VerifySignaturesTestVectorFile structure
let test_file: VerifySignaturesTestVectorFile = serde_json::from_str(&json_content)?;
// Phase 1: parse minimally to detect `expectException` even if typed parsing fails.
let raw: serde_json::Value = serde_json::from_str(&json_content)?;

let expect_exception = raw
.get("tests")
.and_then(|t| t.as_object())
.and_then(|obj| obj.values().next())
.and_then(|tc| tc.get("expectException"))
.and_then(|v| v.as_str())
.map(|s| s.to_owned());
Comment on lines +632 to +638
Copy link
Collaborator

@ArtiomTr ArtiomTr Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't handle json by hand, just update "VerifySignaturesTestVectorFile" struct


// Phase 2: parse into the typed structure.
let test_file: VerifySignaturesTestVectorFile = match serde_json::from_str(&json_content) {
Copy link
Collaborator

@ArtiomTr ArtiomTr Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to handle json errors - nobody structures invalid json as spec test failure. Spec test must always be parsed, if it is not - then it is test failure, not "expected error".

Ok(v) => v,
Err(e) => {
if let Some(ref ex) = expect_exception {
println!(
"\nExpected exception: {} (typed JSON parse failed: {})",
ex, e
);
println!("\n\x1b[32m✓ PASS\x1b[0m\n");
return Ok(());
}
return Err(Box::new(e));
}
};

// Get the first (and only) test case from the file
let (test_name, test_case) = test_file
Expand Down Expand Up @@ -665,23 +689,21 @@ impl TestRunner {
if result {
println!(" \x1b[31m✗ FAIL: Signatures verified successfully but should have failed!\x1b[0m\n");
return Err("Expected signature verification to fail, but it succeeded".into());
} else {
println!(" ✓ Correctly rejected: Invalid signatures detected");
println!("\n\x1b[32m✓ PASS\x1b[0m\n");
}
} else {
// Valid test case - signatures should verify successfully
let result = signed_block.verify_signatures(anchor_state);

if result {
println!(" ✓ All signatures verified successfully");
println!("\n\x1b[32m✓ PASS\x1b[0m\n");
} else {
println!(" \x1b[31m✗ FAIL: Signature verification failed\x1b[0m\n");
return Err("Signature verification failed".into());
}
println!(" ✓ Correctly rejected: Invalid signatures detected");
println!("\n\x1b[32m✓ PASS\x1b[0m\n");
return Ok(());
}

let result = signed_block.verify_signatures(anchor_state);
if !result {
println!(" \x1b[31m✗ FAIL: Signature verification failed\x1b[0m\n");
return Err("Signature verification failed".into());
}

println!(" ✓ All signatures verified successfully");
println!("\n\x1b[32m✓ PASS\x1b[0m\n");
Ok(())
}
}
9 changes: 3 additions & 6 deletions lean_client/containers/tests/test_vectors/verify_signatures.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,6 @@
//! NOTE: Without the `xmss-verify` feature, signature verification only checks
//! structure (attestation count matches signature count, validator indices valid).
//! Full cryptographic verification requires `--features xmss-verify`.
//!
//! IMPORTANT: There is currently a configuration mismatch between leanSpec Python
//! (HASH_LEN_FE=8, 52-byte pubkeys) and leansig Rust (HASH_LEN_FE=7, 48-byte pubkeys).
//! Until this is resolved, the xmss-verify tests will fail with "Invalid public key length".

use std::path::Path;

Expand All @@ -16,10 +12,11 @@ use test_generator::test_resources;
use super::runner::TestRunner;

#[test_resources("test_vectors/verify_signatures/*/verify_signatures/*/*.json")]
fn verify_signatures(spec_file: &str) {
fn verify_signatures(spec_file: &str) -> Result<(), Box<dyn std::error::Error>> {
let test_path = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join(spec_file);

TestRunner::run_verify_signatures_test(test_path).unwrap();
TestRunner::run_verify_signatures_test(test_path)?;
Copy link
Collaborator

@ArtiomTr ArtiomTr Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? just unwrap error, no need to handle them gracefully

Ok(())
}
18 changes: 9 additions & 9 deletions ...y_signatures/devnet/verify_signatures/test_invalid_signatures/test_invalid_signature.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"tests/consensus/devnet/verify_signatures/test_invalid_signatures.py::test_invalid_signature[fork_Devnet][fork_devnet-verify_signatures_test]": {
"network": "Devnet",
"leanEnv": "test",
"leanEnv": "prod",
Copy link
Collaborator

@ArtiomTr ArtiomTr Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't generate tests by hand, update makefile script & use it instead:

.PHONY: generate-test-vectors
generate-test-vectors:
	@rm -rf spec && \
		mkdir -p spec && \
		cd spec && \
		git init && \
		git remote add origin https://github.com/leanEthereum/leanSpec.git && \
		git fetch --depth 1 origin $(LEAN_SPEC_COMMIT) && \
		git switch --detach FETCH_HEAD
-	cd spec && uv run fill --clean --fork=devnet
+	cd spec && uv run fill --clean --fork=devnet --scheme prod
	rm -rf ./test_vectors && mkdir -p ./test_vectors
	cp -r ./spec/fixtures/consensus/* ./test_vectors/

"anchorState": {
"config": {
"genesisTime": 0
Expand Down Expand Up @@ -31,7 +31,7 @@
"validators": {
"data": [
{
"pubkey": "0x8fa00001b5b22a073d21cb60c9465b379f80e74fee9baf1d137eb651ed162900a6f3ec32da74ff09265ea62fc9a5055b2e9b7c7b",
"pubkey": "0x2c7aa65d0b45c01e3ba7155997d3001613775f1e1b85fd1ab65dcd72c65e105f2552c43a311a290af56fa0262dfebe10745ea837",
"index": 0
}
]
Expand All @@ -48,8 +48,8 @@
"block": {
"slot": 1,
"proposerIndex": 0,
"parentRoot": "0xb6f6683bf01027dd60f095f477b8ca38fbe23c18eb02f0aed0b2f34b7a4584f0",
"stateRoot": "0x6c53c1d71203251ceb0abcb1d14a34b53c0760abc1b1ac51d509cbe16d8ceb16",
"parentRoot": "0x438b1cbc01c3c69b14f8853c6463d28b58118798f414f3be472aae4cd77dd572",
"stateRoot": "0x38d7edaf9c08ffb285eb3e1e64456fbe430d4c4a4bab9b0bf29565b74da5c620",
"body": {
"attestations": {
"data": []
Expand All @@ -61,15 +61,15 @@
"data": {
"slot": 1,
"head": {
"root": "0xcdc120c88d251c898fc9b9a1f091b0cb108d9ac82d2784029917c2ac3cee82ee",
"root": "0x6f2ebcd6e5eb1b34823a5fb5867ee63984905cc670722a01a060894b9b2cec3f",
"slot": 1
},
"target": {
"root": "0xcdc120c88d251c898fc9b9a1f091b0cb108d9ac82d2784029917c2ac3cee82ee",
"root": "0x6f2ebcd6e5eb1b34823a5fb5867ee63984905cc670722a01a060894b9b2cec3f",
"slot": 1
},
"source": {
"root": "0xb6f6683bf01027dd60f095f477b8ca38fbe23c18eb02f0aed0b2f34b7a4584f0",
"root": "0x438b1cbc01c3c69b14f8853c6463d28b58118798f414f3be472aae4cd77dd572",
"slot": 0
}
}
Expand Down Expand Up @@ -104,10 +104,10 @@
},
"expectException": "AssertionError",
"_info": {
"hash": "0x0d7dbcbd6fd7a106e16128e65e3650693380131c8864455f3a9e2c1003f710ba",
"hash": "0x8d97e6b6a601e10856ae70720ffad8cd392dab8169fe158054edac0bc0f0e49a",
"comment": "`leanSpec` generated test",
"testId": "tests/consensus/devnet/verify_signatures/test_invalid_signatures.py::test_invalid_signature[fork_Devnet]",
"description": "Test that invalid signatures are properly rejected during verification.\n\n Scenario\n --------\n - Single block at slot 1\n - Proposer attestation has an invalid signature\n - No additional attestations (only proposer attestation)\n\n Expected Behavior\n -----------------\n 1. Proposer's signature in SignedBlockWithAttestation is rejected\n\n Why This Matters\n ----------------\n This test verifies the negative case:\n - Signature verification actually validates cryptographic correctness\n not just structural correctness.\n - Invalid signatures are caught, not silently accepted",
"description": "Test that invalid signatures are properly rejected during verification.\n\nScenario\n--------\n- Single block at slot 1\n- Proposer attestation has an invalid signature\n- No additional attestations (only proposer attestation)\n\nExpected Behavior\n-----------------\n1. Proposer's signature in SignedBlockWithAttestation is rejected\n\nWhy This Matters\n----------------\nThis test verifies the negative case:\n- Signature verification actually validates cryptographic correctness\n not just structural correctness.\n- Invalid signatures are caught, not silently accepted",
"fixtureFormat": "verify_signatures_test"
}
}
Expand Down
1,238 changes: 1,123 additions & 115 deletions ...devnet/verify_signatures/test_valid_signatures/test_proposer_and_attester_signatures.json
View file
Open in desktop

Large diffs are not rendered by default.

Loading
Loading