Skip to content

Port subnet network topology from leanSpec PR #482#249

Merged
pablodeymo merged 4 commits intomainfrom
subnet-network-topology
Mar 25, 2026
Merged

Port subnet network topology from leanSpec PR #482#249
pablodeymo merged 4 commits intomainfrom
subnet-network-topology

Conversation

@pablodeymo
Copy link
Collaborator

Motivation

leanSpec PR #482 (merged 2025-03-25) introduces --aggregate-subnet-ids and changes how nodes subscribe to attestation subnets. The key architectural shift: subnet filtering moves from the fork choice store to the P2P subscription layer.

Before this PR, ethlambda had several gaps:

  • Only the first validator's subnet was used (multi-validator nodes all published/subscribed to one subnet)
  • The store had a hardcoded ATTESTATION_COMMITTEE_COUNT = 1 constant (ignored the CLI parameter)
  • The store performed per-attestation subnet filtering that the spec removed
  • No --aggregate-subnet-ids CLI flag existed
  • Non-aggregator validators didn't subscribe to their subnet (should, for gossipsub mesh health)

Description

Phase 1: CLI (bin/ethlambda/src/main.rs)

  • Add --aggregate-subnet-ids CLI flag (comma-separated, requires = "is_aggregator" via clap)
  • Collect all validator IDs instead of just the first one
  • Pass validator_ids: Vec<u64> and aggregate_subnet_ids to SwarmConfig

Phase 2: P2P layer (crates/net/p2p/src/lib.rs, crates/net/p2p/src/gossipsub/handler.rs)

  • SwarmConfig: validator_id: Option<u64>validator_ids: Vec<u64> + aggregate_subnet_ids: Option<Vec<u64>>
  • BuiltSwarm / P2PServer: single attestation_topicattestation_topics: HashMap<u64, IdentTopic> + attestation_committee_count: u64
  • Multi-subnet subscription logic:
    • All nodes with validators subscribe to their validator subnets (for gossipsub mesh health)
    • Aggregators additionally subscribe to any explicitly requested --aggregate-subnet-ids
    • Aggregator with no validators and no explicit subnets: fallback to subnet 0
    • Non-validator non-aggregator nodes: no attestation subscriptions
  • publish_attestation: routes per-validator to the correct subnet topic (validator_id % committee_count); if the subnet isn't subscribed, constructs the topic on-the-fly for gossipsub fanout
  • Metric lean_attestation_committee_subnet reports the lowest validator subnet (backward-compatible)

Phase 3: Store simplification (crates/blockchain/src/store.rs)

  • Removed ATTESTATION_COMMITTEE_COUNT constant and compute_subnet_id() helper
  • on_gossip_attestation: removed local_validator_ids parameter; stores gossip signatures unconditionally (subnet filtering already handled at P2P layer)
  • on_block / on_block_core: removed local_validator_ids parameter; stores proposer signature unconditionally

Phase 4: Caller updates (crates/blockchain/src/lib.rs, crates/blockchain/tests/signature_spectests.rs)

  • Updated all call sites to match simplified store function signatures

How to test

make fmt    # ✅ passes
make lint   # ✅ passes  
make test   # ✅ all 97 tests pass (forkchoice + signature + STF spec tests)

For devnet testing:

# Single subnet (existing behavior, unchanged)
--attestation-committee-count 1 --is-aggregator

# Multi-subnet with explicit aggregator subscription
--attestation-committee-count 2 --is-aggregator --aggregate-subnet-ids 0,1

# Multi-validator node (subscribes to all validator subnets automatically)
--attestation-committee-count 4 --is-aggregator

Verify via logs:

  • Look for Subscribed to attestation subnet lines (one per subscribed subnet)
  • Published attestation to gossipsub now includes subnet_id field
  • Gossip signatures are stored without subnet filtering (no more silent drops)

Notes

  • Non-aggregator subscription is new behavior: previously non-aggregators never subscribed to attestation subnets. Now they subscribe for mesh health. This slightly increases bandwidth but matches the spec.
  • Hardcoded "devnet0" network name: existing limitation carried forward in the fanout topic construction. Should be addressed separately.
  • Subnet metric: lean_attestation_committee_subnet is a single gauge. With multiple subnets, we report the lowest validator's subnet for backward compatibility. Can be improved in a follow-up.

Related

  • leanSpec PR #482 (upstream spec change)

  Add --aggregate-subnet-ids CLI flag and multi-subnet subscription support.
  Nodes now subscribe to all subnets their validators belong to (for mesh
  health), and aggregators can additionally subscribe to explicit subnets.
  The store no longer filters attestations by subnet — it stores all gossip
  signatures unconditionally, since the P2P layer already ensures only
  relevant subnets are subscribed.

  Key changes:
  - SwarmConfig accepts all validator IDs instead of just the first
  - P2P layer computes subnet set from validators + explicit aggregator IDs
  - publish_attestation routes per-validator to correct subnet topic
  - Store removes ATTESTATION_COMMITTEE_COUNT constant and compute_subnet_id
  - on_gossip_attestation and on_block no longer take local_validator_ids
@github-actions
Copy link

🤖 Kimi Code Review

Security & DoS Risk

  • crates/blockchain/src/store.rs:399-402, 638-646: Removing subnet filtering from on_gossip_attestation and on_block_core eliminates defense-in-depth. If the P2P layer is bypassed or contains a bug that delivers messages from unsubscribed topics, the node will store unbounded signatures, leading to memory exhaustion. Recommendation: Add a maximum capacity limit to insert_gossip_signature or retain a sanity-check filter against the configured committee count.

Input Validation

  • bin/ethlambda/src/main.rs:72-75: aggregate_subnet_ids accepts any u64 without validating that values are < attestation_committee_count. Subscribing to invalid subnets wastes resources and may partition the node from the correct mesh. Recommendation: Add validation in build_swarm or use a custom clap parser to ensure subnet IDs are within range.

Code Duplication & Maintainability

  • crates/net/p2p/src/gossipsub/handler.rs:136-143 vs crates/net/p2p/src/lib.rs:237-239: Topic string construction is duplicated. If the network name or encoding changes, both sites must be updated. Recommendation: Extract a helper function:
    fn attestation_topic(subnet_id: u64, network: &str) -> IdentTopic { ... }

Determinism

  • crates/net/p2p/src/lib.rs:243: Iterating over subscribe_subnets: HashSet produces non-deterministic subscription ordering, complicating debugging and reproducibility. Recommendation: Use BTreeSet or sort the IDs before iterating.

Metrics Accuracy

  • crates/net/p2p/src/lib.rs:232: Reporting only min(validator_subnets) to set_attestation_committee_subnet is misleading when the node subscribes to multiple subnets (e.g., as an aggregator with explicit --aggregate-subnet-ids). The metric no longer reflects actual subscriptions.

Defensive Programming

  • crates/net/p2p/src/gossipsub/handler.rs:126: validator % server.attestation_committee_count assumes attestation_committee_count > 0. While the CLI enforces this, the P2P crate should not rely on CLI validation for panic safety. Recommendation: Use checked_rem or assert at initialization.

Minor

  • crates/net/p2p/src/gossipsub/handler.rs:139: The network variable is hardcoded as "devnet0" but should probably use a constant shared with lib.rs to avoid drift.

Summary: The architectural change to move subnet filtering to the P2P layer is sound, but the PR removes safety boundaries without adding compensating controls (DoS limits) and duplicates topic construction logic. Address the validation and deduplication issues before merging.


Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt

@github-actions
Copy link

🤖 Claude Code Review

Now I have everything I need to write the review.


Review: Port subnet network topology from leanSpec PR #482

Overall: Clean architectural improvement. The shift of subnet filtering from store to P2P layer is correct, and removing the broken ATTESTATION_COMMITTEE_COUNT = 1 constant is overdue. A few issues to address.


Hardcoded "devnet0" in publish fallback path

crates/net/p2p/src/gossipsub/handler.rs:140

.unwrap_or_else(|| {
    let network = "devnet0";
    ...
})

The fallback topic construction duplicates a magic string that already lives in build_swarm (lib.rs:189). Since all validators now subscribe to their own subnets (new behavior in this PR), the fallback path should be unreachable for any validator publishing its own attestation. If it ever is reached (e.g., bug or misconfiguration), a silent topic mismatch will occur — the message goes out on a topic no one subscribed to, with no warning.

At minimum, add a warn! before the unwrap_or_else block:

.unwrap_or_else(|| {
    warn!(subnet_id, "Publishing to unsubscribed attestation subnet via fanout");
    ...
})

The network name should also be derived from config rather than hardcoded here, but that's a pre-existing limitation.


Missing bounds validation for --aggregate-subnet-ids

crates/net/p2p/src/lib.rs:221-228

Subnet IDs from --aggregate-subnet-ids are not validated against attestation_committee_count. A user passing --attestation-committee-count 2 --aggregate-subnet-ids 0,1,5 will silently subscribe to subnet 5, which doesn't exist. Messages on that topic will never arrive, and there's no error or warning.

Consider adding a check after building subscribe_subnets:

if let Some(invalid_id) = subscribe_subnets.iter().find(|&&id| id >= config.attestation_committee_count) {
    // return Err or warn + remove
}

Non-aggregator validators receive-then-discard

crates/blockchain/src/lib.rs:453 and crates/net/p2p/src/lib.rs:213-219

Non-aggregator nodes with validators now subscribe to their validator subnet (new behavior, correct per spec for mesh health). However, the attestation message will travel through the full gossipsub stack → actor mailbox → BlockChainServer → early-return with warn!("Received unaggregated attestation but node is not an aggregator").

This is a known trade-off (noted in PR description) and the behavior is correct. The warn! level is slightly noisy for expected spec-compliant behavior though — a trace! or debug! would be more appropriate here. Worth a follow-up.


Old subnet filtering was effectively a no-op

Worth noting for the record: with the old ATTESTATION_COMMITTEE_COUNT = 1, compute_subnet_id(x) = x % 1 = 0 for all x. So in_our_subnet was always true for any non-empty local_validator_ids. The removed filtering code never actually filtered anything. The new unconditional storage is therefore semantically equivalent to the old behavior for single-subnet deployments, while correctly extending to multi-subnet.


Minor: validator_ids collection order is non-deterministic

bin/ethlambda/src/main.rs:151

let validator_ids: Vec<u64> = validator_keys.keys().copied().collect();

HashMap::keys() has no ordering guarantee. This doesn't affect correctness (subnets are computed per-validator independently), but min() on the resulting subnet set in build_swarm would be deterministic anyway. No action needed — just confirming this is intentional.


Positive observations

  • requires = "is_aggregator" on the clap argument is the right constraint.
  • The fallback subscribe_subnets.insert(0) for aggregators with no validators and no explicit subnets is a sensible default that preserves backward compatibility.
  • on_block_without_verification correctly passes no validator context (&[] → nothing) since spec tests don't need aggregation.
  • Removing the local_validator_ids parameter from on_block and on_gossip_attestation is a clean simplification with no correctness regression given the P2P filtering now in place.
  • The validator_ids: Vec<u64>HashSet<u64> deduplication via .collect() at the point of computing validator_subnets means duplicate validator IDs won't cause duplicate subscriptions.

Automated review by Claude (Anthropic) · sonnet · custom prompt

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 25, 2026

Greptile Summary

This PR ports the subnet network topology from leanSpec PR #482, moving attestation subnet filtering from the fork-choice store to the P2P subscription layer and adding multi-validator / multi-subnet support. The architectural shift is sound and the four-phase decomposition (CLI → P2P config → store simplification → caller updates) is clean.

Key changes:

  • SwarmConfig now accepts validator_ids: Vec<u64> and aggregate_subnet_ids: Option<Vec<u64>>; P2PServer holds a HashMap<u64, IdentTopic> keyed by subnet ID and an attestation_committee_count.
  • Non-aggregator validator nodes now subscribe to their attestation subnets for gossipsub mesh health (new behaviour, noted in PR description).
  • store::on_gossip_attestation and store::on_block_core store signatures unconditionally; the hardcoded ATTESTATION_COMMITTEE_COUNT = 1 constant and compute_subnet_id() helper are removed.
  • P1: Non-aggregator validator nodes now subscribe to attestation subnets and therefore receive attestations, but the existing warn!("Received unaggregated attestation but node is not an aggregator") in BlockChainServer::on_gossip_attestation fires for every one of them — producing continuous log spam for a valid configuration. The guard should be downgraded to trace!.
  • P2: Explicit --aggregate-subnet-ids values are not validated against attestation_committee_count, so an operator can silently subscribe to out-of-range subnet topics.

Confidence Score: 4/5

  • Safe to merge after fixing the non-aggregator warn-spam; no data loss or correctness regressions found.
  • The core architectural change is correct and well-structured. The only concrete bug is the warn! that will fire on every attestation received by non-aggregator validator nodes — noisy but not a correctness issue. The out-of-range subnet validation is a best-practice gap. All 97 tests pass, and the store simplification is straightforwardly correct.
  • crates/blockchain/src/lib.rs — the on_gossip_attestation warn-level guard needs to be addressed before non-aggregator validator nodes can run without log pollution.

Important Files Changed

Filename Overview
crates/net/p2p/src/lib.rs Core P2P refactor: SwarmConfig gains validator_ids: Vec<u64> + aggregate_subnet_ids; multi-subnet subscription logic is correct; explicit subnet IDs lack bounds-checking against attestation_committee_count.
crates/net/p2p/src/gossipsub/handler.rs publish_attestation now routes per-validator to the correct subnet topic via validator_id % committee_count; falls back to on-the-fly fanout topic construction when not subscribed. Hardcoded "devnet0" is an acknowledged pre-existing limitation.
crates/blockchain/src/lib.rs Simplified on_block and on_gossip_attestation call sites; however, the existing warn! in on_gossip_attestation will now fire continuously for non-aggregator validator nodes that subscribe to attestation subnets for mesh health.
crates/blockchain/src/store.rs Removed hardcoded ATTESTATION_COMMITTEE_COUNT and compute_subnet_id; gossip and proposer signatures are now stored unconditionally — subnet filtering correctly delegated to the P2P layer.
bin/ethlambda/src/main.rs Adds --aggregate-subnet-ids CLI flag with requires = "is_aggregator" guard; collects all validator IDs instead of only the first — straightforward and correct.
crates/blockchain/tests/signature_spectests.rs Trivial call-site update removing the now-deleted local_validator_ids parameter from on_block; no issues.

Sequence Diagram

sequenceDiagram
    participant CLI as CLI (main.rs)
    participant Swarm as build_swarm (lib.rs)
    participant P2P as P2PServer
    participant Handler as publish_attestation
    participant BC as BlockChainServer
    participant Store as store.rs

    CLI->>Swarm: SwarmConfig { validator_ids, aggregate_subnet_ids, committee_count }
    Note over Swarm: compute validator_subnets = validator_ids.map(vid % count)
    Note over Swarm: subscribe_subnets = validator_subnets ∪ explicit_ids (if aggregator)
    loop for each subnet_id in subscribe_subnets
        Swarm->>Swarm: gossipsub.subscribe(attestation_subnet_{id})
    end
    Swarm-->>P2P: BuiltSwarm { attestation_topics: HashMap, committee_count }

    Note over P2P: Incoming attestation gossip
    P2P->>BC: new_attestation(signed_attestation)
    BC->>Store: on_gossip_attestation(attestation) [no subnet filter]
    Store-->>BC: Ok (signature stored unconditionally)

    Note over P2P: Outgoing attestation publish
    P2P->>Handler: publish_attestation(attestation)
    Handler->>Handler: subnet_id = validator_id % committee_count
    alt subscribed topic found
        Handler->>P2P: swarm.publish(attestation_topics[subnet_id])
    else not subscribed (fanout)
        Handler->>Handler: construct topic on-the-fly ("devnet0")
        Handler->>P2P: swarm.publish(fanout_topic)
    end
Loading

Comments Outside Diff (1)

  1. crates/blockchain/src/lib.rs, line 452-455 (link)

    P1 Spurious warn! spam for non-aggregator validator nodes

    This PR intentionally subscribes non-aggregator validator nodes to their attestation subnets for gossipsub mesh health (see PR notes). As a result, those nodes will now receive attestations via gossip. Every received attestation will hit this if !self.is_aggregator guard and emit a warn! — once per attestation per slot — producing continuous log noise for a perfectly valid configuration.

    The warn! was written when non-aggregators were never subscribed to attestation topics and receiving one was truly unexpected. Now that subscribing is intentional, this should be downgraded to trace! (or dropped entirely).

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: crates/blockchain/src/lib.rs
    Line: 452-455
    
    Comment:
    **Spurious `warn!` spam for non-aggregator validator nodes**
    
    This PR intentionally subscribes non-aggregator validator nodes to their attestation subnets for gossipsub mesh health (see PR notes). As a result, those nodes will now *receive* attestations via gossip. Every received attestation will hit this `if !self.is_aggregator` guard and emit a `warn!` — once per attestation per slot — producing continuous log noise for a perfectly valid configuration.
    
    The `warn!` was written when non-aggregators were never subscribed to attestation topics and receiving one was truly unexpected. Now that subscribing is intentional, this should be downgraded to `trace!` (or dropped entirely).
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: crates/blockchain/src/lib.rs
Line: 452-455

Comment:
**Spurious `warn!` spam for non-aggregator validator nodes**

This PR intentionally subscribes non-aggregator validator nodes to their attestation subnets for gossipsub mesh health (see PR notes). As a result, those nodes will now *receive* attestations via gossip. Every received attestation will hit this `if !self.is_aggregator` guard and emit a `warn!` — once per attestation per slot — producing continuous log noise for a perfectly valid configuration.

The `warn!` was written when non-aggregators were never subscribed to attestation topics and receiving one was truly unexpected. Now that subscribing is intentional, this should be downgraded to `trace!` (or dropped entirely).

```suggestion
        if !self.is_aggregator {
            trace!("Received unaggregated attestation on non-aggregator node (subscribed for mesh health), ignoring");
            return;
        }
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: crates/net/p2p/src/lib.rs
Line: 221-228

Comment:
**No bounds-check on explicit `aggregate_subnet_ids`**

The `--aggregate-subnet-ids` values provided via CLI are added directly to `subscribe_subnets` without being validated against `attestation_committee_count`. An operator running e.g. `--attestation-committee-count 2 --aggregate-subnet-ids 0,1,5` will silently subscribe to subnet 5, which falls outside the valid range `[0, committee_count)`. This could cause the node to subscribe to a topic that no other peer publishes to, wasting a gossipsub mesh slot, and creates a mismatch between the node's subscriptions and the routing logic in `publish_attestation` (which computes `validator % committee_count`).

Consider adding a validation step here:

```rust
if config.is_aggregator {
    if let Some(ref explicit_ids) = config.aggregate_subnet_ids {
        for &id in explicit_ids {
            if id >= config.attestation_committee_count {
                // return an error or log a warning
                warn!(subnet_id = id, committee_count = config.attestation_committee_count,
                      "aggregate_subnet_ids contains out-of-range subnet ID");
            }
        }
        subscribe_subnets.extend(explicit_ids);
    }
    ...
}
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "Port leanSpec PR #482: subnet filtering ..." | Re-trigger Greptile

Comment on lines 221 to +228
if config.is_aggregator {
swarm
.behaviour_mut()
.gossipsub
.subscribe(&attestation_topic)?;
info!(%attestation_topic_kind, "Subscribed to attestation subnet");
if let Some(ref explicit_ids) = config.aggregate_subnet_ids {
subscribe_subnets.extend(explicit_ids);
}
// Aggregator with no validators and no explicit subnets: fallback to subnet 0
if subscribe_subnets.is_empty() {
subscribe_subnets.insert(0);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 No bounds-check on explicit aggregate_subnet_ids

The --aggregate-subnet-ids values provided via CLI are added directly to subscribe_subnets without being validated against attestation_committee_count. An operator running e.g. --attestation-committee-count 2 --aggregate-subnet-ids 0,1,5 will silently subscribe to subnet 5, which falls outside the valid range [0, committee_count). This could cause the node to subscribe to a topic that no other peer publishes to, wasting a gossipsub mesh slot, and creates a mismatch between the node's subscriptions and the routing logic in publish_attestation (which computes validator % committee_count).

Consider adding a validation step here:

if config.is_aggregator {
    if let Some(ref explicit_ids) = config.aggregate_subnet_ids {
        for &id in explicit_ids {
            if id >= config.attestation_committee_count {
                // return an error or log a warning
                warn!(subnet_id = id, committee_count = config.attestation_committee_count,
                      "aggregate_subnet_ids contains out-of-range subnet ID");
            }
        }
        subscribe_subnets.extend(explicit_ids);
    }
    ...
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/net/p2p/src/lib.rs
Line: 221-228

Comment:
**No bounds-check on explicit `aggregate_subnet_ids`**

The `--aggregate-subnet-ids` values provided via CLI are added directly to `subscribe_subnets` without being validated against `attestation_committee_count`. An operator running e.g. `--attestation-committee-count 2 --aggregate-subnet-ids 0,1,5` will silently subscribe to subnet 5, which falls outside the valid range `[0, committee_count)`. This could cause the node to subscribe to a topic that no other peer publishes to, wasting a gossipsub mesh slot, and creates a mismatch between the node's subscriptions and the routing logic in `publish_attestation` (which computes `validator % committee_count`).

Consider adding a validation step here:

```rust
if config.is_aggregator {
    if let Some(ref explicit_ids) = config.aggregate_subnet_ids {
        for &id in explicit_ids {
            if id >= config.attestation_committee_count {
                // return an error or log a warning
                warn!(subnet_id = id, committee_count = config.attestation_committee_count,
                      "aggregate_subnet_ids contains out-of-range subnet ID");
            }
        }
        subscribe_subnets.extend(explicit_ids);
    }
    ...
}
```

How can I resolve this? If you propose a fix, please make it concise.

  Deduplicates topic string construction between build_swarm and
  publish_attestation, and consolidates the hardcoded devnet0 network
  name into a single constant in gossipsub/messages.rs.
pablodeymo and others added 2 commits March 25, 2026 19:07
Co-authored-by: Tomás Grüner <47506558+MegaRedHand@users.noreply.github.com>
@pablodeymo pablodeymo merged commit bbb9165 into main Mar 25, 2026
2 checks passed
@pablodeymo pablodeymo deleted the subnet-network-topology branch March 25, 2026 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants