Add ephemeral block headers to the history network spec #341

pipermerriam · 2024-09-25T18:46:46Z

fixes #336

should probably depend on #291

What is wrong

We are going to move towards keeping unproven headers in a separate content type than the proven ones. This is based on the discussion in #336

How was it fixed

We will refer to headers near the front of the chain that are not in a stable HistoricalSummaries as "ephemeral" headers. Their only proving path is through tracing the Header.parent_hash through the chain of headers and anchoring to a trusted header. This trusted header must come from an independent oracle (such as the portal beacon network).

All clients in the network will be assumed to have some quantity of these recent headers. The Ping and Pong custom_data field will also be modified to include a new field that signals how many of these recent headers a client is holding.

Clients will need to perform a data migration on their database to remove the previous unproven headers that are no longer valid after this change. Values in the database that previously used the None proof type should be purged, and the subsequent values should be modifed to correct the Union selector if they have been stored with that as part of the data field.

Clients will need to implement a mechanism to purge ephemeral headers as or after they transition out of the most recent epoch boundary.

Clients will need to implement a syncing mechanism to fetch and fill their database with the appropriate window of recent headers that are still within the current unfinalized epoch.

Clients will need to implement an oracle (such as the portal beacon network) to be continually updated on what the most recent block header is, and to use this as the validation anchor for any headers requested this way.

Our hive and testing infrastructure will need to be updated to reflect these changes.

kdeme · 2024-09-26T07:13:49Z

history/history-network.md

@@ -157,7 +160,7 @@ each receipt/transaction and re-rlp-encode it, but only if it is a legacy transa

 HistoricalHashesAccumulatorProof = Vector[Bytes32, 15]

-BlockHeaderProof = Union[None, HistoricalHashesAccumulatorProof]
+BlockHeaderProof = Union[HistoricalHashesAccumulatorProof]


We could for now also leave this None in there, allowing this PR to get merged, before clients get ready to migrate data.

And that would allow us to decide on how to deal with it best (future wise).

kdeme · 2024-09-26T07:14:20Z

history/history-network.md

+ephemeral_headers_key = Container(block_hash: Bytes32, ancestor_count: uint8)
+selector         = 0x04
+
+ephemeral_header_payload = List(ByteList[1024], limit=256)


Suggested change

ephemeral_header_payload = List(ByteList[1024], limit=256)

ephemeral_header_payload = List(ByteList[1024], limit=MAX_EPHEMERAL_HEADER_PAYLOAD)

pipermerriam

I'm considering just moving the limit to 16384 (2**16). Very loose rationale is that at the point where we transition across an epoch boundary, it will take some time for the bridge to gossip new headers into the network.

Broader rationale is:

Allow time for bridges to gossip proven data as epoch boundaries pass by.
Eliminate awkwardness of having to use uint16 for ephemeral_header_count since it allows for a value up to 16384. This way we don't have a weird "dead range" at the top of that value that is invalid and thus would require some kind of special handling.
The proving path for headers deep in the ephemeral chain is expensive requiring proving all the way up to head, which adds a natural disincentive for clients to use this old ephemeral data when they don't have to.

Clients can still of course set this value to 8192 or any value they want. This gives us some freedom of movement. Realistically, the real value might be something like 9000 as it gives a little time for bridge gossip but still constrains this value appropriately.

morph-dev · 2024-09-30T07:20:28Z

history/history-network.md

+
+> Note: The content-id for this data type is not meaningful.
+
+> Note: This message is not valid for Gossip.  Clients **SHOULD** not send or accept gossip messages for this content type.


Instead of "Gossip", should we say "OFFER/ACCEPT"? We can clarify that this includes gossip as well, but that would be implied.

morph-dev · 2024-09-30T07:24:30Z

history/history-network.md

+```python
+# Content and content key
+
+ephemeral_headers_key = Container(block_hash: Bytes32, ancestor_count: uint8)


nit: Should we explicitly declare that ancestor_count can't be bigger than MAX_EPHEMERAL_HEADER_PAYLOAD? Or is it enough that it's implied by the content type?

Or is it ok to request more than 256, but answer shouldn't have more than that?

I'll make these things explicitly stated. They are enforced by the encoding types but worth making them stated limits too.

KolbyML · 2024-10-21T22:44:53Z

This PR changes the custom_payload for history. I propose we deploy this change through #348 which should make upgrading custom_payload easier in the future without the risk of breaking interop and allowing us a window for all implementations to update to the new format before deprecating older versions.

history/history-network.md

pipermerriam · 2025-01-08T14:42:43Z

I think this mechanism still has an unsolved issue.

There is no mechanism in place for getting new headers populated into the network. The current PR disallows GOSSIP/ACCEPT for the ephemeral header data type. I wrote it a long time ago and didn't provide a justification for this ban on GOSSIP, but I assume it was because the payload can't easily have a proof attached to it... maybe... we assume that each client has their own "oracle" that tells them the HEAD hash of the chain.

Under this model, I think things at the very HEAD of the chain become awkward to prove because of race conditions where client A hasn't yet progressed to the very latest block and client B is faster and tries to gossip the very latest block.

Anything 2-3 blocks back from the HEAD is probably easy to prove..

KolbyML · 2025-01-08T15:13:16Z

I think this mechanism still has an unsolved issue.

There is no mechanism in place for getting new headers populated into the network. The current PR disallows GOSSIP/ACCEPT for the ephemeral header data type. I wrote it a long time ago and didn't provide a justification for this ban on GOSSIP, but I assume it was because the payload can't easily have a proof attached to it... maybe... we assume that each client has their own "oracle" that tells them the HEAD hash of the chain.

maybe... we assume that each client has their own "oracle" that tells them the HEAD hash of the chain.

We do assume that they will either have access to Portal Beacon Network node, which would track the head and be constantly receiving the Execution Headers, via its updates

Or running a Full Node in which there also wouldn't be a problem

Under this model, I think things at the very HEAD of the chain become awkward to prove because of race conditions where client A hasn't yet progressed to the very latest block and client B is faster and tries to gossip the very latest block.

We can include an ephemeral header well gossiping the state, this will solve the race condition assuming the node already has latest minus 1 ephemeral headers.

We will need to include headers when gossiping bodies and receipts soon enough to lower latency, I don't see why state content being gossipped at the head of the chain would be any different.

This also avoids tons of nodes doing RFC for headers

Anything 2-3 blocks back from the HEAD is probably easy to prove..

If we include headers well gossiping state or bodies/receipts I think this issue goes away. Assuming that the Portal Client isn't syncing anymore there shouldn't be a problem.

To be clear I think we should ban gossiping ephemeral header data type individually, but if it is bundled with something else I think that is a requirement due to latency concerns going forward.

KolbyML · 2025-01-08T15:15:28Z

Also I think it is import to note all history,state,etc content from LATEST minus 64 blocks would be non-finalized, not just headers. So we would also need to handle side chains in this window for any history or state network content gossiped in this range. And clear bodies and state from the database, if the side chain is discarded.

pipermerriam · 2025-01-08T16:17:38Z

Ok, I think I have what I need to get this PR updated. I'll get that done today.

KolbyML · 2025-01-08T16:34:11Z

Ok, I think I have what I need to get this PR updated. I'll get that done today.

awesome 🚀

kdeme · 2025-01-09T10:46:52Z

history/history-network.md

@@ -236,6 +233,47 @@ content          = SSZ.serialize(block_header_with_proof)
 content_key      = selector + SSZ.serialize(block_number_key)
 ```

+##### Ephemeral Block Headers
+
+This content type represents block headers *near* the HEAD of the chain.  They are provable by tracing through the chain of `header.parent_hash` values.  All nodes in the network are assumed to store some amount of this content.  The `Ping.custom_data` and `Pong.custom_data` fields can be used to learn the number of recent headers that a client makes available.


Should we add a phrase stating that nodes should aim to store at least the latest 8192 ephemeral block headers? (Obviously when syncing they can have less).

kdeme · 2025-01-09T10:48:05Z

history/history-network.md

+
+> Note: This message is not valid for Gossip.  Clients **SHOULD** not send or accept gossip messages for this content type.
+
+> Note: Clients **SHOULD** implement a mechanism to purge headers older than 8192 blocks from their content databases.


It is stated before that the amount is configurable per client. So perhaps we shouldn't mention a specific value here.

The number needs to be 8192 in this context because this is the point where they transition past the ephemeral boundary and should be replaced by the regular content type that has a stable proof.

history/history-network.md

KolbyML

Co-authored-by: kdeme <7857583+kdeme@users.noreply.github.com>

kdeme reviewed Sep 26, 2024

View reviewed changes

pipermerriam commented Sep 26, 2024

View reviewed changes

pipermerriam force-pushed the piper/add-ephemeral-headers branch from ba5ac09 to efd11c8 Compare September 26, 2024 16:03

pipermerriam marked this pull request as ready for review September 26, 2024 16:22

pipermerriam changed the title ~~WIP: Add ephemeral block headers to the history network spec~~ Add ephemeral block headers to the history network spec Sep 26, 2024

morph-dev approved these changes Sep 30, 2024

View reviewed changes

pipermerriam mentioned this pull request Oct 3, 2024

Add HistoricalRootsBlockProof for merge till capella blocks #287

Merged

kdeme mentioned this pull request Oct 16, 2024

Priliminary spec for header gossip network #76

Closed

pipermerriam commented Nov 20, 2024

View reviewed changes

history/history-network.md Show resolved Hide resolved

Add ephemeral block headers to the history network spec

1402aa9

pipermerriam force-pushed the piper/add-ephemeral-headers branch 3 times, most recently from 7bb9e63 to ab54ecf Compare January 8, 2025 19:46

explicit language about the ephemeral_headers_key validity

18f4740

pipermerriam force-pushed the piper/add-ephemeral-headers branch from ab54ecf to 18f4740 Compare January 8, 2025 19:50

kdeme reviewed Jan 9, 2025

View reviewed changes

morph-dev approved these changes Jan 11, 2025

View reviewed changes

KolbyML approved these changes Jan 11, 2025

View reviewed changes

ogenev approved these changes Jan 13, 2025

View reviewed changes

pipermerriam and others added 3 commits January 13, 2025 09:27

Update history/history-network.md

2cc4c8f

Co-authored-by: kdeme <7857583+kdeme@users.noreply.github.com>

Update history/history-network.md

26ea7c0

Co-authored-by: kdeme <7857583+kdeme@users.noreply.github.com>

Add suggestion on how much clients should store

0a096a1

pipermerriam merged commit 0fa815f into ethereum:master Jan 13, 2025
2 checks passed

pipermerriam deleted the piper/add-ephemeral-headers branch January 13, 2025 16:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ephemeral block headers to the history network spec #341

Add ephemeral block headers to the history network spec #341

pipermerriam commented Sep 25, 2024 •

edited

Loading

kdeme Sep 26, 2024

kdeme Sep 26, 2024

pipermerriam left a comment

morph-dev Sep 30, 2024

morph-dev Sep 30, 2024

pipermerriam Sep 30, 2024

KolbyML commented Oct 21, 2024

pipermerriam commented Jan 8, 2025

KolbyML commented Jan 8, 2025

KolbyML commented Jan 8, 2025 •

edited

Loading

pipermerriam commented Jan 8, 2025

KolbyML commented Jan 8, 2025

kdeme Jan 9, 2025

kdeme Jan 9, 2025

pipermerriam Jan 13, 2025

KolbyML left a comment

	ephemeral_header_payload = List(ByteList[1024], limit=256)
	ephemeral_header_payload = List(ByteList[1024], limit=MAX_EPHEMERAL_HEADER_PAYLOAD)


		> Note: The content-id for this data type is not meaningful.

		> Note: This message is not valid for Gossip. Clients SHOULD not send or accept gossip messages for this content type.


		> Note: This message is not valid for Gossip. Clients SHOULD not send or accept gossip messages for this content type.

		> Note: Clients SHOULD implement a mechanism to purge headers older than 8192 blocks from their content databases.

Add ephemeral block headers to the history network spec #341

Add ephemeral block headers to the history network spec #341

Conversation

pipermerriam commented Sep 25, 2024 • edited Loading

What is wrong

How was it fixed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pipermerriam left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KolbyML commented Oct 21, 2024

pipermerriam commented Jan 8, 2025

KolbyML commented Jan 8, 2025

KolbyML commented Jan 8, 2025 • edited Loading

pipermerriam commented Jan 8, 2025

KolbyML commented Jan 8, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KolbyML left a comment

Choose a reason for hiding this comment

pipermerriam commented Sep 25, 2024 •

edited

Loading

KolbyML commented Jan 8, 2025 •

edited

Loading