Skip to content

Commit 674a1c2

Browse files
AliceInHunterlandAliceInHunterland
committed
proposal: NeoFS block storage format
Signed-off-by: Ekaterina Pavlova <[email protected]>
1 parent f329e87 commit 674a1c2

File tree

1 file changed

+115
-0
lines changed

1 file changed

+115
-0
lines changed

‎neofs-blocks.mediawiki

+115
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
<pre>
2+
NEP: TBD
3+
Title: NeoFS block storage format
4+
Author: Ekaterina Pavlova <[email protected]>
5+
Type: Informational
6+
Status: Draft
7+
Created: 2025-04-07
8+
</pre>
9+
10+
==Abstract==
11+
12+
This proposal outlines the specification for storing Neo blockchain blocks within
13+
the NeoFS container.
14+
15+
==Motivation==
16+
17+
Neo node synchronization via P2P requires all headers and blocks (~700 bytes each, over 1 GB per year with
18+
15-second blocks) to remain available, limiting scalability as chains grow. Storing blocks in NeoFS
19+
provides a distributed alternative, reducing local storage, enabling on-demand fetches and allowing to drop
20+
blocks and headers from storage for most nodes.
21+
22+
==Specification==
23+
24+
===Block Storage Schema===
25+
A single NeoFS container is used to store blocks and index files per network. Each container
26+
has network magic attribute (`Magic:56753`). Each block is stored in a binary
27+
form as a separate object with a unique OID and a set of attributes:
28+
* Block: block object identifier with block index value.
29+
* Hash: block hash in the in hex hash representation.
30+
* PrevHash: previous block hash in the in hex hash representation.
31+
* BlockTime: millisecond-precision block creation timestamp.
32+
* Timestamp: second-precision block uploading timestamp.
33+
34+
These attributes are stored as NeoFS object metadata alongside the binary block data and can be extended.
35+
Multibyte data items are always stored in little-endian order, where the low bytes come first.
36+
Its structure is standard Neo block serialization and the object may have additional data after the block.
37+
38+
An [https://rest.fs.neo.org/3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV/Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1 example]
39+
of a block object is as follows:
40+
<pre>
41+
ID: Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1
42+
CID: 3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV
43+
Owner: NVvY1FF67XJ2GTVhy9FqiZGC4jEQtvjmHt
44+
CreatedAt: 28202
45+
Size: 697
46+
HomoHash: 45c98e627910d9b915d58368493789ce49ca194626f16ea5bf6b57bb2cce462921a1d3faf682d252a804b0555ca48905286222ee9209b3ff1ce4677a082ffd4d
47+
Checksum: fa6cedddfec3a61157c4a12e25f81e85c0f92aac70317d6df7fe193f983b4917
48+
Type: REGULAR
49+
Attributes:
50+
Block=1
51+
Primary=0
52+
Hash=5f3fbb43d1e516fe07771e2e873ebc9e2810662401acf603a775aace486220bd
53+
PrevHash=1f4d1defa46faa5e7b9b8d3f79a06bec777d7c26c4aa5f6f5899a291daa87c15
54+
BlockTime=1627894840919
55+
Timestamp=1734362616 (2024-12-16 18:23:36 +0300 MSK)
56+
ID signature:
57+
public key: 02a4920745d86db224c179c936606dc0e4620edad13d568ef036da279352e45f2b
58+
signature: 0443ff20d15952759b5101a7d223d70eb992260fd9ad5aecb404a97448b2ea54bd1ad783b12ccddba0097dd6608b55ccac9215c5715f9589ec6a555542ead6dc00
59+
[Block Binary Data]
60+
</pre>
61+
62+
Each index file is an object containing a constant-sized batch of raw block object
63+
IDs in binary form ordered by block index. Each index file is marked with the
64+
following attributes:
65+
* Index: index file identifier with consecutive file index value.
66+
* IndexSize: the number of OIDs included into index file.
67+
* Timestamp: second-precision index file uploading timestamp.
68+
69+
An [https://rest.fs.neo.org/3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV/4nxxFWRFbNbcfwf5EuftboHQVnnsH6vDw5Dp6mPTuwYN example]
70+
of a index object is as follows:
71+
<pre>
72+
ID: 4nxxFWRFbNbcfwf5EuftboHQVnnsH6vDw5Dp6mPTuwYN
73+
CID: 3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV
74+
Owner: NVvY1FF67XJ2GTVhy9FqiZGC4jEQtvjmHt
75+
CreatedAt: 28202
76+
Size: 4096000
77+
HomoHash: 6d01444340687199c44a3dfaa719954a0f1343108fe7dda6ad03b27eaa4247d87b7bb7cd8de29f09027280cac9490b53261a005c7e8b84ab8e65818439f190d6
78+
Checksum: 57ee0f196c49a7b85b9dc4c5ce97e032511c3c7fe7af57754adf455e48e68b80
79+
Type: REGULAR
80+
Attributes:
81+
Index=1
82+
IndexSize=128000
83+
Timestamp=1734363922 (2024-12-16 18:45:22 +0300 MSK)
84+
ID signature:
85+
public key: 02b6a56095b0c8971bfc7ec0ade0cb97ed7f616360c6dd0d0b93f4ce8d38c64a8d
86+
signature: 046c531bf202429ba86842983f48dfb7693e5ed881e9e9f7347625350a6d10694b011d3b808821b5cfe9e383aef34094356b63e6319383e541ba25a152e84aa2df
87+
[32-bit OID array]
88+
</pre>
89+
90+
===Synchronization Algorithm===
91+
Blocks stored in NeoFS and those synchronized via P2P both use the standard Neo block
92+
serialization format, ensuring compatibility across different synchronization methods.
93+
The NeoFS container serves as verified storage for downloading blocks, with two methods
94+
for block retrieval:
95+
- Index File Search: Searches for index files by index file attribute and reads block
96+
OIDs from index file object-by-object.
97+
- Direct Block Search: Searches blocks one by one directly by block attribute.
98+
99+
Downloaded blocks are inserted into the blockchain using the same logic as in the
100+
P2P synchronization protocol.
101+
102+
==Rationale==
103+
This proposal defines a NeoFS storage schema, building on experiments (e.g., nspcc-dev/neo-go#3496)
104+
and future enhancements (#1526).
105+
106+
== References ==
107+
* [https://github.com/neo-project/neo/issues/3463 Neo Issue#3463]
108+
* [https://github.com/neo-project/neo/issues/2373 Neo Issue#2373]
109+
* [https://github.com/neo-project/neo/issues/1526 Neo Issue#1526]
110+
* [https://github.com/nspcc-dev/neo-go/issues/3496 NeoGo Issue#3496]
111+
* [https://github.com/nspcc-dev/neofs-api/issues/285 NeoFS API Issue#285]
112+
113+
114+
== Implementation ==
115+
* Go: https://github.com/nspcc-dev/neo-go/blob/61992405983ddfb5547ae3afab75c22186dc966f/cli/util/uploader.go

0 commit comments

Comments
 (0)