`GenesisConfig` in a native runtime free world

[bkchr]

The GenesisConfig is currently one of the only things for that we still require the native runtime. It isn't compiled for wasm right now. We also require the native runtime to load a GenesisConfig from a JSON file and as the GenesisConfig can change with newer native runtimes we got the raw chain spec that only contains the key/value storage pairs of the genesis block. When we remove the native runtime we need a new way to handle the GenesisConfig.

I propose that we put the GenesisConfig into the genesis wasm. This means that we start compiling the GenesisConfig for wasm, but I would put this behind some rust feature. So, the normal production builds should not include the GenesisConfig anymore. This is done to reduce the size of the wasm binary. I would also keep supporting the serialization of the GenesisConfig into JSON. If the GenesisConfig is part of the genesis wasm, we have some nice advantages. The first one being that we could keep the JSON serialized GenesisConfig in the chain spec and even distribute this chain spec as we would call into the genesis wasm to parse and generate the genesis block. This would require that we move out the genesis wasm from the actual genesis field in the chain spec to some dedicated field from where we can load the wasm binary. Another advantage would be that we could finally make the chain-spec-builder work properly. Currently this binary isn't really working for chain specs that aren't build from the kitchensink-runtime. The chain-spec-builder could be passed some crate name that belongs to some runtime and then it would generate the chain spec for this runtime. We could also enable certain features in the runtime etc.

To build what I outlined above, we would need the following:

• Make GensisConfig build in wasm. This will require supporting serde in the wasm build to serialize/deserialize the GenesisConfig to/from JSON.
• Add a new runtime api GenesisBuilder (better name should be found) that can be used to build the genesis config, load it etc.
• Change the chain spec to have a dedicated field for the code.
• Conditionally compile GenesisConfig into the wasm binary. By default the GenesisConfig should probably be compiled into the wasm binary. When doing a on-chain build there should exist a feature to disable the GenesisConfig.

Part of: #62

[michalkucharczyk]

Required support for serde addressed in paritytech/substrate#13027

[arkpar]

Make GensisConfig build in wasm. This will require supporting serde in the wasm build to serialize/deserialize the GenesisConfig to/from JSON.

Is anyone really using runtime-formatted JSON specs? All we really need is raw spec which does not require serde.

[ggwpez]

Is anyone really using runtime-formatted JSON specs? All we really need is raw spec which does not require serde.

I think zombienet relies on meddling with the JSON chain spec and other testing tools probably as well.
Also it is very convenient for debugging…

[bkchr]

Yeah and this will open a lot of possibilities. We will be able to ship json chain specs without them breaking, as the wasm file that loads them stays the same. We will be able to write tools that can modify chain specs very easily etc. It improves the user experience quite a lot.

[xlc]

I see it is a nice to have but not essential. We can always load metadata from wasm and use it to convert between the raw values and JSON. polkadot.js makes this relatively easy.

If this feature doesn't need to be part of Substrate, maybe it doesn't need to be part of Substrate?

[bkchr]

I see it is a nice to have but not essential. We can always load metadata from wasm and use it to convert between the raw values and JSON. polkadot.js makes this relatively easy.

Good point, if that works good enough we don't need serde/json support.

[xlc]

Chopsticks already implemented this feature and it works well.

Custom storage override:
https://github.com/AcalaNetwork/chopsticks/blob/master/configs/kusama.yml

Code that convert yaml/json to hex:
https://github.com/AcalaNetwork/chopsticks/blob/master/packages/chopsticks/src/utils/set-storage.ts

[bkchr]

Yeah to hex is easy, the other way around is more the problem. GenesisConfig fields are also not directly the storage items and can be from a different layout and then use custom logic to fill the storage. All in all still something that sounds useful to me. Especially as it is really just much easier for people to interact with it.

We would also then "disable" the GenesisConfig for builds that are going on chain as runtime upgrade to remove serde etc. So, there isn't really any downside.