Compilation to Web Assemby is broken

[phroi]

Compiling any code that uses blake2b-rswith target wasm32-unknown-unknown fails with the following:

user@host:~/ckb-sdk-rust-wasm-test$ sudo apt install llvm-dev libclang-dev clang lld;
user@host:~/ckb-sdk-rust-wasm-test$ cargo build --target wasm32-unknown-unknown;
   Compiling blake2b-rs v0.2.0
The following warnings were emitted during compilation:

warning: BLAKE2/ref/blake2b-ref.c:17:10: fatal error: 'string.h' file not found
warning: #include <string.h>
warning:          ^~~~~~~~~~
warning: 1 error generated.

error: failed to run custom build command for `blake2b-rs v0.2.0`

Caused by:
  process didn't exit successfully: `/home/user/ckb-sdk-rust-wasm-test/target/debug/build/blake2b-rs-fbb694b1e880f308/build-script-build` (exit status: 1)
  --- stdout
  TARGET = Some("wasm32-unknown-unknown")
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_wasm32-unknown-unknown = None
  CC_wasm32_unknown_unknown = None
  TARGET_CC = None
  CC = None
  CFLAGS_wasm32-unknown-unknown = None
  CFLAGS_wasm32_unknown_unknown = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  running: "clang" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=wasm32-unknown-unknown" "-Wall" "-Wextra" "-o" "/home/user/ckb-sdk-rust-wasm-test/target/wasm32-unknown-unknown/debug/build/blake2b-rs-63f6b231df7e98c0/out/BLAKE2/ref/blake2b-ref.o" "-c" "BLAKE2/ref/blake2b-ref.c"
  cargo:warning=BLAKE2/ref/blake2b-ref.c:17:10: fatal error: 'string.h' file not found
  cargo:warning=#include <string.h>
  cargo:warning=         ^~~~~~~~~~
  cargo:warning=1 error generated.
  exit status: 1

  --- stderr


  error occurred: Command "clang" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=wasm32-unknown-unknown" "-Wall" "-Wextra" "-o" "/home/user/ckb-sdk-rust-wasm-test/target/wasm32-unknown-unknown/debug/build/blake2b-rs-63f6b231df7e98c0/out/BLAKE2/ref/blake2b-ref.o" "-c" "BLAKE2/ref/blake2b-ref.c" with args "clang" did not execute successfully (status code exit status: 1).

[phroi]

This error is related to: Status of rust+C bindings for wasm targets

I created a small test repository to replicate this error: https://github.com/phroi/blake2b-rs-wasm-test

[blckngm]

There are pure rust blake2b implementations, e.g. blake2b-simd (this one is also more performant using SIMD), blake2.

[phroi]

Hey @sopium, glad to hear back on this issue!

You are right, still the main underlying Issue is that this library is indirectly imported by ckb-sdk-rust and by most rust CKB codebase.

By having this small bit non wasm compilable, all the rust CKB libraries that import directly or indirectly this library now are non wasm compilable. This means that most of rust CKB codebase is now not non wasm compilable due to this.

That's why I would advice adding a fallback for the WASM target.

Keep up the great work 💪
Phroi

[blckngm]

Seems that ckb-sdk-rust depends on blake2b-rs through ckb-hash. So we'd need to change ckb-hash to use a pure rust blake2b impl to support this.

[blckngm]

But ckb-hash uses blake2b-ref on wasm:

[target.'cfg(target_arch = "wasm32")'.dependencies]
blake2b-ref = "0.2.0"

I'll investigate further later.

[phroi]

Thanks @sopium!! To be honest I opened this issue last October and now I'm using Lumos for doing this job.

The situation may have changed in the meantime, please just double check on it to be sure.

Thanks again,
Phroi

[blckngm]

It's pulled in by sparse-merkle-tree, I've opened an issue there:

nervosnetwork/sparse-merkle-tree#48

[phroi]

Thanks for actually investigating this issue, good to know! 💪