Matthias/conduit

Fork 0

CI: Fix Building with rocksdb #213

New issue

Closed

opened 2022-01-16 23:12:18 +00:00 by jfowl · 16 comments

jfowl commented

2022-01-16 23:12:18 +00:00

(Migrated from gitlab.com)

Since adding rocksdb, the CI fails to compile for arm, armv7 and aarch64.

Below, an log excerpt of the error (for armv7 in this case):

error: failed to run custom build command for `librocksdb-sys v6.20.3`

Caused by:
  process didn't exit successfully: `/home/rust/src/conduit/target/debug/build/librocksdb-sys-68ec048b6f8d2603/build-script-build` (exit status: 101)
  --- stderr
  /usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
  /usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found, err: true
  thread 'main' panicked at 'unable to generate rocksdb bindings: ()', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/build.rs:44:10

The relevant buildscript from rocksdb: https://github.com/rust-rocksdb/rust-rocksdb/blob/master/librocksdb-sys/build.rs#L37

I don't really know how to fix this.

Since adding rocksdb, the CI fails to compile for arm, armv7 and aarch64. Below, an [log excerpt](https://gitlab.com/famedly/conduit/-/jobs/1980441384) of the error (for armv7 in this case): ``` error: failed to run custom build command for `librocksdb-sys v6.20.3` Caused by: process didn't exit successfully: `/home/rust/src/conduit/target/debug/build/librocksdb-sys-68ec048b6f8d2603/build-script-build` (exit status: 101) --- stderr /usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found /usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found, err: true thread 'main' panicked at 'unable to generate rocksdb bindings: ()', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/build.rs:44:10 ``` The relevant buildscript from rocksdb: https://github.com/rust-rocksdb/rust-rocksdb/blob/master/librocksdb-sys/build.rs#L37 I don't really know how to fix this.

jfowl commented

2022-01-16 23:14:10 +00:00

(Migrated from gitlab.com)

changed the description

jfowl commented

2022-01-16 23:25:00 +00:00

(Migrated from gitlab.com)

But I don't know, if something like this would solve our problem, since we want to use musl and not glibc

Related: https://github.com/rust-rocksdb/rust-rocksdb/issues/550 But I don't know, if something like this would solve our problem, since we want to use musl and not glibc

jfowl commented

2022-01-16 23:34:01 +00:00

(Migrated from gitlab.com)

So what appears to fix it:

apt update && apt install -qq gcc-arm-linux-gnueabihf

For armv7: BINDGEN_EXTRA_CLANG_ARGS="-I/usr/arm-linux-gnueabihf/include/" cargo build --target $TARGET

But holy moly, the build script for rocksdb-sys is really slow.

Maybe we can remove some unused compression options @timokoesters ? https://github.com/rust-rocksdb/rust-rocksdb/blob/master/Cargo.toml#L20

So what appears to fix it: ```bash apt update && apt install -qq gcc-arm-linux-gnueabihf ``` For armv7: `BINDGEN_EXTRA_CLANG_ARGS="-I/usr/arm-linux-gnueabihf/include/" cargo build --target $TARGET` But holy moly, the build script for rocksdb-sys is really slow. Maybe we can remove some unused compression options @timokoesters ? https://github.com/rust-rocksdb/rust-rocksdb/blob/master/Cargo.toml#L20

timokoesters commented

2022-01-17 08:26:22 +00:00

(Migrated from gitlab.com)

Good idea, I believe we only need Zstd

jfowl commented

2022-01-17 16:05:50 +00:00

(Migrated from gitlab.com)

rocksdb = { version = "0.17.0", default-features = false, features = ["multi-threaded-cf", "zstd"], optional = true }

It compiles without and creating an account works.
Any more tests needed?

```toml rocksdb = { version = "0.17.0", default-features = false, features = ["multi-threaded-cf", "zstd"], optional = true } ``` It compiles without and creating an account works. Any more tests needed?

jfowl commented

2022-01-17 18:05:43 +00:00

(Migrated from gitlab.com)

Incremental debug build after setting minimal rocksdb features:

Incremental debug build after setting minimal rocksdb features: ![image](/uploads/cd6ed72b71aba11db3396e2fbe44fd76/image.png)

timokoesters commented

2022-01-17 20:53:44 +00:00

(Migrated from gitlab.com)

I think that's okay.

jfowl commented

2022-01-17 22:39:31 +00:00

(Migrated from gitlab.com)

changed title from CI: Fix {-arm targets-} to CI: Fix {+Building with rocksdb+}

changed title from **CI: Fix {-arm targets-}** to **CI: Fix {+Building with rocksdb+}**

jfowl commented

2022-01-17 22:40:10 +00:00

(Migrated from gitlab.com)

Update: Everything is broken.

Apparently the x86 builds included libstdc++.

Rainer provides:

$ ldd conduit-x86_64-unknown-linux-musl 
	linux-vdso.so.1 (0x00007ffd5173f000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f73de52b000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f73de3e8000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f73de21f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f73e0305000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f73de205000)

Might be worth trying to set the -static clang flag

Update: Everything is broken. Apparently the x86 builds included libstdc++. Rainer provides: ``` $ ldd conduit-x86_64-unknown-linux-musl linux-vdso.so.1 (0x00007ffd5173f000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f73de52b000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f73de3e8000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f73de21f000) /lib64/ld-linux-x86-64.so.2 (0x00007f73e0305000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f73de205000) ``` Might be worth trying to set the `-static` clang flag

jfowl commented

2022-01-18 10:19:18 +00:00

(Migrated from gitlab.com)

Nope, -static wasn't it.

BINDGEN_EXTRA_CLANG_ARGS="-static" cargo build --target x86_64-unknown-linux-musl

failed on Alpine.

Nope, `-static` wasn't it. ```bash BINDGEN_EXTRA_CLANG_ARGS="-static" cargo build --target x86_64-unknown-linux-musl ``` failed on Alpine.

timokoesters commented

2022-01-18 22:29:05 +00:00

(Migrated from gitlab.com)

Surprise! Disabling the snappy feature breaks all servers that currently run rocksdb.

Corruption: Unsupported compression method or corrupted compressed block contents: Snappy

Apparently snappy compression will be used even though we did not enable it. We have two options now: Actually use zstd compression and tell all current rocksdb users to do a migration (complicated, you have to migrate to sqlite and then back to a different rocksdb). The second option is to just use snappy compression, which is worse at compression.

Surprise! Disabling the snappy feature breaks all servers that currently run rocksdb. > Corruption: Unsupported compression method or corrupted compressed block contents: Snappy Apparently snappy compression will be used even though we did not enable it. We have two options now: Actually use zstd compression and tell all current rocksdb users to do a migration (complicated, you have to migrate to sqlite and then back to a different rocksdb). The second option is to just use snappy compression, which is worse at compression.

jfowl commented

2022-01-18 23:14:38 +00:00

(Migrated from gitlab.com)

Happy to see the good news stacking up in this issue :D

But in all seriousness: Upgrading from v0.2.0 already was or will be a switch from sqlite to rocksdb, so I would consider it okay to switch to zstd, if it is better.

Happy to see the good news stacking up in this issue :D But in all seriousness: Upgrading from v0.2.0 already was or will be a switch from sqlite to rocksdb, so I would consider it okay to switch to zstd, if it is better.

timokoesters commented

2022-01-19 06:11:05 +00:00

(Migrated from gitlab.com)

mentioned in merge request !259