Skip to content

Commit d811228

Browse files
authored
use zlib-rs for gzip compression in rust code (#15417)
### What does this PR try to resolve? This PR uses `zlib-rs` via the `flate2` crate. It is used for (de)compressing gzip files (e.g. in `cargo package`). Using zlib-rs is significantly faster, and produces output of roughly similar size. For the `windows-bindgen` crate (a very large crate), the speedup is over 60%, or about 2 seconds. ``` > time cargo package -p windows-bindgen --no-verify Packaging windows-bindgen v0.61.0 (/home/folkertdev/rust/windows-rs/crates/libs/bindgen) Updating crates.io index Packaged 76 files, 31.2MiB (8.2MiB compressed) ________________________________________________________ Executed in 3.30 secs fish external usr time 3.19 secs 424.00 micros 3.19 secs sys time 0.05 secs 61.00 micros 0.05 secs > time ~/rust/cargo/target/release/cargo package -p windows-bindgen --no-verify Packaging windows-bindgen v0.61.0 (/home/folkertdev/rust/windows-rs/crates/libs/bindgen) Updating crates.io index Packaged 76 files, 31.2MiB (8.3MiB compressed) ________________________________________________________ Executed in 1.25 secs fish external usr time 1.15 secs 0.00 micros 1.15 secs sys time 0.04 secs 589.00 micros 0.04 secs ``` ### How should we test and review this PR? Generally CI/the test suite should handle correctness. Something to look out for is zlib-rs producing larger binaries than before. So far we're seeing output sizes that are roughly the same (sometimes a bit better, sometimes a bit worse) as the status quo. We've not specifically looked at decompression yet, mostly because we could not come up with a good command to benchmark. In general zlib-rs is much faster than stock zlib there too. ### Additional information For the time being, `cargo` still depends on stock zlib via e.g. `curl` and `git`. As far as I know it is the intention to eventually move away from these C dependencies, at which point the dependency on stock zlib also disappears.
2 parents db99ddc + 97f6819 commit d811228

File tree

4 files changed

+21
-6
lines changed

4 files changed

+21
-6
lines changed

Cargo.lock

+18-3
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

+1-1
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ criterion = { version = "0.5.1", features = ["html_reports"] }
4545
curl = "0.4.47"
4646
curl-sys = "0.4.79"
4747
filetime = "0.2.25"
48-
flate2 = { version = "1.0.35", default-features = false, features = ["zlib"] }
48+
flate2 = { version = "1.1.1", default-features = false, features = ["zlib-rs"] }
4949
git2 = "0.20.0"
5050
git2-curl = "0.21.0"
5151
# When updating this, also see if `gix-transport` further down needs updating or some auth-related tests will fail.

README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ The build will automatically use vendored versions of the following libraries. H
5757
* [`libcurl`](https://curl.se/libcurl/) — Used for network transfers.
5858
* [`libgit2`](https://libgit2.org/) — Used for fetching git dependencies.
5959
* [`libssh2`](https://www.libssh2.org/) — Used for SSH access to git repositories.
60-
* [`libz`](https://zlib.net/) (aka zlib) — Used for data compression.
60+
* [`libz`](https://zlib.net/) (AKA zlib) — Used by the above C libraries for data compression. (Rust code uses [`zlib-rs`](https://github.com/trifectatechfoundation/zlib-rs) instead.)
6161

6262
It is recommended to use the vendored versions as they are the versions that are tested to work with Cargo.
6363

src/doc/src/reference/features.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,7 @@ The [`default` features](#the-default-feature) can be disabled using
187187

188188
```toml
189189
[dependencies]
190-
flate2 = { version = "1.0.3", default-features = false, features = ["zlib"] }
190+
flate2 = { version = "1.0.3", default-features = false, features = ["zlib-rs"] }
191191
```
192192

193193
> **Note**: This may not ensure the default features are disabled. If another

0 commit comments

Comments
 (0)