-
Notifications
You must be signed in to change notification settings - Fork 335
Commit 7e21226
authored
s2: Add LZ4 block converter (#748)
This allows converting compressed LZ4 blocks to S2 (or snappy) blocks without decompression.
LZ4 -> S2 seems to be same size on average.
LZ4 -> Snappy is usually worse.
## Single threaded performance
Speed (excluding LZ4 encoding):
```
BenchmarkLZ4Converter_ConvertBlock/html-32 28237 42827 ns/op 2390.99 MB/s 559.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/urls-32 2138 541816 ns/op 1295.80 MB/s -3943 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/jpg-32 514826 2328 ns/op 52874.24 MB/s 482.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/jpg_200b-32 34821668 33.48 ns/op 5973.00 MB/s 2.000 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/pdf-32 198241 5975 ns/op 17136.81 MB/s 136.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/html4-32 7002 173440 ns/op 2361.63 MB/s 1840 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1-32 5940 196951 ns/op 772.22 MB/s 106.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt2-32 6656 177228 ns/op 706.32 MB/s -1427 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt3-32 2355 510435 ns/op 836.06 MB/s 384.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt4-32 1700 694444 ns/op 693.88 MB/s -9125 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/pb-32 37118 32141 ns/op 3689.60 MB/s 1.000 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/gaviota-32 6961 172253 ns/op 1070.05 MB/s 9303 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_128b-32 19923691 59.82 ns/op 2139.66 MB/s 0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_1000b-32 3180837 375.2 ns/op 2665.40 MB/s 16.00 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_10000b-32 184214 6350 ns/op 1574.70 MB/s 90.00 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_20000b-32 74031 15521 ns/op 1288.54 MB/s -5.000 b_saved 0 B/op 0 allocs/op
```
Assembly speed (amd64)
```
BenchmarkLZ4Converter_ConvertBlock/html-32 47457 24463 ns/op 4185.89 MB/s 559.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/urls-32 3506 330277 ns/op 2125.75 MB/s -3943 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/jpg-32 450177 2718 ns/op 45294.89 MB/s 482.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/jpg_200b-32 76887589 15.52 ns/op 12887.16 MB/s 2.000 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/pdf-32 279540 4322 ns/op 23694.21 MB/s 136.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/html4-32 10000 107485 ns/op 3810.75 MB/s 1840 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1-32 10000 117764 ns/op 1291.47 MB/s 106.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt2-32 10000 100578 ns/op 1244.60 MB/s -1427 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt3-32 3793 313021 ns/op 1363.34 MB/s 384.0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt4-32 2988 399888 ns/op 1204.99 MB/s -9125 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/pb-32 57486 19277 ns/op 6151.76 MB/s 1.000 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/gaviota-32 10000 115641 ns/op 1593.90 MB/s 9303 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_128b-32 38400122 31.21 ns/op 4101.10 MB/s 0 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_1000b-32 6509028 179.9 ns/op 5559.38 MB/s 16.00 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_10000b-32 368212 3244 ns/op 3082.28 MB/s 90.00 b_saved 0 B/op 0 allocs/op
BenchmarkLZ4Converter_ConvertBlock/txt1_20000b-32 141013 8303 ns/op 2408.69 MB/s -5.000 b_saved 0 B/op 0 allocs/op
```
Reference compression speed:
```
BenchmarkCompressBlockReference/html/default-32 14070 82449 ns/op 1241.98 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/urls/default-32 1215 970890 ns/op 723.14 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/jpg/default-32 193770 5904 ns/op 20849.30 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/jpg_200b/default-32 8297767 144.1 ns/op 1387.77 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/pdf/default-32 94203 12694 ns/op 8066.76 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/html4/default-32 12174 97969 ns/op 4180.90 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1/default-32 3613 333851 ns/op 455.56 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt2/default-32 4683 260579 ns/op 480.39 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt3/default-32 1268 947209 ns/op 450.54 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt4/default-32 1083 1097426 ns/op 439.08 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/pb/default-32 18357 64771 ns/op 1830.90 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/gaviota/default-32 3942 295275 ns/op 624.23 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_128b/default-32 11448295 105.7 ns/op 1210.45 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_1000b/default-32 1000000 1021 ns/op 979.26 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_10000b/default-32 116739 10114 ns/op 988.68 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_20000b/default-32 49216 23409 ns/op 854.39 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/html/better-32 6649 174667 ns/op 586.26 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/urls/better-32 627 1905706 ns/op 368.41 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/jpg/better-32 52425 22783 ns/op 5402.88 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/jpg_200b/better-32 2772865 433.3 ns/op 461.61 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/pdf/better-32 9210 127051 ns/op 805.97 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/html4/better-32 5835 201146 ns/op 2036.33 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1/better-32 2034 566702 ns/op 268.38 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt2/better-32 2386 500580 ns/op 250.07 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt3/better-32 758 1556541 ns/op 274.17 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt4/better-32 591 2013515 ns/op 239.31 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/pb/better-32 7836 155117 ns/op 764.51 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/gaviota/better-32 2473 484975 ns/op 380.06 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_128b/better-32 4322678 275.5 ns/op 464.59 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_1000b/better-32 468687 2533 ns/op 394.76 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_10000b/better-32 49606 23720 ns/op 421.59 MB/s 0 B/op 0 allocs/op
BenchmarkCompressBlockReference/txt1_20000b/better-32 14823 81300 ns/op 246.00 MB/s 0 B/op 0 allocs/op
```
Size comparisons (using Go lz4 encoder):
```
=== RUN TestLZ4Converter_ConvertBlock/html
lz4convert_test.go:42: input size: 102400
lz4convert_test.go:43: lz4 size: 21195
lz4convert_test.go:60: lz4->snappy size: 21828
lz4convert_test.go:79: lz4->s2 size: 20636
lz4convert_test.go:91: s2 (default) size: 20865
lz4convert_test.go:95: s2 (better) size: 18969
lz4convert_test.go:97: lz4 -> s2 bytes saved: 559
lz4convert_test.go:98: lz4 -> snappy bytes saved: -633
lz4convert_test.go:99: data -> s2 (default) bytes saved: 330
lz4convert_test.go:100: data -> s2 (better) bytes saved: 2226
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -229
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 1667
--- PASS: TestLZ4Converter_ConvertBlock/html (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/urls
lz4convert_test.go:42: input size: 702087
lz4convert_test.go:43: lz4 size: 292514
lz4convert_test.go:60: lz4->snappy size: 297926
lz4convert_test.go:79: lz4->s2 size: 296457
lz4convert_test.go:91: s2 (default) size: 286538
lz4convert_test.go:95: s2 (better) size: 248076
lz4convert_test.go:97: lz4 -> s2 bytes saved: -3943
lz4convert_test.go:98: lz4 -> snappy bytes saved: -5412
lz4convert_test.go:99: data -> s2 (default) bytes saved: 5976
lz4convert_test.go:100: data -> s2 (better) bytes saved: 44438
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 9919
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 48381
--- PASS: TestLZ4Converter_ConvertBlock/urls (0.01s)
=== RUN TestLZ4Converter_ConvertBlock/jpg
lz4convert_test.go:42: input size: 123093
lz4convert_test.go:43: lz4 size: 123522
lz4convert_test.go:60: lz4->snappy size: 123040
lz4convert_test.go:79: lz4->s2 size: 123040
lz4convert_test.go:91: s2 (default) size: 123097
lz4convert_test.go:95: s2 (better) size: 123097
lz4convert_test.go:97: lz4 -> s2 bytes saved: 482
lz4convert_test.go:98: lz4 -> snappy bytes saved: 482
lz4convert_test.go:99: data -> s2 (default) bytes saved: 425
lz4convert_test.go:100: data -> s2 (better) bytes saved: 425
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -57
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: -57
--- PASS: TestLZ4Converter_ConvertBlock/jpg (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/jpg_200b
lz4convert_test.go:42: input size: 200
lz4convert_test.go:43: lz4 size: 155
lz4convert_test.go:60: lz4->snappy size: 153
lz4convert_test.go:79: lz4->s2 size: 153
lz4convert_test.go:91: s2 (default) size: 153
lz4convert_test.go:95: s2 (better) size: 147
lz4convert_test.go:97: lz4 -> s2 bytes saved: 2
lz4convert_test.go:98: lz4 -> snappy bytes saved: 2
lz4convert_test.go:99: data -> s2 (default) bytes saved: 2
lz4convert_test.go:100: data -> s2 (better) bytes saved: 8
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 0
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 6
--- PASS: TestLZ4Converter_ConvertBlock/jpg_200b (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/pdf
lz4convert_test.go:42: input size: 102400
lz4convert_test.go:43: lz4 size: 83152
lz4convert_test.go:60: lz4->snappy size: 83428
lz4convert_test.go:79: lz4->s2 size: 83016
lz4convert_test.go:91: s2 (default) size: 84199
lz4convert_test.go:95: s2 (better) size: 82884
lz4convert_test.go:97: lz4 -> s2 bytes saved: 136
lz4convert_test.go:98: lz4 -> snappy bytes saved: -276
lz4convert_test.go:99: data -> s2 (default) bytes saved: -1047
lz4convert_test.go:100: data -> s2 (better) bytes saved: 268
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -1183
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 132
--- PASS: TestLZ4Converter_ConvertBlock/pdf (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/html4
lz4convert_test.go:42: input size: 409600
lz4convert_test.go:43: lz4 size: 81908
lz4convert_test.go:60: lz4->snappy size: 84886
lz4convert_test.go:79: lz4->s2 size: 80068
lz4convert_test.go:91: s2 (default) size: 20867
lz4convert_test.go:95: s2 (better) size: 18979
lz4convert_test.go:97: lz4 -> s2 bytes saved: 1840
lz4convert_test.go:98: lz4 -> snappy bytes saved: -2978
lz4convert_test.go:99: data -> s2 (default) bytes saved: 61041
lz4convert_test.go:100: data -> s2 (better) bytes saved: 62929
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 59201
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 61089
--- PASS: TestLZ4Converter_ConvertBlock/html4 (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/txt1
lz4convert_test.go:42: input size: 152089
lz4convert_test.go:43: lz4 size: 79672
lz4convert_test.go:60: lz4->snappy size: 79567
lz4convert_test.go:79: lz4->s2 size: 79566
lz4convert_test.go:91: s2 (default) size: 85931
lz4convert_test.go:95: s2 (better) size: 71608
lz4convert_test.go:97: lz4 -> s2 bytes saved: 106
lz4convert_test.go:98: lz4 -> snappy bytes saved: 105
lz4convert_test.go:99: data -> s2 (default) bytes saved: -6259
lz4convert_test.go:100: data -> s2 (better) bytes saved: 8064
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -6365
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 7958
--- PASS: TestLZ4Converter_ConvertBlock/txt1 (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/txt2
lz4convert_test.go:42: input size: 125179
lz4convert_test.go:43: lz4 size: 70801
lz4convert_test.go:60: lz4->snappy size: 72231
lz4convert_test.go:79: lz4->s2 size: 72228
lz4convert_test.go:91: s2 (default) size: 79572
lz4convert_test.go:95: s2 (better) size: 65938
lz4convert_test.go:97: lz4 -> s2 bytes saved: -1427
lz4convert_test.go:98: lz4 -> snappy bytes saved: -1430
lz4convert_test.go:99: data -> s2 (default) bytes saved: -8771
lz4convert_test.go:100: data -> s2 (better) bytes saved: 4863
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -7344
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 6290
--- PASS: TestLZ4Converter_ConvertBlock/txt2 (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/txt3
lz4convert_test.go:42: input size: 426754
lz4convert_test.go:43: lz4 size: 207038
lz4convert_test.go:60: lz4->snappy size: 206693
lz4convert_test.go:79: lz4->s2 size: 206654
lz4convert_test.go:91: s2 (default) size: 220380
lz4convert_test.go:95: s2 (better) size: 184936
lz4convert_test.go:97: lz4 -> s2 bytes saved: 384
lz4convert_test.go:98: lz4 -> snappy bytes saved: 345
lz4convert_test.go:99: data -> s2 (default) bytes saved: -13342
lz4convert_test.go:100: data -> s2 (better) bytes saved: 22102
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -13726
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 21718
--- PASS: TestLZ4Converter_ConvertBlock/txt3 (0.01s)
=== RUN TestLZ4Converter_ConvertBlock/txt4
lz4convert_test.go:42: input size: 481861
lz4convert_test.go:43: lz4 size: 277731
lz4convert_test.go:60: lz4->snappy size: 286863
lz4convert_test.go:79: lz4->s2 size: 286856
lz4convert_test.go:91: s2 (default) size: 318193
lz4convert_test.go:95: s2 (better) size: 264987
lz4convert_test.go:97: lz4 -> s2 bytes saved: -9125
lz4convert_test.go:98: lz4 -> snappy bytes saved: -9132
lz4convert_test.go:99: data -> s2 (default) bytes saved: -40462
lz4convert_test.go:100: data -> s2 (better) bytes saved: 12744
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -31337
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 21869
--- PASS: TestLZ4Converter_ConvertBlock/txt4 (0.01s)
=== RUN TestLZ4Converter_ConvertBlock/pb
lz4convert_test.go:42: input size: 118588
lz4convert_test.go:43: lz4 size: 19003
lz4convert_test.go:60: lz4->snappy size: 21130
lz4convert_test.go:79: lz4->s2 size: 19002
lz4convert_test.go:91: s2 (default) size: 18603
lz4convert_test.go:95: s2 (better) size: 17686
lz4convert_test.go:97: lz4 -> s2 bytes saved: 1
lz4convert_test.go:98: lz4 -> snappy bytes saved: -2127
lz4convert_test.go:99: data -> s2 (default) bytes saved: 400
lz4convert_test.go:100: data -> s2 (better) bytes saved: 1317
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 399
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 1316
--- PASS: TestLZ4Converter_ConvertBlock/pb (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/gaviota
lz4convert_test.go:42: input size: 184320
lz4convert_test.go:43: lz4 size: 71749
lz4convert_test.go:60: lz4->snappy size: 63392
lz4convert_test.go:79: lz4->s2 size: 62446
lz4convert_test.go:91: s2 (default) size: 65016
lz4convert_test.go:95: s2 (better) size: 55395
lz4convert_test.go:97: lz4 -> s2 bytes saved: 9303
lz4convert_test.go:98: lz4 -> snappy bytes saved: 8357
lz4convert_test.go:99: data -> s2 (default) bytes saved: 6733
lz4convert_test.go:100: data -> s2 (better) bytes saved: 16354
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -2570
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 7051
--- PASS: TestLZ4Converter_ConvertBlock/gaviota (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/txt1_128b
lz4convert_test.go:42: input size: 128
lz4convert_test.go:43: lz4 size: 84
lz4convert_test.go:60: lz4->snappy size: 84
lz4convert_test.go:79: lz4->s2 size: 84
lz4convert_test.go:91: s2 (default) size: 80
lz4convert_test.go:95: s2 (better) size: 76
lz4convert_test.go:97: lz4 -> s2 bytes saved: 0
lz4convert_test.go:98: lz4 -> snappy bytes saved: 0
lz4convert_test.go:99: data -> s2 (default) bytes saved: 4
lz4convert_test.go:100: data -> s2 (better) bytes saved: 8
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 4
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 8
--- PASS: TestLZ4Converter_ConvertBlock/txt1_128b (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/txt1_1000b
lz4convert_test.go:42: input size: 1000
lz4convert_test.go:43: lz4 size: 807
lz4convert_test.go:60: lz4->snappy size: 791
lz4convert_test.go:79: lz4->s2 size: 791
lz4convert_test.go:91: s2 (default) size: 772
lz4convert_test.go:95: s2 (better) size: 744
lz4convert_test.go:97: lz4 -> s2 bytes saved: 16
lz4convert_test.go:98: lz4 -> snappy bytes saved: 16
lz4convert_test.go:99: data -> s2 (default) bytes saved: 35
lz4convert_test.go:100: data -> s2 (better) bytes saved: 63
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 19
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 47
--- PASS: TestLZ4Converter_ConvertBlock/txt1_1000b (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/txt1_10000b
lz4convert_test.go:42: input size: 10000
lz4convert_test.go:43: lz4 size: 6969
lz4convert_test.go:60: lz4->snappy size: 6879
lz4convert_test.go:79: lz4->s2 size: 6879
lz4convert_test.go:91: s2 (default) size: 6931
lz4convert_test.go:95: s2 (better) size: 6216
lz4convert_test.go:97: lz4 -> s2 bytes saved: 90
lz4convert_test.go:98: lz4 -> snappy bytes saved: 90
lz4convert_test.go:99: data -> s2 (default) bytes saved: 38
lz4convert_test.go:100: data -> s2 (better) bytes saved: 753
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -52
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 663
--- PASS: TestLZ4Converter_ConvertBlock/txt1_10000b (0.00s)
=== RUN TestLZ4Converter_ConvertBlock/txt1_20000b
lz4convert_test.go:42: input size: 20000
lz4convert_test.go:43: lz4 size: 12750
lz4convert_test.go:60: lz4->snappy size: 12755
lz4convert_test.go:79: lz4->s2 size: 12755
lz4convert_test.go:91: s2 (default) size: 13513
lz4convert_test.go:95: s2 (better) size: 11489
lz4convert_test.go:97: lz4 -> s2 bytes saved: -5
lz4convert_test.go:98: lz4 -> snappy bytes saved: -5
lz4convert_test.go:99: data -> s2 (default) bytes saved: -763
lz4convert_test.go:100: data -> s2 (better) bytes saved: 1261
lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -758
lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 1266
```1 parent 0793ca1 commit 7e21226Copy full SHA for 7e21226
File tree
Expand file treeCollapse file tree
14 files changed
+9575
-7112
lines changedFilter options
- huff0
- internal/lz4ref
- s2
- _generate
- testdata/fuzz
Expand file treeCollapse file tree
14 files changed
+9575
-7112
lines changed+1-1Lines changed: 1 addition & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
61 | 61 |
| |
62 | 62 |
| |
63 | 63 |
| |
64 |
| - | |
| 64 | + | |
65 | 65 |
| |
66 | 66 |
| |
67 | 67 |
| |
|
internal/lz4ref/LICENSE
Copy file name to clipboard+28Lines changed: 28 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + |
0 commit comments