Skip to content

[SERVER] Support gzip encoding instead of deflate. #757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 28, 2022

Conversation

mgautierfr
Copy link
Member

The compress function is copied from httplib

Fix #753

@codecov
Copy link

codecov bot commented Apr 22, 2022

Codecov Report

Merging #757 (c52a903) into master (dc42f83) will increase coverage by 0.15%.
The diff coverage is 100.00%.

❗ Current head c52a903 differs from pull request most recent head fba0f09. Consider uploading reports for the commit fba0f09 to get more accurate results

@@            Coverage Diff             @@
##           master     #757      +/-   ##
==========================================
+ Coverage   61.09%   61.25%   +0.15%     
==========================================
  Files          58       58              
  Lines        3789     3802      +13     
  Branches     2080     2084       +4     
==========================================
+ Hits         2315     2329      +14     
+ Misses       1473     1472       -1     
  Partials        1        1              
Impacted Files Coverage Δ
src/server/internalServer.cpp 82.54% <ø> (ø)
src/server/request_context.cpp 86.73% <100.00%> (ø)
src/server/request_context.h 100.00% <100.00%> (ø)
src/server/response.cpp 92.11% <100.00%> (+0.88%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc42f83...fba0f09. Read the comment docs.

Copy link
Collaborator

@veloman-yunkan veloman-yunkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since our unit tests don't check the correctness of compressed content, did you verify that the code actually works with the browsers?


#define KIWIX_MIN_CONTENT_SIZE_TO_DEFLATE 100

#define KIWIX_MIN_CONTENT_SIZE_TO_COMPRESS 100
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this threshold is too low - why trying to compress payload that is smaller than the header section?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably yes. It is a pretty old magic value and it seems it has been introduce as some kind of workaround for a segfault kiwix/kiwix-tools@4c79cfc

The question is what would be the good value ? I've seen different values but it seems that MTU is a good limit (https://www.computerworld.com/article/2693941/why-it-doesn-t-make-sense-to-gzip-all-content-from-your-web-server.html). So we can move this to 1400 Bytes.

Copy link
Collaborator

@veloman-yunkan veloman-yunkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with the code but I didn't build and manually test it (since the unit tests don't exhaustively validate this change). Before merging, this functionality must be tested via kiwix-serve and the browser development console (in order to verify that kiwix-serve sends gzip-compressed content and the browser is able to decompress it).

@mgautierfr
Copy link
Member Author

Before merging, this functionality must be tested via kiwix-serve and the browser development console (in order to verify that kiwix-serve sends gzip-compressed content and the browser is able to decompress it).

Done

Capture d’écran du 2022-04-27 11-40-15

Everything works correctly, as far as I can see on my browser

@veloman-yunkan
Copy link
Collaborator

Before merging, this functionality must be tested via kiwix-serve and the browser development console (in order to verify that kiwix-serve sends gzip-compressed content and the browser is able to decompress it).

Done

Capture d’écran du 2022-04-27 11-40-15

Everything works correctly, as far as I can see on my browser

Perfect! Why don't you merge this PR then?

@mgautierfr mgautierfr merged commit f90cc39 into master Apr 28, 2022
@mgautierfr mgautierfr deleted the gzip_compression branch April 28, 2022 12:36
@mgautierfr
Copy link
Member Author

Perfect! Why don't you merge this PR then?

Because I was waiting for CI to pass after my rebase-fixup and switch to something else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[URGENT] Use gzip instead of deflate
2 participants