Skip to content

Add logic to track rendering area of various PDF ops #19043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 58 commits into
base: master
Choose a base branch
from

Conversation

nicolo-ribaudo
Copy link
Contributor

@nicolo-ribaudo nicolo-ribaudo commented Nov 14, 2024

I started working towards #6419. This PR introduces the logic to track where different elements of the PDF are rendered, and hooks it up to the debugger since @calixteman mentioned that it would be useful.

I'm marking this as draft because there are a few changes I need to make:

  • change the various methods in canvas.js to receive the index as a param, rather than returning a function that takes the index
  • clean up the "dependencies tracking", since currently it's all over the place. Ideally most of this logic should be self-contained in CanvasRecorder, so that when not recording it doesn't have a performance impact.
  • improve the dependency tracking (so far I'm only tracking some of them)
  • do not track extra dependencies (for example, a stroke path doesn't depend on the fill color)
  • track object dependencies
  • fix image dependencies tracking for transform (currently there is a .setTransform that makes it get lost)

However, I'd love to receive feedback on the direction.

Commit 1:

Add logic to track rendering area of various PDF ops

This commit is a first step towards #6419, and it can also help with
#13287. To support rendering part of a page, we will need to
first compute which ops can affect what is visible in that part of
the page.

This commit adds logic to track "group of ops" with their respective
bounding boxes. Each group eather corresponds to a single op or
to a range, and it can have dependencies earlier in the ops list that
are not contiguous to the range.

Consider the following example:

0. setFillRGBColor
1. beginText
2. showText "Hello"
3. endText
4. constructPath [...]
5. eoFill

here we have two groups: the text (range 1-3) and the path (range 4-5).
Each of them has a corresponding bounding box, and a dependency
on the op at index 0.

This tracking happens when first rendering a PDF: we wrap the canvas
with a "canvas recorder" that has the same API, but with additional
methods to mark the start/end of a group.

Commit 2:

Hook up the ops bbox logic to the pdf debugger

When using the pdf debugger, when hovering over a step now:

  • it highlights the steps in the same groups
  • it highlights the steps that they depend on
  • it highlights on the PDF itself the bounding box

This is an example of what the debugger integration looks like (note: I couldn't figure out how to make my cursor show up in the recording 😅 I'm moving it over the steps list):

Screen.Recording.2024-11-14.at.16.35.58.mov

By default it doesn't show all the bounding boxes because on some PDFs it's too much noise, but if you click on the checkbox then it shows the boxes and you can click on a box to scroll into view the corresponding ops.

@nicolo-ribaudo
Copy link
Contributor Author

master...nicolo-ribaudo:pdf.js:draw-page-portion-optimized is a branch merging this PR together with #19128. In the video below you can see that it first renders in the background a low-resolution image "the old way" taking 12 seconds, and then it renders the "detail view" on top taking only 1.4 seconds and only running one fifth of the PDF operations :)

Screen.Recording.2024-12-17.at.18.10.30.mp4

Still keeping this as draft because there are significant bugs (in the PDF I'm using for testing, it often skips rendering some pieces of text even if they are visible on screen, or it renders some paths with the wrong color), but it's nice to see some progress.

@bobsingor
Copy link

Very good progress on this! This is a feature that the community is waiting a long time for. Can't wait to see more progress on this.

@nicolo-ribaudo nicolo-ribaudo force-pushed the compute-bounding-boxes branch from 8184a06 to cad8d31 Compare June 1, 2025 16:43
@nicolo-ribaudo
Copy link
Contributor Author

nicolo-ribaudo commented Jun 2, 2025

Update!

  • I've reworked the dependency tracking to be based on PDF operations rather than on canvas operations. Doing it on canvas operations originally seemed cleaner, but it introduces a lot of complexity because each PDF op calls many canvas ops, and they read state from the canvas in a way that caused the tracking logic to loose information of where that state was originally coming from.
  • I've now hooked it up to the "detail view" logic, so that we record dependencies/bboxes while rendering the background page and then use that information when rendering the detail view.

This video shows how we are skipping some ops while rendering the detail view as we scroll around the page :)

Screen.Recording.2025-06-02.at.16.00.49.mov

The main missing task is that I have to properly hook this logic up to the reftests, maybe rendering a fraction of the page with the logic and checking that it matches the same fraction of the page with the unoptimized rendering. Once this is done, I can go through the failing tests one by one and add the missing tracking.

@nicolo-ribaudo nicolo-ribaudo force-pushed the compute-bounding-boxes branch 3 times, most recently from c294316 to 740b221 Compare June 9, 2025 17:36
Add missing recordFullPageBBox to CanvasNestedDependencyTracker
Mark pr8808-partial as known partial mismatch
Mark issue12295-partial as known partial mismatch
Use full page bbox for type 3 fonts with no bbox
Track bbox of paintImageMaskXObjectGroup
Mark bug1365930-partial as known partial mismatch
Use CanvasNestedDependencyTracker for type3 fonts
Track paintInlineImageXObjectGroup bbox before restoring ctx
@nicolo-ribaudo nicolo-ribaudo force-pushed the compute-bounding-boxes branch from da1864f to e938c4e Compare June 24, 2025 08:47
Mark `issue1905-partial` as a known mismatch (it's not visible, at the edge)
Track bbox in paintChar when patternFill/patternStroke
Mark issue1466-partial as known partial mismatch
Avoid multiple CanvasNestedDependencyTracker
Mark bug1898802-partial as known mismatch
Fix bbox computing of text drawn by clipping
Mark bug887152-partial as known partial mismatch
Mark issue4926-partial as known mismatch
Handle unbalanced save/restore in type3 fonts
Ensure that there is a bbox in type3 font operations
Track fill dependencies for shadingFill
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: High priority
Development

Successfully merging this pull request may close these issues.

3 participants