Skip to content

feat(ui): canvas auto-mask #7163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 53 commits into from
Oct 23, 2024
Merged

Conversation

psychedelicious
Copy link
Collaborator

@psychedelicious psychedelicious commented Oct 22, 2024

Summary

  • Add auto-mask (via Segment Anything v1) functionality to canvas. Users can add foreground, background and neutral points to mask images. Raster and Control layers are supported.
  • Add node to apply a tensor mask to an image.
  • Update segment anything node and pipeline to support input points (previously only supported bounding boxes).
  • The CanvasSegmentAnythingModule class is similar to CanvasEntityFilterer, but implemented separately to allow for interactions on the canvas, which would have been tricky to add to CanvasEntityFilterer. In the future, the SAM module could be used as a blueprint for filters that need canvas interactions.
  • To support the new canvas module, the handling of canvas tools was reworked so that each tool manages its own interaction events. This change comprises a good chunk of this PR.
  • Consolidate isolatedFilteringPreview and isolatedTransformingPreview settings into a single isolatedLayerPreview. It is now used for auto-masking, filtering and transforming.
  • Rename autoProcessFilter to autoProcess. It is now used for auto-masking and filtering.
  • Updated quite a few docstrings and canvas debug/repr outputs.
  • Swapped out unmaintained dev dependency ts-toolbelt for type-fest. Was gonna use a util type from it but eventually figured a way to not need the type. Oh well, good to update the dependency anyways.

Related Issues / Discussions

QA Instructions

YOLO

Merge Plan

n/a

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)

@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files frontend-deps PRs that change frontend dependencies frontend PRs that change frontend files labels Oct 22, 2024
`ts-toolbelt` is unmaintained while `type-fest` is very actively maintained. Both provide similar TS utilities.
Move brush tool event logic to its class.
It's essentially a noop but I think it makes sense to have a module for each tool...
@psychedelicious psychedelicious force-pushed the psyche/feat/ui/canvas-sam-mask branch from 15a06ec to f73aaf5 Compare October 23, 2024 03:49
It's used for more than filters now.
@psychedelicious psychedelicious merged commit fdbc244 into main Oct 23, 2024
14 checks passed
@psychedelicious psychedelicious deleted the psyche/feat/ui/canvas-sam-mask branch October 23, 2024 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend PRs that change backend files frontend PRs that change frontend files frontend-deps PRs that change frontend dependencies invocations PRs that change invocations python PRs that change python files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[enhancement]: Implement Auto Masking feature in Canvas.
2 participants