Skip to content

Commit d550559

Browse files
author
Kevin Turner
committed
fix: improve gguf performance with torch.compile
pytorch 2.7 does not implement `set.__contains__`, so make this a list instead. See pytorch/pytorch#145761
1 parent 9170576 commit d550559

File tree

1 file changed

+2
-1
lines changed
  • invokeai/backend/quantization/gguf

1 file changed

+2
-1
lines changed

invokeai/backend/quantization/gguf/utils.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@
55
import gguf
66
import torch
77

8-
TORCH_COMPATIBLE_QTYPES = {None, gguf.GGMLQuantizationType.F32, gguf.GGMLQuantizationType.F16}
8+
# should not be a Set until this is resolved: https://github.com/pytorch/pytorch/issues/145761
9+
TORCH_COMPATIBLE_QTYPES = [None, gguf.GGMLQuantizationType.F32, gguf.GGMLQuantizationType.F16]
910

1011
# K Quants #
1112
QK_K = 256

0 commit comments

Comments
 (0)