Open
Description
Name and Version
b5602
Operating systems
Mac
GGML backends
Metal
Hardware
iPad M1, M2
Models
qwen2.5 1.5b, seems other model are has same problem
Problem description & steps to reproduce
run the project in the read iPad device
First Bad Commit
metal : use F32 attention accumulators in FA kernels
Relevant log output
llama_model_load_from_file_impl: using device Metal (Apple M1 GPU) - 54 MiB free
llama_model_loader: loaded meta data with 26 key-value pairs and 339 tensors from /private/var/containers/Bundle/Application/F38DADD0-126F-4CA0-9B82-23B7EA53CF74/qwen2_5.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = qwen2.5-1.5b-instruct
llama_model_loader: - kv 3: general.version str = v0.1
llama_model_loader: - kv 4: general.finetune str = qwen2.5-1.5b-instruct
llama_model_loader: - kv 5: general.size_label str = 1.8B
llama_model_loader: - kv 6: qwen2.block_count u32 = 28
llama_model_loader: - kv 7: qwen2.context_length u32 = 32768
llama_model_loader: - kv 8: qwen2.embedding_length u32 = 1536
llama_model_loader: - kv 9: qwen2.feed_forward_length u32 = 8960
llama_model_loader: - kv 10: qwen2.attention.head_count u32 = 12
llama_model_loader: - kv 11: qwen2.attention.head_count_kv u32 = 2
llama_model_loader: - kv 12: qwen2.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 13: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 14: general.file_type u32 = 7
llama_model_loader: - kv 15: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 16: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 17: tokenizer.ggml.tokens arr[str,151936] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 18: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 19: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 21: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 22: tokenizer.ggml.bos_token_id u32 = 151643
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
llama_model_loader: - kv 25: general.quantization_version u32 = 2
llama_model_loader: - type f32: 141 tensors
llama_model_loader: - type q8_0: 198 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q8_0
print_info: file size = 1.76 GiB (8.50 BPW)
init_tokenizer: initializing tokenizer for type 2
load: control token: 151659 '<|fim_prefix|>' is not marked as EOG
load: control token: 151656 '<|video_pad|>' is not marked as EOG
load: control token: 151655 '<|image_pad|>' is not marked as EOG
load: control token: 151653 '<|vision_end|>' is not marked as EOG
load: control token: 151652 '<|vision_start|>' is not marked as EOG
load: control token: 151651 '<|quad_end|>' is not marked as EOG
load: control token: 151649 '<|box_end|>' is not marked as EOG
load: control token: 151648 '<|box_start|>' is not marked as EOG
load: control token: 151646 '<|object_ref_start|>' is not marked as EOG
load: control token: 151644 '<|im_start|>' is not marked as EOG
load: control token: 151661 '<|fim_suffix|>' is not marked as EOG
load: control token: 151647 '<|object_ref_end|>' is not marked as EOG
load: control token: 151660 '<|fim_middle|>' is not marked as EOG
load: control token: 151654 '<|vision_pad|>' is not marked as EOG
load: control token: 151650 '<|quad_start|>' is not marked as EOG
load: special tokens cache size = 22
load: token to piece cache size = 0.9310 MB
print_info: arch = qwen2
print_info: vocab_only = 0
print_info: n_ctx_train = 32768
print_info: n_embd = 1536
print_info: n_layer = 28
print_info: n_head = 12
print_info: n_head_kv = 2
print_info: n_rot = 128
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 6
print_info: n_embd_k_gqa = 256
print_info: n_embd_v_gqa = 256
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 8960
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: causal attn = 1
print_info: pooling type = -1
print_info: rope type = 2
print_info: rope scaling = linear
print_info: freq_base_train = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 32768
print_info: rope_finetuned = unknown
print_info: ssm_d_conv = 0
print_info: ssm_d_inner = 0
print_info: ssm_d_state = 0
print_info: ssm_dt_rank = 0
print_info: ssm_dt_b_c_rms = 0
print_info: model type = 1.5B
print_info: model params = 1.78 B
print_info: general.name = qwen2.5-1.5b-instruct
print_info: vocab type = BPE
print_info: n_vocab = 151936
print_info: n_merges = 151387
print_info: BOS token = 151643 '<|endoftext|>'
print_info: EOS token = 151645 '<|im_end|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: PAD token = 151643 '<|endoftext|>'
print_info: LF token = 198 'Ċ'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: layer 0 assigned to device Metal, is_swa = 0
load_tensors: layer 1 assigned to device Metal, is_swa = 0
load_tensors: layer 2 assigned to device Metal, is_swa = 0
load_tensors: layer 3 assigned to device Metal, is_swa = 0
load_tensors: layer 4 assigned to device Metal, is_swa = 0
load_tensors: layer 5 assigned to device Metal, is_swa = 0
load_tensors: layer 6 assigned to device Metal, is_swa = 0
load_tensors: layer 7 assigned to device Metal, is_swa = 0
load_tensors: layer 8 assigned to device Metal, is_swa = 0
load_tensors: layer 9 assigned to device Metal, is_swa = 0
load_tensors: layer 10 assigned to device Metal, is_swa = 0
load_tensors: layer 11 assigned to device Metal, is_swa = 0
load_tensors: layer 12 assigned to device Metal, is_swa = 0
load_tensors: layer 13 assigned to device Metal, is_swa = 0
load_tensors: layer 14 assigned to device Metal, is_swa = 0
load_tensors: layer 15 assigned to device Metal, is_swa = 0
load_tensors: layer 16 assigned to device Metal, is_swa = 0
load_tensors: layer 17 assigned to device Metal, is_swa = 0
load_tensors: layer 18 assigned to device Metal, is_swa = 0
load_tensors: layer 19 assigned to device Metal, is_swa = 0
load_tensors: layer 20 assigned to device Metal, is_swa = 0
load_tensors: layer 21 assigned to device Metal, is_swa = 0
load_tensors: layer 22 assigned to device Metal, is_swa = 0
load_tensors: layer 23 assigned to device Metal, is_swa = 0
load_tensors: layer 24 assigned to device Metal, is_swa = 0
load_tensors: layer 25 assigned to device Metal, is_swa = 0
load_tensors: layer 26 assigned to device Metal, is_swa = 0
load_tensors: layer 27 assigned to device Metal, is_swa = 0
load_tensors: layer 28 assigned to device Metal, is_swa = 0
load_tensors: tensor 'token_embd.weight' (q8_0) (and 0 others) cannot be used with preferred buffer type CPU_AARCH64, using CPU instead
ggml_backend_metal_log_allocated_size: allocated buffer, size = 1801.11 MiB, ( 7207.81 / 5461.34)
ggml_backend_metal_log_allocated_size: warning: current allocated size is greater than the recommended max working set size
load_tensors: offloading 28 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 29/29 layers to GPU
load_tensors: CPU_Mapped model buffer size = 236.47 MiB
load_tensors: Metal_Mapped model buffer size = 1801.09 MiB
Using 6 threads
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 5120
llama_context: n_ctx_per_seq = 5120
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = 0
llama_context: freq_base = 1000000.0
llama_context: freq_scale = 1
llama_context: n_ctx_per_seq (5120) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
ggml_metal_init: allocating
ggml_metal_init: picking default device: Apple M1 GPU
ggml_metal_init: GPU name: Apple M1 GPU
ggml_metal_init: GPU family: MTLGPUFamilyApple7 (1007)
ggml_metal_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_init: GPU family: MTLGPUFamilyMetal3 (5001)
ggml_metal_init: simdgroup reduction = true
ggml_metal_init: simdgroup matrix mul. = true
ggml_metal_init: has residency sets = false
ggml_metal_init: has bfloat = true
ggml_metal_init: use bfloat = true
ggml_metal_init: hasUnifiedMemory = true
ggml_metal_init: recommendedMaxWorkingSetSize = 5726.63 MB
ggml_metal_init: loaded kernel_add 0x12cc58120 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_add_row 0x12cc58b40 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_sub 0x12cc59500 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_sub_row 0x12cc59ec0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul 0x12cc5a880 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_row 0x12cc5b240 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_div 0x12cc641e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_div_row 0x12cc642a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_repeat_f32 0x12cc64c60 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_repeat_f16 0x12cc652c0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_repeat_i32 0x12cc65920 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_repeat_i16 0x12cc65f80 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_scale 0x12cc665e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_scale_4 0x12cc66640 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_clamp 0x12cc666a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_tanh 0x12cc66700 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_relu 0x12cc66760 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_sigmoid 0x12cc667c0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_gelu 0x12cc66820 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_gelu_4 0x12cc66880 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_gelu_erf 0x12cc668e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_gelu_erf_4 0x12cc66940 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_gelu_quick 0x12cc669a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_gelu_quick_4 0x12cc66a00 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_silu 0x12cc66a60 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_silu_4 0x12cc66ac0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_elu 0x12cc66b20 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_soft_max_f16 0x12cc66b80 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_soft_max_f16_4 0x12cc66ee0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_soft_max_f32 0x12cc67240 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_soft_max_f32_4 0x12cc675a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_diag_mask_inf 0x12cc67900 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_diag_mask_inf_8 0x12cc67a80 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_f32 0x12cc67c00 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_f16 0x12cca8240 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_bf16 0x12cca8300 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q4_0 0x12cca8660 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q4_1 0x12cca89c0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q5_0 0x12cca8d20 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q5_1 0x12cca9080 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q8_0 0x12cca93e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q2_K 0x12cca9740 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q3_K 0x12cca9aa0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q4_K 0x12cca9e00 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q5_K 0x12ccaa160 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_q6_K 0x12ccaa4c0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq2_xxs 0x12ccaa820 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq2_xs 0x12ccaab80 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq3_xxs 0x12ccaaee0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq3_s 0x12ccab240 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq2_s 0x12ccab5a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq1_s 0x12ccab900 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq1_m 0x12ccabc60 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq4_nl 0x12cd042a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_iq4_xs 0x12cd04360 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_get_rows_i32 0x12cd046c0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rms_norm 0x12cd04a20 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_l2_norm 0x12cd04c00 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_group_norm 0x12cd04de0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_norm 0x12cd05140 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_ssm_conv_f32 0x12cd05320 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_ssm_scan_f32 0x12cd05980 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rwkv_wkv6_f32 0x12cd06220 | th_max = 384 | th_width = 32
ggml_metal_init: loaded kernel_rwkv_wkv7_f32 0x12cd06280 | th_max = 448 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_f32_f32 0x12cd062e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_bf16_f32 0x12cd06a00 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_bf16_f32_1row 0x12cd07120 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_bf16_f32_l4 0x12cd07840 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_bf16_bf16 0x12cd2c600 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_f16_f32 0x12cd2c6c0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_f16_f32_1row 0x12cd2cde0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_f16_f32_l4 0x12cd2d500 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_f16_f16 0x12cd2dc20 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q4_0_f32 0x12cd2e340 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q4_1_f32 0x12cd2ea60 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q5_0_f32 0x12cd2f180 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q5_1_f32 0x12cd2f8a0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q8_0_f32 0x12cd48660 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_f16_f32_r1_2 0x12cd48720 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_f16_f32_r1_3 0x12cd48f60 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_f16_f32_r1_4 0x12cd497a0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_f16_f32_r1_5 0x12cd49fe0 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_0_f32_r1_2 0x12cd4a820 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_0_f32_r1_3 0x12cd4b060 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_0_f32_r1_4 0x12cd54060 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_0_f32_r1_5 0x12cd54120 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_1_f32_r1_2 0x12cd54960 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_1_f32_r1_3 0x12cd551a0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_1_f32_r1_4 0x12cd559e0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_1_f32_r1_5 0x12cd56220 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_0_f32_r1_2 0x12cd56a60 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_0_f32_r1_3 0x12cd572a0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_0_f32_r1_4 0x12cd642a0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_0_f32_r1_5 0x12cd64360 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_1_f32_r1_2 0x12cd64ba0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_1_f32_r1_3 0x12cd653e0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_1_f32_r1_4 0x12cd65c20 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_1_f32_r1_5 0x12cd66460 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q8_0_f32_r1_2 0x12cd66ca0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q8_0_f32_r1_3 0x12cd674e0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q8_0_f32_r1_4 0x12cd744e0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q8_0_f32_r1_5 0x12cd745a0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_K_f32_r1_2 0x12cd74de0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_K_f32_r1_3 0x12cd75620 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_K_f32_r1_4 0x12cd75e60 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q4_K_f32_r1_5 0x12cd766a0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_K_f32_r1_2 0x12cd76ee0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_K_f32_r1_3 0x12cd77720 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_K_f32_r1_4 0x12cd84720 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q5_K_f32_r1_5 0x12cd847e0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q6_K_f32_r1_2 0x12cd85020 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q6_K_f32_r1_3 0x12cd85860 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q6_K_f32_r1_4 0x12cd860a0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_q6_K_f32_r1_5 0x12cd868e0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_iq4_nl_f32_r1_2 0x12cd87120 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_iq4_nl_f32_r1_3 0x12cd90120 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_iq4_nl_f32_r1_4 0x12cd901e0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_ext_iq4_nl_f32_r1_5 0x12cd90a20 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q2_K_f32 0x12cd91260 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q3_K_f32 0x12cd91980 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q4_K_f32 0x12cd920a0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q5_K_f32 0x12cd927c0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_q6_K_f32 0x12cd92ee0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq2_xxs_f32 0x12cd93600 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq2_xs_f32 0x12cdb83c0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq3_xxs_f32 0x12cdb8480 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq3_s_f32 0x12cdb8ba0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq2_s_f32 0x12cdb92c0 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq1_s_f32 0x12cdb99e0 | th_max = 448 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq1_m_f32 0x12cdba100 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq4_nl_f32 0x12cdba820 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_iq4_xs_f32 0x12cdbaf40 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_f32_f32 0x12cdbb660 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_f16_f32 0x12cdec4e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_bf16_f32 0x12cdec5a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q4_0_f32 0x12cdecd20 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q4_1_f32 0x12cded4a0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q5_0_f32 0x12cdedc20 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q5_1_f32 0x12cdee3a0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q8_0_f32 0x12cdeeb20 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q2_K_f32 0x12cdef2a0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q3_K_f32 0x12ce04120 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q4_K_f32 0x12ce041e0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q5_K_f32 0x12ce04960 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_q6_K_f32 0x12ce050e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq2_xxs_f32 0x12ce05860 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq2_xs_f32 0x12ce05fe0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq3_xxs_f32 0x12ce06760 | th_max = 704 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq3_s_f32 0x12ce06ee0 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq2_s_f32 0x12ce07660 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq1_s_f32 0x12ce344e0 | th_max = 448 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq1_m_f32 0x12ce345a0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq4_nl_f32 0x12ce34d20 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mv_id_iq4_xs_f32 0x12ce354a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_f32_f32 0x12ce35c20 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_f16_f32 0x12ce361c0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_bf16_f32 0x12ce36760 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q4_0_f32 0x12ce36d00 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q4_1_f32 0x12ce372a0 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q5_0_f32 0x12ce37840 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q5_1_f32 0x12ce64300 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q8_0_f32 0x12ce643c0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q2_K_f32 0x12ce64960 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q3_K_f32 0x12ce64f00 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q4_K_f32 0x12ce654a0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q5_K_f32 0x12ce65a40 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_q6_K_f32 0x12ce65fe0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq2_xxs_f32 0x12ce66580 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq2_xs_f32 0x12ce66b20 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq3_xxs_f32 0x12ce670c0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq3_s_f32 0x12ce67660 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq2_s_f32 0x12ceb8120 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq1_s_f32 0x12ceb81e0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq1_m_f32 0x12ceb8780 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq4_nl_f32 0x12ceb8d20 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_iq4_xs_f32 0x12ceb92c0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_map0_f16 0x12ceb9860 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_map1_f32 0x12ceb9bc0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_f32_f16 0x12ceb9f20 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_f16_f16 0x12ceba4c0 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_bf16_f16 0x12cebaa60 | th_max = 768 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q4_0_f16 0x12cebb000 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q4_1_f16 0x12cebb5a0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q5_0_f16 0x12cef4060 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q5_1_f16 0x12cef4120 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q8_0_f16 0x12cef46c0 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q2_K_f16 0x12cef4c60 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q3_K_f16 0x12cef5200 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q4_K_f16 0x12cef57a0 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q5_K_f16 0x12cef5d40 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_q6_K_f16 0x12cef62e0 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq2_xxs_f16 0x12cef6880 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq2_xs_f16 0x12cef6e20 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq3_xxs_f16 0x12cef73c0 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq3_s_f16 0x12cef7960 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq2_s_f16 0x12cf34420 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq1_s_f16 0x12cf344e0 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq1_m_f16 0x12cf34a80 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq4_nl_f16 0x12cf35020 | th_max = 896 | th_width = 32
ggml_metal_init: loaded kernel_mul_mm_id_iq4_xs_f16 0x12cf355c0 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_rope_norm_f32 0x12cf35b60 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rope_norm_f16 0x12cf366a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rope_multi_f32 0x12cf371e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rope_multi_f16 0x12cf547e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rope_vision_f32 0x12cf548a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rope_vision_f16 0x12cf553e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rope_neox_f32 0x12cf55f20 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_rope_neox_f16 0x12cf56a60 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_im2col_f16 0x12cf575a0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_im2col_f32 0x12cf74120 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_im2col_ext_f16 0x12cf741e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_im2col_ext_f32 0x12cf747e0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_conv_transpose_1d_f32_f32 0x12cf74de0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_conv_transpose_1d_f16_f32 0x12cf75080 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_upscale_f32 0x12cf75320 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_pad_f32 0x12cf75b00 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_pad_reflect_1d_f32 0x12cf76160 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_timestep_embedding_f32 0x12cf76880 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_arange_f32 0x12cf76a00 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_argsort_f32_i32_asc 0x12cf76b80 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_argsort_f32_i32_desc 0x12cf76ca0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_leaky_relu_f32 0x12cf76dc0 | th_max = 1024 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_h64 0x12cf76e80 | th_max = 640 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_h80 0x12cfb80c0 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_h96 0x12cfb8180 | th_max = 576 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_h112 0x12cfb8ae0 | th_max = 512 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_h128 0x12cfb9440 | th_max = 512 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_h192 0x12cfb9da0 | th_max = 448 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_hk192_hv128 0x12cfba700 | th_max = 512 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_h256 0x12cfbb060 | th_max = 832 | th_width = 32
ggml_metal_init: loaded kernel_flash_attn_ext_f16_hk576_hv512 0x0 | th_max = 0 | th_width = 0
ggml_metal_init: error: load pipeline error: Error Domain=AGXMetal13_3 Code=3 "Compute function exceeds available stack space" UserInfo={NSLocalizedDescription=Compute function exceeds available stack space}
ggml_backend_metal_device_init: error: failed to allocate context
llama_init_from_model: failed to initialize the context: failed to initialize Metal backend