VK_NV_cooperative_matrix2

[Mushoz]

What subset of the VK_NV_cooperative_matrix2 extension does llama.cpp use for the vulkan backend? The CooperativeMatrixConversionsNV functionality support has recently landed in Mesa (See: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34793 ) And I was wondering if that is sufficient for llama.cpp to run the more optimized coopmat2 codepath and flash attention.

[0cc4m]

No, that is not enough. My guess is that we have to wait for Khronos to create a generic VK_KHR_cooperative_matrix2 extension based on the Nvidia one before it gets implemented in a usable way on AMD/Intel.