How do I turn on the geometry shader?

[tmonk22]

_traits = vsg::WindowTraits::create();
_traits->deviceFeatures->get().geometryShader = VK_TRUE;

I set an empty geometry shader and crash when call viewer->compile

[robertosfield]

When you say empty geometry shader it's too open ended to know exactly what you've done, but clearly something in setting up the shader modules associated with the shaders. In general if you don't need to use a shader stage you simply don't assign it.

Often it's useful to see correct code to compare against. A quick search of vsgExamples shoes the vsgcompilemanager test enabled geometryShader:

grep -r geometryShader .
./tests/vsgcompilemanager/vsgcompilemanager.cpp:    requestFeatures->get().geometryShader = VK_TRUE; // for gl_PrimitiveID

I didn't write this test so can't say exactly what it does off the top of my head. The test runs just fine on my Kubuntu 24.04 + Geforce 2060.

[tmonk22]

vert shader

#version 450
#extension GL_ARB_separate_shader_objects : enable
layout(push_constant) uniform PushConstants { mat4 projection; mat4 modelView; };
layout(location = 0) in vec3 vertex;
out gl_PerVertex { vec4 gl_Position; };
void main() {
gl_Position = (projection * modelView) * vec4(vertex, 1.0);
}

geom shader
#version 450
layout(points) in;
layout(triangle_strip, max_vertices = 4) out;
//layout(location = 0) out vec2 outTexCoord;

void main() {
    vec4 center = gl_in[0].gl_Position;
    float size = 0.01;
    vec4 positions[4] = vec4[](
        center + vec4(-size, -size, 0.0, 0.0), 
        center + vec4( size, -size, 0.0, 0.0), 
        center + vec4(-size,  size, 0.0, 0.0), 
        center + vec4( size,  size, 0.0, 0.0)  
    );
    vec2 texCoords[4] = vec2[](
        vec2(0, 0),
        vec2(1, 0),
        vec2(0, 1),
        vec2(1, 1)
    );
    for (int i = 0; i < 4; i++) {
        gl_Position = positions[i];
        //outTexCoord = texCoords[i];
        EmitVertex();
    }
    EndPrimitive();
}

frag shader
#version 450

//layout(location = 0) in vec2 inTexCoord;
layout(location = 0) out vec4 outColor;

void main() {
    outColor = vec4(1,0,0,1);
  return ;
}

_traits = vsg::WindowTraits::create();
auto requestFeatures = _traits->deviceFeatures =
vsg::DeviceFeatures::create();
requestFeatures->get().geometryShader = VK_TRUE; // for gl_Primitiv

auto vertexShader =
    vsg::read_cast<vsg::ShaderStage>(vertexShaderFilename, _options);
auto fragmentShader =
    vsg::read_cast<vsg::ShaderStage>(fragmentShaderFilename, _options);
auto geometryShader =
    vsg::read_cast<vsg::ShaderStage>(geometryShaderFilename, _options);

auto shaderSet = vsg::ShaderSet::create(
    vsg::ShaderStages{vertexShader, fragmentShader, geometryShader});
shaderSet->addPushConstantRange("pc", "", VK_SHADER_STAGE_VERTEX_BIT, 0,
                                128);
shaderSet->addAttributeBinding("vertex", "", 0, VK_FORMAT_R32G32B32_SFLOAT,
                               vsg::vec3Array::create(1));
_pointPipelineConfigurator =
    vsg::GraphicsPipelineConfigurator::create(shaderSet);
struct SetPipelineStates : public vsg::Visitor {
  void apply(vsg::Object& object) override { object.traverse(*this); }

  void apply(vsg::InputAssemblyState& ias) override {
    ias.topology = VK_PRIMITIVE_TOPOLOGY_POINT_LIST;
  }
};
vsg::visit<SetPipelineStates>(_pointPipelineConfigurator);

_pointPipelineConfigurator->enableArray(
    "vertex", VK_VERTEX_INPUT_RATE_VERTEX, 12, VK_FORMAT_R32G32B32_SFLOAT);

_pointPipelineConfigurator->init();

That's how I use it, and he's interrupted in the same place,the system is windows

[AnyOldName3]

0xC0000005 is an access violation, the Windows equivalent of a segfault, and 0x0000000000000000 is mentioned, which would be a 64-bit null pointer, something definitely unsafe to dereference. It's a safe bet that something somewhere (likely one of the values in the VkShaderModuleCreateInfo instance) is a nullptr when it's not legal for it to be.

[tmonk22]

I used it the way described above, and if I remove the geometry shader, everything works fine

[robertosfield]

Could you check whether your VSG version has shader compilation support enabled? This requires the VSG to be compiled with compile with glslang.

You can check whether shader compilation is supported via the CMake var VSG_SUPPORTS_ShaderCompiler or the include/vsg/core/Version.h VSG_SUPPORTS_ShaderCompiler define.

[tmonk22]

It should be supported

[robertosfield]

I don't have any other ideas what is going wrong on your system at this point, so it'll have to be a case of just turning over stone by stone till you find what is wrong.

Broadly it looks to be an issue with getting all the required shader modules compiled from source into SPIRV and then passed on to the creation of the VkShaderModuleCreateInfo, so putting debugging code or breakpoints into the src/vsg/state/ShaderModule.cpp and src/vsg/utils/ShaderCompiler.cpp might help illuminate when the null is being introduced.

The vsg::ShaderCompiler also has a debug messages that you will not see at level notification level by if you up the notification level to DEBUG you'll see them:

vsg::Logger::instance()->level = vsg::Logger::LOGGER_DEBUG;

This might illuminate what might be going on.

Helping debug this type of thing remote is really hard, what would help others like myself would be an example program that reproduces the problem. What can work well is programmers creating a small example program that can go in vsgExamples that doesn't yet work on their system, share that then others can figure out what is wrong, fix it, then the whole community will have an example program that illustrates a feature and it can also be used as a form of unit test to make sure that feature still works as intended.

[tmonk22]

After confirming that it should be a shader issue, an extension needs to be added, and at the same time it needs to ensure that the output of the vertex shader is consistent with the geometry shader, this is a test code for the geometry shader, I don't know the specific process of adding it to vsgExample, I posted the code.

#include <vsg/all.h>
#include <iostream>

const char* vertexShaderSource = R"(
#version 450
#extension GL_ARB_separate_shader_objects : enable
layout(push_constant) uniform PushConstants { mat4 projection; mat4 modelView; };
layout(location = 0) in vec3 vertex;

void main() {
gl_Position =  vec4(vertex.xy,1, 1.0);
}
)";

const char* geomShaderSource = R"(
#version 450
#extension GL_ARB_separate_shader_objects : enable
layout(points) in;
layout(triangle_strip, max_vertices = 4) out;
void main() {
    vec4 center = gl_in[0].gl_Position;
    float size = 0.1;

    vec2 offsets[4] = vec2[](
     vec2( 1.0, -1.0),
     vec2(-1.0, -1.0),  
     vec2( 1.0,  1.0),
     vec2(-1.0,  1.0)
    );

    for (int i = 0; i < 4; ++i) {
        gl_Position =vec4(offsets[i], 1, 1);
        gl_PointSize = 1.0;
        EmitVertex();
    }

    EndPrimitive();
}
)";

const char* fragmentShaderSource = R"(
#version 450
layout(location = 0) out vec4 outColor;

void main() {
    outColor = vec4(1.0,0,0, 1.0);
}

)";


int main(int argc, char** argv) {
  try {
    auto options = vsg::Options::create();
    options->paths.push_back("../data");

    // 创建窗口
    auto windowTraits = vsg::WindowTraits::create();
    //auto features = windowTraits->deviceFeatures =
    //    vsg::DeviceFeatures::create();
    //features->get().geometryShader = VK_TRUE;
    windowTraits->windowTitle = "VSG Point Sprite Quad";
    auto windows = vsg::Window::create(windowTraits);

    // 顶点数据（一个点）
    auto stateGroup = vsg::StateGroup::create();
    {
      auto vertexShader = vsg::ShaderStage::create(VK_SHADER_STAGE_VERTEX_BIT,
                                                   "main",
                               vertexShaderSource);
      auto fragmentShader = vsg::ShaderStage::create(VK_SHADER_STAGE_FRAGMENT_BIT,
                                                     "main",
                               fragmentShaderSource);
      auto geometryShader = vsg::ShaderStage::create(VK_SHADER_STAGE_GEOMETRY_BIT,
                                                     "main", geomShaderSource);

      auto shaderSet = vsg::ShaderSet::create(
          vsg::ShaderStages{vertexShader, fragmentShader, geometryShader});
      shaderSet->addPushConstantRange("pc", "", VK_SHADER_STAGE_VERTEX_BIT, 0,
                                      128);
      shaderSet->addAttributeBinding("vertex", "", 0,
                                     VK_FORMAT_R32G32B32_SFLOAT,
                                     vsg::vec3Array::create(1));

      auto pointPipelineConfigurator =
          vsg::GraphicsPipelineConfigurator::create(shaderSet);
      struct SetPipelineStates : public vsg::Visitor {
        void apply(vsg::Object& object) override { object.traverse(*this); }

        void apply(vsg::InputAssemblyState& ias) override {
          ias.topology = VK_PRIMITIVE_TOPOLOGY_POINT_LIST;
        }
      };
      vsg::visit<SetPipelineStates>(pointPipelineConfigurator);

      pointPipelineConfigurator->enableArray("vertex",
                                             VK_VERTEX_INPUT_RATE_VERTEX, 12,
                                             VK_FORMAT_R32G32B32_SFLOAT);
      pointPipelineConfigurator->init();
      pointPipelineConfigurator->copyTo(stateGroup);
    }

    vsg::ref_ptr<vsg::vec3Array> vertices =
        vsg::vec3Array::create({{0.0f, 0.0f, 0.0f}});
    auto vertexData = vsg::DataList{vertices};
    auto vertexDraw = vsg::VertexDraw::create();

    vertexDraw->assignArrays(vertexData);
    vertexDraw->vertexCount = vertices->size();
    vertexDraw->instanceCount = 1;
    stateGroup->addChild(vertexDraw);

    auto sceneGraph = vsg::Group::create();
    sceneGraph->addChild(stateGroup);

    auto lookAt = vsg::LookAt::create(vsg::dvec3{150, 175, 200},
                                      vsg::dvec3{0, 0, 0}, vsg::dvec3{0, 0, 1});
    auto perspective = vsg::Perspective::create();
    auto viewportState = vsg::ViewportState::create(windows->extent2D());
    auto camera = vsg::Camera::create(perspective, lookAt, viewportState);

    /// Viewer and CommandGraph
    auto viewer = vsg::Viewer::create();
    viewer->addWindow(windows);
    viewer->addEventHandler(vsg::CloseHandler::create(viewer));
    viewer->addEventHandler(vsg::Trackball::create(camera));

    auto commandGraph =
        vsg::createCommandGraphForView(windows, camera, sceneGraph);
    viewer->addRecordAndSubmitTaskAndPresentation({commandGraph});
    viewer->compile();

    while (viewer->advanceToNextFrame()) {
      viewer->handleEvents();
      viewer->update();
      viewer->recordAndSubmit();
      viewer->present();
    }
  } catch (const std::exception& ex) {
    std::cerr << ex.what() << std::endl;
  }

  return 0;
}

[AnyOldName3]

On my machine that doesn't crash and makes a red window, which looks like what it's intended to do. Is this your code after it's been fixed rather than the original version that wasn't working?