loadImage(...) and createGraphics(..., P2D) have different y-coordinates

[clankill3r]

Most appropriate sub-area of Processing 4?

OpenGL

Processing version

4.4.4

Operating system

MacOSX

Steps to reproduce this

PShader texCoordShader;
PGraphics pgP2D, pgDefault;

void setup() {
  size(800, 400, P2D);
  
  String fragShader = 
    "#ifdef GL_ES\n" +
    "precision mediump float;\n" +
    "#endif\n" +
    "varying vec4 vertTexCoord;\n" +
    "void main() {\n" +
    "  gl_FragColor = vec4(vertTexCoord.st, 0.0, 1.0);\n" +
    "}";


  saveStrings("frag.glsl", split(fragShader, "\n"));
  texCoordShader = loadShader("frag.glsl");//, "vert.glsl");

  pgP2D = createGraphics(400, 400, P2D);
  pgDefault = createGraphics(400, 400); // Default JAVA2D renderer

  pgP2D.beginDraw();
  pgP2D.endDraw();

  pgDefault.beginDraw();
  pgDefault.endDraw();
}

void draw() {
  background(0);
  shader(texCoordShader);
  image(pgP2D, 0, 0);
  image(pgDefault, 400, 0);
}

Additional context

I'm working on a keystone library that is using a shader.
Because a image has different coordinates then a graphics created with P2D I need to know what the user is giving.
Not only that, it complicates the shader math way more then I would like.

To give an example:

The difference between those is P2D and default renderer (both in a sketch that is P2D).
Notice that the text is correct in both cases!!!

This does not just apply for PGraphics, using loadImage is different as well (cause also baked by a BufferedImage).
Having a mix of both of them really complicate things.
Complaining is easy however. I will dig into seeing if I can find a solution.

Would you like to work on the issue?

Trying...

[SableRaf]

Hi @clankill3r and tanks for reporting this!

I haven't had time to investigate closely, but I wonder if it could be related to this issue (#891) about the y axis. We made a fix for it in 4.4.2 which had to be reverted in 4.4.4 as it caused some unexpected side effects.

Is it possible you started work on your library using 4.4.2 or 4.4.3 and only recently updated to 4.4.4?

[clankill3r]

Hi @SableRaf I checked out #891, thanks for pointing that out. It's seems to be not related.
I'm on processing 4.4.4 but I checked in 4.3 as well and it happens there too.
I think the issue is really old but that most users won't ever notice cause the amount of people using shaders is low and then the change of producing a shader where this affects you is low as well.

Right now (speculation: )I think that on the GPU the pixels are in reverse order but that it doesn't matter much (it actually does), because the vertexCoords are reversed in y axis as well countering the first error.

(With DEFAULT I mean the not P2D, I don't think this is strictly JAVA2D so I just call it DEFAULT).

Some findings so far, calcTriangleNormal seems to produce the same result for P2D and DEFAULT.
In PGraphicsOpenGL there is copyManyAttribs where one of the things it does is:

PApplet.arrayCopy(in.texcoords, 2 * i0,
                        polyTexCoords, 2 * firstPolyVertex, 2 * nvert);

I checked the result against P2D and DEFAULT and they are similar.

My next step will be to invest uniform mat4 transformMatrix; and attribute vec4 position; in the vertex shader.

Also I think I might now a way to work around this bug, but it would still be nice to fix it for the cleanness of P5.

[clankill3r]

Ok, I give up. It's such a intense layercake that I decided to go with this:

static public void render(..., boolean flip_y_because_of_processing_issue_1134) {

[clankill3r]

There is one more thing I would like to point out in case it helps someone.

When using loadPixels on a P2D then nativeToJavaARGB is being called which also flips the y axis.
Which is good. But for the default, there is no javaARGBToNative to be found anywhere. So it seems that keeping the order to not confirming to the OpenGL standard but flipping the texcoords instead was a cheap hack made in the past that sadly enough doesn't work in all cases.