Understanding the Screen Share Protocol

The screen share protocol in Ulterius works by looking for difference in individual frames of the desktop. These changes are then sent to the client as patches which can be stitched into the canvas.

Screen capture data is packed into the following binary format

-----Body-----
Guid (byte[16])
Bounds X (int32)
Bounds Y (int32)
Bounds Top (int32)
Bounds Bottom (int32)
Bounds Left (int32)
Bounds Right (int32)
Jpeg Patch (byte[<remaining>])
----------

For example, the web client uses this parse function:

export function unpackFrameData(data: Uint8Array) {
    let fv = new jDataView(data, 0, data.length, true)
    //let uid = fv.getString(16)
    let x = fv.getInt32(16)
    let y = fv.getInt32()
    let top = fv.getInt32()
    let bottom = fv.getInt32()
    let left = fv.getInt32()
    let right = fv.getInt32()
    let image = decompressData(data.subarray(16 + (4*6)))
    return {
        x, y, top, bottom, left, right, image
    }
}

When starting an Ulterius screen share stream these patches will be sent to you in real time.