MP4 ATOM is not found for some cameras

[attilafustos]

@Cleric-K

There is a known problem, some cameras produce mp4 headers than are not handled properly by the script.
I tried to debug it but my mp4 knowledge is far from yours.
It's been a while when I tried to debug it, but I think it was this line return struct.unpack('>I4s', buf)
The SMO, InstaOne has this problem. Also people reported unsupported content exported from Premiere.
If/when you have some time and want to take a look I can send you some samples.

[Cleric-K]

Yes, if you can, upload a problematic .mp4 somewhere and I'll see if I can figure something out. I'm no expert in mp4 also. My first encounter with mp4 internals was when I was writing for this project.

[attilafustos]

https://drive.google.com/drive/folders/1sFGhRZLtbnE34xKENX3WsA2uaAirdPHg?usp=sharing

I uploaded to my g drive a video (MIKE1049.MP4) I received from a user.

Thank you

[attilafustos]

[attilafustos]

@Cleric-K
I found some info. I always get mdat size = 1 for those videos.

Among them, size indicates the size of the entire box, including the header part. If the box is large (for example, the mdat box that holds the specific video data), the size of the uint32 is exceeded, the size is set to 1, and the next 8-bit uint64 is used to store the size.

https://www.programmersought.com/article/9912137617/

[Cleric-K]

Yes, I've just pushed a test gui_temp branch. Can you take a look?

This is not a complete solution however. It should read 64bit mdat OK but if the data is really larger than 4GB it won't work. Other things have to be modified also in the other samples which store the samples offset.

[Cleric-K]

The atom stco stores the sample offsets. These offsets are from the beginning of the file. As such, if some sample is positioned beyond the 4GB mark, atom co64 is used. There's no support for this atom at this time. If someone hits that limit it'll need to be implemented.

See https://developer.apple.com/standards/qtff-2001.pdf for info on the atoms.

[attilafustos]

Just checked the output result on the test file from my g drive. The result is a black video without content. The header seems ok though because no error is reported.

[Cleric-K]

Oops. Looks like this is actually the case. This file uses co64 instead of stco. It'll need to be implemented.

[Cleric-K]

You may change this in process.py

for stco in moov.find(b'co64'):
        count = struct.unpack('>I', stco.data[4:8])[0]
        offset_format = '>{}Q'.format(count)
        offsets = struct.unpack(offset_format, stco.data[8:])
        # translate offsets to new origin
        offsets = [o + mdat_offset_diff for o in offsets]
        stco.data[8:] = struct.pack(offset_format, *offsets)

It's just the co64 and the Q in the the format.

[attilafustos]

Yes. It works like that.
So I should detect somehow if the file uses stco or co64 and adapt the code ?

[Cleric-K]

Yeah, I can do it if you want. The generation of offsets for the GPMF should also be touched.

[attilafustos]

Yeah, I can do it if you want. The generation of offsets for the GPMF should also be touched.

I'd like that. I am sure the 3K facebook users would be grateful for it.

[attilafustos]

I will gladly post your paypal.me address on the facebook group to raise some beer money for your work. Just let me know.

[Cleric-K]

I've just pushed. Haven't tested anything though. But it should be clear from the code what the goal is.
Thank you for paypal suggestion! Not needed. I'm happy that this project is moving forward.

[attilafustos]

Looks good. Thank you!

[Cleric-K]

Great! You can delete the temp branch when you merge it to yours.

Good luck!