It worked, thanks. September 29, 2008 6:09 PM   Subscribe

Thanks to jepler, pixel detctive

Several Mefites tried to help me with a very specfic video problem I was having this past summer. While the posted answers did not directly lead to the tidy solution, this comment by jepler:

"The bits of each pixel must be interleaved in some way that is not yet apparent."

...kept knocking around in my head, and eventually got me thinking in a different direction.

Eventually I did find out how those bits were laid out - and they were interleaved in some [very preculiar] way.

So, thanks.
posted by shino-boy to MetaFilter-Related at 6:09 PM (26 comments total) 2 users marked this as a favorite

posted by Kattullus at 6:25 PM on September 29, 2008 [1 favorite]

The bits of each pixel must be interleaved in some way that is not yet apparent.

That's what she said! Is your uh, is your wife interested in... photography, ay guv? 'Photographs, ay', he asked him knowlingly? Wink wink, nudge nudge, a nod's as good as a wink to a blind bat, ay?
posted by XMLicious at 6:52 PM on September 29, 2008

I didn't understand a word you just said.
posted by yhbc at 6:56 PM on September 29, 2008

My friend has a pool.
posted by turgid dahlia at 7:39 PM on September 29, 2008 [1 favorite]

Is Obama a hipster?
posted by Rumple at 7:43 PM on September 29, 2008

posted by puke & cry at 7:46 PM on September 29, 2008

posted by Rumple at 8:07 PM on September 29, 2008

He fixes radios by thinking!
posted by lukemeister at 8:20 PM on September 29, 2008 [1 favorite]

Sorry, meant: "pixel detective".
posted by shino-boy at 8:54 PM on September 29, 2008

When did everyone turn into an idiot? Did I miss a memo?
posted by erebora at 9:07 PM on September 29, 2008

jepler: defective pixel pixel detective.
posted by clearly at 9:08 PM on September 29, 2008

I'm wondering the same thing, erebora; is this Slashdot all of a sudden?

posted by hattifattener at 11:07 PM on September 29, 2008

Is there something in the air? What is wrong with all you people over there? You can't all possibly be drunk at the same time, can you?
posted by dg at 12:53 AM on September 30, 2008

Q: What's brown and sounds like a bell?
posted by jbickers at 4:09 AM on September 30, 2008 [1 favorite]

Sorry, meant: "pixel detective".

Now this very preculiar thread makes sense.
posted by Rock Steady at 4:45 AM on September 30, 2008

So... now I'm curious, shino-boy, what was the cause of the problem after all and how did you fix it?
posted by Kattullus at 5:10 AM on September 30, 2008

Yeah, ditto Kattullus. If you're going to go all metatalk on us, at least make with the 10-bit encoding skinny, because I'm all curious now.

Also, you should throw a note in the thread itself; it's still open.
posted by cortex (staff) at 7:45 AM on September 30, 2008

That's some very "preculiar" spelling you've got going there.

I've got nothing.
posted by Sys Rq at 10:14 AM on September 30, 2008

Ugh, my "sprell checking" is atrocious.

For the curious, gruesome details follow:

Quick recap - I bought a camera mod that records 8bit and 10bit uncompressed video files, mod developer supplied an app to render the files, app never worked on 10bit files, company was sold, developer went offline, broken app abandoned and unsupported. I wondered if I could write a QuickTime codec to read the video files directly, bypassing the mod company's render app altogether.

The main problem with the 10bit version of the codec was that I couldn't figure out how the camera mod was saving the red/green/blue channels of each pixel.

jepler confirmed that some type of pattern was repeating every 15 bytes in the 10bit files. He and other MeFites also noticed that the math wasn't adding up - the filesize should equal frames times resolution times bitdepth. But it wasn't.

So my technique of trying to see the 10bit pattern by viewing the video file's data directly was clearly not working.

"But what if the bits of each pixel were interleaved in some CRAZY NON-INTUITIVE way?" I thought and decided to give it all one last, brute force attempt.

Making a bunch of assumptions I wrote my own app to generate 10bit video files in my camera mod's format. I figured that the first 15 bytes of my test video file should equal the first four pixels on the first scanline of each frame. Then I set the first byte (of the fifteen) to 255 and everything else to 0.

And here comes the tricky part - I then passed this file to the company's render app to see what came out. Now, we know that the render app doesn't work on 10bit files, which is true. But it will take a 10bit file and render out 8bit from it (how tacky!). I figured that even though I was feeding in 10bit files and getting out 8bit files, if I could see where the high and low bits were getting rendered, I could determine where the render app was expecting them. Then I could figure out the pattern.

The first time I tried it, the render app complained about a "missing tail byte". ??? So I added one with a value of 0. And then the render app rendered out my test file. This tail byte accounts for the odd filesize that jepler and others had noticed, each 10bit file has one extra byte tacked on to the end. I have no idea what it does. I think of this byte like an appendix, perhaps long ago it served some useful purpose. But now? Meh.

So the workflow became: Set each one of the first fifteen bytes, write out a 10bit test file, render it, examine the output .mov, look at the first four pixels, chart where the bits went.

I was able to account for all the RGB bits of the first four pixels. Each channel's 10 bits are always laid out over two non-adjacent, non-consecutive bytes. I have no idea why it's like that.

I got the pattern and, after some very cool advice from the QuickTime mail list, plugged it back into my codec. It works. Yay.
posted by shino-boy at 11:56 AM on September 30, 2008 [2 favorites]

posted by cortex (staff) at 12:19 PM on September 30, 2008

So, what is the pattern? Might be fun to come up with the most plausible rationale.
posted by flabdablet at 5:39 AM on October 1, 2008

flabdablet, I decided to scan my workbook because I couldn't write out the pattern in a readable format for this comment section.

The bytes are in big-endian order. Each block of 15 bytes equals 4 RGB, 10bits/channel pixels.

This scan shows the pattern as a block of 15 bytes. "r1 lo" would mean pixel #1, red channel, least significant bytes. "b4 hi" would represent pixel #4's blue channel, most significant bits.

This scan was more useful for me, it shows the pixels on the left hand and the corresponding byte positions on the right.

Don't worry if you can't read my chicken-scratch writing. I can' read it either! :)
posted by shino-boy at 9:18 PM on October 1, 2008

OK. So I've hopefully correctly translated your first diagram into a list of the bits in the first five bytes. Each bit has a three-character code, the first character being the pixel number the bit belongs to from 1 to 4, the second character being the color channel within that pixel (r, g or b), and the third character being the bit number within that color channel (0 for lsb, 9 for msb). I list the bits within each byte with the msb (bit 7) on the left, and the lsb (bit 0) on the right, as is conventional for binary numbers. The list looks like this:

1r7 1r6 1r5 1r4 1r3 1r2 1r1 1r0
1g5 1g4 1g3 1g2 1g1 1g0 1r9 1r8
4b3 4b2 4b1 4b0 1g9 1g8 1g7 1g6
2r1 2r0 4b9 4b8 4b7 4b6 4b5 4b4
2r9 2r8 2r7 2r6 2r5 2r4 2r3 2r2
2g7 2g6 2g5 2g4 2g3 2g2 2g1 2g0
3b5 3b4 3b3 3b2 3b1 3b0 2g9 2g8
3r3 3r2 3r1 3r0 3b9 3b8 3b7 3b6
3g1 3g0 3r9 3r8 3r7 3r6 3r5 3r4
3g9 3g8 3g7 3g6 3g5 3g4 3g3 3g2
2b7 2b6 2b5 2b4 2b3 2b2 2b1 2b0
4r5 4r4 4r3 4r2 4r1 4r0 2b9 2b8
4g3 4g2 4g1 4g0 4r9 4r8 4r7 4r6
1b1 1b0 4g9 4g8 4g7 4g6 4g5 4g4
1b9 1b8 1b7 1b6 1b5 1b4 1b3 1b2

That's a straightforward little-endian packing of the following sequence of 10-bit values, each identified by a pixel number and a color channel code:

1r 1g 4b
2r 2g 3b
3r 3g 2b
4r 4g 1b

So it's not a random interleaving, just an odd one. It's weird how the blue channel's pixel order is flipped compared to the red and green channels. I wonder if this is an artifact of the physical layout of the camera's sensor chip?
posted by flabdablet at 11:54 PM on October 1, 2008 [1 favorite]

I'm not sure about the sensor chip. I do know that the 8bit files are written out with a standard RGB interleave:

1r 1g 1b 2r 2g 2b 3r 3g 3b [...]

All of this stuff is new to me as I'm neither a programmer nor QuickTime engineer by trade.

I just want all my bits, is that so wrong?

Thank you for taking the time to formalize the pattern more clearly, it's helpful to me.
posted by shino-boy at 9:32 AM on October 2, 2008

I'm guessing that somewhere inside the camera there's a 128-bit register that presents 120 bits of pixel data exactly as they come off the sensor plus 8 unused bits grouped at one end; I'm guessing that if you read the bytes in that register in order from lsb to msb and leave out the unused byte, you see the weird interleaved 10-bit pattern; I'm guessing that the byte stream for 8-bit files is produced by a software routine that "cooks" the data read from that register, discarding the two lowest bits from each 10-bit value and rearranging them in a sensible order; and I'm guessing that the camera mod software doesn't bother rearranging the bits, but saves them exactly as they're read out of the register.

It's all guesswork, but I have seen similar kinds of weirdness in other embedded systems.
posted by flabdablet at 8:50 PM on October 2, 2008

oh, hey, it's like a callout but in a good way. hi, I'm glad my little bit of analysis was helpful to you.
posted by jepler at 5:18 AM on October 3, 2008

« Older Your Favorite Attitude Toward Sentimental Stories...   |   Travelfilter DOA? Newer »

You are not logged in, either login or create an account to post comments