Search code examples
htmlcssimage-processingpngjpeg

Displaying raw pixel array on a web page using image tag


I am trying to figure out the best way to put a raw pixel array into an image tag. The pixels are served from a server that does not have a png or jpg compression library so the raw array comes in via an HTTP request. I can control the return headers so I put a mime type in the response. I'd like to do:

<img src="http://myserver.com/id/" />

But I don't think I can do that. I could use the src="data:XXXXXX;base64,http://myserver.com/id/" if that works, but I need to know what to do with XXXX.

Another idea I've had is using svg if I can set an image equal to SVG. Not sure if i'd have to wrap each pixel in an element.

Maybe there is a way to do this with CSS?

I can write the data to canvas element with js pretty easily, but I was hoping to have a non-js way.

I can do some minor manipulation of the binary structure of the data coming out of the server, so if there is an easy way to tell the jpg or png format that this is uncompressed data, I could do that...I just don't have the horsepower or the time to translate to the png or jpg libraries to the (blockchain) based language I'm having to use.


Solution

  • Relevant links: the zlib RFC, the DEFLATE RFC, and the PNG spec. You may wish to consult these while reading.

    Let's make a PNG! Start with the image data as an RGBA image, for example for a 2x2 image:

    12 34 56 78  9a bc de f0
    cd ef 01 23  45 67 89 ab 
    

    Specify filters

    For each row of the image, add a null byte to the beginning to set the PNG filter to none. For example,

    00  12 34 56 78  9a bc de f0 # row 1 of image
    00  cd ef 01 23  45 67 89 ab # row 2 of image
    

    DEFLATE encode it

    While this is a compression format, we don't actually need to compress it! DEFLATE provides a way to encode data without compressing it (see section 3.2.4 of the DEFLATE spec). Find the length of the filtered image data from above as a 16-bit integer (if it is bigger than 65535, then split the image data into 65535-sized chunks, and do this step for each chunk). Create an empty array to hold the DEFLATE data stream. Insert that size as a 16-bit integer in big-endian format into the currently empty DEFLATE data stream. Next, append the bitwise inverse of the length. Finally, insert the filtered image data into the data stream.

    Wrap it in zlib

    Next, we need to zlib-encode the data. Start with an empty zlib data stream, and insert 2 null bytes for the header. Next, insert the DEFLATEd data from above. Finally, insert the Adler32 checksum of the uncompressed data. Specifically, the Adler32 checksum should be created from the filtered image data.

    Wrap it in a PNG format

    Finally, let's wrap this into a PNG. Replace the width and height values, specify the length of the IDAT chunk as the length of the zlib-encoded data, and replace the CRCs of the IDHR and IDAT chunks by taking the CRC32 checksum of the the chunk name concatenated with the data.

    # PNG signature
    137 80 78 71 13 10 26 10
    
    # IDHR chunk
    00 00 00 0d # IDHR length
    73 72 68 82 # IDHR type
    00 00 00 02 # width: COMPUTE THIS
    00 00 00 02 # height: COMPUTE THIS
    08 # bit depth: 8
    06 # colour type: truecolour with alpha
    00 # compression method: zlib
    00 # filter method: normal
    00 # no interlacing
    55 55 55 55 # CRC: COMPUTE THIS from "IDHR" + data
    
    # IDAT chunk
    55 55 55 55 # IDAT length: COMPUTE THIS
    73 68 65 84 # IDAT type
    [zlib data]
    55 55 55 55 # CRC: COMPUTE THIS from the "IDAT" + data
    
    # IEND chunk
    00 00 00 00 # IEND length
    73 69 78 68 # IEND type
    AE 42 60 82 # CRC (always the same value, doesn't need to be computed)