How to multiply two images with different encoding

Question

I have two textures map, one albedo and another ambient occlusion. The albedo one is srgb encoded .jpg whereas the ambient occlusion is linear encoded .jpg.

Now, I want to load these two images (preferably in node.js) and multiply their rgb values evenly(0.5 weight) and output the image in .jpg format with sRGB encoding.

I tried to simply read and write a linear encoded normal map using the sharp library(npm, node.js) as a test, with the following code, but the output image looks slightly darker now.

import sharp from 'sharp';

const img = sharp('assets/normal.jpg');
const processedImg = img
  .resize(1024)
  .jpeg({ quality: 100 });
processedImg.toFile('assets/__normal.jpg');

Even, the metadata() function on the image says the image is in srgb space, but I had exported the maps from Quixel Bridge and I know that those are linear encoded, still the metadata returns that they are in srgb space.

I can't find any hints from the sharp.js documentation on how to force change the input file encodings.

Basically, I want to replicate this operation in blender, but using code in node.js or c++.

I can use some other library, if recommened.

I am even open to c or c++ solutions if it can't be done in nodejs or gets complicated.

Thank you in advance.

Answer 1

Well I found out a trick with which we can trick sharp to work with linear encodings.

So, we know that our file is linear encoded. But since it has no metadata, sharp assumes it to be sRGB encoded. So what can we do? Hmm..

sharp(ifile)
    .pipelineColourspace('srgb')
    .toColourspace('srgb')
    .toBuffer();

We say to sharp that please while processing our file don't gamma correct our linear encoded file(i.e. sharp would think its sRGB, and would have applied gamma correction) to linear (which it normally does), but instead work with supposedly ``sRGB(i.e. linear```) values.

.pipelineColourspace('srgb') tells sharp to do the image-processing with sRGB values, and since sharp wrongly assumed that our file is sRGB, it doesn't gamma correct our file, cause it thinks it already in the required format.

.toColourspace('srgb') tells sharp to output the iamge as srgb values, now again since to sharp our pipeline was sRGB it doesnt gamma correct it again, and just simply spits the buffer received from pipeline.

This way we tell sharp to avoid applying gamma correction on our image. Cool.

Now lets answer the whole question, on how to multiply srgb albedo, linear ao, and output to a sRGB image.

export const multiplyTexture = async (albedo: Buffer, ao: Buffer) => {
  return sharp(albedo)
    .pipelineColorspace('linear')
    .composite([
      {
        input: await sharp(ao)
          .pipelineColourspace('srgb')
          .toColourspace('srgb')
          .toBuffer(),
        blend: 'multiply',
        gravity: 'center',
      },
    ])
    .toColorspace('srgb')
    .toBuffer();
};