Overlapping images with transparency with BaseBitmap

[ferdinand]

Hey @sasha_janvier,

I today spent some time with this but I have not yet an answer yet, as I myself ran into some troubles with this, both from the classic and maxon API side. May I ask on which version of Cinema 4D you are? I assume 2024.2.0?

Regarding your questions:

1. Scaling images is possible in the classic Image API through BaseBitmap::ScaleIt. The maxon Image API also has image scaling methods, but they are unfortunately not public.
2. There are no image rotation methods in the classic or maxon Image API, both in their private and public parts.

There is something fishy going on with alphas in 2024.2, at least I can neither get the PNG nor the TIF image loader to do what I think they should do, no matter if I use the classic or maxon API. I will have to do more testing in the next days. I will answer here latest until Friday the 2nd.

Cheers,
Ferdinand

[sasha_janvier]

Thank you very much @ferdinand. It's both relieving and admittedly a bit annoying to have confirmation that something fishy is going on with alphas in 2024.2 given the time I've put into this so far, but ultimately, I'm just glad I brought this issue to you and the team's attention.

And yes, I am using 2024.2.

Regarding your answers, while I knew of the "BaseBitmap::ScaleIt", I guess what I was really asking was whether the Image API still would allow me to do pixel-per-pixel transformations such as scaling/rotating (like I am currently doing with the BaseBitmap class), but upon closer inspection, I feel like the answer to this question would lean towards "Yes".

Thanks again @ferdinand and very much looking forward to your testing and update. I really appreciate it your time and assistance.

[ferdinand]

@sasha_janvier said in Overlapping images with transparency with BaseBitmap:

I guess what I was really asking was whether the Image API still would allow me to do pixel-per-pixel transformations such as scaling/rotating.

Yes that would be possible, at least in one direction. You can create a BaseBitmap, then get its underlying ImageRef, manipulate the image data with the Image API to you hearts content, and then at any point return to the classic API BaseBitmap interface and use its methods. BaseBitmap::GetImageRef returns a reference to the internal ImageInterface used by the BaseBitmap to store its data.

The Cinema 4D API has two worlds as you surely have noticed by now, the so called classic API, that is the code that has been written since version R6-ish of Cinema 4D and mostly uses old-school C++ concepts, and then there is the so called maxon API, that is code that started out around R20 (there is no clear starting point) which uses a more modern approach to C++ (templating, yay!).

Because the current code base of Cinema 4D is about 25 years old, we often cannot replace all code of something in the classic API which we want to replace with something in the maxon API. We then rewrite the original type as an adapter (a 'wrapper') which under the hood uses the new maxon API type. Examples for this are images (BaseBitmap, ImageInterface), URLs/paths (Filename, UrlInterface), or color profiles (ColorProfile, ColorProfileInterface).

In some cases it is then possible to get the underlying maxon API data for a classic API wrapper, e.g., get the ImageRef for a BaseBitmap and with that use both worlds in tandem. However, it usually is not possible to go the other way around, construct a classic API type instance around an existing maxon API type.

So, when you start out by loading an image with BaseBitmap::InitWith, you can use both worlds. But if you start out with maxon::ImageTextureInterface::LoadTexture you cannot.

Cheers,
Ferdinand

[ferdinand]

FYI: I wrote a mail to the developers who own the image stuff, it might take some time before it has taken its route.

[sasha_janvier]

Thank you very very much once again, @ferdinand. I really appreciate the thorough explanation, as well as you sending an email to the team at Maxon.

In the meantime, I'll keep using the BaseBitmap interface and use an image with no alpha. Not an ideal situation, but at least I can progress with my plugin in the meantime.

Thanks a million once again. Very much looking forward to see what Maxon says and how we can resolve this issue.

[ferdinand]

Hello @sasha_janvier,

please excuse the longer waiting time. I have now a solution, but it is unfortunately not a pretty one. What I implied above, simply rely on the Image API and either use MultipassBitmap or ImageTextureInterface to load and layer images does not work in the context of images with embedded alpha information. What also does not work is using high level methods such as BaseBitmap::Get/SetPixelCnt to read and write data, as just for pure loading and copying, you cannot simply treat things as COLORMODE::ARGB in this case. Find my litany of failed attempts at the very end. One of the developers of the Image API had a go at this too and could not make it work either using MultipassBitmap and pure loading.

Solution

What you did in your initial posting was quite close to what you have to do, and is the only thing that works for me. To not make this thread enormous, I have ignored your sub-questions of scaling and rotating images. Please open a new thread for them if you still need help there.

What you could do in addition to my code below, is combine it with BaseBitmap::Get/SetPixelCnt, but then deal with the color and alpha bitmap on its own, you cannot just try to flat-out copy/write things as ARGB. You could also use MultipassBitmap by writing data simply into layers so that you do not have to do the blending on your own. Regarding the predefined blending functions you asked for, there is maxon::BlendColor but that just linearly interpolates two colors. There is gfx_image_blend_functions.h which more what you need but I did not end up using it.

When loading images with embeded alphas in the Picture Viewer of Cinema 4D, we must enable Image Transparency which is by default off for some reason.

Cheers,
Ferdinand

Inputs

Output

Code

#include "apibasemath.h"
#include "c4d_basebitmap.h"

#include "maxon/url.h"

maxon::Result<void> BlendBitmap()
{ 
  // This is the exit point for errors in this function, e.g., iferr_return or return maxon::...Error. 
  // It only works because the return type of this method is of the scheme Result<T>, i.e., it returns
  // an error or an instance of T, here void.
  iferr_scope;

  // If we want to use error handling but do not want this function to be of type Result<void> but
  // for example bool, we would do this:
  //iferr_scope_handler{
  //	// Not necessary but nicer, dumps the error prefixed by the function name to the console.
  //	DiagnosticOutput("@: @", MAXON_FUNCTIONNAME, err);
  //	return false;
  //};

  // Define the image size and two input and one output file next to this source file.
  const Int32 size = 256;
  const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory();
  const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return;
  const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return;
  const maxon::Url urlResult = (directory + "out_blend.png"_s) iferr_return;
  
  // Auto allocate the bitmap into which we will layer things and add an alpha channel. Auto 
  // allocation is scope based, i.e., #composition will be destroyed when this function is exited.
  // When we wanted to pass #composition to the outside and give ownership to the caller, we would
  // use BaseBittmap::Alloc instead.
  AutoAlloc<BaseBitmap> composition;
  if (composition->Init(size, size, 24) != IMAGERESULT::OK)
    return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not load file into bitmap."_s);

  BaseBitmap* const compChannel = composition->AddChannel(true, false);

  // Defines a function to layer #filePath into #composition.
  auto AddLayer = [&compChannel](BaseBitmap* const composition, const maxon::Url& filePath) -> maxon::Result<void>
  {
    // Error scope for this lambda and create the bitmap to load #filePath into.
    iferr_scope;

    AutoAlloc<BaseBitmap> source;
    if (source->Init(MaxonConvert(filePath)) != IMAGERESULT::OK)
      return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not load file into bitmap."_s);

    // Figure out the the smaller of the two images #composition and #source.
    const Int32 copyWidth = Min(composition->GetBw(), source->GetBw());
    const Int32 copyHeight = Min(composition->GetBh(), source->GetBh());

    if (copyWidth < 1 || copyHeight < 1)
      return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION, "Composition or source has null width or height."_s);

    // Make sure the source has an alpha channel.
    BaseBitmap* sourceChannel = source->GetInternalChannel();
    if (!sourceChannel)
      sourceChannel = source->AddChannel(true, false);

    // Two variables to hold pixel alpha values from the comp and the source and a constant to
    // convert pixel values from the [0, 255] to the [0, 1] interval.
    UInt16 a0, a1;
    const Float32 floatConversion = 255.99f;

    // Here comes the slow part, we really have to iterate pixel by pixel so that we can splice
    // channels.
    for (Int32 y = 0; y < copyHeight; y++)
    {
      for (Int32 x = 0; x < copyHeight; x++)
      {
        // Get the color and alpha component for the current pixel of both #comp and #source
        const Vector32 compColor = composition->GetPixelDirect(x, y);
        const Vector32 sourceColor = source->GetPixelDirect(x, y);
        composition->GetAlphaPixel(compChannel, x, y, &a0);
        source->GetAlphaPixel(sourceChannel, x, y, &a1);

        // Convert the alpha values to the [0, 1] interval.
        const Float32 compAlpha = Float32(a0) / floatConversion;
        const Float32 sourceAlpha = Float32(a1) / floatConversion;

        // Compute the sum of the alphas, i.e., the current alpha value at (x, y) when both images
        // are layered. And compute a scaling factor for normalized alpha values, i.e., alpha values
        // where the BG and FG always sum to 1.
        const Float32 alphaSum = Clamp01(compAlpha + sourceAlpha);
        const Float32 alphaFactor = 1.f / alphaSum;
        
        // Compute the final color and the final alpha value. I am not a big expert on all the bitmap
        // stuff, but this seems to be the most sensible way to do this.
        Vector32 color = (compColor * compAlpha * alphaFactor + 
                          sourceColor * sourceAlpha * alphaFactor);
        Int32 alphaInt = ClampValue(Int32(alphaSum * floatConversion), 0, 255);

        // Comment out to see what is happening, will of course make things very slow.
        // ApplicationOutput("x:@, y:@, a0:@, a1:@, alpha:@ (@)", x, y, a0, a1, alphaSum, alphaInt);

        // Write the pixel in the comp.
        composition->SetPixel(x, y, color.x, color.y, color.z);
        composition->SetAlphaPixel(compChannel, x, y, alphaInt);
      }
    }

    return maxon::OK;
  };

  // Add a layer for urlTex0 and then one for urlTex1.
  AddLayer(composition, urlTex0) iferr_return;
  AddLayer(composition, urlTex1) iferr_return;

  // Display the image in the Picture Viewer and save it as a PNG.
  ShowBitmap(composition);
  composition->Save(MaxonConvert(urlResult), FILTER_PNG, nullptr, SAVEBIT::ALPHA);

  // In the maxon API, Result<void> functions return maxon::OK when everything went well.
  return maxon::OK;
}

Graveyard

I have put this here so that future readers can see what as of 2024.2 and its following release DOES NOT WORK. All this code is non-functional regarding loading embedded alphas:

#include "apibasemath.h"
#include "c4d_basebitmap.h"

#include "maxon/gfx_image.h"
#include "maxon/url.h"
#include "maxon/mediasession_image_export_psd.h"

maxon::Result<void> ConstructMultipassBitmap1()
{
  // Does not work in the alpha aspect of lading images.
  iferr_scope;
  
  const Int32 size = 256;
  const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory();
  const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return;
  const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return;
  const maxon::Url urlResult = (directory + "out_multi1.psd"_s) iferr_return;
  
  MultipassBitmap* composition = MultipassBitmap::Alloc(size, size, COLORMODE::ARGB);
  if (!composition)
    return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION);
  
  finally {
    MultipassBitmap::Free(composition);
  };
  
  for (maxon::Url path : {urlTex0, urlTex1})
  {
    AutoAlloc<BaseBitmap> source;
    if (source->Init(MaxonConvert(path)) != IMAGERESULT::OK)
      return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION);
  
    MultipassBitmap* const layer = composition->AddLayer(nullptr, COLORMODE::ARGB);
    MultipassBitmap::AllocWrapper(source)->CopyTo(layer);

    //const BaseBitmap* const sourceAlpha = source->GetInternalChannel();
    //BaseBitmap* layerAlpha = layer->GetInternalChannel();
    //if (!layerAlpha)
    //	layerAlpha = layer->AddChannel(true, false);

    //if (sourceAlpha && layerAlpha)
    //	sourceAlpha->CopyTo(layerAlpha);
    //else
    //	ApplicationOutput("@: Could not copy alpha channel.", MAXON_FUNCTIONNAME);
  
    layer->SetParameter(MPBTYPE::SHOW, true);
    layer->SetParameter(MPBTYPE::SAVE, true);
  }
  
  ShowBitmap(composition);
  composition->Save(MaxonConvert(urlResult), FILTER_PSD, nullptr, SAVEBIT::MULTILAYER | SAVEBIT::ALPHA);
  
  return maxon::OK;
}

maxon::Result<void> ConstructMultipassBitmap2()
{
    // Does not work in the alpha aspect of lading images.
  iferr_scope;

  const Int32 size = 256;
  const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory();
  const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return;
  const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return;
  const maxon::Url urlResult = (directory + "out_multi2.psd"_s) iferr_return;

  MultipassBitmap* composition = MultipassBitmap::Alloc(size, size, COLORMODE::ARGB);
  if (!composition)
    return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION);

  finally {
    MultipassBitmap::Free(composition);
  };

  auto AddLayer = [](
    MultipassBitmap* const composition, const maxon::Url& filePath) -> maxon::Result<MultipassBitmap* const>
  {
    iferr_scope;

    if (!composition)
      return maxon::NullptrError(MAXON_SOURCE_LOCATION, "Invalid composition pointer."_s);

    AutoAlloc<BaseBitmap> source;
    if (source->Init(MaxonConvert(filePath)) != IMAGERESULT::OK)
      return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not load file into bitmap."_s);

    MultipassBitmap* const layer = composition->AddLayer(nullptr, COLORMODE::ARGB);
    const Int32 copyWidth = Min(source->GetBw(), layer->GetBw());
    const Int32 copyHeight = Min(source->GetBh(), layer->GetBh());

    if (copyWidth < 1 || copyHeight < 1)
      return maxon::IllegalArgumentError(MAXON_SOURCE_LOCATION, "Composition or source has null width or height."_s);

    iferr (UChar * buffer = NewMem(UChar, (COLORBYTES_ARGB * copyWidth)))
      return maxon::OutOfMemoryError(MAXON_SOURCE_LOCATION, "Could not allocate copy buffer"_s);

    BaseBitmap* const sourceAlpha = source->GetInternalChannel();
    sourceAlpha->Save(Filename("e:\\alpha.psd"), FILTER_PSD, nullptr, SAVEBIT::NONE);

    for (Int32 y = 0; y < copyHeight; y++)
    {
      source->GetPixelCnt(0, y, copyWidth, buffer, COLORBYTES_ARGB, COLORMODE::ARGB, PIXELCNT::NONE);
      layer->SetPixelCnt(0, y, copyWidth, buffer, COLORBYTES_ARGB, COLORMODE::ARGB, PIXELCNT::NONE);
    }

    DeleteMem(buffer);

    return layer;
  };

  AddLayer(composition, urlTex0) iferr_return;
  AddLayer(composition, urlTex1) iferr_return;

  ShowBitmap(composition);
  composition->Save(MaxonConvert(urlResult), FILTER_PSD, nullptr, SAVEBIT::MULTILAYER | SAVEBIT::ALPHA);

  return maxon::OK;
}

maxon::Result<void> ConstructImageTexture()
{
  // Does not work in the alpha aspect of lading images.

  iferr_scope;

  const maxon::Url directory = maxon::Url(maxon::String(MAXON_FILE)).GetDirectory();
  const maxon::Url urlTex0 = (directory + "bar0.png"_s) iferr_return;
  const maxon::Url urlTex1 = (directory + "bar1.png"_s) iferr_return;
  const maxon::Url urlResult = (directory + "out_imgtexture.psd"_s) iferr_return;

  const maxon::ImageTextureRef tex0 = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return;
  const maxon::ImageTextureRef tex1 = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return;

  tex0.Load(urlTex0, maxon::TimeValue(), maxon::MEDIASESSIONFLAGS::NONE) iferr_return;
  tex1.Load(urlTex1, maxon::TimeValue(), maxon::MEDIASESSIONFLAGS::NONE) iferr_return;

  maxon::ImageBaseRef tex0img = tex0.GetFirstChild(maxon::ConstDataPtr(maxon::IMAGEHIERARCHY::IMAGE));
  maxon::ImageBaseRef tex1img = tex1.GetFirstChild(maxon::ConstDataPtr(maxon::IMAGEHIERARCHY::IMAGE));
  tex0img.Remove();
  tex1img.Remove();

  const maxon::ImageTextureRef layers = maxon::ImageTextureClasses::TEXTURE().Create() iferr_return;
  layers.AddChildren(maxon::IMAGEHIERARCHY::IMAGE, tex0img, maxon::ImageBaseRef()) iferr_return;
  layers.AddChildren(maxon::IMAGEHIERARCHY::IMAGE, tex1img, tex0img) iferr_return;

  const maxon::MediaOutputUrlRef psd = maxon::ImageSaverClasses::Psd().Create() iferr_return;
  maxon::MediaSessionRef session;

  layers.Save(urlResult, psd, maxon::MEDIASESSIONFLAGS::NONE, &session) iferr_return;
  session.Close() iferr_return;

  return maxon::OK;
}

[sasha_janvier]

Absolutely wonderful, @ferdinand. I can't thank you and the team at Maxon enough. No worries whatsoever about the delay, you guys make up for it by 10 folds when you come back with a response.

Thank you again. I did indeed notice some similarities with the approach I pasted in my first message. I will try out your proposed implementation later today. I'm very eager to get the alpha channel working.

[sasha_janvier]

Hey @ferdinand,

Just out of curiosity, I was wondering if there was any hopes for the Image API to handle transparency one day? As you've stated yourself (and as I'm experiencing myself while testing my plugin everyday), iterating pixel by pixel in the goal of splicing channels is very slow when dealing with a lot of semi-transparent images.

Of course, I am gonna stick with the approach you generously proposed as it works, but a faster alternative would eventually be very welcome.

Thank you!

[ferdinand]

Hey @sasha_janvier,

Thank you for reaching out to us. To lead with a direct answer, no there are currently no plans to extend the Image API in that fashion.

And just to be clear, both the Image API and the classic API image types support alphas and multi layer alphas, it is only that the special case of manually assembling a multi layered image with alphas out of loaded files is not something we ever really needed and therefore implemented. BodyPaint that functionality but does it with its own code that is not public.

The only way to make things faster would be to help yourself.

Write things in rows

You could try to do what I did in the working example and mix it with GetPixelCnt and SetPixelCnt (I used it in the Graveyard in ConstructMultipassBitmap2). The idea would be still to write a BaseBitmap and not a MultipassBitmap but write data row by row and not pixel by pixel. But since we must blend the pixels ourself, we would still need a bit which blends each row pixel by pixel. Something like this (this is pseudo code):

int y; // The current row we are writing.

// Buffers for the RGB pixel data of a row.
Uchar* bufferA;
Uchar* bufferB;
Uchar* bufferOut;

// Get the data for one row for both images.
imageA->GetPixelCnt(0, y, copyWidth, bufferA, COLORBYTES_RGB, COLORMODE::RGB, PIXELCNT::NONE);
imageB->GetPixelCnt(0, y, copyWidth, bufferB, COLORBYTES_RGB, COLORMODE::RGB, PIXELCNT::NONE);
// Blend both rows into one buffer, BlendBuffers would have to iterate pixel by pixel over both rows.
BlendBuffers(bufferA, bufferB, bufferOut);
// Write the buffer into the output image.
output->SetPixelCnt(0, y, copyWidth, bufferOut, COLORBYTES_ARGB, COLORMODE::ARGB, PIXELCNT::NONE);

// Do the same for alphas ...

The problem with this is that doing this will likely eat a good portion of the performance benefit that GetPixelCnt\SetPixelCnt yields over the pixel-by-pixel getters and setters because we still call BlendBuffers.

Just use parallelization

Since you say that you have many images to blend, another way out could be simply paralleization. Depending on your image data parallelization might also be necessary. When you have 200 4k images to blend, then writing data in rows offers a theoretical speed-up by the factor of 4096. When you are however trying to blend 1E6 32px images, writing in rows alone will only yield a theoretical speed-up by the factor of 32. The smaller the images and the more you have, the more likely it is that you will need parallelization on top of writing things in rows.

For paralleization you could use jobs (a bit overkill in this case IMHO) or ParallelFor::Dynamic which is our flavor of a parallelized loop.

Poke again at the Image API

After I answered your topic here, I realized a flaw with what I did in the pure Image API code in ConstructImageTexture. The problem when I wrote this was that BaseImageInterface:: AddChildren only allows you to add ImageRef children to an image, and not ImageTextureRef children. But to load and save files you need this type, i.e., you end up with two ImageTextureRef you want to insert into a ImageTextureRef (which you cannot). Which is why I had to poke around in the internals of the loaded files with GetFirstChild. I never spend much time on exploring how the GetFirstChild image is composed, i.e., simply assumed it had an alpha child. I am pretty sure that when you slam your head hard enough against the wall here, you can make this work. But unfortunately, many things in the Image API have never been documented, so you often have to reverse engineer things.

In the end this is all very speculative. I do not really know the context of what you are doing. How many files to blend?, size of the files?, done in one batch or spread over the runtime of a user session?, etc. pp.

Parallelization is often not the best option to make something faster, but it is pretty straight forward. You should however talk to us before you do that unless you feel confident in knowing what you do, as you can also make Cinema 4D slower when you flood its job-queue with too many jobs.

Cheers,
Ferdinand

[sasha_janvier]

As always, thanks a million for your incredibly thorough answer, @ferdinand.

How many files to blend?, size of the files?, done in one batch or spread over the runtime of a user session?, etc. pp.

The exact number of drawn bitmap images depends on the settings set by the user through my plugin's UI, but on average, I would roughly estimate the number of drawn bitmap files to be between 200 and 500. The bitmap's dimensions are set to be 256px by 256px and the drawing process is all done in one batch upon the user triggering the plugin's primary action button.

Thanks for the heads up about the possibility of flooding Cinema 4D's job-queue with too many jobs. This is something I will be very mindful of and aim to prevent in any way I can.

I am extremely grateful for your generous support, but because you've presented multiple methods and concepts I was completely unfamiliar with, you'll have to bear with me as I carefully research and revise them over the week-end before opting for a solution that seems ideal for my specific case.

Thanks again for everything, @ferdinand. I will get back to you with my findings!

Cheers

[ferdinand]

Hey @sasha_janvier,

This seems to be a case where parallelization should work well. What I forgot to say in my last posting is that maxon::ParallelImage could be a good option for you here to replace writing in rows with Get/SetPixelCnt.

• Use maxon::ParallelImage to parallelize reading and writing pixels in your input and output (for a singular result image) at once.
• Use maxon::ParallelFor::Dynamic or jobs to run multiple pixel writing tasks at once. Make sure to use indeed ::Dynamic and not ::Static and use the default constructor for maxon::Granularity, i.e., leave it completely up to Cinema 4D to decide how much bandwidth it will give you.

I would recommend starting out with the pixel writing parallelization first, as it will likely yield the largest performance benefit. Maybe this will be then already performant enough for you and you won't need the (likely more complicated) second step with ParallelFor.

I would also recommend writing a type/struct with which you can more easily express a set of to be blended bitmaps, so that is more easy for you to expose the data to your worker lambdas for ParallelImage and ParallelFor.

When you run into problems, feel free to ask questions, we know that the maxon API can sometimes be a steep hill to climb for newcomers. When you run into problems with parallelization, I would however have to ask you to open a new thread, as we are reaching here off-topic territories in this thread.

Cheers,
Ferdinand

[sasha_janvier]

Thank you very kindly, @ferdinand! I had already started exploring the maxon::ParallelImage class as I suspected it to be the ideal path forward for my case, so it's great to have my suspicions be validated!

I will make sure to start a new thread if I have any questions related to this class.

Thank you very much once again. Your assistance has been indispensable!

Cheers