c4d.documents.RenderDocument does not apply OCIO Viewtransform

hSchoenberger

Hi

this is working as intended I would say

The View Transform is applied everywhere else.
That is the issue. I cannot generate the same output as C4D with a script.
If I Batch render the scene via commandline or if I start to render the output files via the Picture Viewer.

So either everything else is broken
or it is just that this script render does not work the same way as all other file outputs in Cinema4D.
And I assume the Batch render and the Picture Viewer render work as customers expect it (otherwise you would have fixed it already).

If you render .exr files, then you have an option to "Bake View Transform".
If you render any other file format, this option is unchangeable and always activated (internally).
And for both output types it does not work to bake the View Transform into the file if I try it via script.

ferdinand

Hey @hSchoenberger,

I was just polite with my wording when I said 'I would say'. The view transform and the display space are not intended to be baked into images, as this would be the exact opposite of what OCIO is trying to achieve. Please read the docs I have linked to understand what OCIO is trying to do on a technical level.

The bake view transform option RDATA_BAKE_OCIO_VIEW_TRANSFORM (there is also a hidden setting RDATA_BAKE_OCIO_VIEW_TRANSFORM_RENDER) is a speciality option which is only available for 32 bit per channel color depth images. It has nothing to do with exr files, other than that these are 32 bit by default.

Showing RenderDocument sourced images in the Picture Viewer is currently broken and will be fixed in a future release. When you think otherwise, I would point you to Support Procedures: Reporting Bugs & Crashes for how to report a bug. In this case you would have for example to demonstrate that an image rendered manually looks different in Photoshop or another image app, than rendering the same document with the same settings with RenderDocument and displaying that image in that alien application.

What we will add in a future release, is the Python OCIO API which will also contain a function to bake a view and display transform into an image; to for example display it in the Picture Viewer of Cinema 4D. But renderings operate in render space and store colors in the output profile of the image they are rendered into. The display and view transform are intentionally irrelevant here, as you otherwise cannot have other apps looking at the same render data with a different display color space and view transform; which in a sentence is the purpose of OCIO.

Cheers,
Ferdinand

hSchoenberger

Hi

Please read the docs I have linked to understand what OCIO is trying to do on a technical level.

I know the different levels of OCIO as I have already implemented OCIO via C++ into an application.
You are right AS LONG AS you are working within your pipeline, the view transform is applied in the display function of every application. E.g. In compositing like Nuke.
But once you render a final image which is not intended to be loaded in any other application afterwards (an image send to the client),
then view transform has to be applied.
And the customer I am working for uses C4D to render final images as well.
This is why they need the view transform baked into the rendered image.

It has nothing to do with exr files

Ok, then I correct: If you render 8bit files, then "Bake View Transform" is enabled by default internally.
And if you render 32bit, then you can choose to enable "Bake View Transform" or not.
Nevertheless it does not work for rendering a frame via script.

Showing RenderDocument sourced images in the Picture Viewer

I am not loading anything into the Picture Viewer.
I stated examples how you can render to a file with C4D.
I was refering to this menu option:

Support Procedures: Reporting Bugs & Crashes

Ok, then I have to report a bug that the "Bake View Transform" does not work with a scripted render.
I assumed there is some kind of undocumented option/flag missing for the script functions.
But as it seems OCIO was not tested for a script render and therefore it is a bug
I send you the customers scene file and rendered images.

ferdinand

Hello @hSchoenberger,

You actually have to cooperate with us. I already pointed you to our docs and could point also to the official OCIO docs, which will also explain you what a view is, and that is does not make much sense to bake this into images by default as this would literally downgrade the quality of your data (as ACES SDR video is emulating a low-grade consumer SDR video displaying device and your data pipeline is meant to be HDR).

Cinema 4D as many other applications also offers the option to bake in the view-display transform in some scenarios for multiple reasons. Cinema 4D is not baking in the view transform everywhere, nor is everything broken as you wrote above. The image will contain the render data in render space transformed to the image profile you have chosen for the rendering. When you have one of the baking options enabled, and you think you have found a bug, then please provide precise reproduction steps; Support Procedures: Reporting Bugs & Crashes.

You won't get very far here with being imprecise and confrontative at the same time. I always try to help external developers, especially when the initial information is a bit lackluster, as I know it can be hard for users to communicate their problems. But you make this really hard. You did not even mention that you have one of the baking options enabled in your first posting - and I am now only inferring this from your postings - and then continue with wild statements ("everything is broken") or arguing with me.

It is obviously not impossible that there is a bug in Cinema 4D. But you have to make yourself understood. We will not go on wild goose hunts for bugs without clear instructions on what is not working and reproduction steps. In your case that could be a scene file and a complete Script Manager script which renders that file to an image on disk, which then looks different.

So, please let's start over here, and provide a clear (and ideally short) step -by-step instruction of what is going wrong for you, accompanied by meaningful executable code.

Cheers,
Ferdinand

hSchoenberger

Hi

We will not go on wild goose hunts for bugs without clear instructions on what is not working and reproduction steps

At first, I just wanted to know if I am missing any flag/option in the script.
Of course I would have send you repro steps if it turns out to be a bug. But it got into a discussion not about how C4D works, more about how OCIO should work.

You actually have to cooperate with us

I am trying. But it is difficult.
You are telling me again and again that I do not know anything about OCIO!

Have you thought that what you stated might be slightly wrong?
e.g. your first statement that started the theory versus reality discussion:

A rendering does not contain the view transform, as this would basically be against everything OCIO is about

As I cannot argue with information. Simple Repro steps to show why I told you you are not 100% right:

Open C4D, new scene.
Add some object.
Set a filename for "Regular Image", but keep all other settings to the default.
Render to disk in any way.
The View Transform is baked. And I have NOT enabled any setting. This is the default behavior of C4D. And you cannot disable it.

Cinema 4D is not baking in the view transform everywhere

As you can see in this example.

Back to the main issue "script versus other C4D functionality"
Reproduction steps:
250306_OcioScript.zip

Open the scene.

Render with Baked OCIO:
2) Execute "Render to Picture Viewer" (rename the output).
3) Execute "Add to Render Queue", start it (rename the output).
4) Render via commandline (rename the output).

Render without Baked OCIO:
5) Execute the script (rename the output).

Request:
No matter how C4D saves a rendered frame, the file written should always look the same.

refuse to read
which will also explain you what a view is

OK, If you want to hear it from me: The view transform was "invented" to mimic a different display type like a movie screen. This is done not only to have the option to export it for different devices, it is done to have a more physical correct compositing as well (linear space).
But - as I already stated - within your pipeline only (Your OCIO graph columns Texturing, Rendering, Editing).
Not for the final image (column Products).

About your OCIO knowledge:
Let me ask you a question about the OCIO graph you posted: In the third column "Editing" you see the last box states the render in ACEScg. A customer can not display an ACEScg image on his screen. Just try to upload an ACEScg image to Youtube, use it on any website, burn it on a blueray, ...
In your graph: The editing ACEScg box is linked into a "sRGB" and "DCI-P3" color space. How do you think ACEScg is converted into sRGB? You have to bake a transform into the final image.

continue with wild statements ("everything is broken")

That was a simple conclusion....
I never assumed that anyone thinks the right answer is "everything is broken" to the issue.... I simply showed you the alternatives.

Let me explain: You state that it does not make any sense that the view transform is baked into the frame. It is wrong to do that. It should never be applied. So such a behavior is a bug in your eyes. And I took this statement and refer to it as "broken".

BUT: C4D does that by default.
And you cannot disable it (for 8bit output).

And that gets to the conclusion:
A) Either C4D does it wrong since forever (bug/broken).
It should not bake the view transform.
B) C4D does it right.
There is a good reason C4D bakes the view transform.
Customers creates a new scene, renders it and the image saved is the same as he has seen in C4D.
And that leads to the fact that the script render does it wrong.

You did not even mention that you have one of the baking options enabled

I did not enable anything.
C4D default

hSchoenberger

So, please let's start over here,

Ok, so can we now agree that we are both professional developers and we both both about OCIO.
And get back to the issue that our end customer wants to render frames via a script and not via UI functions.
But still get the same output file.

ferdinand

Thanks, I will have a look, at glance there indeed seems something to be broken.

edit: So, there is unfortunately another bug in Render Document where the image profile of the and the render settings and render bitmap is not correctly respected in OCIO. There is no way to fix this for externals right now in Python. In C++ one might be able to fix this by using MultipassBitmap::SetTempColorProfile (to write the render settings profile like that into the bitmap), I did not try yet.

The only way to solve this for third parties right now, would be one of the other ways to invoke a rendering. Such as a simple CallCommand or using the render queue a.k.a. BatchRender in the API.

I have moved this into Bugs.

Cheers,
Ferdinand

An example for using the renderer queue (which is more or less just a fancier variant of CallCommand). What to do concretely, depends on what you want to achieve. The BatchRender is meant for rendering multiple things at once, its big 'disadvantage' is that files must be saved to disk. The advantage is that with a thread one can realize non-blocking rendering with it. With a simple CallCommand one could realize rendering volatile unsaved document changes, but a non-blocking rendering will here be impossible.

import c4d
import os
import mxutils

doc: c4d.documents.BaseDocument  # The currently active document.

def main() -> None:
    """Called by Cinema 4D when the script is being executed.
    """
    if not doc.GetDocumentPath():
        raise RuntimeError("Please save the document before rendering.")
    
    docPath: str = os.path.join(doc.GetDocumentPath(), doc.GetDocumentName())
    renderPath: str = os.path.join(doc.GetDocumentPath(), "render.jpg")
    rData: c4d.documents.RenderData = doc.GetActiveRenderData()

    # Make sure a bitmap is being saved to disk.
    rData[c4d.RDATA_SAVEIMAGE] = True
    rData[c4d.RDATA_PATH] = renderPath
    rData[c4d.RDATA_FORMAT] = c4d.FILTER_JPG

    # Save the file, the batch renderer can only render files from disk.
    c4d.documents.SaveDocument(doc, docPath, c4d.SAVEDOCUMENTFLAGS_0, c4d.FORMAT_C4DEXPORT)

    # Get the render size.
    xRes: int = int(rData[c4d.RDATA_XRES_VIRTUAL])
    yRes: int = int(rData[c4d.RDATA_YRES_VIRTUAL])
    print(f"{docPath = }, {renderPath = }, {xRes = }, {yRes = }")
    
    # Get the batch render queue.
    queue: c4d.documents.BatchRender = c4d.documents.GetBatchRender()
    if queue.IsRendering():
        raise RuntimeError("Rendering is already in progress.")
    
    # Disable all elements in it which are not the current document.
    alreadyContained: bool = False
    for i in range(queue.GetElementCount()):
        if queue.GetElement(i) == docPath:
            alreadyContained = True
            queue.EnableElement(i, True)
        else:
            queue.EnableElement(i, False)
    
    # Add the current document to the queue if it is not already in it.
    if not alreadyContained:
        queue.AddFile(docPath, 0)
        queue.EnableElement(0, True)

    # Start the rendering.
    @mxutils.SET_STATUS("Rendering...", doSpin=True)
    def doRender() -> None:
        queue.SetRendering(c4d.BR_START)
        while queue.IsRendering():
            pass
        return
    
    doRender()

    print("Done")

if __name__ == '__main__':
    main()

hSchoenberger

Hi @ferdinand

It is a blocking render using Commandline.exe.
And it always output files to disk.

BatchRender would probably change the render queue items if C4D is started by an artist as well.
And the artist might use the render queue for testing/preview.
We can perhaps relink the user prefs folder on Windows, but not for MacOS.

On the other side, CallCommand is a wrapper to emulate a button click in the UI as far as I understand.
And the UI is not available in Commandline.exe.

We have to do some tests.

ferdinand

@hSchoenberger said in c4d.documents.RenderDocument does not apply OCIO Viewtransform:

It is a blocking render using Commandline.exe [...] And it always output files to disk.

Are these questions? Renderings in themself are not blocking, as they happen in their own render thread. What can be blocking, is the action that invokes the rendering, as many things happen by default on the main thread of Cinema 4D in Python; e.g., Script Manger scripts run on the main thread. E.g., a c4d.documents.RenderDocument call on its own will only exit once the rendering is done, which makes sense as you otherwise could not interact with the finished rendered bitmap. Similar things apply to the BatchRender. But you can make the BatchRender not blocking by wrapping it in a thread (and I think you can also do that with RenderDocument in Python, but I never tried there). You cannot do this with a CallCommand, they will always be blocking. I.e., when you invoke the lines below, "Hello world!" will only be printed once the rendering is done.

c4d.CallCommand(12099)
print("Hello world!")

And more important, since this is blocking the main thread, the user could also not interact with the GUI. When you are in a headless Cinema 4D instance like c4dpy or the CommandLine (not quite sure how you use the CommandLine in this context), then this is of course not really important, as you do not care about having to wait for the renderer to finish.

On the other side, CallCommand is a wrapper to emulate a button click in the UI as far as I understand.

Sort of, but at the same time also not really. CallCommand just invokes a command, more specifically the CommandData::Execute function of the command that has the passed plugin ID. But commands are in the GUI often associated with buttons. But that does not mean that you cannot invoke commands in a headless Cinema 4D, i.e., CallCommand and co. work just fine there.

But you cannot run Script Manager scripts in the CommandLine (which you seem to imply). The only Python code which will run in a CommandLine are plugins or scripting elements. With c4dpy you can also run Python code that is not a plugin or embedded into a document in a headless environment. Unless Maxime implemented the -script argument for the CommandLine, but I do not see any indication that he did.

BatchRender would probably change the render queue items if C4D is started by an artist as well [...]

Depends on where this runs. When you implement this in a non-blocking manner, and let this run in a Cinema 4D.exe on which an artist is working, then, yes, this could lead to conflicts when incorrectly implemented (not having a proper while queue.IsRendering() loop). As the artist would have to stop the current batch to make changes, and IsRendering would then be False. But when you start another Cinema 4D.exe, or CommandLine.exe, or c4dpy.exe on the same machine, they would all have their own render queue.

They question would be how sensible such setup would be when you are using Redshift or pretty much any other GPU renderer, as they tend to try to claim all VRAM resources when initialized. So, when you start two GPU render jobs on the same machine, you end up with horrible performance in both.

I would really advise against trying to do this, letting GPU render slaves run on a machine an artist is actively working on. Not only artist and render slave renders will have terrible performance, also stuff like particles, fluids, and other simulation stuff, as they all want all your VRAM. The common practice of enlisting unused artist workstations as render slaves is of course fine.

Cheers,
Ferdinand

hSchoenberger

@ferdinand

It is a blocking render using Commandline.exe
Are these questions?

An information.
In reply to your statements "The advantage is that with a thread one can realize non-blocking rendering with it" and to use "CallCommand".

But when you start another Cinema 4D.exe, or CommandLine.exe, or c4dpy.exe on the same machine,
they would all have their own render queue.

But if a new instance of C4D does not use the same render queue settings, how does it come that the same render queue is availabIe if I restart C4D?
Thats what I mean. The render queue settings/items are not bound to the executable instance, they are bound to the user settings.
Which means the artist has to clear the render queue every time he starts C4D.
And he looses old items on C4D restart.
But that would be an issue on MacOS only.
And still better than wrong frames.

(not quite sure how you use the CommandLine in this context)
The only Python code which will run in a CommandLine are plugins

Yes, you did not offer to run scripts directly, therefore we have written a plugin.
Our plugin works for 11 years now, since C4D version R16.
(We still have to support older C4D versions as well, but perhaps we create some new connection with a version switch in the future. But for now adding other features is more important)

Renderings in themself are not blocking

I know, I am familiar with the multi-threading of renderers.

Your function doRender() has already implemented blocking.
So blocking is not an issue.

letting GPU render slaves run on a machine an artist is actively working on

I know. We are working with Redshift since the first beta version.
I did not mean that 2 GPUs jobs run at the same time.
(With the exception that we offer the option to split your GPUs if you have multiple installed)
But it can run at night.
And there are CPU renderer which can be assigned to some cores of the machine while an artist is working