Multithreading/waiting for render to complete

brian.michael

I am working on a script that will do some things, and then render to picture viewer, and then do more things and render to picture viewer again. I know I could save out files and submit to the render server or deadline - and that is an option - but I would like/really be nice to have this to be an option as well. So - below is a sudo code of what I have so far.

class WaitForRender(c4d.threading.C4DThread):

    def Main(self):
        bc = c4d.BaseContainer()
        time.sleep(1) #added this to see if it will wait a couple seconds for render to start before checking, but locks up everything
        while c4d.CheckIsRunning(c4d.CHECKISRUNNING_EXTERNALRENDERING):
            if self.TestBreak():
                break
            time.sleep(1)
            self.End()
        
class DoRender:
    def start_render(self):
        self.thread = WaitForRender(self)
        #wait for render to be done.
        self.thread.Start() # start up the waiting thread.
        c4d.CallCommand(self.PICTURE_VIEWER_COMMAND)  # Render to Picture Viewer
        c4d.EventAdd()  #tried this to see if it affects render/force render to start now - doesn't seem to.
        
        self.thread.Wait(False) #didnt really understand this, both true and false seem to have the same effect
        #close the thread
        thread.End()
        
    def process_shots(self):
          for loop through renderlist:
                self.start_render()

It seems Picture Viewer does not start until the whole script is finished - and holds up on the threaded sleep commands. I also tried to put the whole script into a thread, and call the picture viewer command - but picture viewer wouldn't seem to run/start when called from a threaded command. I am new to cinema, and their API's so I am probably missing something.

ferdinand

Welcome to the Maxon developers forum and its community, it is great to have you with us!

Getting Started

Before creating your next postings, we would recommend making yourself accustomed with our forum and support procedures. You did not do anything wrong, we point all new users to these rules.

Forum Overview: Provides a broad overview of the fundamental structure and rules of this forum, such as the purpose of the different sub-forums or the fact that we will ban users who engage in hate speech or harassment.
Support Procedures: Provides a more in detail overview of how we provide technical support for APIs here. This topic will tell you how to ask good questions and limits of our technical support.
Forum Features: Provides an overview of the technical features of this forum, such as Markdown markup or file uploads.

It is strongly recommended to read the first two topics carefully, especially the section Support Procedures: Asking Questions.

About your First Question

Puh, there are quite a few things here, as it is not quite clear to me what you want to achieve.

First of all, you should be aware that C4DThread is a mechanism to make things non-blocking, not a mechanism to speed up things. The threading module description goes a bit more into detail with this. Especially the terms multithreading or multiprocessing are usually tied to so-called true parallelism, C4DThread and the Python API does not offer that.
While you can technically run multiple renderings at once in most render engines, it is often not very advisable to do so these days. The primary reason for this is GPU rendering and how render instances behave - they try to grab as much VRAM as they can when being spawned. Trying to run multiple renders processes at once then results in a giant race condition with the different processes fighting over the system resources.
All rendering is by default threaded, as you could not do anything while rendering otherwise.
You misuse the type C4DThread or I do not understand what you are trying to do there. The payload of the thread, i.e., the to be threaded code has to go into C4DThread.Main or must be invoked from there. See threading_basic_13.py for a super basic example. You seem to be under the impression that all the code you sandwich between your self.thread.Start() and thread.End() call will be threaded, e.g., your c4d.CallCommand call. That is not the case.
GUI operations will always run on the main thread. There are also certain other things you cannot do off main thread, like for example invoke an event. See the Threading Manual for details.
Any main thread call to a plugin or script is blocking, that is why it is the main thread There are two large categories of how functions in your code can be called by Cinema: On-main-thread and off-main-thread. A script manager script, a CommandData.Execute, everything in a Gedialog, or (most of the time) a NodeData.Message will for example run entirely on the main thread. Everything that is tied to scene evaluation, e.g., a ObjectData::GetVirtualObjects, a TagData::Execute, or the main() function in a Python Generator object will run in parallel off main thread. Main thread functions are by definition blocking, no matter how many EventAdd or DrawViews you sprinkle in, everything is paused while your code runs (that is why your are not bound by the threading restrictions on the main thread). Only when you give control back to Cinema 4D, when you return your function call, the machinery that is Cinema 4D resumes its work and starts chweing on your modifications and inputs.

When you just want to run multiple renderings in succession, the right tool would be either using c4d.documents.BatchRender or c4d.documents.RenderDocument. There is also the teams/net interface, but that is much more complicated. It seems likely that you want to use the BatchRender here, as it is a tool to 'Run renderings as if you were a user'. RenderDocument is more a means to programmatically access the output of rendering a document, and does not one-to-one emulate the process of rendering a document/scene as a user might experience it.

Finally, I simply do not understand what you are trying to do here. When you just want to run some command, you do not need a thread to check for it to finish. You are on the main thread, things are blocking by nature, your code will only advance once the CallCommand is done. I have attached at the end an example for threading.

Cheers,
Ferdinand

Result

{0: 0.5731085252403424, 1: 0.9450752097714332, 2: 0.647703922179293, 3: 0.9509944959671278, 4: 0.5241290607800987}

Code

import c4d
import random
import time

class ComputeThread(c4d.threading.C4DThread):
    """Realizes a thread that computes a result.
    """
    def __init__(self, num: int):
        """Initializes the thread and the result.
        """
        c4d.threading.C4DThread.__init__(self)
        self._num: int = int(num)
        self._result: float | None = None

    def Main(self):
        """Runs the thread.

        All code that is executed in the thread is in this method or called from this method.
        """
        # We wait a bit to simulate a computation that takes some time and then use a random number 
        # as result. Python's random module is not thread-safe, but in this case it does not matter.
        time.sleep(3)
        self._result: int = random.random() 

def main() -> None:
    """Some execution context on the main thread, here a script manager's script main function.
    """
    # We create a bunch of threads and start them.
    threadPool: list[ComputeThread] = [ComputeThread(i) for i in range(5)]
    for thread in threadPool:
        thread.Start()

    # We wait for all threads to finish. This checking of IsRunning() is the key pattern of using 
    # threads. Sometimes people also talk about joining threads, but in the end this just means
    # that we wait on the main thread for our parallel threads to finish (which are not truly 
    # parallel in Python). This pattern is usually using a while loop somewhere to either check
    # an IsRunning() method or to check if a pool of threads is empty.
    results: dict[int, float] = {}
    # We loop over our thread pool until it is empty.
    while threadPool:
        # First we check if any thread is done and store the result.
        for thread in threadPool:
            if not thread.IsRunning():
                thread.End()
                results[thread._num] = thread._result
        # Then we remove all threads that are done from the pool.
        threadPool = [thread for thread in threadPool if thread.IsRunning()]
        # We wait a bit to not spam the threads with checks.
        time. Sleep(1)
            
    # Once the loop finishes, all threads are done and we can work with the results.
    print(results)

if __name__ == "__main__":
    main()

brian.michael

The goal of the script is that there will be multiple items to set in my script, that then will take those items and loop through rendering. Batch, and deadline are on the list, but I also wanted something that the user is use as they use it all the time - the picture viewer. It would be sorta like Render all takes, but I would set the options between takes/ a bunch of different settings.

In my script I was thinking that I would start a thread that will wait and watch the Picture viewer, and when done change some settings and do it again. I was trying to thread the waiting part of the script, and not the running picture viewer part. Problem is that the picture viewer doesn't do anything until the main thread is done - which makes sense. but when the main thread is done, it seems to kill my threaded thread if I remove thread.Wait() and thread.End().

Looking into c4d.documents.RenderDocument looks promising but I still cant find how to get the feedback of seeing the render as it renders in picture viewer. Played quickly with RENDERFLAGS_CREATE_PICTUREVIEWER, and RENDERFLAGS_OPEN_PICTUREVIEWER - but neither opened the picture viewer. Only thing that did was - c4d.bitmaps.ShowBitmap(bmp) after the single frame was done.

ferdinand

Hey @brian-michael,

Thank you for the additional information. To run multiple renders in succession, you should use the batch renderer. RenderDocument is not the right tool for what you want to do, as already hinted at in my last posting. Both the batch renderer and RenderDocument do not allow you to inspect the image progress of a rendering, such information is not exposed by the API.

I can also only stress again that as long as your functions runs (given it is a main thread function), no GUI operation can occur. The main thread is called main thread because it is a singular lane. When your code runs, nothing else can run. To me it still sounds a bit like that you do not yet understand that.

You can have a look at this thread for a more concrete example of how to make something not blocking with threads. But all this threading is overkill in this case IMHO. All you need, is a GeDialog opened as DLG_TYPE_ASYNC and there hook into the timer and use the timer to check whatever you want to check (has the rendering finished for example).

Cheers,
Ferdinand

Result

Code

"""Demonstrates how to periodically check something with a dialog.
"""
import c4d

doc: c4d.documents.BaseDocument  # The currently active document.
op: c4d.BaseObject | None  # The primary selected object in `doc`. Can be `None`.

class RenderDialog(c4d.gui.GeDialog):
    """Realizes a dialog which starts a rendering in the PV and catches its end.
    """
    ID_BUTTON: int = 1000  # The ID of the button in the dialog.

    def CreateLayout(self) -> bool:
        """Called by Cinema 4D to create the layout of the dialog.

        We just add a button to the dialog.
        """
        self.SetTitle("Render Dialog")
        self.AddButton(self.ID_BUTTON, c4d.BFH_CENTER, name="Render")

        return True
    
    def Command(self, id: int, msg: c4d.BaseContainer) -> bool:
        """Called by Cinema 4D when a button is clicked in the dialog.
        """
        if id == self.ID_BUTTON and not c4d.CheckIsRunning(c4d.CHECKISRUNNING_EXTERNALRENDERING):
            c4d.CallCommand(12099)
            self.SetTimer(250)

        return True
    
    def Timer(self, msg: c4d.BaseContainer) -> bool:
        """Called by Cinema 4D when the timer is triggered.
        """
        if not c4d.CheckIsRunning(c4d.CHECKISRUNNING_EXTERNALRENDERING):
            self.SetTimer(0)
            c4d.gui.MessageDialog("Rendering complete!")

        return True


dlg: RenderDialog  = RenderDialog()
if __name__ == '__main__':
    # Please do not do what I am doing here in production code, open an async dialog from a script
    # manager script. This is just for demonstration purposes, as it will result in a dangling dialog.
    # In production code, you must have an owning entity for such async dialog, e.g., a command plugin.
    dlg.Open(c4d.DLG_TYPE_ASYNC, defaultw=200, default=100)

brian.michael

Thank you so much! That was it. I was thinking I would try to create this standalone, before moving it into creating a GUI, and I didn't know about the Timer in GeDialog. This is now working, and all I needed to do is remove the multithreading and change it to the timer.