Python Api: Document Events

[paulgolter]

Hey all,

My goal is pretty simple:

In Python I want to execute callback functions for a couple of different events. Mostly document events such as:

• Document Saved
• Document Loaded
• Object / Node created
• Object / Node deleted

I am very confused how I can achieve this.

I understand that cinema has different kinds of plugins such as Message, Object, CommandData plugins and more. Each of those plugins has some function which you can re-implement to receive messages. So far so good.

I tried to use all of those plugins, but could not achieve my goal.
From what I read in the message system manual it seems like i would need to listen to the event with the id: c4d.MSG_DOCUMENTINFO, which can be received among other plugins by a CommandData plugin.

My super simple CommandData plugins looks like this:

class MyCommandDataPlugin(c4d.plugins.CommandData):
    def Message(self, type: int, data: Any):
        if type == c4d.MSG_DOCUMENTINFO:
            logger.debug("Document event: %i data: %s", type, data)
        return True

This does not log anything if I save / open a document tough.

The next topic, I could not event test yet, is that on top of the 'generic' c4d.MSG_DOCUMENTINFO event type there seem to be more document specific events such as:
c4d.MSG_DOCUMENTINFO_TYPE_SAVE_AFTER
c4d.MSG_DOCUMENTINFO_TYPE_LOAD

I have no idea where and how I can check for those either.

I would really appreciate some help on this topic.

[ferdinand]

Hey @paulgolter,

Welcome to the Maxon developers forum and its community, it is great to have you with us!

Getting Started

Before creating your next postings, we would recommend making yourself accustomed with our forum and support procedures. You did not do anything wrong, we point all new users to these rules.

• Forum Overview: Provides a broad overview of the fundamental structure and rules of this forum, such as the purpose of the different sub-forums or the fact that we will ban users who engage in hate speech or harassment.
• Support Procedures: Provides a more in detail overview of how we provide technical support for APIs here. This topic will tell you how to ask good questions and limits of our technical support.
• Forum Features: Provides an overview of the technical features of this forum, such as Markdown markup or file uploads.

It is strongly recommended to read the first two topics carefully, especially the section Support Procedures: Asking Questions.

About your First Question

Okay, there is quite a bit to unpack here and I am probably not going to answer everything in one posting. The most important thing would be for me: What are you actually trying to do? There is probably an easier, or more "maxonic" way of doing what you are trying to do.

The good thing is that you have read the message system manual, and that you have found this posting of mine.

• To detect generic scene changes you will have to listen for EVMSG_CHANGE and track changes yourself as I have shown there. This has not changed.
  • There are technically the BaseList2D.AddEventNotification methods, but they are private for a reason as you can quite easily crash Cinema with them. And you also need to own a node implementation for them, as you are notified via NodeData::Message. Most importantly, you cannot track objects being added this way (you cannot attach to something that does not yet exist), and you are only notified when the actual object is deleted, not when its parent is (and with it your node). So, this is very likely not the right option for you, and instead you must use EVMSG_CHANGE.
• While CommandData technically receives node messages, it is only a very small subset which is being broadcasted to command data (which makes sense as a command is not derived from NodeData). You will not be able to receive MSG_DOCUMENTINFO messages in a command. You will need a proper node for this, such as an object, tag, material, shader, or scene hook (C++ only). A scene hook would be the natural choice for you are probably trying to do. Note that some NodeData type plugins such as a PreferenceData plugin will not receive this message either (because someone once thought this would make sense).
• As to how message families work, see my code example below.

So, long story short, to react to scene changes, you will need an entity that implements a CoreMessage method such as a MessageData plugin. To react to a scene being loaded or saved, you would normally use a SceneHookData plugin (sort of a MessageData plugin for node the message stream - not quite true but close enough). In Python you do not have this option, and would have to resort to hacks like adding a hidden object or tag to scene to do your bidding. Which can become a bit complicated. It is probably better that you tell us what you want to do.

Cheers,
Ferdinand

PS: The messages are also documented, which will explain you how the message data is composed. It also tells you if the message can be sent, recieved or both. I.e., if you can listen for it in NodeData.Message, or if you can send it with for example C4DAtom.Message.

Result:

Code:

"""Demonstrates the idea of message families in the classic API of Cinema 4D.

Place this code in a Python Generator object and run the script. And save the scene to see the output.
"""

import c4d
from mxutils import CheckType

def main() -> c4d.BaseObject:
    """Called by Cinema 4D to retrieve the cache of the Python Generator object.
    """
    return c4d.BaseObject(c4d.Ocube)

def message(mid: int, mdata: any) -> bool:
    """Called by Cinema 4D to convey events to the node.

    This is the counterpart to NodeData.Message() in a full blown plugin.
    """
    # There are multiple message families in the classic API of Cinema 4D, as for example
    # MSG_DOCUMENTINFO, MSG_MULTI_RENDERNOTIFICATION, MSG_MULTI_SCRIPTINFO and many more. The idea
    # of a message family is that you are notified about the general event, e.g.,"something with
    # the document happened", and then must retrieve the more specific event from the message data.
    # How this is done varies from event to event, because while the event data is usually a dict
    # in Python, in C++ the event data are custom structs or classes.
    if mid == c4d.MSG_DOCUMENTINFO:
        # Get the message subtype from the message data, #CheckType just makes sure that the data
        # is indeed a dict.
        mtype: int = CheckType(mdata, dict).get("type", c4d.NOTOK)
        if mtype == c4d.NOTOK:
            raise RuntimeError(f"Encountered invalid message data in: {mdata}")

        # Now we can handle the specific event.
        if mtype == c4d.MSG_DOCUMENTINFO_TYPE_SAVE_BEFORE:
            print ("Before save")
        elif mtype == c4d.MSG_DOCUMENTINFO_TYPE_SAVE_AFTER:
            print ("After save")
        # ...

    return True

[paulgolter]

Hey @ferdinand,

Thanks for your detailed answer first of all!

To give a simplified background of what I am trying to do:
I have a server on the web that stores information about my current cinema session. It should be as reactive as possible. The idea is to update the information on that server sparsely once certain cinema events are triggered to avoid having to send a full update in a specified time interval.

You pointed out the EVMSG_CHANGE event. As I understand this is being triggered on almost all events (eg. if c4d.EventAdd() is being called). This would be way to spammy for me and I would resort using a timer with a fixed interval and scan the scene for the information that I am interested in and code my own sparse change detection logic.

• You will not be able to receive MSG_DOCUMENTINFO messages in a command

Thanks for the info, I think I was confused about this because in the doc it says:

The following messages can be received in the Message methods of CommandData, FalloffData, NodeData, SculptBrushToolData, and ToolData interfaces and their derived interfaces.

• You will need a proper node for this, such as an object, tag, material, shader, or scene hook (C++ only)

I have read about the scene hooks as well, unfortunately as you said, they are only available in C++.

Adding a 'custom hidden' node that is able to catch some of the events that I am interested in sounds like a solution to me tough. At least for some events.

Could you give me a headstart how I would go about adding a custom node in my plugin that can catch events and making sure this node always exists when I start up cinema or load a new scene?

All the best

[ferdinand]

Hey,

You pointed out the EVMSG_CHANGE event. As I understand this is being triggered on almost all events (eg. if c4d.EventAdd() is being called). This would be way to spammy for me and I would resort using a timer with a fixed interval and scan the scene for the information that I am interested in and code my own sparse change detection logic.

Yes, EVMSG_CHANGE is spammy, and it is not 'like' EventAdd, it IS EventAdd, or more precisely the message that signals that the event queue has been consumed. The only way to do what you want to do to, is to write something like I have shown in the light lister thread. There is unfortunately no free lunch here. This is the common way to implement this, I would advise against a timer based approach.

Thanks for the info, I think I was confused about this because in the doc it says:

The following messages can be received in the Message methods of CommandData, FalloffData, NodeData, SculptBrushToolData, and ToolData interfaces and their derived interfaces.

Yeah, I understand. I wrote these docs, and I will clarify a bit the scope of CommandData::Message. But in general, our message system is not too well documented, and I cannot so easily reverse engineer all that information. I know for example that CommandData receives MSG_COMMANDINFORMATION and that a c4d.plugins.PreferenceData does not receive MSG_DOCUMENTINFO. But there are literally 100's of these micro-decisions in the message system which have never been documented. When you work with messages, you must embrace the workflow of simply checking which messages you get and which you do not (just print out the message IDs and data in a Message function; that is why the docs list the int values of messages).

Could you give me a headstart how I would go about adding a custom node in my plugin that can catch events and making sure this node always exists when I start up cinema or load a new scene?

Well, I cannot write a full example, but the general idea would be:

1. Implement some kind of node, an object is probably the best choice as it is easy to inject. Let's call it MyObject which is of type c4d.plugins.ObjectData. There you implement NodeData.Message and handle whatever message you want to handle.
2. Now you need something that (a) injects that node into every scene and (b) makes sure that this is only done once. I.e., you have to fake a scene hook, because that is more or less what a scene hook does, it is a node type which is present exactly once in every scene. There is no way to do this in a really nice manner, you will always end up with something semi-hacky. I would use a MessageData plugin and then check on each EVMSG_CHANGE if the active document contains an instance of my little dummy object. With NBIT_OHIDE you can hide an object from the object manager.

The devil is here in the detail, as you might want for example propagate information from your dummy node to other entities. Which is all not rocket science when you are an experienced developer, but I see people often struggle with this: Propagating data in a scene. We just very recently had this topic where we talked about this very subject.

They key thing is to get here started, you can start from any direction, the MessageData spider in the web, or the little derpy dummy node. You could for example try to write a MessageData plugin which injects a Cube object (with a special name or a unique ID - when you want to do it 'right' - to make it unique) into every open scene exactly once. And then later replace the cube with your actual payload. That is at least how I would start.

We will help you when you get stuck, but you must show us executable code. And in the end you must be able to climb this hill on your own, we will help you along the way, but you must take the steps yourself.

Last but not least, I would contemplate if that is all really what you want and have to do. Your task description was still a bit cloudy with:

I have a server on the web that stores information about my current cinema session. It should be as reactive as possible. The idea is to update the information on that server sparsely once certain cinema events are triggered to avoid having to send a full update in a specified time interval.

This sound a bit like that you want to have a full web view of a document. Which is possible in Python, but not a trivial task as you will also need intimate knowledge of our scene graphs (2025 will make this a bit easier, as we finally added there abstracted scene traversal).

To simplify the task, you could for example say that this does not work automatically in each scene, but that the user must add a special WebConnector object to the scene which contains controls for how and what is synchronized to the webserver. This would be a more 'maxonic' way to tackle things, as Cinema 4D is very (generator) object centric. You could then still implement another plugin which allows the user to toggle such object being added by default to each open scene.

Cheers,
Ferdinand

[paulgolter]

Hey @ferdinand

Thanks again for the detailed answer. I was able to solve it somehow with your tips.

The solution I have now is a combination of 2 systems.

01. Object Cache Refresher

I have a c4d.plugins.MessageData plugin that runs every 2 seconds and compares a cached list of object ids with the current objects ids in the scene. That way I can detect what was deleted and added.

I am using the creator id of each object that you described in this post which was useful.

class ObjectCacheRefresher(c4d.plugins.MessageData):

    def GetTimer(self):
        return 2000

    def CoreMessage(self, id: int, bc: c4d.BaseContainer):
        if id == c4d.MSG_TIMER:
            refresh_object_id_cache()
        return True

02. Custom Node

On top of this I added a custom node that can listen to document events in its Message function.

class CustomNode(c4d.plugins.ObjectData):

    def Message(self, node: c4d.GeListNode, type: int, data: Any):
        if type == c4d.MSG_DOCUMENTINFO:

            logger.debug("Document event: %i data: %s", type, data)

            if data["type"] == c4d.MSG_DOCUMENTINFO_TYPE_LOAD:
                on_scene_loaded()

            if data["type"] == c4d.MSG_DOCUMENTINFO_TYPE_SAVE_AFTER:
                on_scene_save()

            if data["type"] == c4d.MSG_DOCUMENTINFO_TYPE_SETACTIVE:
                on_scene_activate()

            if data["type"] == c4d.MSG_DOCUMENTINFO_TYPE_MERGE:
                on_scene_merge()
        return True

This node needs to be present in the scene tough. I solved this as you suggested with an other MessageData plugin that ensures this is node is added in the scene once.

This plugin also uses a timer instead of the EVMSG_CHANGE event, as this was sufficient for me and feels less spammy.

With custom_node.ChangeNBit(c4d.NBIT_OHIDE, c4d.NBITCONTROL_SET) i could hide the node as you noted.

class NodeAdder(c4d.plugins.MessageData):

    def GetTimer(self):
        return 2000

    def CoreMessage(self, id: int, bc: c4d.BaseContainer):
        if id == c4d.MSG_TIMER:
            ensure_event_listener_node()
        return True

TODO

What I am currently still trying to figure out:

1. Detect object rename events. In the current approach, I am using the creator id which does not seem to change when the node is renamed.

2. Detect parameter changes. I would like to detect if certain paramters of certain nodes changed . Including user data parameters.

If you have any input on these 2 points I would love to hear them.

Thanks again

[ferdinand]

Hey @paulgolter,

good to hear that you found your solution. A timer is however not necessary from the Cinema side of things when you implement the scene traversal correctly; you can just evaluate the scene on each EVMSG_CHANGE and there comes no big performance penalty with that.

1. I assume you mean MAXON_CREATOR_ID with 'the creator id', right? I am not quite sure how the rest of the sentence is meant as it is the whole purpose of this marker to never change. Not only for renaming events, but also everything else. MAXON_CREATOR_ID is just an quasi-alias for GeMarker (C++ only)*. A marker identifies an object over its creation UNIX epoch time, the mac-address of the machine it was created on, and some secret sauce. MAXON_CREATOR_ID is a UUID/GUID that encodes the identity of a node in the sense of what a user would consider "the same node". A cube object a user created five days ago in the document "foo.c4d" is in the eyes of the user still "the same" node, no matter how often the document has be reloaded, how often the user changed the parameters of the cube, and how often Cinema 4D shoved data round in memory. A GeMarker or MAXON_CREATOR_ID are persistent over reallocation boundaries (Cinema reallocates things in memory a lot), serialization boundaries, and of course also data write boundaries. I described the subject in the other thread at length. The MAXON_CREATOR_ID is literally the 16b memory dump of the marker.
2. Take a look at the script from the other thread, it also does exactly that, including capturing when a light is being renamed.

Cheers,
Ferdinand

*: The warning in the docs there is not true, markers do NOT change over documents being reloaded. Cloning a node will however change its marker, unless one uses COPYFLAGS_PRIVATE_IDENTMARKER. I probably should fix that, not sure who has put that there.

Cloning with COPYFLAGS_NONE
Original: 2cf05d77cc4718072431ab4c00000000
Clone   : 2cf05d77cc4718072d3bab4c00000000
Equal   : False
Cloning with COPYFLAGS_PRIVATE_IDENTMARKER
Original: 2cf05d77cc4718072431ab4c00000000
Clone   : 2cf05d77cc4718072431ab4c00000000
Equal   : True

import c4d
import mxutils

op: c4d.BaseObject | None  # The primary selected object in `doc`. Can be `None`.

def main() -> None:
    """Called by Cinema 4D when the script is being executed.
    """
    if not op:
        raise ValueError("Please select an object.")

    uuid: str = bytes(op.FindUniqueID(c4d.MAXON_CREATOR_ID)).hex()

    # Normal cloning will entail a new UUID, as what is happening under the hood, is that a new node is allocated,
    # and then 'its data' is copied over from the original node. Which will not entail the marker to be copied over.
    other: c4d.BaseObject = mxutils.CheckType(op.GetClone(c4d.COPYFLAGS_NONE), c4d.BaseObject)
    otherUuid: str = bytes(other.FindUniqueID(c4d.MAXON_CREATOR_ID)).hex()

    print ("Cloning with COPYFLAGS_NONE")
    print (f"Original: {uuid}")
    print (f"Clone   : {otherUuid}")
    print (f"Equal   : {uuid == otherUuid}")

    # But with COPYFLAGS_PRIVATE_IDENTMARKER, the marker is copied over and the UUID will stay the same. This is more
    # or less what Cinema 4D does when its shoves nodes around in memory and wants them to stay "the same".
    otherDeep: c4d.BaseObject = mxutils.CheckType(op.GetClone(c4d.COPYFLAGS_PRIVATE_IDENTMARKER), c4d.BaseObject)
    otherDeepUuid: str = bytes(otherDeep.FindUniqueID(c4d.MAXON_CREATOR_ID)).hex()

    print ("Cloning with COPYFLAGS_PRIVATE_IDENTMARKER")
    print (f"Original: {uuid}")
    print (f"Clone   : {otherDeepUuid}")
    print (f"Equal   : {uuid == otherDeepUuid}")

    # You can save this file and the scene you run it on (and close and reopen it), the output for #op and #otherDeep 
    # will always be the same.


if __name__ == '__main__':
    main()

[paulgolter]

Hey @ferdinand

A timer is however not necessary from the Cinema side of things when you implement the scene traversal correctly; you can just evaluate the scene on each EVMSG_CHANGE and there comes no big performance penalty with that.

Can you be more specific how one does scene traversal correct or incorrect in python?

For example to get all object of a certain type I currently have this code:

def recurse_hierarchy(
    obj: c4d.BaseObject, hierarchy_list: List, obj_type: Optional[int] = None
) -> None:
    while obj:
        if not obj_type or obj.CheckType(obj_type):
            hierarchy_list.append(obj)
        recurse_hierarchy(obj.GetDown(), hierarchy_list, obj_type)
        obj = obj.GetNext()

# Get all cameras 
doc = c4d.documents.GetActiveDocument()
obj_list = []
recurse_hierarchy(doc.GetFirstObject(), obj_list, obj_type=c4d.Ocamera)

About the efficieny I was thinking if I traverse the scene everytime EVMSG_CHANGE is being fired i do it way more often compared to doing it every 3 seconds or so, which would be enough for me. Correct me if I'm wrong!

I assume you mean MAXON_CREATOR_ID with 'the creator id', right?

Yes that is what I meant. Thanks for the detailed explanation.

Take a look at the script from the other thread, it also does exactly that, including capturing when a light is being renamed.

Ah yes I kind of missed the part where you also store a hash sum of each object in the scene cache with: obj.GetDirty(c4d.DIRTYFLAGS_DATA).

I implemented this as well, and if anything changes with the node I re-query the node information I am intersted in, which works. Thanks!

[ferdinand]

Hey @paulgolter,

yes, this is okay. What you should not do, is traverse the whole scene graph, i.e., every object, shader, material, scene hook, track, curve, key etc that are to be found in a scene; just to get hold of all cube objects for example. For performance critical operations, we should always only traverse what we really need . But you just traverse the objects which is totally fine.

I personally would avoid recursion, passing around a list, and not using an iterator, as this all makes things slower. Especially recursion/function calls are not cheap in Python. For the next major release we added some premade scene traversal functions in Python. I have attached one of the functions which will be exposed below.

But your function is probably fine too, this is all very nitpicky. You do not use full recursion which is the most important thing. Everything else is probably nerd-talk.

Cheers,
Ferdinand

def IterateTree(node: c4d.GeListNode | None, yieldSiblings: bool = False, 
                rewindToFirstSibling: bool = False, rewindToRoot: bool = False) -> typing.Iterator[c4d.GeListNode]:
    """Yields nodes that are hierarchically related to `node`.

    .. note:: This function walks a node tree in a purely iterative manner, which makes it fast and robust for medium to
        large trees. For small trees (<= 100 nodes) `RecurseTree` is slightly faster. This function is usually the 
        faster and safer choice compared to `RecurseTree` as the performance gain for large trees is significant 
        while the loss for small trees is negligible.

    .. note:: To also yield non-hierarchical relationships, use `RecurseGraph` instead.

    :param node: The root node to start iterating from.
    :type node: c4d.GeListNode or None
    :param yieldSiblings: Whether to yield the next siblings of the current node, defaults to `False`.
    :type yieldSiblings: bool, optional
    :param rewindToFirstSibling: Whether to rewind the node to its first sibling before the iteration 
        is carried out, defaults to `False`.
    :type rewindToFirstSibling: bool, optional
    :param rewindToRoot: Whether to rewind the node to its root before the iteration is carried out,
        defaults to `False`.
    :type rewindToRoot: bool, optional
    :return: An iterator that yields the descendants of the passed `node`.
    :rtype: typing.Iterator[c4d.GeListNode]
    :raises RuntimeError: If the UUID of a node could not be retrieved.

    :Example:

    .. code-block:: python

        import c4d
        from mxutils import CheckType, IterateTree

        # For the object tree:
        #
        #   A
        #   |___B
        #   |   |___C
        #   |   |___D
        #   |   |   |___E
        #   |   |___F
        #   G

        # Find the object named "D" in the scene.
        d: c4d.BaseObject = CheckType(doc.FindObject("D"))

        # Will yield D and E.
        for node in IterateTree(d):
            print (node)

        # Will yield D, E, and F. But not C as we start out at D and #yieldSiblings only looks
        # downwards, at next siblings and not previous ones.
        for node in IterateTree(d, yieldSiblings=True):
            print (node)

        # Will yield C, D, E, and F. Because we rewind to the first sibling of D - which is C.
        for node in IterateTree(d, yieldSiblings=True, rewindToFirstSibling=True):
            print (node)

        # Will yield A, B, C, D, E, and F. But not G, as we do not yield siblings.
        for node in IterNodeTree(d, rewindToRoot=True):
            print (node)

        # Will always yield the whole tree, no matter where we start.
        for node in IterateTree(d, True, True, True):
            print (node)
        
    """
    def iter(node: c4d.GeListNode) -> typing.Iterator[c4d.GeListNode]:
        """Iterates over the descendants of the passed `node`.
        """
        # The visited nodes and the node to stop iteration at.
        visited: dict[bytes, c4d.GeListNode] = {}
        terminal: c4d.GeListNode = node

        # Walking a tree iteratively is faster for medium to large trees than a recursive or 
        # semi-recursive traversal. See: https://developers.maxon.net/forum/topic/13684
        while node:
            # We could use GeListNode.__hash__ here, but it turns out that it is much slower than
            # doing it manually with MAXON_CREATOR_ID (sic!). Probably because of the overhead of
            # __hash__ hashing the UUID into an integer?
            nodeUuid: bytes = bytes(node.FindUniqueID(c4d.MAXON_CREATOR_ID))
            if nodeUuid is None:
                raise RuntimeError(f"Could not retrieve UUID for {node}.")

            # Yield the node when it has not been encountered before.
            if visited.get(nodeUuid) is None:
                yield node
                visited[nodeUuid] = True

            # Walk the graph in a depth first fashion.
            getDown: c4d.GeListNode | None = node.GetDown()
            getNext: c4d.GeListNode | None = node.GetNext()
            getDownUuid: bytes | None = getDown.FindUniqueID(c4d.MAXON_CREATOR_ID) if getDown else None
            if getDown and getDownUuid is None:
                raise RuntimeError(f"Could not retrieve UUID for {getDown}.")
            
            if getDown and visited.get(bytes(getDownUuid)) is None:
                node = getDown
            elif node == terminal:
                break
            elif getNext:
                node = getNext
            else:
                node = node.GetUp()

    # --- End of iter() ----------------------------------------------------------------------------

    if not isinstance(node, c4d.GeListNode):
        return
    
    # Set the starting node.
    # if rewindToRoot:
    #    node = GetRootNode(node)

    # if rewindToFirstSibling:
    #    node = GetFirstSiblingNode(node)

    # Traverse the hierarchy.
    while node:
        yield from iter(node)
        if not yieldSiblings:
            return
        
        node = node.GetNext()