Issue collecting all the shaders in a material

John_Do

Hi,

I'm writing a function to get all the shaders in a material, but I have a specific case where it doesn't work as expected and skip some shaders in the process.

Here is the code :

import c4d


def iter_tree(doc, node, shaders=None):
    if shaders is None:
        shaders = []
    if node.GetChildren():
        for i in node.GetChildren():
            if i not in shaders:
                shaders.append(i)
            iter_tree(doc, i, shaders=shaders)
    return shaders


def main():
    doc = c4d.documents.GetActiveDocument()
    mtls = doc.GetActiveMaterials()

    doc.StartUndo()

    if mtls:
        for mtl in mtls:
            shaders = []
            # Looking for Corona Physical Mtl or Corona Legacy Mtl
            if mtl.CheckType(1056306) or mtl.CheckType(1032100):
                mtl_bc = mtl.GetDataInstance()
                for id, value in mtl_bc:
                    if isinstance(value, c4d.BaseShader):
                        node = value
                        if node not in shaders:
                            shaders.append(node)
                        if node.GetDown():
                            nodes = iter_tree(doc, node)
                            shaders.extend(nodes)

                print([i.GetName() for i in shaders])

    doc.EndUndo()
    c4d.EventAdd()


# Execute main()
if __name__ == '__main__':
    main()

And here is the material's shader tree :

You can see in the ouput that the top branch of the Fusion shader is missing ( AO and Filter are missing in the list below) :

['Fusion', 'Color', 'CoronaNormal', 'Projector', 'Normal', 'Roughness']

If I target specifically the Fusion shader, the output miss the branch connected to the Blend Channel :

['Fusion', 'Color']

Two important notes :

If I loop over the shaders description ids and values, the children are right there

mtl = <c4d.BaseMaterial object called Material/Corona Physical with ID 1056306 at 2329339889344>

Fusion base container data listed below :

íd:1002, value:1.0
íd:1001, value:2003
íd:1008, value:<c4d.BaseShader object called Color/Color with ID 5832 at 2329339846464>
íd:1006, value:<c4d.BaseShader object called Filter/Filter with ID 1011128 at 2329339886144>
íd:1007, value:None
íd:9101004, value:0
íd:9101001, value:128
íd:9101002, value:128
íd:1041671, value:Vector(1, 0.9, 0.4)
íd:520000000, value:<class 'AttributeError'>
íd:1003, value:0

but if I use the Corona Add Existing... command which open a tree view of the material, the missing shaders are missing here too

So should I process the shaders description's values to collect the shaders rather than iterating over the material's shader tree ?

Thank you

HerrMay

Hi @John_Do,

I could imagine that you stumble upon the same problem I did back then. Have a look at this post.

@ferdinand helped me back then. The key gotcha was that I didn't account for the branching nature of a shader tree. I have pasted the code to succesfully traverse such shader tree down below.

Cheers,
Sebastian

import c4d

doc: c4d.documents.BaseDocument  # The active document

def TraverseShaders(node: c4d.BaseList2D) -> c4d.BaseShader:
    """Yields all shaders that are hierarchical or shader branch descendants of #node.

    Semi-iterative traversal is okay in this case, at least I did not bother with implementing it fully
    iteratively here.

    In case it is unclear, you can throw any kind of node at this, BaseObject, BaseMaterial, 
    BaseShader, etc., to discover shaders which a descendants of them.
    """
    if node is None:
        return

    while node:
        if isinstance(node, c4d.BaseShader):
            yield node

        # The shader branch traversal.
        for descendant in TraverseShaders(node.GetFirstShader()):
            yield descendant 

        # The hierarchical traversal.
        for descendant in TraverseShaders(node.GetDown()):
            yield descendant

        node = node.GetNext()
        
def main() -> None:
    """
    """
    material: c4d.BaseMaterial = doc.GetActiveMaterial()
    print (f"{material = }\n")
    for shader in TraverseShaders(material):
        print (shader)

if __name__ == '__main__':
    main()

m_adam

Hi @John_Do the answers provided by @HerrMay is correct thanks a lot to him.

If this does not work I would invite you to reach the corona support teams, since Maxon can't provide support for 3rd party render and they may have some corner cases we do not know about.

Cheers,
Maxime.

John_Do

Thanks you for the link and the code @HerrMay !

These functions are working fine at first, no shader is missing, pretty much the contrary !
It seems it is 'branching' a bit too much and output all the shaders in the scene and not just the ones in the active material. I guess I have to start the loop one level deeper or something similar.

EDIT
So starting the loop from the first shader in the material work, it no longer output shaders parented to other materials. But now it loops over some shaders up to 3 times, which I guess comes from the way the function traverse the shader tree.
However I'm noticing that several shaders have the same adress in memory, how can it be possible ? What does it mean ?

mtl = <c4d.BaseMaterial object called Material/Corona Physical with ID 1056306 at 1365502093504>

<c4d.BaseShader object called Fusion/Fusion with ID 1011109 at 1362487764672> # Same shader different adress
<c4d.BaseShader object called Filter/Filter with ID 1011128 at 1362487747904> # Different shader same adress 
 <c4d.BaseShader object called AO/Bitmap with ID 5833 at 1362487740608>
<c4d.BaseShader object called Color/Color with ID 5832 at 1362487747904> # Different shader same adress 
<c4d.BaseShader object called CoronaNormal/Normal with ID 1035405 at 1362487720000>
<c4d.BaseShader object called Projector/Projector with ID 1011115 at 1362487747904>
<c4d.BaseShader object called Normal/Bitmap with ID 5833 at 1362487766080>
<c4d.BaseShader object called Roughness/Bitmap with ID 5833 at 1362487747904>
<c4d.BaseShader object called Fusion/Fusion with ID 1011109 at 1362487766976> # Same shader different adress
<c4d.BaseShader object called Filter/Filter with ID 1011128 at 1362487747904>
<c4d.BaseShader object called AO/Bitmap with ID 5833 at 1362487766080>
<c4d.BaseShader object called Color/Color with ID 5832 at 1362487747904>
<c4d.BaseShader object called Roughness/Bitmap with ID 5833 at 1362487815488>
<c4d.BaseShader object called Fusion/Fusion with ID 1011109 at 1362487766976>
<c4d.BaseShader object called Filter/Filter with ID 1011128 at 1362487743936>
<c4d.BaseShader object called AO/Bitmap with ID 5833 at 1362504263360>
<c4d.BaseShader object called Color/Color with ID 5832 at 1362487743936>

John_Do

I'm not a developer but even so I'm guessing it's ugly and hacky, but I've found a solution.
Adding the elements in a set eliminate duplicate shaders and returns me the expected result.

import c4d

doc: c4d.documents.BaseDocument  # The active document

def TraverseShaders(node):
    """Yields all shaders that are hierarchical or shader branch descendants of #node.

    Semi-iterative traversal is okay in this case, at least I did not bother with implementing it fully
    iteratively here.

    In case it is unclear, you can throw any kind of node at this, BaseObject, BaseMaterial, 
    BaseShader, etc., to discover shaders which a descendants of them.
    """

    if node is None:
        return

    while node:
        if isinstance(node, c4d.BaseShader):
                yield node
            

        # The shader branch traversal.
        for descendant in TraverseShaders(node.GetFirstShader()):
                yield descendant 
                

        # The hierarchical traversal.
        for descendant in TraverseShaders(node.GetDown()):
                yield descendant 

        node = node.GetNext()
        
def main():
    """
    """
    mtls = doc.GetActiveMaterials()
    
    if mtls:
        for mtl in mtls:
            print (f"{mtl = }\n")
            # Looking for Corona Physical Mtl or Corona Legacy Mtl
            if mtl.CheckType(1056306) or mtl.CheckType(1032100):
                mtl_bc = mtl.GetDataInstance()
                shaders = set()
                for bcid, value in mtl_bc:
                    if isinstance(value, c4d.BaseShader):
                        for shader in TraverseShaders(value):
                            shaders.add(shader)
                if shaders:
                    for i in shaders:
                        print(i)
                del shaders

    

if __name__ == '__main__':
    main()

Now, if you know a better way to do this, please share it, I'm eager to learn.

HerrMay

Hi @John_Do,

the multiple loop of some shaders can indeed come from the way the TraverseShaders function iterates the shader tree. Additionally theat function has another drawback as its iterating the tree utilizing recursion which can lead to problems on heavy scenes.

Find below a script that uses an iterative approach of walking a shader tree. As I don't have access to Corona I could only test it for standard c4d materials.

Cheers,
Sebastian

import c4d

doc: c4d.documents.BaseDocument  # The active document

def iter_shaders(node):
    """Credit belongs to @ferdinand from the Plugincafe. I added only the part with the material and First Shader checking.
    
    Yields all descendants of ``node`` in a truly iterative fashion.

    The passed node itself is yielded as the first node and the node graph is
    being traversed in depth first fashion.

    This will not fail even on the most complex scenes due to truly
    hierarchical iteration. The lookup table to do this, is here solved with
    a dictionary which yields favorable look-up times in especially larger
    scenes but results in a more convoluted code. The look-up could
    also be solved with a list and then searching in the form ``if node in
    lookupTable`` in it, resulting in cleaner code but worse runtime metrics
    due to the difference in lookup times between list and dict collections.
    """
    if not node:
        return

    # The lookup dictionary and a terminal node which is required due to the
    # fact that this is truly iterative, and we otherwise would leak into the
    # ancestors and siblings of the input node. The terminal node could be
    # set to a different node, for example ``node.GetUp()`` to also include
    # siblings of the passed in node.
    visisted = {}
    terminator = node

    while node:

        if isinstance(node, c4d.Material) and not node.GetFirstShader():
           break
        
        if isinstance(node, c4d.Material) and node.GetFirstShader():
            node = node.GetFirstShader()
        
        # C4DAtom is not natively hashable, i.e., cannot be stored as a key
        # in a dict, so we have to hash them by their unique id.
        node_uuid = node.FindUniqueID(c4d.MAXON_CREATOR_ID)
        if not node_uuid:
            raise RuntimeError("Could not retrieve UUID for {}.".format(node))

        # Yield the node when it has not been encountered before.
        if not visisted.get(bytes(node_uuid)):
            yield node
            visisted[bytes(node_uuid)] = True

        # Attempt to get the first child of the node and hash it.
        child = node.GetDown()

        if child:
            child_uuid = child.FindUniqueID(c4d.MAXON_CREATOR_ID)
            if not child_uuid:
                raise RuntimeError("Could not retrieve UUID for {}.".format(child))

        # Walk the graph in a depth first fashion.
        if child and not visisted.get(bytes(child_uuid)):
            node = child

        elif node == terminator:
            break

        elif node.GetNext():
            node = node.GetNext()

        else:
            node = node.GetUp()
        
def main():
    
    materials = doc.GetActiveMaterials()
    tab = " \t"

    for material in materials:
    
        # Step over non Corona Physical materials or Corona Legacy materials.
        if material.GetType() not in (1056306, 1032100):
            continue
        
        print (f"{material = }")

        for shader in iter_shaders(material):
            print (f"{tab}{shader = }")
            
        print(end="\n")

if __name__ == '__main__':
    main()

John_Do

Thank you for the script @HerrMay it works great on a Cinema 4D Standard Material !

Without recursion I find this one easier to understand, I just had to change the class check to from c4d.Material to c4d.BaseMaterial to made it working with Corona.

One thing though I'm not sure to understand is the terminator thing, with this in the function :

terminator = node

and this a little further in the loop

        elif node == terminator:
            break

Since the terminator assignation is done out of the loop, does it mean it is equal to the material in this case, so when the loop is getting back to the first assigned element = the material, it ends ?

HerrMay

Hi @John_Do,

great to hear that I could be of help and that it works for you.

Yes, I find it myself often enough hard to understand recursive code as well. I guess it's true what they say.
"To understand recursion one must first understand recursion."

You guessed correctly. The variable terminator is doing nothing in this case. As does the equality check against node. It is (for each loop) assigned to the incoming node which is the current material that is processed in each loop cycle. So node and terminator are essentially the same in this context.

Cheers,
Sebastian

m_adam

Just in case with release 2024.2 the debugger is working for script and plugins. You can find documentation about how to set it up in https://developers.maxon.net/docs/py/2024_2_0/manuals/manual_py_vscode_integration.html#visual-studio-code-integration-manual.

Once you have the debugger attached you can then place breakpoint in your code and go step by step and inspect all variables at any time, this may help you to have a better understanding about the logic.

Cheers,
Maxime.

John_Do

@m_adam Thanks it's a nice addition, I will look into this

John_Do

Hi,

One last thing I've noticed.

The last code shared by @HerrMay works great on all materials but the one I'm using to test my script, it's odd.
On this particular material, the function consistently skip the shader which is in the Blend Channel of the Fusion shader. Even the Mask shader in the Mask channel is discovered and printed.

I've ran the debugger on the function :

when the loop state is on the Color shader, the function is going up on the parent Fusion shader as expected
but then the elif node.GetNext() condition isn't verified (?!) so it skips the last branch and go back to the material

The thing is it doesn't happen with other materials nor other shaders :

I've made this particular material's shader tree more complex and all the shaders are seen, even when a node has several children.
A brand new material with the same shader setup is not a problem.

There are two cases where the function scan the shader correctly and output the expected result :

When using the recursive function written by @ferdinand
Disconnecting and reconnecting the Filter shader from the Fusion shader. If I run the function again after doing that, the AO and Filter shaders are discovered and printed correctly.

Do you have any idea of what it is happening here ?
I'm planning to use this function for several commands and I don't like the possibility that it could fail in some cases, even if it is some niche ones.