Apparently incorrect matrices in Python effector when subject to deformer

vhardy

Hello,
Please find attached a file which illustrates how (I think) I am getting incorrect matrices in a Python deformer. I have created a test file to illustrate the issue (attached below at the very end).
My Python effector (Python Direction Effector) computes the 'lighting' at a given clone based on the clone's normal and the infinite light's direction.
Here is the code for the Python Effector, also in the attached c4d file. It scales and colors the clone (from 0 to 1 and red to yellow) depending on the product of the clone's normal and the light. So a scale of 1 and yellow for full light, and a scale of 0 and red for no light.

import c4d
import typing
from c4d.utils import Neighbor
import math

op: c4d.BaseObject  # The Python Effector object containing this code.
gen: c4d.BaseObject  # The MoGraph Generator executing `op`.
doc: c4d.documents.BaseDocument  # The document `op` and `gen` are contained in.

def main() -> bool:
    data: c4d.modules.mograph.RMoData = c4d.modules.mograph.GeGetMoData(op)
    if data is None:
        return False

    light = op[c4d.ID_USERDATA, 1]
    if light is None:
        print("No light found in the effector's first user data field")
        return False

    mg_light: c4d.Matrix = light.GetMg()
    light_direction_global = -mg_light.v3.GetNormalized()

    # The Effector's MoData has a matrix per clone. Each clone therefore has its
    # position and orientation in that matrix. The clone's normal is the
    # orientation (v3)
    matrices: list = data.GetArray(c4d.MODATA_MATRIX)

    strength = op[c4d.ID_MG_BASEEFFECTOR_STRENGTH]

    n_clones = len(matrices)

    # Initialize color array
    colors = [c4d.Vector(0.0, 0.0, 0.0) for _ in range(n_clones)]

    for i in range(n_clones):
        normal_global = matrices[i].v3.GetNormalized()
        w = max(0.0, normal_global.Dot(light_direction_global))

        # Interpolate color from red to yellow based on illumination
        colors[i] = c4d.Vector(1, w, 0)  # Red to Yellow

        matrices[i].Scale(1 - strength + strength * w)

    # Set the color array to the MoData
    data.SetArray(c4d.MODATA_COLOR, colors, op[c4d.FIELDS].HasContent())
    data.SetArray(c4d.MODATA_MATRIX, matrices, op[c4d.FIELDS].HasContent())

    return True

This Python Effector works in a lot of cases I tested. However, in some circumstances, it seems the matrices I get in the Effector (which seem incorrect) do not match the actual final placement of clones (which seem correct).
The attached test file stems from my efforts to have clones along a cylinder that have a nice helix distribution. To achieve that, I have a first set up (under 'vertical capsule':

A first single helix cloner. This clones simple small spheres. To have the normal I need on these, the cloner has a step effect which rotates the clones so that the normal is perpendicular to the helix center axis.
A second cloner, which simply replicates the single helix cloner vertically, so that I have the density I am looking for.

So far so good. If I apply my effector to that set up as is, I get the expected result. In the attached image, you can see that under 'vertical capsule', I have the right lighting for my spheres, congruent with my normals.
Now, I want to deform my top cloner with a Spline Wrap. To do that, I insert the deformer as a sibling of my "multiple helixes' cloner. To make sure that I can visualize my normals on the clones, I have a little blue arrow in my cloner which is aligned along the z-axis. What I see:

Correct normals in both deformed and non deformed situations.
Correct lighting from the effector in non deformed situations. I conclude my effector is using the right clone matrices.
Incorrect light from the effector in deformed situation. I conclude my effector is not using the right clone matrices.
Display issue in the viewport.

I have attached the test file as well as screen captures which illustrate my issue.

My question is: is there something I should be doing in the Python Effector code that I am missing to get the proper normals. The cloner is displaying the right normals, so I suspect the issue is in what the effector does or accesses. I have noticed that in some cases, the MoData can be queried from both the effector and the cloner, but I am not sure what the difference is and did not find information about that. My test showed that in an Effector, the proper method seems to access MoData through the effector (in c4d.modules.mograph.GeGetMoData(op) above).

I hope my questions is formulated well enough. If not, please shoot and I'll improve / correct.
Thanks in advance for your help.

Image 1 Note: the image below illustrates that the normals are correct in the cloners (even deformed). However, the Effector does not produce the right result (under 'snake issue').

Image 2 Note: under 'snake issue', the viewport rendering does not structurally match the image viewer rendering.
Screenshot 2025-03-06 at 4.30.10 PM.png
normal-effector-deformer-debug-v2.c4d

ferdinand

Hey @vhardy,

I do not want to discourage you, as clearly have put some effort into the presentation and construction of your file. But the first thing I would point out is that your file is way-way-way too complex. I have a relatively beefy machine here at work, and your file is quite sluggish here. But even when you work on NASAs root node itself, such complex files are not very ideal for debugging, as evident by you getting here overwhelmed by your own file.

Please also note that we might refuse support in future cases where no effort is visible to break down a problem into a tangible example, and more over no concrete question aside "this is not working, please fix this". See our Support Procedures: How to ask Questions for details.

Issues with your Rig

The first thing I did, was throw out all the fluff in your file, ending up with this (one could simplify this further but I did not), I also removed the particle matrix transform in your effector code.

With that it became pretty quickly apparent, that your rig cannot really work, as the left rig is basically the same as the right, only that a spline wrap deformer is applied as the last element. And particles and your Python effector can obviously not react to being deformed after they did their work.

Furthermore, the apparent alignment of the clones on the right with their spline is only a hack, it is achieved by the mysterious step effector which just hardcodes a rotation transform which matches with rotation of the helix spline. This will obviously not work in any other cases than when the clones are arranged such linear cloning/rotation axis.

How to Fix

Okay, we figured out what is going wrong here, let's fix that. The first thing we must do, is create the 'snake' (as you named it), i.e., the nested inner helix winding around an outer helix as a singular geometry. I did it like this, I just wrap one helix around the other:

Another issue with your rig is that you try to clone onto splines. Which in itself is fine, but you also want to align the clones along the spline. Along what you probably consider the "normals" of the spline. The normals of a spline are not what most artists think they are, we once talked about it here. The MoGraph cloner also uses parallel transport when "Align Clones" is checked, but with a sweep object we have more control and its parallel transport is also just better. So I created a sweep object onto which we will clone.

Now we are pretty much done, I wasted a few minutes with pondering why my colors were not what I thought they should be and then wrote me this little Python tag with its draw function, to see where the clone normals point, to find out that the alignment option seems to swizzle axis, and we having to pick the i/v1/x axis in this case and not k/v3/z as one would expect.

When we now put everything together, we end up with this, the colors on "my" rig are inverted compared to yours, as I did not bother fixing your "correct" case.

What I did not fix either, is your math. You do not take the position of the light source into account, which does not make that much sense to me for a directed not-infinitely-distant light source as you use there, but maybe I am misunderstanding your goals.

Cheers,
Ferdinand

The updated file: normal-effector-deformer-debug-v3.c4d

I once crated this scene nodes effector which might be close to what you want to do, and might give you some ideas (I used there ray casting to get the self-shadowing effect):

edit: to clarify myself you do not need ray casting to take the position of your light source into account, you only need that for self shadowing as shown in the example below.

1718830521440-specular2.c4d

A very long time ago I held this presentation. The slides and presentation are in German, but the code example is in English. I implemented then a "Local Thickness" tag as Python Tag, using ray casting. So, when you want to implement some fancy lighting effector, you could just get the particle position, then cast a ray against a mesh, and measure the angle between your light ray and the hit pos and its normal on the mesh - to get 'proper' lighting.

coburg_raycasting_narrative.c4d

vhardy

@ferdinand Hello,
My question was whether or not I was doing something incorrect in my effector code, and after reading your message, I understand that what I did not account for was the processing order of the generators, effectors and deformers. The effector being called before deformers apply, I could not expect to get the result I wanted. Thanks for making that clear to me.

I worked on the example you sent because I wanted to make sure my effector worked generally, i.e., with any geometry that has its normal along the positive z axis, and avoid having to special-case my effector to a particular cloner it applies to. I illustrated this in normal-effector-axis-test-v1.c4d:
Screenshot 2025-03-07 at 2.37.43 PM.png
The effector you sent and the original one are applied to simple geometry. On the left is the original effector (working with z-axis), on the right in the effector in the example you sent which compensated for a rotation in the cloner (which uses the negative x-axis).

I modified the example so I can use the original effector working with the regular surface normal (on Z) with the technique your sent for generating geometry that let's me control the normals as I need them. This is the attached normal-effector-axis-test-v2.c4d

Screenshot 2025-03-07 at 2.46.42 PM.png

On the left is the geometry you sent with my effector, just modifying the cloner to not apply a 90 degree rotation on H. The colors are the opposite because I have yellow for light and red for dark and yours has the opposite as you explained in your message.

@ferdinand said in Apparently incorrect matrices in Python effector when subject to deformer:

You do not take the position of the light source into account, , which does not make that much sense to me for a directed not-infinitely-distant light source as you use there, but maybe I am misunderstanding your goals.

Yes, that is correct, I am using an infinite light in my project, so that is why I keep my effector using the direction of the light (also an infinite light in my project / test file).

Thanks for the explanations, examples and pointers. The examples on ray tracing are of particular interest as I have a poor man's version of that in the full version of my effector which works as I was hoping, but can probably be improved in many ways. So that will be helpful. I have also not gotten into Scene Nodes effectors, so that will be a good opportunity to dig into a new area. Lots to learn!

Kind regards,
-v

normal-effector-axis-test-v1.c4d
normal-effector-axis-test-v2.c4d

ferdinand

Hey @vhardy,

I am happy to see that you made progress! Please excuse me for being a bit direct, but is here still an open question? Because I am not so sure.

When I would work on something, it would be your lighting model because not taking the light source position for a directed light source you seem to want to emulate, will often look "wrong". There are many ways you can do this, here is a simple one in pseudo code:

def calculate_illumination(p: c4d.Vector, pn: c4d.Vector, lp: c4d.Vector, ln: c4d.Vector, lt: float, li: float) -> float:
    """Calculates the illumination intensity for point #p for a given a spotlight.

    Parameters:
        p (c4d.Vector): The point to illuminate.
        pn (c4d.Vector): The normal at the point to illuminate.
        lp (c4d.Vector): The position of the spotlight.
        ln (c4d.Vector): The orientation of the spotlight.
        lt (float): The angle theta of the spotlight cone (in radians).
        li (float): The diffuse light intensity.

    Returns:
        float: The illumination intensity at point p.
    """
    # A vector pointing from the light source to the point.
    q: c4d.Vector = (p - lp).GetNormalized()

    # Calculate the dot product between the q and ln normal, i.e., the cosine of the angle spanned
    # between the light-to-point normal and the spot light orientation normal.
    gamma: float = q.Dot(ln.GetNormalized())

    # Our point is in the cone of light.
    if gamma > math.cos(lt):
        # Basically what you were doing.
        intensity: float = li * max(0, q.Dot(pn.GetNormalized()))
    # It is not.
    else:
        intensity = 0

    return intensity

You could of course make this fancier by for example not having a harsh contrast between in-cone and not-in-cone via the a bit brutish if gamma > math.cos(lt) and instead blend smoothly or by using a more complex lighting model.

Or you can just use ray casting whose costs due to the very few rays you will cast here will be negligible, and then have right away full self shadowing (but would then need access to mesh against which to ray cast).

Cheers,
Ferdinand

vhardy

@ferdinand You are correct, there was no further question, I was just closing the loop in case someone else ran into something similar. Thanks for the additional suggestions.
-v