Unexpected "refresh" behavior when manipulating linked object

Havremunken

Hi there,
I am creating a C++ plugin that lays out some text objects in a table structure according to some input data and parameters set by the user. In general, this works; I input data, I set parameters, I get a table. Here is a simple screenshot just for basic context.

There are a few things at play here. My object generator has laid out the numbers 1-9 according to the parameters set by the user (in this case, me). As you can see, there are also some splines around the characters. These splines come in handy as "evidence" later. There are two REAL parameters for horizontal and vertical "padding" - this is the space between the bounding box of the characters (1-9) and the splines surrounding them. If padding was 0, then the splines would touch the numbers.
There are also two other objects above the output of my plugin. A text spline, and a mograph text object. These are used as template objects - in this case, you can see that the 3D mograph text object is used as a template for each cell data. I actually clone this text object and change the text contents. In the UI for my generator plugin, I have a link field defined as such:

LINK MT_HEADERTEXTTEMPLATE				{
	ACCEPT  { Osplinetext; Omograph_text; }
}

The idea is that the user can drag a text object (spline or mograph) that has the correct font, size, spacing, 3d depth etc., so that the numbers 1-9 look like what they want. If the user does not drag anything into the link field, I just create a basic text spline. The object is created with this simple code:

BaseObject* text_object = cell->font == nullptr
                    ? BaseObject::Alloc(Osplinetext)
                    : static_cast<BaseObject*>(cell->font->GetClone(COPYFLAGS::NONE, nullptr));

cell->font in this case is a reference to the object in the MT_HEADERTEXTTEMPLATE field that I get in the SetDParameter() of my generator object.

In my GetVirtualObjects() implementation I of course try to be a good citizen and check if we can use the cache or if I need to recalculate my layout. In order to create the splines you see in the screenshot I need to measure the size of each text object (they won't typically contain just 1-9, there will be lots of different stuff in there). When either the data changes, or the text object linked, I run what I call the "measure" pass in my layout algorithm. This creates a new document, and creates a clone of all the text objects in this new document, in order to measure the bounding box of the text object. This information is then cached, so that we only run this measure process when we need to. For instance, the font size of the linked text object changing means we have to re-run measure. The padding between the text objects and the splines around it does not; So we save some expensive calculation when we can.

The start of my GetVirtualObjects() looks like this:

Bool linksDirty = false;

if(_layout_settings.font_template != nullptr)
{
	int dirty = _layout_settings.font_template->GetDirty(DIRTYFLAGS::ALL);
	if(dirty != _last_font_template_dirty_cs)
	{
		_last_font_template_dirty_cs = dirty;
		linksDirty = true;
	}
}

Bool isDirty = op->CheckCache(hh) || op->IsDirty(DIRTYFLAGS::DATA);
if (!isDirty && !linksDirty)
	return op->GetCache();

_layout_settings.font_template is again the pointer to the linked text object. _last_font_template_dirty_cs is just a local variable for now that is used to check to see if the linked object has been changed. If it has, well, then no cache for us, and we go ahead and create the object structure normally.

And now finally we have arrived at my problem. My object (the numbers 1-9 in this case) are not always refreshed on screen, depending on how I change parameters.

If I change any of my own parameters (such as the input data, the linked text object, horizontal or vertical padding between the text and the splines) then my object is refreshed continually. If I drag a padding number, I get realtime updates.
If I select the linked text object in the C4D Object Manager, activate the Height field, type in a new number, and then press enter, on screen I see the following:

Just to be very specific - GetVirtualObjects() has run fully here - the splines are laid out correctly for the next font size. Using ApplicationOutput and the debugger I have verified that I am indeed creating the text objects along with the spline objects we see on screen. However, they are nowhere to be seen... that is, until I click somewhere in the C4D UI that causes it to update. Maybe I click my object in the Object Manager. Or anywhere really outside the Attribute Manager. This causes the view to refresh and I once more have the expected layout visible:

(I used different text Height values here to produce the screenshots, so the font size varies, that is not a layout issue - the splines ARE correct in every instance)
It is even more visible if I drag the text Height of the font object (either when the font object is selected in the Object Manager directly, or when I drill down into the linked text field property on my own object). While I drag Height left and right to increase and decrease the font size, I see the actual text object resize in realtime, AND I see my splines resize themselves to accomodate the new text size. However, I do not see any text objects, despite my GetVirtualObjects() returning them as part of my object structure. Until I let go of the mouse - in that case the text objects are instantly back and visible again.
If I put my text cursor in the Height parameter of the text box and use the arrow up/down keys to change the value, again I see the splines moving, but no text objects. If I tab to another field, still no text objects. If I press enter the text objects appear.

Now given that I don't know if my GetVirtualObjects() is being called while the user is scrubbing some value, or if it is "settled", I am not behaving any differently. The only thing changing with my GetVirtualObjects() implementation is the caching code I showed at the beginning - either it returns the cached object, or it does not. When the user is scrubbing a value, the cache is NOT returned - I have verified this with the debugger and ApplicationOutput() also.

If I select my object plugin object in the Object Manager and press C to make it an editable object, the whole structure is there exactly as I create it in my code. Text objects and splines inside some nulls to keep things organized. Of course I can't change it to an editable object in the middle of the user scrubbing parameter values, but I have verified through debugging output that I am returning the correct - same - object structure on every invocation. The text objects just do not appear while I am messing with parameters.

Am I perhaps missing some message that I should handle to tell the system to completely redraw my objects? I can't imagine, because why would the splines be redrawn in that case?

I will happily show my code, but I don't know which part of it would be useful to figure out why this happens (or doesn't happen). My GetVirtualObjects() gets kinda complex because of several layout operations happening, but if there is anything in there you need to see, let me know, and I will post it. I just can't imagine why Cinema 4D would draw the splines I return as part of my object structure, and not the text objects...

Thanks for any insights!

ferdinand

Hello @Havremunken,

Thank you for reaching out to us. It is impossible to answer such a long question faithfully, please have a look at our Support Procedures: Asking Questions section. I understand that you put quite some effort into your posting and we appreciate that but it is just too much. Please keep it shorter in the future.

About you Question

The core question seems to be that GVO is not updating your object as you would expect it to when the user is manipulating an input object expressed by a BaseLink. And this happens despite you already manually including the dirty state of the linked object in your GVO dirty calculations.

You might here have to also overwrite NodeData::CheckDirty using the same code you already use to calculate your isDirty in GVO. But in CheckDirty you do not return a bool but flag yourself dirty when you want to update your object (so that an execution method like GVO is called later).

Your should still keep the dirty checking in your GVO.

Cheers,
Ferdinand

Havremunken

Hi Ferdinand,

First, sorry for the long question; I tried to keep it as short as possible, while still including all the information about what goes wrong, and what my plugin does during this process, to avoid a situation where your natural response would be "Well, did you do A, B and C" (basic stuff). I will work on compressing further and getting to the point quicker in the future. Not my strong side.

I will attempt to override NodeData::CheckDirty like you specify and post results. I still find it strange that GVO is called and evaluated (as proven by the splines updating), but I don't know the workings of the SDK well enough; If a proper dirty check is what is needed, that will be great. Thanks!

ferdinand

Hey @Havremunken,

yes, that is indeed a bit strange. When we talked this morning about your case, we actually had internally a bit of a difference of opinion. My point was that handling the dirty state in GVO is enough while @m_adam stressed the importance of CheckDirty. For Python I at least know for sure that GVO will be called on each scene execution and it is then up to you to cache things or not. In C++, CheckDirty might kick in in some cases but I personally am a bit skeptical, but I might be wrong. But it won't hurt to implement it, as you should do it anyway so that other entities can evaluate your plugins true dirty state.

When CheckDirty does not help, we will probably need your code. You say your GVO does run when you modify the linked object? This could also be a priority issue. Have you tried reordering your plugin and the linked object in the OM. So, that the linked object is once before and once after the your plugin? If that is the problem, you might have to manually rebuild the cache of the linked object. But I would be surprised if that is the case.

Cheers,
Ferdinand

ferdinand

Hey,

When you decide to post your code, please also make sure to have a look at Support Procedures: Reporting Bugs & Crashes, so that we can reproduce your problem. As always, when you do not want to share things publicly, you can send them to us via sdk_support(at)maxon(dot)net. But you can also uploads ZIPs and other files up to a size of 32 MB to the forum.

Cheers,
Ferdinand

Havremunken

Hi @ferdinand - I just got in front of the dev PC again and before I changed any code, I wanted to test something based on what you said - it is something that did not even cross my mind during my own testing - the question of priority. Apparently, up until now, my object has always been "below" (physically under) the text object in the Object Manager. This gives the behavior of not seeing the complete object while changing the linked one.

However, if I reorder the objects in the Object Manager, so that my object is on top, and the linked text object is on the bottom, and start scrubbing the Height of the text object, I now see my complete object change in realtime! I.e. not just the splines anymore, but the texts 1-9 update and resize while I scrub!

From your answer, I take it that this means that there is a caching order issue. So basically when my GVO is called, I should force the linked object to update the cache before I actually start cloning it? If my understanding is correct, do you have a short code snippet that demonstrates how I can force the linked object to update its' cache?

Thanks for helping me stumble closer to a solution!

ferdinand

Hey @Havremunken,

yeah, that is really one of the oddities of Cinema 4D that it has no dependency graph in the classic API. You have two options here:

Always build the caches of your inputs yourself, so that you always operate on 'fresh' data, even when you are in line before your inputs. This is flawed because when your inputs have dependencies themselves you will be in a world of hurt. The general idea is: Clone your inputs, copy them into a dummy document and BaseDocument::ExecutePasses the caches you need. Beside the dependencies problem (which is hard to solve due to a lack of a dependency graph), this is also not the fastest thing.
Implement ObjectData::AddToExecution. Here you can (sort of) define the order in which your object is evaluated. This is tied to the execution priorities. The problem is here that execution priorities are a little bit chaotic. Below is an example from our own code base, where we shift the priority of the particle emitter object much earlier into the pipeline from EXECUTIONPRIORITY_GENERATOR (5000) to EXECUTIONPRIORITY_ANIMATION (2000) plus some magic offset. In your case it might be already enough to set the priority to EXECUTIONPRIORITY_GENERATOR + 1 so that your plugin is always evaluated after all your spline inputs. But as you can see by the code example, even we ourself are sometimes a bit at lost, because you can easily break things with this.

What confuses me a bit is that you claim that it works when you have your object at the top of the Object Manager hierarchy while I would have expected the opposite. In the end, you would still have to provide your project so that we can have a look ourself. It is hard to diagnose such problems at a distance.

Cheers,
Ferdinand

Example:

```cpp
Bool ParticleObject::AddToExecution(BaseObject* op, PriorityList* list)
{
	// TODO: (???) check for somebody to give an answer on this. Probably not possible to change, 
	// because it will change scene evaluation order and thus appearance. expression based changes 
	// to the start position of the emitter will lag one frame behind when the animation is reverted. 
	// Maybe change this to EXECUTIONPRIORITY_EXPRESSION + 500 or EXECUTIONPRIORITY_DYNAMICS?
	list->Add(op, EXECUTIONPRIORITY_ANIMATION + 500, EXECUTIONFLAGS::NONE);
	return true;
}

Havremunken

Thanks again @ferdinand !

First; I did confirm that I wasn't dreaming. This works properly when my object is above the text object in the Object Manager. When the text object is above, it doesn't work. We get the splines resizing, but not the cloned text objects (1-9 in my original example). My understanding of how C4D generally works (top down within the same priority group, basically) also says this should be the other way around. As long as the text object is changed first (when I change it's Height property), then creating my clones etc. should be good to go. Instead it is the opposite.

Let us add some spice to that equation; I added AddToExecution (and after tearing some hair out - added the OBJECT_CALL_ADDEXECUTION to my registration, that wasn't trivial to find) :

Bool mt::AddToExecution(BaseObject* op, PriorityList* list)
{
	list->Add(op, 2147483647, EXECUTIONFLAGS::NONE);

	return true;
}

Note the very big number in there. Biggest I could find. I tried setting it to everything from EXECUTIONPRIORITY_INITIAL to EXECUTIONPRIORITY_GENERATOR to EXECUTIONPRIORITY_GENERATOR + 500 to the max value. This did NOT change the way this behaves. Only the order of the objects in the Object Manager changes this behavior as far as I can find. And this is still "opposite" of what we would expect.

One further thing to note - I did override CheckDirty as well, and made it as close as possible to the code in the start of GVO. However, CheckDirty is not passed a HierarchyHelp object (unlike GVO), so I can't really call op->CheckCache(). However, it seems like this is a moot point because according to my debugger, CheckDirty is never actually called by C4D. And unlike the case with OBJECT_CALL_ADDEXECUTION, I could not find a flag to set to register for it. Is there anything special I need to do in order to "activate" the CheckDirty() checking?

I realize this probably means you need to take a look at my code - I am convinced I am doing something "bad" that causes this priority of the object orders to be the only way that works. I will try to clean up the code a bit - it is an unholy mess right now, obviously programmed by someone who learned C++ 25 years ago, didn't use it much since (spent my time in C#) AND is trying to learn the idiomatic way to do things in the C4D SDK at the same time. Perhaps this cleanup will even fix the problem for all I know.

I will keep you posted and eventually send you the code if I can't figure out why it behaves this way!

Thanks again for the help so far!

ferdinand

Hello @Havremunken,

no, you did everything correctly, the upper integer boundary is a bit overkill, but should work. Regarding CheckDirty, no, you do not have to enable something, you just replace the default implementation (which does nothing). And you do not need HierarchyHelp, my instructions might have been a bit misleading. You are evaluating/setting your own dirtiness here, so if there is a cache or not is irrelevant for you, you just implement additional factors which should make you dirty. As shown by the example in the docs, you could for example tie your own dirtiness to a change in time.

void CheckDirty(BaseObject* op, const BaseDocument* doc)
{
  if (doc->GetTime()!=cached_time)
  {
    cached_time = doc->GetTime();
    op->SetDirty(DIRTYFLAGS::DATA);
  }
}

In your case you would probably just also make yourself dirty when your font object is dirty.

And yes, please share your code, I could speculate into other directions - you could also try NodeData::GetAccessedObjects - but that will lead to nothing when I send you off into random directions without a clear sense of what your code is doing.

Cheers,
Ferdinand

Havremunken

@ferdinand After a lot of time forced to work on other projects, I have looked further into this, and sent you an email. I hope you don't see it until after the holiday season.