Hey @BruceC,
uff, that is a lot of points. I'll try to comb through most of it, but I do not have that much time today. Feel free to ask for clarifications where I left out things.
Currently, I do this in the tag plugin's Execute() function. The tag plugin remembers the vertex color tag's pointer (initialized as nullptr) and dirty checksum (by C4DAtom::GetDirty(), and initialized as 0).
Never user pointers to long term keep track of scene elements. Something not being nullptr does not mean you point to valid data. The node you point to could be long deleted, and then you point to garbage and access attempts will then result in access violations and a crash. Either use a BaseLink to store a reference to a scene element, or use a weak pointer. I.e., a pointer which becomes invalid when the pointed data is being deleted. You can for example see here how to use a maxon::WeakRawPtr.
However, I found ExecutePasses() called in PrerollToTime() triggers the tag plugin's Execute() function (I think this is the cloned tag in the cloned document), and Execute() function tries to read the linked vertex color tag at the speficied frame again, so PrerollToTime() -> ExecutePasses() is called again, and triggers a new cloned tag in a new cloned document to run its Execute(), and so on.
Yes, that is true and a common problem (not only in Cinema 4D). The superficial reason is that my preroll code example also passes True for the third argument expressions (effectively API slang for tags). When you would pass there false, your tag would not be executed again. But also all other tags would not be executed (which is likely not what you want). The solution to this can be registering a plugin ID, e.g., ID_FOO_PLUGIN_IS_COMPUTE_PASS. When you then clone your document, or execute the passes on an existing document, you write under that ID for example a bool into the data container of the document. Your tags Execute(tag, ...) then gets the document from tag and checks for that flag being present, to stop what it does that causes the infinite update loop. When operating on a document not owned by you, you would then delete that flag or set it to false after you manually invoked the passes, so that future updates made by Cinema 4D (which owns and operates the document) then take the 'normal' route.
The better solution is to generally design things in a manner that such infinite update loops cannot happen in the first place. I do not understand your problem well enough to give more concrete advice here. These scenarios are also not always avoidable.
According to the sample code PrerollToTime(), the lastFrame (the target frame) must not be earlier than the current scene's time.
The preroll function was just an example, not something you must follow to the letter. There is also a little bug I just see now, it should be of course if (lastFrame < firstFrame - 1). But the general idea is, that when you have a scene with simulations, i.e., stuff like Pyro or particles, where the state of the current frame depends on the last frame, you must basically cycle through all the correct scene states to get to that state. You can of course also jump to a previous point, but then you do not have to sim from NOW to FUTURE but from 0 to PREVIOUS.
Prerolling is not necessary when the scene does not contain any simulations, e.g., just key framed position animations, all simulations are cached, or you do not care about simulations and their impact on the rest of scene. Given what kind of plugins you do develop, you likely want to preroll.
⚠ Just to stress this again: Prerolling can be extremely expensive. When the user has a scene with 300 frames of Pyro simulations worth 30 minutes of computing, and you try to preroll half of it, your preroll function will run for 15 minutes.What to do here, really depends on the exact case. Generally you try to avoid having to reach into the future or the past, as this is often a big no-no. The most common "pro" solution to that is caching. E.g., that you write yourself a vertex color cache in your case. When you only need to reach into the past, and this only applies in rendering, you could also effectively hide this from the user, as you must have passed and naturally calculated that value before (and therefore need no explicit cache recording).
Will answer the (2) portion on Monday, running a bit out of Friday time right now.
Cheers,
Ferdinand