How to store "cached" data in the document from the scenehook?

ferdinand

I am going to answer in a code block. As writing text without code is here probably pretty pointless and confusing.

using namespace cinema;

// This line instantiates a node of type #Tassetnode.
BaseList2D* asset = BaseList2D::Alloc(Tassetnode);

// It is conceptually very similar to this line where we instantiate a cube object.
BaseObject* const op = BaseObject::Alloc(Ocube);

// #Ocube and #Tassetnode are type symbols or in other words the plugin IDs with which these nodes
// have been registered.

// Plugin ID for the generic asset node and at the same time its branch ID. T is not only the prefix
// for tag types but also generic node types, e.g., Tbasedocument, Tpluginlayer, Tgelistnode, etc.
static const Int32 Tassetnode = 1030001;

// Register the plugin hook, here a genric node which we instantiate above.
if (!RegisterNodePlugin(Tassetnode, "My Generic Node Type"_s, 
    PLUGINFLAG_SMALLNODE, MyAssetData::Alloc, icon, 0, nullptr))
    // Do something on failure ...

// A hook is the driving interface, you could also say 'controller' in modern terms, of a node in a
// scene. So, one could do this comparison:
//
//    Data Layer (user facing)                                        Logic Layer (system facing)
//
//    BaseObject (Ocube)      ---- GeListNode::GetNodeData()¹ ---->   CubeObjectData                
//    BaseList2D (Tassetnode) <-------- NodeData::Get()² ----------   MyAssetData
//
// I.e., you have a frontend/data layer that is the scene graph of Cinema 4D and it contains the 
// tangible content of a scene, but it effectively only holds the data of the scene (not 100% true
// but close enough). The logic is separated into its own layer, the plugin hooks third party and
// Maxon developers write. You can switch between these two layers with the methods GetNodeData(1)
// and Get(2) of the respective interfaces.
//
BaseObject* op;              // An instance of Ocube

// Get the plugin hook for an Ocube instance. We can do the access in different levels of verboseness/
// concreteness. We generally do not advise accessing hooks for types you do not own. The last line
// is the most concrete form and impossible in the public API as the cube object implementation is
// not public (and is also not named like that as primitives are abstracted).
NodeData* const plainHook = op->GetNodeData<NodeData>();
ObjectData* const objectHook = op->GetNodeData<ObjectData>();
CubeObjectData* const concreteHook = op->GetNodeData<CubeObjectData>();

// The way back from a hook is then this (usually called inside of methods of the hook or passed in
// as an argument to hook methods).

// Get the node that represents the hook. It requires implementation knowledge to know which hook can
// be cast to what, technically that can be abstracted with information stored in the hook, but there
// is no builtin automated way.
GeListNode* const node = hook->Get();
BaseObject* const cube = static_cast<BaseObject*>(node);

I hope this sheds some light on how scene elements and their plugin hooks correspond.

Cheers,
Ferdinand

Havremunken

Hi Ferdinand,

Thank you again! I see I misunderstood the point of the alloc parameter, as I was also missing the connection between the scene hook (owner of the branch I guess) and the asset nodes making up the branch. I love the explanation of the different layers as an added bonus.

Havremunken

This is perhaps not a big deal, but while implementing this, I met some resistance from Visual Studio while overriding GetBranchInfo in my scene hook. It did not match the signature from your example. Ok, so detective time - my scene hook inherits from SceneHookData, which has no GetBranchInfo of its own, this comes from it inheriting from NodeData, and this class does have a GetBranchInfo. However, the signature does not match entirely. I am on 2024 still, just in case this changed for 2025. This is the signature:

virtual maxon::Result<Bool> GetBranchInfo(const GeListNode* node, const maxon::ValueReceiver<const BranchInfo&>& info, GETBRANCHINFO flags) const;

Ok, sligtly different, but... It is supposed to return a bool? Ok, that is a bit confusing, in your example you returned the number of elements we filled in. Ok, let me look at the docs in the source file (c4d_nodedata.h):

/// @return The number of BranchInfo elements filled in the @formatParam{info} array.

Ok, that is clear as mud. So I am not supposed to return a number, then, but just true if I actually put something in there? How does it learn the number of items, or doesn't it need to?

A minor point is that in your code example (in your first post) you write it as FlowTag::GetBranchInfo - I guess I can safely assume this was copied from another source example where a FlowTag class was implemented, so the name "FlowTag" has no meaning in this case?

From your example I get the sense that the MyAssetData is created in a way where one instance of the class represents one piece of data. In my case, my variant of this asset node would contain one URL and one content string. Then I use the list starting with the AssetHead to store each url+content combo I have in my cache, is this correct? So my version of AddAsset would take these two as params?

I am also trying to understand the choice to "feed" the asset object by sending it a message, instead of for instance calling Load directly. Is this because when you BaseList2D::Alloc(Tassetnode); you don't actually get direct access to the instance of the MyAssetData class itself, so we're in the Data Layer instead of in the Logic Layer, and have to use this mechanism?

Hopefully final question in this round (at least I'm not one of the many spammers that keep trying their luck on the forum, huh?): When I create an object that causes some data to be cached using this system, and save the C4D file, then later reopen it - I understand that C4D reads this data from the file, and I imagine that the searching lambda in your GetAssetHead is what locates this again the first time it is called (when _assetHead is not already set). Fairly soon after opening the file, C4D will call GetVirtualObjects on my main plugin, and it will ask the scene hook for the data - am I correct in assuming that this would be when the scene hook calls GetAssetHead(), reads all the MyAssetData (or whatever I end up calling them) nodes, and finally has his cache so he can return data to the generator plugin?

I hope these questions are not too annoying, I am just trying to understand the code rather than just copy & paste it (or feed it to an AI).

Thanks!

ferdinand

Hey @Havremunken,

first of all, questions are never annoying to us. You are right, I incorrectly used there an outdated signature of NodeData::GetBranchInfo. I removed my pseudo code example from above, as I decided today, that I will just write a code example for branching for the SDK (the Asset Container thing from below which has the custom branch net.maxonexample.branch.assetcontainer with a cube and a sphere object stored in it).

Just posting here so that you do not think we overlooked you, I will post the example on Monday, as I still have to make sure the example works properly in all aspects. For now I will sail of into the weekend

Cheers,
Ferdinand

Havremunken

Thank you so much, Ferdinand, I look forward to that! And enjoy the well-deserved weekend in the mean time!

ferdinand

Hey @Havremunken,

I have posted the code example which will be shipped with an upcoming C++ SDK here. The example does not do 100% what you are trying to do, the node type implementing the custom branch is for example an object and not a scene hook, as I had to keep common applicability a bit in mind.

But it realizes a node which inernalizes other scene elements in a custom branch of itself, and should get you at least going. When you still need help with your specific case, please ask questions in this thread. When you have generic branching questions tied to the example, please use the other thread.

Cheers,
Ferdinand

Havremunken

"Thank you" seems poor and inadequate for that fantastic example, but I don't have the English vocabulary to go beyond that!

I have read through the example once, and I think I understand it. Since this has been a real Monday (tm), I'm going to let this simmer in the brain for a day or two, and then use the techniques you demonstrate to add GetBranchInfo etc. to my scene hook. Both the example and the video were great at explaining the steps needed to get "extra" information into the document, and are much appreciated!

So while it does not express the full extent of my gratitude, once again - thank you!

Havremunken

Hi Ferdinand,

I have worked on fitting your example into my project. As you know, there are two major differences that I needed to account for; first, I am doing this in a Scene Hook, and second, I am not keeping Cinema 4D primitives in my BranchInfo, I am keeping URLs and their contents. So I had to adapt.

First I implemented GetBranchInfo in my scene hook. It is more or less a copy of yours.

maxon::Result<Bool> GetBranchInfo(const GeListNode* node, const maxon::ValueReceiver<const BranchInfo&>& info, GETBRANCHINFO flags) const override
{
	iferr_scope;
	yield_scope;

	NodeData::GetBranchInfo (node, info, flags) yield_return;

	if (!(flags & GETBRANCHINFO::ONLYWITHCHILDREN) || _assetHead->GetFirst())
	{
		info (
			BranchInfo{
				MAXON_REMOVE_CONST (this)->_assetHead,
				"clever.unique.name.changed.to.protect.the.innocent"_s,
				ID_MORPHINETABLE_URLCACHE_ASSETNODE,
				BRANCHINFOFLAGS::NONE
			}) yield_return;
	}

	return true;
}

This will come into play a little later.

In my scenehook I also implemented the Read, Write and CopyTo (I understand how these go together, but will CopyTo ever really be used in a scene hook? I placed a breakpoint in the function and it didn't get called yet).

First of all; Since I need something to store my data in, I decided on creating a NodeData subclass like this - the simplest implementation I could imagine getting away with:

class UrlCacheAssetData : public NodeData
{
	INSTANCEOF (UrlCacheAssetData, NodeData)

private:
	maxon::String _url;
	maxon::String _data;

public:
	static NodeData* Alloc()
	{
		return NewObjClear (UrlCacheAssetData);
	}

	static Bool RegisterAssetDataPlugin()
	{
		return RegisterNodePlugin (ID_MORPHINETABLE_URLCACHE_ASSETNODE, "UrlCacheAssetDataPlugin"_s,
		                           PLUGINFLAG_SMALLNODE, UrlCacheAssetData::Alloc, nullptr, 0, nullptr);
	}

	void SetData(String url, String data)
	{
		_url = url;
		_data = data;
	}

	String GetUrl() const
	{
		return _url;
	}

	String GetData() const
	{
		return _data;
	}
};

So when the user enters a new URL into my plugin, I have added code to add a new asset as such:

auto node = AllocSmallListNode (ID_MORPHINETABLE_URLCACHE_ASSETNODE);
auto const assetData = node->GetNodeData<UrlCacheAssetData>();
if (assetData != nullptr)
{
	assetData->SetData (*task->url, loadedData);
	_assetHead->InsertLast (node);
	_currentCacheSize++;
}

_assetHead is declared the same way you did in your example. _currentCacheSize is a running count of the number of asset nodes we have.

Ok, time to save a document (with a plugin that has added at least one item in the cache). Here is my Write implementation for the scene hook:

Bool Write(const GeListNode* node, HyperFile* hf) const override
{
	if (!_assetHead || !hf)
		return false;

	if (!hf->WriteInt32 (_currentCacheSize))
		return false;

	auto current = _assetHead->GetFirst();
	while (current)
	{
		auto const assetData = current->GetNodeData<UrlCacheAssetData>();
		if (assetData != nullptr)
		{
			if (!hf->WriteString (assetData->GetUrl()))
				return false;
			if (!hf->WriteString (assetData->GetData()))
				return false;
		}
		current = current->GetNext();
	}
	return SceneHookData::Write (node, hf);
}

As you see, my thought is that I first write an integer saying how many url/content pairs I have, then I loop through the asset nodes and write them to the file. I couldn't think of another way to do this, as I do not have the luxury of dealing with objects that already have the WriteObject() functions implemented. I am of course expecting that my way is not the optimal way, or even correct.

I step through it using the debugger and all seems to work the way I expect. Ok, I close the file and then try to reopen it using C4D.

Here is my Read implementation:

Bool Read(GeListNode* node, HyperFile* hf, Int32 level) override
{
	SUPER::Read (node, hf, level);

	iferr_scope_handler
	{
		return false;
	};

	if (!hf || !_assetHead)
		return false;
	_assetHead->FlushAll();

	if (!hf->ReadInt32 (&_currentCacheSize))
		return false;

	for (auto i = 0; i < _currentCacheSize; i++)
	{
		auto newNode = AllocSmallListNode (ID_MORPHINETABLE_URLCACHE_ASSETNODE);
		auto const assetData = newNode->GetNodeData<UrlCacheAssetData>();
		if (assetData != nullptr)
		{
			String url;
			if (!hf->ReadString (&url))
				return false;
			String data;
			if (!hf->ReadString (&data))
				return false;
			assetData->SetData (url, data);
			_assetHead->InsertLast (newNode);
		}
	}

	return true;
}

I step through it using the debugger and again it works as expected; I read the integer specifying the number of cache items, then loop through that amount of times to read the items. The items are read successfully, containing the expected data! Amazing!

However, at some point after my Read method is exited, I get a crash. This crash happens while Cinema is executing my GetBranchInfo() code above, specifically the info(...) yield return part. At this point it gets a little difficult for me to look into. I understand that a value is pointing to the wrong place, with a very suspicious value (the "opposite" of a null pointer, so to speak), but I can't tell where this value is coming from.

Exception details:

Exception thrown at 0x00007FF8E8427C10 (c4d_base.xdl64) in Cinema 4D.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.

This is refering to a location in delegate.h in the SDK where the following code can be seen:

//----------------------------------------------------------------------------------------
/// Forwards the call (invokes the stub function for a callable).
//----------------------------------------------------------------------------------------
MAXON_ATTRIBUTE_FORCE_INLINE RESULT operator ()(ARGS... args) const
{
	StubPtrType stub = StubPtrType(_stubPtr);
#ifdef PRIVATE_MAXON_MTABLE_PTMF
	return (reinterpret_cast<Delegate*>(_objectPtr)->*stub)(std::forward<ARGS>(args)...);
#else
	return stub(_objectPtr, std::forward<ARGS>(args)...);
#endif
}

Line 761-772 in my 2024 SDK. I don't know where exactly the 0xFFF... is coming from but it is not the stub/_stubPtr and the exception happens in the line with the reinterpret_cast.

The call stack does not tell me a lot apart from this happening while Cinema is running my GetBranchInfo:

I realize of course that debugging this based on a forum post is hopeless - unless the 0xFFFF... value could be caused by only one very well known thing, I expect you would need my complete code for this? It changed quite a bit since last time.

I am quite sure that there is something that is not properly initialized or something along those lines, since this happens only when I try to open a document saved with my branch info in it. Oh, and for completeness, my asset head is declared as a private instance variable in my scene hook like this:

AutoAlloc<GeListHead> _assetHead;

And I also borrowed from your init code and put it in the same class.

Bool Init(GeListNode* node, Bool isCloneInit) override
{
	if (!_assetHead)
		return false;

	_assetHead->SetParent (node);

	return SUPER::Init (node, isCloneInit);
}

If you want me to send you the code, just let me know. If you want me to change or test anything else, the same. And if I am completely off track with my asset implementation, doing something that couldn't possibly work - please don't be afraid to tell me. I'm a big boy, I can take it.

Thanks again!

ferdinand

Hey, just as an FYI, I have seen your question(s) here. I will try to answer within the week but have a rather loaded plate right now. I will answer latest next week.

Thank you for your understanding,
Ferdinand

ferdinand

Hey @Havremunken,

In my scene hook I also implemented the Read, Write and CopyTo (I understand how these go together, but will CopyTo ever really be used in a scene hook? I placed a breakpoint in the function and it didn't get called yet).

Yes, you really must do this. NodeData::CopyTo is the plugin layer endpoint for C4DAtom::GetClone and Cinema 4D clones scenes and scene elements all the time. A good example is rendering, for everything but in editor renderings, Cinema 4D will clone the whole document which is about to be rendered. When you then do not implement NodeData::CopyTo your scene hook in the cloned scene will not have its internal data, e.g., the GeListHead that holds the assets will be nullptr. And while it is true that scene hooks are unlikely to be cloned directly due to to their singleton-like nature, there is no guarantee that not some backend system does clone scene hooks for some technical reason. You having set a breakpoint does not mean much, as it is literally impossible to test all the scenarios where Cinema 4D wants to draw a copy of your node. This can then all lead to access violations and more.

Ok, time to save a document ...

In your code, I do not see you serializing your custom branch and the manual allocation (and serializing) of child nodes is not so good:

Bool Read(GeListNode* node, HyperFile* hf, Int32 level) override
{
	SUPER::Read (node, hf, level);

	/// ...

	for (auto i = 0; i < _currentCacheSize; i++)
	{   
    // Big NO-NO, never manually allocate nodes in deserialization. Technically possible but should
    // be avoided.
		auto newNode = AllocSmallListNode (ID_MORPHINETABLE_URLCACHE_ASSETNODE);
		auto const assetData = newNode->GetNodeData<UrlCacheAssetData>();
		if (assetData != nullptr)
		{
			String url;
			if (!hf->ReadString (&url))
				return false;
			String data;
			if (!hf->ReadString (&data))
				return false;
			assetData->SetData (url, data);
      // not good, `newNode` has a different GeMarker (think of it as the UUID of the node) than the 
      // node in the original scene. you might also miss data which Cinema 4D has written into that
      // original node.
			_assetHead->InsertLast (newNode);
		}
	}

	return true;
}

You must really must do what I show in my example, this will automatically serialize all the data stored in the branch. As I write in my example C4DAtom::ReadObject and WriteObject should be taken with grain of salt. 'Object' is meant here in the sense of 'element'. You can write every type of C4DAtom with this, e.g., also our custom GeListHead and all its children.

 Bool Read(GeListNode* node, HyperFile* hf, Int32 level)
  {
    // Call the base implementation, more formality than necessity in our case.
    SUPER::Read(node, hf, level);

    if (!_assetHead->ReadObject(hf, true)) // Deserialize the data of our custom branch.
      return false;

    return true;
  }

  /// @brief Called by Cinema 4D when it serializes #node in a document that is being saved
  /// to let the plugin do custom serialization of data into #hf.
  // ----------------------------------------------------------------------------------------------
  Bool Write(const GeListNode* node, HyperFile* hf) const
  {
    // Call the base implementation, more formality than necessity in our case.
    SUPER::Write(node, hf);

    if (!_assetHead->WriteObject(hf)) // Serialize the data of our custom branch.
      return false;

    return true;
  }

When you have then specialized data in the nodes in your branch (UrlCacheAssetData in your case), you can do three things:

BAD: Store the extra data with the scene hook as you do in your code where you write the strings into the hyper file chunk of the scene hook. While this technically works and can be done intentionally for performance reasons, storing data from children in their parent is not so good, as the tends to become very complicated very quickly.
BETTER: Store the data in the hyper file chunk of each node which owns the data, i.e., overwrite UrlCacheAssetData::Read, ::Write, and ::CopyTo.
CODING-GOD: When you only want to store atomic data which can be expressed in a data container and you own the node implementation, I would just store it there, as you then do not have to do all the serialization dance in UrlCacheAssetData. I.e., just call BaseList2D::GetDataInstance on a node for your UrlCacheAssetData (obtainable via NodeData::Get() from inside UrlCacheAssetData and als often passed in under labels such as node, op, tag, etc. depending on the NodeData type). E.g., this:

// PSEUDO CODE, not compiled or tested, take with a grain of salt.

// A node whose implementation we own, i.e., we own the address space of its data container and can
// write everywhere we want to.
BaseList2D* asset = BaseList2D::Alloc(ID_MORPHINETABLE_URLCACHE_ASSETNODE);

// Just write the data into the node.
BaseContainer* data = asset->GetDataInstance();
data->SetString(ID_MORPHINETABLE_URL, "www.google.com"_s);
data->SetString(ID_MORPHINETABLE_DATA, "Bob's your uncle."_s);

// Insert the node into the branch of your scene hook. Its data will now be stored with the scene.
// You can of course also go the inverse route to read data or modify the data by getting the node
// and then getting its data container again.

// We technically can also do the same for nodes where we do not own the implementation. Here we just
// store the data in a custom container under a custom plugin ID.

// We do not own the cube impl.
BaseList2D* cube = BaseList2D::Alloc(Ocube);

// A custom plugin ID we registered to store alien data in a collision free manner in Ocube.
const Int32 pid = 123456789;

// Create a container and just write some data into it, the IDs do not matter here.
BaseContainer bc (pid);
bc.SetString(0, "www.google.com");
bc.SetString(0, "Bob's your uncle.");

// Get the data container of the node.
BaseContainer* data = cube->GetDataInstance();

// Write the data making sure we do not overwrite native data.

// There is no alien data at all.
GeData nativeData;
if (data.FindIndex(pid, nativeData) == NOTOK)
  data.SetContainer(pid, bc);
// There is already data and it is of type #BaseContainer, and it its a container marked with the
// id #pid, so it is one of our containers, we can safely overwrite things.
else if (nativeData.GetType() == DA_CONTAINER && nativeData.GetContainer().GetId() == pid)
  data.SetContainer(pid, bc);
// There is already native data at #pid and it is not ours, we are screwed. This is not impossible
// to happen as some nodes dynamically write data into their data container (just as we did above
// for #ID_MORPHINETABLE_URLCACHE_ASSETNODE), but it is quite unlikely to happen in the range of
// plugin IDs. 
else
  CrashAndPretendItWasNotOurFault();

Finally, 0xFFFFFFFFFFFFFFFF does have a special meaning. It is the end of the 64bit address space and operating systems/debuggers usually use this address when an access violations occurs. I.e., you have pointer p which points to some point in memory (and is therefore not the nullptr) which holds a FooThing. We then try to call p->Bar() (i.e., FooThing::Bar). But the FooThing at p is long deleted, the memory at p with the size of FooThing is either empty or does nor match the layout of FooThing. An access violation happens, the fireworks begin. TLDR: 0xFFFFFFFFFFFFFFFF is an alias for a corrupted/dangling pointer and manual memory management is the idea of a mad man.

In your case probably something with the irregular instantiated ID_MORPHINETABLE_URLCACHE_ASSETNODE node from your NodeData::Free is going wrong. Or something else, I cannot really tell without the full source code. When you share code, please ideally share your project, not just some .h/.cpp files. You can share code confidentially via [email protected].

Cheers,
Ferdinand