OpenUSD (pxr) library in c4d python

llealloo

Hello, I am curious how one might try to load the pxr library in c4d python as part of a plugin? I have loaded the pxr library (from usd-core) into the shared third party python libs directory as mentioned here.

It seems that there is some conflict (crash) if I try to:
import pxr
followed by:
from pxr import Usd <- this is where the crash occurs.

I am quite certain this is due to c4d's native (C++) version of USD already being loaded into memory (or one of its dependencies such as TBB). This thread here gives me some clues, but not a solve so far. By the way, I am using macOS. This issue on github also provides more clues.

Thanks for your help, and I wish everyone a happy 2024!

Leah

llealloo

I forgot to mention some further information that might be helpful...

I am aware the usd-core requires Python >=3.6, <3.11 and that 2024 uses Python 3.11. I have tried loading it in the same fashion with earlier versions of c4d (2023) also with a crash when trying from pxr import Usd

ThomasB

@llealloo
Your link is not working, with the folder example in your first post.
Have you tried to place it into a separate folder in your res-folder or directly into your plugin folder and write the path into to the sys-path?
That's just a few lines of code:

Folder structure:

plugin folder.
- res folder
- your module
- plugin.pyp

import os
import sys

# if the module is placed directly into the plugin folder - add the folder to the python search path
folder = os.path.dirname(__file__)  # your plugin folder
if folder not in sys.path:
    sys.path.insert(0, folder)

import yourmodule

But I would paste the module into the res-folder

Folder structure:

plugin folder.
- res folder
- - your module
- plugin.pyp

import os
import sys

# add the  res  folder to the python search path
folder = os.path.join(os.path.dirname(__file__), "res")  # res folder
if folder not in sys.path:
    sys.path.insert(0, folder)

import yourmodule

llealloo

Hi Thomas, thanks for the response!

The previous link mentioned references the docs on external python libraries:
https://developers.maxon.net/docs/py/2023_2/manuals/introduction/python_libraries.html

I have been trying to put the pxr library in:
~/Library/Preferences/Maxon/python/python39/libs for 2023
~/Library/Preferences/Maxon/python/python311/libs for 2024
(as outlined in the docs)

I can get the pxr lib to load in python successfully with import pxr, so I know it is seeing the library. The issue is that when I try to load any of the library's modules, it deadlocks c4d.

It should be fairly easy to reproduce on your end if you grab pxr from usd-core at https://pypi.org/project/usd-core/

Thanks again for the help!

Leah

llealloo

Once again, I think this issue stems from the preexisting USD/pxr library loaded natively in c4d for USD import/export. Likely, it is the TBB dependency getting loaded twice into memory that is crashing c4d on macOS.

Also as a side note, it should be possible to load this library for use in scripts outside of a plugin, too, in which case the directories I previously mentioned would be the top choice so that c4d automatically adds the library to the python sys.path.

ThomasB

@llealloo
That wasn't a one-way ticket recommendation, but just to test whether the problem still exists...

I can't even get the usd library installed. pip shows me errors even via venv in pycharm as well as with their .whl files.

Back to your proplem:

Maybe it doesn't work properly because the USD module needs some dependencies that are not installed or some modules are not supported in C4D python...to my knowledge not all modules of the standard library are supported in c4d python

llealloo

Understood! Thanks again for looking into this with me

Okay, to pip install the usd-core library, you will want to make sure your venv is using Python 3.9 or 3.10. This should also work well with c4d 2023.1.3 which uses its own version of Python 3.9. I do understand that the Python version included as part of the c4d install is potentially modified by maxon, and I am hoping that the pxr library will be compatible as long as the major version matches.

Also fwiw, the usd-core version of the pxr library is a "core" distribution of USD that doesn't require additional dependencies to be installed; it comes prepackaged for use as a Python API and does not need other things to be installed. The only bit usd-core requires is that the Python version you are pip installing from is >=3.6, <3.11.

What I have done is setup a venv with Python 3.9, pip installed usd-core, and moved the concomitantly downloaded pxr library into c4d python's supported external library directory. import pxr works, but trying to load the from pxr import Usd results in a deadlock of c4d 2023.1.3, and 2024 (though I am not expecting it to work that well given that 2024 runs Python 3.11 which is not fully supported by usd-core)

llealloo

Ideally, the USD/pxr library in use by c4d's native USD import/export io plugin could be loaded and used by c4d python. It is my hope that in a future version of C4D this will be possible

i_mazlov

Hi Leah,

In Cinema 2023 (as it is the last package shipped with the supported by the usd library python3.9) I quickly tried installing usd-core package using c4dpy as discussed in How to install pip and numpy for Cinema 4D 2024 on MacOS? and it haven't given me any crash.

However, regardless of that please note that according to our Support Procedures we cannot provide support for the third party libraries.

Cheers,
Ilia

llealloo

Hi @i_mazlov, thanks for your response! Did you try:

import pxr

followed by :

from pxr import Usd

I was also able to successfully "install" it, but actually loading the Usd module in the pxr lib was what gave me the crash. Thanks so much for putting eyes on this.

llealloo

Did c4d deadlock when you tried:

from pxr import Usd

So very curious. Thanks again for your help!

Firefly2021

Hi llealloo,
I installed usd-core at C4D R26 by powershell in this folder \Maxon Cinema 4D R26\resource\modules\python\libs\python39.win64.framework, and the following test code works fine and can produce the output usd file.
Hope it may give you more clue.

import c4d
import pxr
# following code from GTC_usd_introduction.ipynb------------------------------
from pxr import Usd, UsdGeom
# Create a tempory stage in memory
stage = Usd.Stage.CreateInMemory('SampleLayer.usda')

# Create a transform and add a sphere as mesh data
xformPrim = UsdGeom.Xform.Define(stage, '/MySphere')

# Set a translation
UsdGeom.XformCommonAPI(xformPrim).SetTranslate((7,8,9))

spherePrim = UsdGeom.Sphere.Define(stage, '/MySphere/MeshData')

# Get the sphere as a generic prim
sphere = stage.GetPrimAtPath('/MySphere/MeshData')

# Get the extent and radius parameters for the prim
radiusAttr = sphere.GetAttribute('radius')
extentAttr = sphere.GetAttribute('extent')

# Access the sphere schema to set the color
colorAttr = spherePrim.GetDisplayColorAttr()

# Set the radius to 2
radiusAttr.Set(2)

# Expand the extents to match the new radius 
extentAttr.Set(extentAttr.Get()*2)

# Make the sphere blue
colorAttr.Set([(0,0,1)])

# Print out the stage
print(stage.GetRootLayer().ExportToString())

# Save the resulting layer
stage.GetRootLayer().Export('SampleLayer.usda')

llealloo

@Firefly2021 this is amazing! thanks so much!

I admittedly have not tested installation of this lib on windows, only macOS. I think it very well might be a macOS issue if this is any clue:
https://github.com/PixarAnimationStudios/OpenUSD/issues/1479

Quoting from the above issue on github:

The USD library cannot be loaded twice into the same process on the MacOS (and probably Linux, not tested yet). It does work on Windows.

This is a continuation of #1341, which was the first problem we ran into. We found an acceptable workaround for the TBB deadlock.

The new problem is that when the second copy of USD gets loaded into the process, it exits the process with
FATAL ERROR: [TF_DEBUG_ENVIRONMENT_SYMBOL] multiple symbol definitions
from line 96 in pxr/base/tf/debug.cpp

What happens is approximately this:

Cinema 4D R23 loads its own USD module, a shared library that links dynamically to libusd_ms.dylib, which does its registration.

C4D loads our plugin (shared library), which loads another of our shared libraries that is statically linked to USD

The copy of USD in our shared library registers its classes, but then it also triggers the registration in C4D's libusd_ms.dylib a second time, which causes the TF_FATAL_ERROR in debug.cpp.

This seems to be related to the DYLD load callback that gets registered by InstallDyldCallbacks() in pxr\base\arch\attributes.cpp. The first copy's callback gets called for the second copy that gets loaded into the process.

i_mazlov

Hi Leah,

I have checked this on Windows platform and yes it works with actually using the library (in my case creating and saving a stage with a single xform object).

I'll have a quick look for this issue on the OSX platform once I have such opportunity, but I won't give you any promises, since as I've already mentioned to you that support for the 3rd party libraries is out of scope of our support.

Anyways, thank you for sharing your findings with the community here, highly appreciated!

Cheers,
Ilia

llealloo

Thanks so much Ilia!

I really appreciate you taking a look even though it's technically out of scope.

A lofty question... Is there any interest among developers there at maxon to expose the usd/pxr library that c4d ships with (albeit internally for USD io) natively within c4d python? That would be a most ideal scenario that I'm sure many folks starting to build pipelines with USD in c4d would love to have... Just thought I'd mention it