Solved Bug R23? Viewport Render Aborts with Physical Renderer

Hi I did a small ObjectPlugin Test,

I created a temp_doc and loaded a file into with textures.
The textures do not work in this examples because file size is too big to upload here.
I created a container(null) put the clone of a specific polygon object in the temp_doc into this null and returned the null.
Just to test if it returnes at least the polygon-object without materials.

When I render in viewport Standard, Redshift and Corona are rendering, just Physical Renderer aborts the rendering.
This is just in versions R23 and below I guess.

Here is the code:

import c4d
from c4d import plugins, bitmaps
import os

PLUGIN_ID = 1234567  # Just for testing


class Testcity(plugins.ObjectData):

    def __init__(self):
        self.SetOptimizeCache(True)

    def Init(self, op):
        return True

    def GetVirtualObjects(self, op, hh):
        model = os.path.join(model_path, "build_0.c4d")
        new_doc = c4d.documents.BaseDocument()
        c4d.documents.MergeDocument(new_doc, model, loadflags=c4d.SCENEFILTER_OBJECTS | c4d.SCENEFILTER_MATERIALS)
        model = new_doc.GetFirstObject().GetDown().GetDown().GetClone()
        null = c4d.BaseObject(c4d.Onull)
        model.InsertUnder(null)
        return null


if __name__ == "__main__":
    path, file = os.path.split(__file__)
    file = "icon.tif"
    new_path = os.path.join(path, "res", file)
    model_path = os.path.join(path, "res", "models", "models", "0_Standard", "build_0")
    bitmap = bitmaps.BaseBitmap()
    bitmap.InitWith(new_path)
    plugins.RegisterObjectPlugin(id=PLUGIN_ID, str="testcity", g=Testcity, description="testcity", icon=bitmap,
                                 info=c4d.OBJECT_GENERATOR)

PluginFile
testcity.zip

Hello @ThomasB,

Thank you for reaching out to us. This is not a bug, at least not a big one. You clone from a document with materials but do not include these materials in the document. Your cache then contains a broken material reference. Just CTSO your object and you will see that it contains such unresolving material link.

That Physical does not render generators which contain unresolving material references in their caches is probably strictly speaking a bug. But generators should not include any form of material reference in their caches. Because that requires the object managing its own materials which tends to end up in a mess. In your GetVirtualObjects (GVO), you cannot insert materials into the document the node is contained in due to threading restrictions.

Also you loading a document in GVO is a really bad idea. Imagine a user having 100 of your city-objects in a scene with 100 keyframes each. That means 10,000 file access operations, which could be one, yikes! But we are also not on the MT in GVO so the method is executed in parallel, which easily could lead to access violations when two objects are trying to read the same file at the same time.

Cheers,
Ferdinand

File: testcity.zip

Result: I just modified your source file and removed all material links there and your plugin will render in the viewport with Physical.

Code:

import os
import c4d

PLUGIN_ID = 1234567  # Just for testing

class Testcity(c4d.plugins.ObjectData):
    """
    """
    ROOT, _ = os.path.split(__file__)

    # The files to load.
    FILE_PATHS = {
        "build_0": os.path.join(
            ROOT, "res", "models", "models", "0_Standard", "build_0", "build_0.c4d")
    }
    # And the cache.
    DOCUMENT_CACHE = {}

    @classmethod
    def CheckDocumentCache(cls):
        """Rebuilds the document cache.
        """
        if isinstance(cls.DOCUMENT_CACHE, dict) and len(cls.DOCUMENT_CACHE) > 1:
            isDirty = False
            for key, value in cls.DOCUMENT_CACHE.items():
                if not isinstance(value, c4d.documents.BaseDocument) or not value.IsAlive():
                    isDirty = True
                    break
            if not isDirty:
                return

        for key, value in cls.FILE_PATHS.items():
            if not os.path.exists(value):
                raise OSError(f"File path '{value}' does not exist.")

            cls.DOCUMENT_CACHE[key] = c4d.documents.LoadDocument(value, c4d.SCENEFILTER_OBJECTS)

    def __init__(self):
        """
        """
        self.SetOptimizeCache(True)

    def Init(self, node):
        """
        """
        self.CheckDocumentCache()
        return True

    def GetVirtualObjects(self, op, hh):
        """
        """
        self.CheckDocumentCache()
        doc = self.DOCUMENT_CACHE.get("build_0", None)
        if not doc or not doc.IsAlive():
            return c4d.BaseObject(c4d.Onull)

        # When doing such chained calls which can fail use error handling.
        try:
            clone = doc.GetFirstObject().GetDown().GetDown().GetClone()
        except:
            return c4d.BaseObject(c4d.Onull)

        null = c4d.BaseObject(c4d.Onull)

        # Something went really wrong, either #doc is not what we think it is or we cannot even
        # allocate a null object anymore. Indicate a memory error by returning #False.
        if None in (clone, null):
            return False

        clone.InsertUnder(null)
        return null

if __name__ == "__main__":
    path, file = os.path.split(__file__)
    file = "icon.tif"
    new_path = os.path.join(path, "res", file)
    model_path = os.path.join(path, "res", "models", "models", "0_Standard", "build_0")
    bitmap = c4d.bitmaps.BaseBitmap()
    bitmap.InitWith(new_path)
    c4d.plugins.RegisterObjectPlugin(
        id=PLUGIN_ID, str="testcity", g=Testcity, description="otestcity", icon=bitmap,
        info=c4d.OBJECT_GENERATOR)

Hello @ThomasB,

Thank you for reaching out to us. This is not a bug, at least not a big one. You clone from a document with materials but do not include these materials in the document. Your cache then contains a broken material reference. Just CTSO your object and you will see that it contains such unresolving material link.

That Physical does not render generators which contain unresolving material references in their caches is probably strictly speaking a bug. But generators should not include any form of material reference in their caches. Because that requires the object managing its own materials which tends to end up in a mess. In your GetVirtualObjects (GVO), you cannot insert materials into the document the node is contained in due to threading restrictions.

Also you loading a document in GVO is a really bad idea. Imagine a user having 100 of your city-objects in a scene with 100 keyframes each. That means 10,000 file access operations, which could be one, yikes! But we are also not on the MT in GVO so the method is executed in parallel, which easily could lead to access violations when two objects are trying to read the same file at the same time.

Cheers,
Ferdinand

File: testcity.zip

Result: I just modified your source file and removed all material links there and your plugin will render in the viewport with Physical.

Code:

import os
import c4d

PLUGIN_ID = 1234567  # Just for testing

class Testcity(c4d.plugins.ObjectData):
    """
    """
    ROOT, _ = os.path.split(__file__)

    # The files to load.
    FILE_PATHS = {
        "build_0": os.path.join(
            ROOT, "res", "models", "models", "0_Standard", "build_0", "build_0.c4d")
    }
    # And the cache.
    DOCUMENT_CACHE = {}

    @classmethod
    def CheckDocumentCache(cls):
        """Rebuilds the document cache.
        """
        if isinstance(cls.DOCUMENT_CACHE, dict) and len(cls.DOCUMENT_CACHE) > 1:
            isDirty = False
            for key, value in cls.DOCUMENT_CACHE.items():
                if not isinstance(value, c4d.documents.BaseDocument) or not value.IsAlive():
                    isDirty = True
                    break
            if not isDirty:
                return

        for key, value in cls.FILE_PATHS.items():
            if not os.path.exists(value):
                raise OSError(f"File path '{value}' does not exist.")

            cls.DOCUMENT_CACHE[key] = c4d.documents.LoadDocument(value, c4d.SCENEFILTER_OBJECTS)

    def __init__(self):
        """
        """
        self.SetOptimizeCache(True)

    def Init(self, node):
        """
        """
        self.CheckDocumentCache()
        return True

    def GetVirtualObjects(self, op, hh):
        """
        """
        self.CheckDocumentCache()
        doc = self.DOCUMENT_CACHE.get("build_0", None)
        if not doc or not doc.IsAlive():
            return c4d.BaseObject(c4d.Onull)

        # When doing such chained calls which can fail use error handling.
        try:
            clone = doc.GetFirstObject().GetDown().GetDown().GetClone()
        except:
            return c4d.BaseObject(c4d.Onull)

        null = c4d.BaseObject(c4d.Onull)

        # Something went really wrong, either #doc is not what we think it is or we cannot even
        # allocate a null object anymore. Indicate a memory error by returning #False.
        if None in (clone, null):
            return False

        clone.InsertUnder(null)
        return null

if __name__ == "__main__":
    path, file = os.path.split(__file__)
    file = "icon.tif"
    new_path = os.path.join(path, "res", file)
    model_path = os.path.join(path, "res", "models", "models", "0_Standard", "build_0")
    bitmap = c4d.bitmaps.BaseBitmap()
    bitmap.InitWith(new_path)
    c4d.plugins.RegisterObjectPlugin(
        id=PLUGIN_ID, str="testcity", g=Testcity, description="otestcity", icon=bitmap,
        info=c4d.OBJECT_GENERATOR)

@ferdinand
thank you ferdinand the plugin is just a small example of the original. the orginal is much more complicated and with textures.
so thank you for this hint, with broken mat references.

I am aware of that, that it always loads the document. I was unsure about that too. So your CacheMethod handles that...right.

So first I did this all in the Message() Method in the main Thread the doc creation and so forth...after the user chose a model and pressed refresh.So I just switched because it did not work so there must be a bug in my old version. I loaded the doc into a membervariable like you in the CachMethod.....hmmm

Well thank you so far Ferdinad...

Hello @ThomasB ,

without further questions or postings, we will consider this topic as solved by Friday, the 11th of august 2023 and flag it accordingly.

Thank you for your understanding,
Maxon SDK Group