UNSOLVED Object Plugin self.OptimizeCache(False) . How to refresh internal calculations

ThomasB

Hi,
in my Object Plugin I have set the Cache to False in my __init__method

self.OptimizeCache(False)

The problem I have now is. In my code there is a blink function which does some calculation when a specific parameter in the Plugin UI is checked.
Internally it exchanges the reference object in the instances which were returned. And so calculates a looping blink behavior depending on the frame in the timeline.
Since nothing is changing in the UI of the Plugin the blink thing doesn't work if OptimizeCache = True

Is there any possibility to refresh the state of the returned object in another way ...I am not yet so familiar with the message system.
EventAdd() doesn't work.

My current workflow is:
I set OptimizeCache to False if the parameter is checked otherwise it is True.

Best Regards
Tom

ferdinand

Hey @thomasb,

Thank you for reaching out to us. While I understand the general idea of your question, some aspects of it are ambiguous for me. I will answer your main question abstractly, and then list out the things I do not understand below.

Abstract Answer

Generally speaking, BaseObject.OptimizeCache is one of the very few cases where the Python API deviates substantially from the C++ API, as the C++ API does not have such method. The method ties the call frequency of ObjectData.GetVirtualObjects to the parameter dirtyness of the BaseObject instance representing the plugin hook. But nothing prevents you from doing this yourself and adding custom checks. There is an example in the PyRoundedTube code, although the relevant comment is a bit cloudy, I personally would have commented it like this instead:

# When there is no cache or the cache is invalid - CheckCache() returns True for invalid or missing 
# caches and False for valid ones -, or one of the node parameters has changed, then consider the 
# plugin node to be dirty. When the node is NOT dirty in this sense, then return the existing cache.
# This is an approximation of what `ObjectData.SetOptimizeCache(True)` does.
dirty: bool = op.CheckCache(hierarchyhelp) or op.IsDirty(c4d.DIRTYFLAGS_DATA)
if not dirty:
    return op.GetCache(hierarchyhelp)

# Start building and returning a new cache here ...

And doing the dirtyness evaluation manually is usually only interesting when one deviates from the
behaviour of ObjectData.SetOptimizeCache(True), as for example by making the cache frame dependent:

# Stores the last frame for which this plugin instance has build the cache.
self._lastCacheFrame: int 

# Get the node document and the current frame.
doc: c4d.documents.BaseDocument = op.GetDocument()
currentFrame: int = doc.GetTime().GetFrame(doc.GetFps())

# Consider the node dirty when it has no or an invalid cache, is parameter dirty, or the document 
# frame has changed since the last time the cache has been built. One could also store _lastFrame
# inside the data container of the node itself, to have a more robust evaluation. I personally would
# consider this simpler form of storing it on the plugin hook instance to be okay too, as doing it
# inside the container will be non-trival because one has then to avoid dirtyness feedback loops.
dirty: bool = (op.CheckCache(hierarchyhelp) or 
               op.IsDirty(c4d.DIRTYFLAGS_DATA) or 
               self._lastCacheFrame != currentFrame)
if not dirty:
    return op.GetCache(hierarchyhelp)

# Start building and returning a new cache here ...
self._lastCacheFrame = currentFrame

The evaluation usually takes place at the top of ObjectData.GetVirtualObjects as shown in the rounded tube example, but one can also use ObjectData.AddToExecution to manage how often GetVirtualObjects is being called from an earlier point. But in most cases doing it from GetVirtualObjects should be sufficient.

Ambiguities in the Question

blink function [...] a looping blink behavior depending on the [current] frame

I assume you want the cache to be frame dependent, i.e., a new frame means a new cache. But it is not clear to me what you mean by 'blink function or behaviour' specifically.

Internally it exchanges the reference object in the instances which were returned.

I do not understand what you are talking about here. Are you talking about returning the cache of a node when it is not dirty? In general, GVO should never return references. In Python is everything a reference when one takes the term literally, but GVO should not return a node tree which is being used in and owned by another context (i.e., what is commonly dubbed as 'reference'). So, you should not attach the cache output of GVO to for example MyPluginHook._lastCache or have the same cache appear more than once in a scene graph.

I set OptimizeCache to False if the parameter is checked otherwise it is True.

You should not toggle the caching behaviour of a node in such manner. The method is meant to be used once in the __init__ method of the plugin hook. Ignoring this will not induce crashes or malformed data, but one can miss cache updates when switching the behaviour at runtime.

Cheers,
Ferdinand

ThomasB

Sorry Ferdinand for the bad explanation.

The plugin creates Led blocks with 35 Led's each block. And every Led is an c4d.OInstance Object. Depending what LED is on it gets either a dummy object with the luminous material or one with a non luminous material.
Internally I have written specific algorithms to let the Led's blink in different variations.
However, this replacement of these dummy objects in the c4d.Oinstance objects is not displayed if optimize cache is True, because nothing will be changed in the plugin's UI.
So either I set OptimizeCache to False, but then the performance and frame rate drops rapidly.
The user can also turn led's off manually and set Keyframes for this......
But to ensure good framerate and performance the user should also have the opportunity to turn the optmizie cache to True....like it is in the PythonGenerator....

But I looked for a way which gives me better performance as I get with SetOptimizeCache(False)

I already tried the method

dirty: bool = op.CheckCache(hierarchyhelp) or op.IsDirty(c4d.DIRTYFLAGS_DATA)
if not dirty:
    return op.GetCache(hierarchyhelp)

but this doesn´t work because cinema4d doesn't recognize this blinking and exchange in the c4d.Oinstance objects.
so that is the reason why I set optmizeCache to False when the user activates the blinking function, as you can do in the python generator

Best Regards

ferdinand

Hey @thomasb,

Thank you for the additional information. I unfortunately still do not really understand what you are doing. I get the LED part, but where I am a bit lost is the following part:

The plugin creates Led blocks with 35 Led's each block. And every Led is an c4d.OInstance Object. Depending what LED is on it gets either a dummy object with the luminous material or one with a non luminous material. Internally I have written specific algorithms to let the Led's blink in different variations.

When you want to turn an LED on or off, are you rebuilding the cache or are you trying to do some instance target switcheroo? It sounds a bit like the latter, which is of course a violation of cache integrity. A cache is static by definition, having this setup for example would be 'illegal':

Null.0 (Null)
   Sphere.0 (Sphere)

MyPlugin.0
    MyPlugin.0.Cache
        myPlugin.0 (Null)
            Instance.0 (Instance: Null.0)
            Instance.1 (Instance: Null.0)
            Instance.2 (Instance: Null.0)

I.e, a cache which uses nodes of the "public" graph to drive instances. Cinema 4D will not crash when you do this, but caches are by definition static, and replacing Sphere.0 in Null.0 with for example a Cube Object instance will have no impact on the cache of MyPlugin.0. For the same reason, putting a look-at-camera tag on an object inside a tag will have no effect either. Cinema 4D considers caches to be immutable.

You can use instances inside your cache to save memory, but the instance sources must be then inside your cache, and you cannot use instances to animate things. Caches are static by nature, and it sounds to me like you want to replace your cache every frame when you want an 'animation'. The reason is that you are violating two ObjectData patterns here:

ObjectData caches should not yield animations directly/automatically. E.g., let an object oscillate in size every 1 sec by 100%. When you want an object to blink in such fashion, implement a 'Blink' parameter going from 0% to 100% and let the user animate it.
ObjectData caches should not contain materials.

Possible Solutions

Write a tag that drives the luminosity of N materials assigned to your plugin (on the "outside" not in the cache). Each instance of your object requires N unique materials, where N is the number of LEDs your plugin returns. The plugin would have to return the LEDs as a single polygon object and assign polygon selection tags to it. The materials can reach with their selection name inside the cache. You would need almost no cache rebuilds with this variant. The tag could grab its setting from its hosting LED-object when you want to unify the interface.
Write an LED shader which has a link parameter to the LED object it is meant render. The idea is then tu use the shader in the luminance channel of a singular material assigned to the LED object instance the shader paramater links to. When the shader is polled, you determine if the sample point should be on or off. It is very likely that you will need C++ to implement this. Because you will either have to convert a UVW sample point to object space or you will have to implement a volume shader, both things are not possible in Python. The first one is technically possible by implementing the conversion yourself, but that will be very slow. But even if this would be not the case, the remaining math to infer if the that point in object space should be lit or not would probably be still very slow as a Python shader.

Cheers,
Ferdinand

ThomasB

@ferdinand

Thanks Ferdinand,

I additionally implemented such a blink paramater the user can animate Led's on and off
the internal structure of the led construction:

MyPluginNode
    Container(Null-Object)
        Dummies(Null-Object) (invisible in Editor and Render View)
            LED-Object(e.g. disc) with luminous-Material (Material was created and is live)
            LED-Object(e.g. disc) with non-luminous-Material and empty texture tag
        Block-Container(Multi-Instance-Object Referenz: Dummies)
            Block_1 (Null-Object)
                Led_1 (Render-Instance-Object / Reference: eg. LED-Object with luminous-Material
                Led_2 (Render-Instance-Object) / Reference: eg. LED-Object with luminous-Material
                Led_3 (Render-Instance-Object) etc.
                ...
                Led_35 (Render-Instance-Object)
            Block_2 (Null-Object)
                Led_1 (Render-Instance-Object)
                Led_2 (Render-Instance-Object)
                Led_3 (Render-Instance-Object)
                ...
                Led_35 (Render-Instance-Object)
                
            .....
            Block_12
                .....

The Container(Null-Object) is returned
I created so to say my own Cloner with Multiinstances

The blink algorithm just exchanges the Reference in the LED instance object in the Blocks.
The plugin has two Baselist Links for a luminous and non luminous material and if you insert some, one Dummies Object gets the luminous the other one the non-luminous Material
You write the Text you want in a text-field and an internal led-font-dictionary with the indexes decides which of the 35 led's per block and letter are getting the luminous dummy or the non luminous dummy.
If the user doesn't insert a luminous material , the led's just show the returned geometry because the reference material link in the texture tag of the luminous dummy object is empty.

Yes maybe I remove the blink algorithm and let the user just animate the blink manually.
Or I try this Tag thing, can I use a python tag?
I do not know yet how to create such a tag-plugin with code together with my Object-Plugin (blame) and also I do not know how to write a shader . Not yet

Best Regards

ferdinand

Hey @thomasb,

Thank you for the clarification. Yeah, this setup requires you modifying the cache. So, the slow performance version with disabling the optimization is the best you can do when approaching things in such brutish manner.

FYI: I do not have much time this week, so this is all the help you will get this week from me, but I am happy to help you next week if you still need help then.

Things you can do:

Turning off the optimization will calculate the cache every time Cinema 4D asks for it. Depending on how your blinking works, you might not have to calculate the cache every frame. Just determine when a new cache is needed and when not, as demonstrated in my first posting.
When in 99% of the cases 99% of your old invalid cache is still good, nothing prevents you from either caching expensive to compute parts yourself or modifying the existing cache and return that as the new one. Changing a selection state is such an example of where 99.9% of the expensive work is still valid.
It would be quite easy to do, when selection tags could reach into caches, but they cannot. So, you cannot have a selection tag on a generator (in Python) which indexes elements of the cache. But you can have a selection tag inside the cache which is referenced by for example a material on the generator holding the cache.
With this knowledge, you can:
- Write a solution following (2.) where everything happens in the object, but in most cases, you just modify an existing cache instead of creating a new one.
- Do the same, but here you use a tag to modify the cache. This is a little bit dicey, as you should not mess with caches. But in this specific form, where we only change the selection state of a selection tag inside the cache, it should be okay. All other external cache modifications are off limits and can lead to crashes when you do not know what you are doing. In a nicer variant, you would implement the tag as a TagData plugin, but I provided a simple Python programming tag version below.
The shader solution is not viable in Python; it will be too slow. You will also need quite some math knowledge and reverse engineering skills, as you would have to sort of reimplement texture mapping.

PS: In your more complex setup, this could mean that you just change the materials on things. Although consolidating things inside caches is always advantageous. The more generators your cache contains, the more expensive it will be to evaluate the cache of your object. When possible, it is always better to return a single polygon object as your cache result, or at least a tree which contains only null objects and polygon objects, and no generator objects as the Sphere object, instance-objects, cloners, etc., i.e., things which must be cached themselves.

When the cache for an object is being built, all that stuff is converted to polygons anyway. But when you return one hundred instance objects which reference a sphere generator each, cinema will have to build 102 caches in total: one for your object, one for the sphere object, and one hundred for the instance objects. When you just return one polygon object which contains all the geometry, Cinema 4D must build only one cache. In Python this fact is a little bit mitigated by the slowness of Python, and it can be advantageous to push things to C++, but your cache is too complicated IMHO. Just construct your LED once, then build the cache for it, copy the cache thirty-five times, modify the position and material of each copy, and return these thirty-five copies under a null object as your object cache.

Cheers,
Ferdinand

File: led.c4d
Result:

Python Generator object:

import c4d

op: c4d.BaseObject # The Python Generator object containing this code.

def main() -> c4d.BaseObject:
    """Returns a clone of the polygon object linked in its first user data field.
    """
    source: c4d.PolygonObject = op[c4d.ID_USERDATA, 1]
    if not isinstance(source, c4d.PolygonObject):
        return c4d.BaseObject(c4d.Onull)

    clone: c4d.PolygonObject = source.GetClone(c4d.COPYFLAGS_NO_HIERARCHY | c4d.COPYFLAGS_NO_BITS)
    clone.SetMg(c4d.Matrix())
    return clone

Python Programming tag:

import c4d

doc: c4d.documents.BaseDocument # The document evaluating this tag.
op: c4d.BaseTag # They Python Programming tag containing this code.

def main():
    """Reaches into the cache of its host object and modifies it.
    """
    # Get the host object, its cache, and find the polygon selection tag on it.
    obj: c4d.BaseObject = op.GetMain()
    if not isinstance(obj, c4d.BaseObject):
        return

    cache: c4d.BaseObject = obj.GetCache()
    if not isinstance(cache, c4d.PolygonObject):
        return

    tag: c4d.SelectionTag = cache.GetTag(c4d.Tpolygonselection)
    if not isinstance(tag, c4d.SelectionTag):
        return

    # Get the current document frame and the selection of the tag and flush it. In practice you could
    # also make this parameter driven, but for expressions, tags, it is also fine to make things
    # automatic as such.
    frame: int = doc.GetTime().GetFrame(doc.GetFps())
    bs: c4d.BaseSelect = tag.GetBaseSelect()
    bs.DeselectAll()

    # Define the indices of 100 cap polygons, and pick the polygon which matches the current time.
    states: list[int] = [n for n in range(4, 599, 6)]
    i: int = states[frame % 100]

    # Set the new selected element.
    bs.Select(i)