Why are GUIDs not globally unique?

On 08/08/2017 at 02:32, xxxxxxxx wrote:

User Information:
Cinema 4D Version:    
Platform:   Windows  ;   
Language(s) :     C++  ;

---------
Hey guys,

I am currently doing a lot of scene parsing and need to identify objects between different passes.

I noticed, that GUID (GetGUID()) are actually not globally unique. I've found objects with the same GUID in different caches (GetCache()). In this example, they are all different Cube objects (apparently with the same setting, but at different locations/hierarchy in the scene). Different caches have a PolygonObject with the identical GUID. So I cannot rely on the GUID to identify them!

I now have so many different approaches to persistently and globally identify objects across different parsing passes but it's now at a state that has only fringe cases and no working normal case.

How can I uniqely identify objects in Cinema 4D that survives document cloning and generator updates?

I am parsing the Scene using a Hierarchy derived class.

Here's a excerpt form my log: Notice how 4b1f714b50e82730 as GUID shows up several times.

"    Cubes                                 <Null>                   [tachyon id: 9]                                    (IP: 0)                               GUID: 2d8cafd00775eb9"
"        Cube                             <Cube>                   [tachyon id: 10]                                    (IP: 0)   (control object)            GUID: f477d54c6fb6c0e2"
"            Cube                          <Polygon>               [tachyon id: 11]                             CACHED                                        GUID: 99be06871203b0ed"
"        Cube.1                            <Cube>                   [tachyon id: 12]                                    (IP: 0)   (control object)            GUID: e7bf04001d4713f7"
"            Cube.3                        <Cube>                   [tachyon id: 13]                                    (IP: 0)   (control object)            GUID: 291b1c6c76497667"
"               Cube.3                    <Polygon>               [tachyon id: 14]                             CACHED                                ADDED   GUID: 4b1f714b50e82730"
"            Cube.1                        <Polygon>               [tachyon id: 15]                             CACHED                                        GUID: edb96c535e457807"
"        Cube.2                            <Cube>                   [tachyon id: 16]                                    (IP: 0)   (control object)            GUID: fde0836b0541a662"
"            Cube.3                        <Cube>                   [tachyon id: 17]                                    (IP: 0)   (control object)            GUID: 8a1bf116c387044a"
"               Cube.3                    <Polygon>               [tachyon id: 14]                             CACHED                                ADDED   GUID: 4b1f714b50e82730"
"            Cube.2                        <Polygon>               [tachyon id: 15]                             CACHED                                ADDED   GUID: edb96c535e457807"
"    Disc                                  <Disc>                   [tachyon id: 18]                                    (IP: 0)   (control object)            GUID: f4e712ea7f50171e"
"        Disc                             <Polygon>               [tachyon id: 19]                             CACHED                                        GUID: 91dcd02f1ff6391d"
"    Transforms                            <Null>                   [tachyon id: 20]                                    (IP: 0)                               GUID: 5e81f406e2c53450"
"        roated_cube                      <Polygon>               [tachyon id: 21]                                    (IP: 0)                               GUID: 97d092171cec6a79"
"            rotated_cube_top              <Polygon>               [tachyon id: 22]                                    (IP: 0)                               GUID: b3f5d4b7071c2e5e"
"        roated_cube                      <Polygon>               [tachyon id: 23]                                    (IP: 0)                               GUID: a397640e1f0359b6"
"            rotated_cube_top              <Polygon>               [tachyon id: 24]                                    (IP: 0)                               GUID: 83a0776a00658095"
"        roated_cube                      <Polygon>               [tachyon id: 25]                                    (IP: 0)                               GUID: 5f31384399b1069a"
"            rotated_cube_top              <Polygon>               [tachyon id: 26]                                    (IP: 0)                               GUID: ae898e933fbe4981"
"        roated_cube                      <Polygon>               [tachyon id: 27]                                    (IP: 0)                               GUID: f3c2a611f0abfa94"
"            rotated_cube_top              <Polygon>               [tachyon id: 28]                                    (IP: 0)                               GUID: 7b159faec312eae9"
"    Instances                             <Null>                   [tachyon id: 29]                                    (IP: 0)                               GUID: 73c064e77b243bd6"
"        Platonic                          <Polygon>               [tachyon id: 30]                                    (IP: 0)                               GUID: 1330ea8421bf03f7"
"        Platonic Instance                 <Instance>               [tachyon id: 216]                                    (IP: 0)                               GUID: 1239cddc0645b377"
"        Platonic Instance                 <Instance>               [tachyon id: 217]                                    (IP: 0)                               GUID: d6d49c38675cfe5b"
"    Cloth Surface                         <Cloth Surface>          [tachyon id: 31]                                    (IP: 0)   (control object)            GUID: bda8395b7e07eedc"
"        cloth base-1.1                    <Polygon>               [tachyon id: 32]                                    (IP: 0)   (control object)            GUID: 7be07e6f7c1fc70c"
"        cloth base-1.1                    <Null>                   [tachyon id: 5]                             CACHED                                        GUID: 171a5cd795410cdf"
"            cloth base-1.1               <Polygon>               [tachyon id: 33]                             CACHED (IP: 1)                               GUID: a65e706f4f5413d7"
"    Subdivision Surface                   <Subdivision Surface>    [tachyon id: 34]                                    (IP: 0)   (control object)            GUID: 5af996a4082be520"
"        subdiv-base                      <Cube>                   [tachyon id: 35]                                    (IP: 0)   (control object)            GUID: f22ea4d687d56aec"
"            subdiv-base                   <Polygon>               [tachyon id: 14]                             CACHED            (control object)            GUID: 4b1f714b50e82730"
"        subdiv-base                      <Polygon>               [tachyon id: 36]                             CACHED                                        GUID: 9a14fc7b254e33cc"
"    displaced-plane-parent               <Null>                   [tachyon id: 37]                                    (IP: 0)                               GUID: e546b3bdebf8fc7"
"        displaced-plane                   <Plane>                  [tachyon id: 38]                                    (IP: 0)   (control object)            GUID: 8ba76ad96aee805e"
"            Twist                         <Twist>                  [tachyon id: 39]                                    (IP: 0)                               GUID: e7f35e39d62a0f70"
"            Displacer                     <Displacer>              [tachyon id: 39]                     (dirty)        (IP: 0)                               GUID: dac77b01a3468647"
"            displaced-plane               <Polygon>               [tachyon id: 40]                             CACHED            (control object)            GUID: 6234c4a36df28952"
"               displaced-plane           <Polygon>               [tachyon id: 41]                     (dirty) CACHED                                        GUID: e234129dad8b3e5b"
"    Visibility                            <Null>                   [tachyon id: 42]                                    (IP: 0)                               GUID: a04fc4d58d77da9c"
"        vis.cube.1                        <Cube>                   [tachyon id: 43]                                    (IP: 0)   (control object)            GUID: af852c6b6888025a"
"            vis.cube.2                    <Cube>                   [tachyon id: 44] (hidden)                            (IP: 0)   (control object)            GUID: 11599b0d69eb05ef"
"               vis.cube.3               <Cube>                   [tachyon id: 45] (hidden)                            (IP: 0)   (control object)            GUID: 3194f419e734775a"
"                    vis.cube.3            <Polygon>               [tachyon id: 14] (hidden)                    CACHED                                        GUID: 4b1f714b50e82730"
"               vis.cube.4               <Cube>                   [tachyon id: 45]                                    (IP: 0)   (control object)            GUID: fc6388f59208063f"
"                    vis.cube.4            <Polygon>               [tachyon id: 14]                             CACHED                                        GUID: 4b1f714b50e82730"
"               vis.cube.5               <Cube>                   [tachyon id: 45] (hidden)                            (IP: 0)   (control object)            GUID: a14399de90fd95c5"
"                    vis.cube.5            <Polygon>               [tachyon id: 14] (hidden)                    CACHED                                        GUID: 4b1f714b50e82730"
"               vis.cube.2               <Polygon>               [tachyon id: 14] (hidden)                    CACHED                                        GUID: 4b1f714b50e82730"
"            vis.cube.1                    <Polygon>               [tachyon id: 14]                             CACHED                                ADDED   GUID: 4b1f714b50e82730"

On 08/08/2017 at 04:09, xxxxxxxx wrote:

I think you are looking for this kind of thing.
https://plugincafe.maxon.net/topic/9825/13230_unique-identification-of-scene-elements

But if you want the cache object to be the same than the live object you have to add you own GeMarker.

On 08/08/2017 at 04:37, xxxxxxxx wrote:

Thanks! Unfortuantely I've been through that.

GeMarker get changed all the time. Especially when cloning the scene and when generators re-create their cache objects. So the marker is of no use to identify an object after "something" happened to the scene.

The FindUniqueID() returns the Marker.

I've implemented my own UniqueID() where I enumerate objects, but that again gets lost with objects in caches.

So I started to cache my own IDs for cache objects or objects that are created by Control Objects (provided they assign a UniqueIP(), which is not always true).

Here is another typical "offender" in Cinema 4D. The Figure object. Many generated objects share the same GUID, most objects have the same UniqueIP.

"Figure                                    <Figure>                 [tachyon id: 1]                                    (IP: 0)   (control object)            GUID: bdb68961803fe8c3"
"    Figure                                <Polygon>               [tachyon id: 2]                             CACHED                                        GUID: 193a4baabce7ca18"
"        Left Thigh                        <Polygon>               [tachyon id: 3]                             CACHED (IP: 1)                               GUID: acf48aadc2a85fa3"
"            Left Shin                     <Polygon>               [tachyon id: 4]                             CACHED (IP: 1)                      ADDED   GUID: c8cea1d09fd46bf"
"               Left Foot                 <Polygon>               [tachyon id: 5]                             CACHED (IP: 1)                               GUID: 12763bb2f406aaaa"
"                    Left Foot Effector    <Polygon>               [tachyon id: 6]                             CACHED (IP: 1)                               GUID: 7b4ff7c0ab59ef1a"
"                    Left Joint            <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"               Left Joint               <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"            Left Joint                    <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"        Right Thigh                      <Polygon>               [tachyon id: 8]                             CACHED (IP: 2)                               GUID: 28988867afadd6a7"
"            Right Shin                    <Polygon>               [tachyon id: 4]                             CACHED (IP: 1)                      ADDED   GUID: c8cea1d09fd46bf"
"               Right Foot               <Polygon>               [tachyon id: 5]                             CACHED (IP: 1)                      ADDED   GUID: 12763bb2f406aaaa"
"                    Right Foot Effector   <Polygon>               [tachyon id: 6]                             CACHED (IP: 1)                               GUID: 7b4ff7c0ab59ef1a"
"                    Right Joint           <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"               Right Joint               <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"            Right Joint                   <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"        Upper Body                        <Polygon>               [tachyon id: 9]                             CACHED (IP: 3)                               GUID: 95132978ec08bc2a"
"            Left Upper Arm               <Polygon>               [tachyon id: 4]                             CACHED (IP: 1)                      ADDED   GUID: c8cea1d09fd46bf"
"               Left Lower Arm            <Polygon>               [tachyon id: 4]                             CACHED (IP: 1)                               GUID: c8cea1d09fd46bf"
"                    Left Hand             <Polygon>               [tachyon id: 10]                             CACHED (IP: 1)                               GUID: e10af12c73f4115e"
"                        Left Hand Effector<Polygon>               [tachyon id: 6]                             CACHED (IP: 1)                               GUID: 7b4ff7c0ab59ef1a"
"                        Left Joint        <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"                    Left Joint            <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"               Left Joint               <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"            Right Upper Arm               <Polygon>               [tachyon id: 11]                             CACHED (IP: 2)                               GUID: 88e0e8d764f8cfbb"
"               Right Lower Arm           <Polygon>               [tachyon id: 4]                             CACHED (IP: 1)                      ADDED   GUID: c8cea1d09fd46bf"
"                    Right Hand            <Polygon>               [tachyon id: 10]                             CACHED (IP: 1)                      ADDED   GUID: e10af12c73f4115e"
"                        Right Hand Effector<Polygon>               [tachyon id: 6]                             CACHED (IP: 1)                               GUID: 7b4ff7c0ab59ef1a"
"                        Right Joint       <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"                    Right Joint           <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"               Right Joint               <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                               GUID: 11a45cd55e45476f"
"            Head                          <Polygon>               [tachyon id: 9]                             CACHED (IP: 3)                      ADDED   GUID: 95132978ec08bc2a"
"               Head Effector             <Polygon>               [tachyon id: 6]                             CACHED (IP: 1)                               GUID: 7b4ff7c0ab59ef1a"
"               Neck                      <Polygon>               [tachyon id: 7]                             CACHED (IP: 2)                      ADDED   GUID: 11a45cd55e45476f"
"            Joint                         <Polygon>               [tachyon id: 12]                             CACHED (IP: 4)                               GUID: 5b8cb8aa2da463f4"

On 08/08/2017 at 04:43, xxxxxxxx wrote:

Crap, I just noticed that the UniqueIP is assigned uniquely per hierarchy level. That might help but makes it more involved. I'll try that.

Still in my first example the IP is no help.

On 08/08/2017 at 05:57, xxxxxxxx wrote:

Haha, Cinema is really inconsistent with IPs, GUIDs etc.

Putting a hierarchy inside a subdivision surface object makes them all control objects but puts the resulting geometry inside the SDS cache but never cares to assigns uniqueIPs.

All uniqueIPs are 0, duplicate GUIDs for armor_MDL and armor_MDL.1

It's getting really hard (ie. actually impossible) to identify objects over several passes.

"        Null                             <Null>                   [tachyon id: 85]                             CACHED                                        GUID: 171a5cd795410cdf"
"            Belts_MDL                     <Null>                   [tachyon id: 86]                             CACHED (IP: 0)                               GUID: 3534142a6ce566d"
"               Belts_MDL                 <Polygon>               [tachyon id: 87]                             CACHED (IP: 0)                               GUID: de3b7097a4626466"
"            body_mdl                      <Null>                   [tachyon id: 86]                             CACHED (IP: 0)                               GUID: 3534142a6ce566d"
"               body_mdl                  <Polygon>               [tachyon id: 88]                             CACHED (IP: 0)                               GUID: 353ef4cca795f026"
"               Poses: body_mdl           <Null>                   [tachyon id: 86]                             CACHED (IP: 0)                               GUID: 3534142a6ce566d"
"                    body_morph_6          <Polygon>               [tachyon id: 88] (hidden)                    CACHED (IP: 0)                               GUID: 353ef4cca795f026"
"                    body_morph_4          <Polygon>               [tachyon id: 88] (hidden)                    CACHED (IP: 0)                               GUID: 353ef4cca795f026"
"                    body_morph_3          <Polygon>               [tachyon id: 88] (hidden)                    CACHED (IP: 0)                               GUID: 353ef4cca795f026"
"                    body_morph_2          <Polygon>               [tachyon id: 88] (hidden)                    CACHED (IP: 0)                               GUID: 353ef4cca795f026"
"                    body_morph_1          <Polygon>               [tachyon id: 88] (hidden)                    CACHED (IP: 0)                               GUID: 353ef4cca795f026"
"                    Default: body_mdl     <Polygon>               [tachyon id: 88] (hidden)                    CACHED (IP: 0)                               GUID: 353ef4cca795f026"
"            armor_MDL.1                   <Null>                   [tachyon id: 86]                             CACHED (IP: 0)                               GUID: 3534142a6ce566d"
"               armor_MDL.1               <Polygon>               [tachyon id: 89]                             CACHED (IP: 0)                               GUID: d39c0b252a5c2853"
"            armor_MDL                     <Null>                   [tachyon id: 86]                             CACHED (IP: 0)                               GUID: 3534142a6ce566d"
"               armor_MDL                 <Polygon>               [tachyon id: 89]                             CACHED (IP: 0)                      ADDED   GUID: d39c0b252a5c2853"
"            stuff_mdl                     <Null>                   [tachyon id: 86]                             CACHED (IP: 0)                               GUID: 3534142a6ce566d"
"               stuff_mdl                 <Polygon>               [tachyon id: 90]                             CACHED (IP: 0)                               GUID: 1f51476382fb6469"
"            wasa                          <Null>                   [tachyon id: 86]                             CACHED (IP: 0)                               GUID: 3534142a6ce566d"
"               wasa                      <Polygon>               [tachyon id: 91]                             CACHED (IP: 0)                               GUID: 7629de13d5354fdc"

On 08/08/2017 at 06:46, xxxxxxxx wrote:

IDs of objects in the cache are only consistent if the generator re-uses the cache. Otherwise, a new
object is allocated which will have a new UniqueID. GetUniqueIP() appears to be set manually by the
generator, where the IDs are unique in the domain of the objects that the generator creates virtually.

Not sure exactly whether all generator plugins do this properly or whether this is done automatically,
I know that the Cloner object for instance assign proper IDs starting from 0 using (presumable
manually using SetUniqueIP() after a clone is created).

On 08/08/2017 at 08:27, xxxxxxxx wrote:

I have spent 2 months on the topic now so I have a pretty good grasp of how Cinema 4D manages objects. It's the fringe cases that drive me nuts now.

Cinema has these methods to "identify" objects:
* GeMarker
* UniqueID (maps to GeMarker for Maxon, plugins can use custom Data)
* UniqueIP (control objects enumerate created children, where sub-objects can have their own set of UniqueIP again)
* GUID (which should be globally unique, but actually different objects have the same GUID)

All these IDs have a theoretical concept that is regretfully broken in certain cases.

GeMarker changes frequently. Cache re-builds, document cloning (ie. for rendering), etc. It does not live very long.

Vendor specific UniqueID: caches that rebuild get rid of objects so the ID gets lost.

GUID: is actually not unique.

UniqueIP: depending on the generator and the existing hierarchy this is implemented differently (unique IPs for all created objects; unique IPs for only one level in the hierarchy; no IPs at all - either not set or 0)

Also, control objects generally create their objects within the cache. But not all. Subdivision Surface Object marks the children as control objects and puts resulting, subdivided PolygonObjects in its cache but neglects to identify them. The associated control object (the original input PolygonObject) is not the parent of the result like in other cases.

I've got dozens of scenes, with thousands of objects, working. It's these fringe cases like the subdivision surface object, that defy documentation and common logic :-(

Cheers,
Martin

EDIT: I'm still hoping I am missing something simple ;-)

On 13/08/2017 at 05:15, xxxxxxxx wrote:

Why do you need that?

1. To track object updates inside the renderer?  Seems to be no way using IDs. They are different in Editor or PV modes. There were topics by Frozen Tarzan about this.
2. Do you need to make Variation shaders depending on object IDs? If this case there is an option of using the hash of object names in the hierarchy for this purpose. The hash changes if an object is moved from one level to another of the hierarch. However, when objects are created then by default C4D assign new unique names. 
   It is possible to implement object tags with autogenerated IDs that are saved with them. The tricky part is when the objects are copy-pasted and when objects are removed. This should be resolved in the ideal case. Maybe, using time of creation, intercepting copy-paste events and etc. etc. All of this seems to be a waste of time for such a small task. Easier is to make a button with "Regenerate Unique IDs" :)

On 14/08/2017 at 00:36, xxxxxxxx wrote:

Hi Aaron,

Thanks for your reply.

1. Yes, tacking objects across different rendering calls and also for animation. Frozen Tarzan was a team member who started doing this a while back.

2. No, not for a shader. Hashing the path/names is also no option as object can have the same names. Hashing the path would be vulnerable to changed object order in one single hierarchy. So you would have to track position within hierarchy as well. Again, this is very vulnerable.

I am generating IDs on each render pass for objects that have not been identified already. The problem is, in Cinema 4D most objects that are visible to the renderer do not exist in the "normal" hierarchy but are part of caches or deform caches. For example, if you put a character into a Subdivision Object, none of the original objects will make it to the renderer. They are all marked as control objects. Other than with other control objects, their resulting geometry is not found in their own caches though but in the Subdivsion Object. Unfortunately, the Subdivision Object does not create unique IPs as other generators. So there is no way to identify the created geometry in a persistent way.

To use an object tag (hidden and automatically managed) was one option I was thinking about but using the AddUniqueID() method is easier. This is also usable for objects that are not able to have Tags.

My problem is really to identify objects in some edge cases. So far I have it working for most situations. Even for instances of complete hierarchies. Objects like the Subdivision Object seem to work quite differently and in contrast to what the documentation recommends that third party developers should do.

As for the waste of time: We have render times of a few milliseconds, so each fraction of a second I can save transferring scene data is worth it in our opinion. In the end, the goal is to render the scene live ;-)

Thanks for your help.
Martin

On 14/08/2017 at 06:16, xxxxxxxx wrote:

Hi Martin, first of all thanks for writing us.

I strongly apologize for getting later than expect to you with reference to uniquely identifying objects in Cinema but it's taking me longer than expected to formulate a comprehensive answer (touching different cases) to prove that it's indeed possible and consistent with how Cinema is internally designed.

I kindly ask you for a little more patience in order to reorder ideas and get back once and (maybe) for all on the topic.

Best, Riccardo

On 14/08/2017 at 13:06, xxxxxxxx wrote:

Hi Assoc,

Btw, have a look at a problem as in this thread: 
https://plugincafe.maxon.net/topic/580/13734_posemorph-tag-always-increases-getdirty-counter

What is relevant for your case is that in some cases the object generated caches/addresses  may change very frequently. Without even any user change. The object IDs may break rapidly.

Yes, using hierarchical name hashing has dissadvantage that names on the same level can be same and object ID change if we move the object up/down through the hierarchy. But good thing is that by default Cinema makes them different :)

One more idea to think about for you is to assign object IDs with some clever method until certain level (until a generator like subdiv or cloner). And starting from the generator and deeper (when we work with caches and objects that depend on scattering parameters) we may assign depth-first order enumeration for children.

I look after this topic too to get more options for my goals.
Btw, I have a problem. Not very huge however. It's simply impossible to get any object ID in a method

Bool NodeData::InitGLImage(BaseMaterial* mat, BaseDocument* doc, BaseThread* th, BaseBitmap* bmp, Int32 doccolorspace, Bool linearworkflow)
{}

This method is called to get a texture 256x256 for object color in the viewport. So the custom Variation shaders look the same in the Viewport. And if you look for what objects this material is assigned too then there can be several.

On 15/08/2017 at 07:43, xxxxxxxx wrote:

funny that this thread is active.. i was just looking for smth like this...

im trying to store an object using a string, to find it again later
obviously the name is not enough, there can be multiple objects with the same name
so is there a possibilty to get an id or index for an object?
GetGUID() gives me a number, but how can i select this object again if i have the GUID?
note: im using this only for real objects (no generators, cache or whatever)
is GUID safe for that?

and... if you str(object) you get a number.... 0x.... is this usable or just a memory position?

On 15/08/2017 at 23:44, xxxxxxxx wrote:

Hi Aaron,

in InitGlImage you don't have an object reference AFAIK. I was briefly looking into that. My conclusion was, I needed to overwrite GLDraw() which is not documented. From what I saw you need to build your own GLSL shaders with some undocumented methods. There are some GL examples in the SDK that can get you a step or two further but ultimately without documentation how GLSL viewport shaders are build by Cinema 4D this needs quite some effort in reverse engineering ;-)

Cheers,
Martin

On 15/08/2017 at 23:48, xxxxxxxx wrote:

Hi Zeorge,

str(object) will give you undefined behavious. It's converting the memory content of the object to a string. Just don't do that!

I would try GetGUID(). I have found cases were objects receive the same GUID (see above) but that seems to be restricted to caches. The method suggested in the SDK is GetMarker(). This will be unique for each object. It will change frequently though. If you need the IDs for only one pass parsing through the scene and have no requirement for them to be persistent, Marker should work fine.

On 15/08/2017 at 23:52, xxxxxxxx wrote:

BTW: I am assigning IDs to objects in a first parsing pass and build my own cache of IDs for all objects in caches based on their Control Object parent and their IPs. Where this is not possible, I try GUID and where that fails I resort to Marker to differtiate between objects (though, no longer can identify them).

It seems that different generators behave differently. Putting a character in a subdivision objects yields quite different structural results than using ie. Cloners. Using deformers again has a different structure.

And I haven't even started to talk about instances of hierarchies. They pose their own set of challenge that I am about to finish.

On 15/08/2017 at 23:52, xxxxxxxx wrote:

Thanks Riccardo.

On 18/08/2017 at 04:25, xxxxxxxx wrote:

Hi Martin, first and foremost thansk for your patience.

This week has been busy not to have been cope with reordering my findings in a short time and I also wanted to extend my research to reduce the "corner case" risk.

Cinema 4D API offers basically two means to uniquely identify objects: BaseObject::GetGUID() and BaseList2D::GetMarker() which respectively access the object's unique ID and object's GeMarker.

The intent of my research has been to demonstrate that GUID(s) is indeed a reliable way to track objects in Cinema across different user interaction scenarios and aided by GeMarker(s) they offer a complete way to track objects uniquely around Cinema.

Scanning a very simple scene and printing the IDs and GeMarkers info belonging to the objects present provides the following table:

===================================================================================================  
  [ { GeMarker CRC /  GetMarkerStampEx  } |     GetGUID        ]                                     
  [ { 000459316928 / 0014570499 # 07496 }                      ] TraversalTest01.c4d  
  [ { 003110699520 / 0014622782 # 07807 } | 0x739249CE036A085C ][0]Cube.A : Cube, 5159-0/0/0/0-0/1  
  [ { 000611561600 / 0002058671 # 09006 } | 0x4B1F714B50E82730 ][0][C]Cube.A : Polygon, 5100-0/0/0/0-1/1  
  [ { 003620385536 / 0063246992 # 13019 } | 0x29738F0DAFFED8E7 ][1]+Sphere.A : Sphere, 5160-0/0/0/0-0/1  
  [ { 002984169472 / 0002058671 # 09014 } | 0x51A98A6F2EF6E7EF ][1]+[C]Sphere.A : Polygon, 5100-0/0/0/0-1/1  
  [ { 003436609536 / 0002058671 # 09023 } | 0xD1A95C51EE8F50E6 ][1]+[C][D]Sphere.A : Polygon, 5100-0/0/0/0-1/1  
  [ { 000174035360 / 0088455968 # 08560 } | 0x05B40FA55FC2F5DF ][2]++Bulge : Bulge, 5129-0/0/0/0-0/1  
  [ { 003097937408 / 0063290296 # 13696 } | 0x4EE5F93031C05A1A ][0]Sphere.C : Sphere, 5160-0/0/0/0-0/1  
  [ { 002138721536 / 0002058672 # 09030 } | 0x51A98A6F2EF6E7EF ][0][C]Sphere.C : Polygon, 5100-0/0/0/0-1/1  
  [ { 003100995840 / 0014593669 # 07682 } | 0x2879AC49124B66C3 ][1]+Cube.C : Cube, 5159-0/0/0/0-0/1  
  [ { 000041026420 / 0002058672 # 09039 } | 0x4B1F714B50E82730 ][1]+[C]Cube.C : Polygon, 5100-0/0/0/0-1/1  
  [ { 002939917824 / 0014594104 # 07733 } | 0x06B497DDDB536CC5 ][0]Cube.D : Cube, 5159-0/0/0/0-0/1  
  [ { 002548005888 / 0002058672 # 09047 } | 0x646E97C2D42A60BE ][0][C]Cube.D : Polygon, 5100-0/0/0/0-1/1  
  [ { 001481651072 / 0002058672 # 09085 } | 0xE46E41FC1453D7B7 ][0][C][D]Cube.D : Polygon, 5100-0/0/0/0-1/1  
  [ { 000965783296 / 0088388848 # 08324 } | 0x062A5B18C994B055 ][1]+Bend : Bend, 5128-0/0/0/0-0/1  
  [ { 004036197120 / 0014594509 # 07747 } | 0x42F618087566F85F ][1]+Cube.E : Cube, 5159-0/0/0/0-0/1  
  [ { 001382765824 / 0002058672 # 09055 } | 0x646E97C2D42A60BE ][1]+[C]Cube.E : Polygon, 5100-0/0/0/0-1/1  
  [ { 002404496384 / 0002058672 # 09078 } | 0xE46E41FC1453D7B7 ][1]+[C][D]Cube.E : Polygon, 5100-0/0/0/0-1/1  
  [ { 002284037888 / 0088342184 # 08167 } | 0x7E14190FED9A5DE4 ][2]++Twist : Twist, 5134-0/0/0/0-0/1  
  [ { 001033116672 / 0014594922 # 07761 } | 0x1ACC3DCCC83D6EE4 ][2]++Cube.F : Cube, 5159-0/0/0/0-0/1  
  [ { 001744077952 / 0002058672 # 09063 } | 0x4B1F714B50E82730 ][2]++[C]Cube.F : Polygon, 5100-0/0/0/0-1/1  
  [ { 002722147328 / 0002058672 # 09071 } | 0xCB1FA77590919039 ][2]++[C][D]Cube.F : Polygon, 5100-0/0/0/0-1/1

In the table above the info after GUID is as follows: [ n]+++[C][D]<name> : <type name>, <type>-<other status flags>

• [n]: n equals the nesting level in the hierarchy;
• +++: graphical representation of the nesting level;
• [C][D]: C refers to a cache object, D refers to a deform cache object;
• <name>: refers to the object name;
• <type name>: refers to the object's type name;
• <type>: refers to the object type ID;
• <other status flags>: other flags.
  In order to validate the GUID and GeMarker generation mechanism and verify how it reacts upon scene modification a few interaction scenarios have been run (in cascade) on a simple test scene with identification data printed at each step.

The test scenes are available here:

• ObjectsIdentification_SimpleTests.c4d
• ObjectsIdentification_MographTests.c4d
• ObjectsIdentification_ParticlesTests.c4d
  The complete logs of the tests run are available here (the MoGraph and CA tests include examples scene took from the Content Browser) :
• ObjectsIdentification_SimpleTests.txt
• ObjectsIdentification_MographTests.txt
• ObjectsIdentification_ParticlesTests.txt
• ObjectsIdentification_CATests.txt
  _ NOTE: The value for GeMarker is obtained by computing the CRC32 based on the values returned by GeMarker::GetMemory()._

...
    bl->GetMarker().GetMemory(data, size);
    const UInt32 crc32 = ZipFile::CalcCRC32(data, size);
    markerString = String::FloatToString(Float32(crc32), 12, 0)
...

_ NOTE: The value for the GUID is obtained by using String::HexToString conversion on the returned value from GetGUID. Although the value returned looks like a memory address it is not._

...
    const UInt64 guid = obj->GetGUID();  
  const String guidString = String::HexToString(guid);
...

Opening the file TraversalTest_01.c4d leads to the following results:

• all GeMarker(s) (as well as the GetMarkerStampEx value pair(s)) belonging to generators are unique;
• all GeMarker(s) belonging to caches (standard and deform) are unique;
• all GUID(s) belonging to generators are unique;
• all GUID(s) belonging to caches (standard and deform) with equal number of points and polygons are equal but different from their generator's GUID;
• equal GUID(s) belonging to cache objects (sharing same number of points/polygons) are different from the GUID(s) of the deform cache objects (which are equal as well).

Rendering View (Ctrl-R) leads to the following results:
_ NOTE: _During a rendering process only object caches representing the final geometry of the objects are provided in the VolumeData() instance which means the ID data is extracted from the objects actually used by the Renderer.
To retrieve the initial generators it's required to implement a recursive function using BaseObject::GetCacheParent() to iterate up to the parent generator. Objects listed as "CacheP" are the root-cache parent of a cache object.

• all GeMarker(s) belonging to caches (standard and deform) found in the VolumeData instance are equal to those in the source scene;
• all GeMarker(s) belonging to generators (parents of the caches found in the VolumeData) are equal to those in the source scene;
• all GUID(s) belonging to generators and caches (standard and deform) found in the VolumeData instance are equal to those in the source scene;

_
_
Rendering via IRR (Alt-R) leads to the following results:
NOTE: During a rendering process only object caches representing the final geometry of the objects are provided in the VolumeData() instance which means the ID data is extracted from the objects actually used by the Renderer.
_To retrieve the initial generators it's required to implement a recursive function using BaseObject::GetCacheParent() to iterate up to the parent generator.  _ Objects listed as "CacheP" are the root-cache parent of a cache object.
IRR clones the active scene to run the interactive rendering in a separate thread and leave Cinema 4D free to interact with the user.

• all GeMarker(s) belonging to caches (standard and deform) found in the VolumeData instance are new and unique;
• all GeMarker(s) belonging to generators (parents of the caches found in the VolumeData) are equal to those in the source scene;
• all GUID(s) belonging to generator and caches (standard and deform) found in the VolumeData instance are equal to those in the source scene;

Rendering in PV (Ctrl-R) leads to the following results:
_ NOTE: During a rendering process only object caches representing the final geometry of the objects are provided in the VolumeData() instance which means the ID data is extracted from the objects actually used by the Renderer. _
_To retrieve the initial generators it's required to implement a recursive function using BaseObject::GetCacheParent() to iterate up to the parent generator.  _ Objects listed as "CacheP" are the root-cache parent of a cache object.
Render to PV clones the active scene to run the rendering in a separate thread and leave Cinema 4D free to interact with the user.

• all GeMarker(s) belonging to caches (standard and deform) found in the VolumeData instance are new and unique;
• all GeMarker(s) belonging to generators (parents of the caches found in the VolumeData) are equal to those in the source scene;
• all GUID(s) belonging to generator and caches (standard and deform) are new and unique;

Cloning the scene (via simple Python script) leads to the following results:

• all GeMarker(s) belonging to cloned generators are new and unique;
• all GeMarker(s) belonging to cloned caches (standard and deform) are new and unique;
• all GUID(s) belonging to cloned generators are new and unique;
• all GUID(s) belonging to cloned caches (standard and deform) with equal number of points and polygons are equal in the cloned scene and are also equal to the corresponding ones in the source scene.

_
_
Copy &pasting active document's objects into a new document leads to the same results brought by Cloning the scene;

Swapping from cloned scene to source scene leads to the following results:

• all GeMarker(s) belonging to generators are unchanged;
• all GeMarker(s) belonging to caches (standard and deform) are new and unique (because caches are regenerated during document swapping);
• all GUID(s) belonging to generators are unchanged (even the object affected by the change);
• all GUID(s) belonging to caches (standard and deform) are unchanged.
• all GUID(s) belonging to caches (standard and deform) with equal number of points and polygons are equal.

Changing a parameter affecting geometry representation (Fillet "ON" in Cube.C) leads to the following results:

• all GeMarker(s) belonging to generators are unchanged;
• GeMarker(s) belonging to unmodified caches (standard and deform) are unchanged;
• GeMarker(s) belonging to modified caches (standard and deform) are new and unique;
• all the GUID(s) belonging to generators are unchanged (even the object affected by the change);
• GUID(s) belonging to unmodified caches (standard and deform) are unchanged;
• all GUID(s) belonging to caches (standard and deform) with equal number of points and polygons are equal;
• GUID(s) belonging to modified caches (standard and deform) are new and unique.

Undoing the parameter change affecting geometry representation leads to the following results:

• all GeMarker(s) belonging to generators are unchanged;
• GeMarker(s) belonging to unmodified caches (standard and deform) are unchanged;
• GeMarker(s) belonging to modified caches (standard and deform) are new and unique;
• all the GUID(s) belonging to generators are unchanged (even the object affected by the change);
• GUID(s) belonging to unmodified caches (standard and deform) are unchanged;
• GUID(s) belonging to modified caches (standard and deform) are new and unique.

Saving the scene (overwriting existing file) leads to the following results:

• all GeMarker(s) are unchanged;
• all GUID(s) are unchanged;
• Closing Cinema 4D and opening the overwritten scene leads to the following results:
• all GeMarker(s) belonging to generators are unchanged;
• all GeMarker(s) belonging to caches (standard and deform) are new and unique;
• all the GUID(s) belonging to generators and caches (standard and deform) are unchanged.

Saving the scene to a new file leads to the following results:

• all GeMarker(s) are unchanged;
• all GUID(s) are unchanged;

Closing Cinema 4D and opening the new saved scene leads to the following results:

• all GeMarker(s) belonging to generators are unchanged;
• all GeMarker(s) belonging to caches (standard and deform) are new and unique;
• all the GUID(s) belonging to generators and caches (standard and deform) are unchanged.

Re-opening the initial scene leads to the following results:

• all GeMarker(s) belonging to generators are unchanged;
• all GeMarker(s) belonging to caches (standard and deform) are new and unique;
• all the GUID(s) belonging to generators and caches (standard and deform) are unchanged.

In the end:

1. Using GUID(s) to identify objects across multiple scenes in Cinema 4D has proven to be efficient and effective. It should be noted that the tracking should not occur on cache level (where multiple objects can share the same GUID) but on generator level. All the tests conducted so far have shown no corner case where GUID(s) are failing.
2. When rendering instead is taken into consideration, GeMarker(s) are the way to go since GeMarker was designed exactly to identify Cinema4D objects in a non GUI context. A few distinctions should take place depending on the different rendering scenarios:
   * if rendering is delivered in viewport ("Render View" / Ctrl-R) the scene is not cloned and all the GUID(s)/GeMarker(s) found in the VolumeData are exactly matching those used in the "source" scene;
   *   if rendering is delivered via IRR ("Interactive Region Render" / Alt-R) the scene is cloned and whilst GeMarker(s) for cache are regenerated, all GUID(s) and GeMarker(s) for generators are retained;
   *   if rendering is delivered via PV ("Render to Picture Viewer" / Shift-R) the scene is cloned and only GeMarker(s) for generators are retained. It should be also noted that GUID(s) and GeMarker(s) used in the rendering cloned scene are consistent across time which means that a particle or a mograph instance created in a specific frame retains the ID (both GUID and GeMarker) assigned for the whole lifespan of the item.
3. It's recommended, in the end, to rely on GUID(s) to identify objects whilst, when rendering takes place, it's recommended to use GeMarker(s) which has proven to be the only way to uniquely identify objects across both the "source scene" and the actually data used in rendering process.

Looking forward your thought or comments (or even better corner case), give best.
Riccardo

On 18/08/2017 at 05:30, xxxxxxxx wrote:

Wow, thanks Riccardo! Please pin this topic somewhere. :)

On 18/08/2017 at 06:23, xxxxxxxx wrote:

Actually the plan is to add the info to the SDK docs.

On 18/08/2017 at 06:49, xxxxxxxx wrote:

Hi Riccardo,

thank you for your extensive research.

Unfortunately, your findings confirm my own research:

• GUIDs are not unique as duplicated of the same GUID can appear in caches

• GeMarker change when the generator updates objects. Thus loosing tracking between different render calls

Objects that are repesented in the scene work fine. Unfortunately, a typical Cinema 4D scene's geometry that needs to be rendered is mostly found in caches and deform caches. Generators and deformers cannot be rendered directly.

The generated geometry cannot be reliably tracked/identified. Identical GUIDs are used for different objects in caches. GeMarker change when the geometry is changed.

Generators should make use of unique IPs so that objects can be reliably tracked across different frames. Unfortunately, for example the Subdivsion Object does not generate IPs for its generated objects in the Cache. Putting a hierarchy of objects into a Subdivision object has two results: all objects are marked as control object. Control objects themselves indicate that they are not to be used directly but that they generate the data that should be used. Typically, this data is found in the cache of the object itself. The Subdivision Object exhibits different behaviour. All the generated geometry is put in it's own cache, regardless where the control object for that geometry. This way, there is no reliable way to deduct the control object that originally was responsible for creating the data.

The problem I have is not getting to the actual geometry to render but to keep track of which object generated what and to only update the changes. With rendering times of a few milliseconds it ridiculous to spend dozens of secoonds or even a few minutes to prepare the frame.

In conclusion: Ultimately, GUIDs and GeMarker can be used to differentiate between objects. Identification is not possible though. Also, UniqueIPs - a proposed method to track/identify genearated objects during animation - is not consistenly used by generators.

• The same GUID is used for different objects in caches. Visible geometry for many scenes can be found exclusively in caches.

• GeMarker changes when the geometry for generated objects change, making it impossible to identify the object between two consecutive renderings.

• Cloning documents changes GeMarkers. Cloning is happening frequently when rendering.

• The proposed mmethod to track generated object by using their UniqueIP fails as generators like the Subdivision Object does not assign UniqueIPs

BTW: I need to identify objects not only between consecutive renderings but I also track changes "LIVE" to immediately update the realtime rendering. I am not using VolumeData in the VideoPost Plugin for this very reason.

I am using Hierarchy class derived classes to traverse the scene and also handle instances myself.

Here is a scene that is showing some of the duplicate GUID problems and also shows how Subdivision Object is handling caches and UniqueIPs differently: https://drive.google.com/file/d/0BwFHFNK-2yFAMGpIVkRodDlxbzg/view?usp=sharing

Image of resulting test-rendering here: https://drive.google.com/file/d/0BwFHFNK-2yFAcjFlaGJ1NG5wb28/view?usp=sharing

Edit: Here is my log of the scene: https://drive.google.com/file/d/0BwFHFNK-2yFAUVhtU2ZOT21LV1E/view?usp=sharing
For IP I print the cache parent ID and the IP. ID in general is the assigned ID I created. (cached) has a CacheParent is not part of the scene the user sees in the object manager.

I am having troubles with identification in this scene mostly due to the nesting of instances and putting everything into a Subdivision Object. One Holder_01 "instance of an instance of an instance" ;-) and a couple of Holder_02 objects are eluding my efforts to identify them.

Thanks,
Martin

BTW: I am out of the office next week. So I will not be able to respond in a timely fashion.