Crash on Exit

On 18/05/2017 at 12:43, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   R18 
Platform:      
Language(s) :       PYTHON  ;

---------
I got a pretty weird crash. The crash occurs only when sds tag already exist into the mesh.

About the script usage. Make a box. Make it editable. Make an edge selection into it.

import c4d
  
  
def get_all_objs(op, data=None) :
    """ get a list of all obj in the scene """
    if not data:
        data = list()
  
    while op:
        data.append(op)
  
        data += get_all_objs(op.GetDown(), None)
        op = op.GetNext()
  
    return data
  
  
def reinitialize(obj) :
    tag = obj.GetTag(c4d.Tsds)
    #probably crash come from here. Since it's only bug when an SDS is already into the mesh.
    if tag:
        new_tag = c4d.BaseTag(c4d.Tsds)
        obj.InsertTag(new_tag)
        tag.Remove()
  
    if not tag:
        tag = c4d.BaseTag(c4d.Tsds)
        obj.InsertTag(tag)
  
  
def make_sds(obj) :
    selection_tag = obj.GetTag(c4d.Tedgeselection)
  
    if not selection_tag:
        return
  
    doc.SetActiveObject(obj)
  
    # Set To edge mdoe
    doc.SetMode(c4d.Medges)
  
    # Get selected edge
    bs = obj.GetEdgeS()
    bs_copy = bs.GetClone()  # for retrieve it later
  
    # Set the weight
    tool = c4d.plugins.FindPlugin(1007573, c4d.PLUGINTYPE_TOOL)
    tool[c4d.MDATA_SDSWEIGHT_MODE] = 0
    tool[c4d.MDATA_SDSWEIGHT_STRENGTH] = 1
    c4d.CallButton(tool, c4d.MDATA_SDSWEIGTH_SETSTRENGTH)
  
    # reset edge selection
    bs_copy.CopyTo(obj.GetEdgeS())
  
  
def main() :
    objs = get_all_objs(doc.GetFirstObject())
  
    save_mode = doc.GetMode()
    save_obj = doc.GetActiveObjects(1)
  
    for obj in objs:
        make_sds(obj)
  
    # reset doc
    first = True
    for o in save_obj:
        if first:
            doc.SetSelection(o, c4d.SELECTION_NEW)
            first = False
        else:
            doc.SetSelection(o, c4d.SELECTION_ADD)
  
    doc.SetMode(save_mode)
    c4d.EventAdd()
  
    objs = get_all_objs(doc.GetFirstObject())
    for obj in objs:
        reinitialize(obj)
  
    c4d.EventAdd()
  
  
if __name__ == '__main__':
    main()

If you execute only one time. and quit the scene. C4D will not crash.
If you execute this script two time. C4d will crash on the exit.

If you only use make_sds. It not crash
If you only use reinitialize. It not crash
If you use reinitialize before make_sds. It not crash.

I think the crash come from this part since it only happend if a c4d.Tsds tag already exist on the mesh.

    if tag:
        new_tag = c4d.BaseTag(c4d.Tsds)
        obj.InsertTag(new_tag)
        tag.Remove()

I already found a workaround, and I changed my reinitialize function and now is not crashing anymore. So it's just for you support guys.

On 19/05/2017 at 07:04, xxxxxxxx wrote:

Hi,

I'm not done, yet, investigating the issue. If I swap the Remove() and InsertTag() in your last snippet, I can't reproduce the issue anymore. So this may be a workaround for you as well. I'd also be interested in your workaround, but only if you find the time of course.

But for the actual reason, I will need to do some more investigation next week.

On 19/05/2017 at 07:28, xxxxxxxx wrote:

My workaround is to select all edge then use make_sds with a weight of 0.

Originally posted by xxxxxxxx

If I swap the Remove() and InsertTag() in your last snippet, I can't reproduce the issue anymore.

Haha I didn't notice this. It's very weird.

Good luck ;)

On 19/05/2017 at 09:38, xxxxxxxx wrote:

Not weird at all. I've seen this a billion, zillion, gazillion times.
In fact. I've run into this so may times that any time I create a new object in a script. The hairs on my neck stand straight up.

When you create an object (objs, tags, materials) with code it creates it first in memory. Then that object needs to be physically added to the scene and cached.
There is a small time lag between when the object exists in memory....and when it actually exists in the scene and you can safely use it.
This is where you can run into problems with linear code.
The next line of code can easily execute before your new object is ready.

Aside from strange crashes. The more common problem you'll likely run into is a "None" return value when trying to edit your new object.
By re-ordering the code progression. You can often get around this problem (but not always).
So you have to be very careful about creating and deleting things when using linear code like a script.
You can't always just make and delete things at will.
You need to order things so that C4D can finish creating and cache your new object.

This is why people use the Message() method in plugins to create and delete things.
The Message() method runs more than once. So it can be used like a callback function to check if your new object actually does exist and is ready to be used. And only do something if it does actually exist in the scene.
Linear code like scripts can't do this. So you need to be very, very mindful about how and when you're creating and deleting things.
You can't just blindly create and delete things any old way.

-ScottA

On 19/05/2017 at 10:53, xxxxxxxx wrote:

I have to strongly object to what Scott is saying. The behaviour of the API is consistent, you just need
to know how it is behaving.

Originally posted by xxxxxxxx

There is a small time lag between when the object exists in memory....and when it actually exists in the scene and you can safely use it.

There is no time lag. As long as your code runs in the main thread (which is the case for scripts,
CommandData and MessageData plugins), there is zero race conditions, transaction- or
ready-time when modifying the scene. You insert an object, and it is inserted. You remove an object,
and it is removed immediately. As the .Remove() function returns, the object is no longer part of the
hierarchy it used to be.

Originally posted by xxxxxxxx

Aside from strange crashes. The more common problem you'll likely run into is a "None" return value when trying to edit your new object.

What I suppose you are referring to, seeing as you talked about the "cache" before, is that a primitive
object returns "None" from BaseObject.GetCache() after you created it.

obj = c4d.BaseObject(c4d.Ocube)
print(obj.GetCache())  # None

This is due to the fact that the object did not have a chance to generate any cached objects within
ObjectData.GetVirtualObjects(), yet. The cache will be created when Cinema 4D requests the
object to do so, for example when the viewport is updated (and the object may choose to return the
cached objects from the previous evaluation), but also when you use BaseDocument.ExecutePasses()
or c4d.DrawViews() with the right flags.
Alternatively, you can use SendModelingCommand() with MCOMMAND_MAKEEDITABLE, but AFAIR
this does give you a valid version of what the object's cache would look like, but the result is
not actually assigned to the object's cache (thus, GetCache() would still return "None").

Regards,
Nik

On 19/05/2017 at 16:24, xxxxxxxx wrote:

^No Niklas.
The object is simply not there, ready to be used. Right after you make a call to create it.
This doesn't just apply to this particular code case. It happens in many other scenarios without delete being involved.
This happens all of the time. It's not a rare issue at all.

For example.
Write an ObjectData plugin that creates new materials and changes it's attributes using linear code.
It will probably crash. Because the new material is not there yet when you try to change it. Even though you've executed the create code before hand.
But... if you put that creation code the Message() method. And you use a condition to check for it. The code will work just fine.
The reason for this is because the Message() method runs multiple times.
On the first, and possibly second, Message() execution. The material will return None. Hence the source of the crash.
But typically by the 3rd pass, the object(in this case a material) will finally be there ready for use.
This is similar to how a callback function is used to safely get a return value.
Hence. This is a time based issue.

There are dozens of cases where this problem rears it's ugly head.
Most scripting cases are not crashes. But instead you will get "None" from a new object right after you've created it with code. It happens all of the time.

As another example.
If you create too many hairs using a script. You'll get a crash.
But if you use sleep() you can prevent the crash.
Once again. This is a time based problem. It can always be solved by giving C4D more time to finish creating your new object. Or by checking if the object is actually there and ready by asking for it over and over again in some sort of function that runs multiple times before using it.

Some objects are more prone to this construction problem than others.
The primitive objects(cube, sphere, etc) are not usually a problem here. They seem to be light and create very fast.
But other objects like materials, or even things like UserData can be prone to this "I'm not ready to be used yet" problem.
The typical work around for linear code is to re-order the code. Or use a temp document, or the SMC to do some of the work.
These "tricks" are needed because object creation is not always safe. And there are times when the object isn't there and ready to be used yet. Even after you call to create it.

If you don't like the word time used to describe this issue. Then give it another name that makes you feel more comfortable. Time makes the most sense to me. Because the problem is always solved by giving C4D more time before using the new object.

-ScottA

On 22/05/2017 at 09:55, xxxxxxxx wrote:

I have to agree to Niklas... and am a bit surprised about how this thread derailed...
And personally I do not like these very general and derogative statements.

Anyway, I think, what you perceive as time problems, is actually wrong use of the API or doing forbidden things and I'd probably call these threading issues. Unfortunately your example descriptions are very general and I can only assume, what might be going wrong.

For example:

Write an ObjectData plugin that creates new materials and changes it's attributes using linear code.
It will probably crash. Because the new material is not there yet when you try to change it. Even though you've executed the create code before hand.
But... if you put that creation code the Message() method. And you use a condition to check for it. The code will work just fine.
...

Am I right, you are creating materials in GetVirtualObjects()? This is a forbidden action, see the last paragraph in the article about Threading, chapter "Forbidden Actions".

But I think, it makes little sense in me imagining what may go wrong with your examples. I suggest you open up one thread per example, maybe even posting some example code, but at least stating, in which function you do what, so we can take a look and either fix the bugs in C4D or provide you with suggestions/ideas, where to fix your code. Maybe there are already older threads existing on some of these, then it's of course fine to point us to these, instead of opening new ones. I still hope we will be able to provide explanations for C4D.

On 22/05/2017 at 11:45, xxxxxxxx wrote:

The OP commented that C4D acted weird. And I simply replied that this is not a weird thing.
And it's something you will probably run into often. Under various scenarios.
There are rules in C4D that can cause problems when trying to created objects in C4D with code. We can't just create and delete things any old time, or any old way we want.

I gave two valid examples. But there are lots more. The forum is loaded with examples where you create an object, but then it's not available.
Just randomly picking something out of my notes. Here's another example:

    BaseObject *obj = doc->GetActiveObject();  
  if(!obj) return false;  
  
  BaseObject *clonerObj = BaseObject::Alloc(1018544);   //Create a cloner object  
  if(!clonerObj) return false;  
  
  //Make the active object a child of the cloner object to generate it's hidden modata tag  
  clonerObj->InsertBefore(obj);  
  
  obj->Remove();  
  obj->InsertUnder(clonerObj);  
  obj->Message(MSG_UPDATE);  
  
  //At this point C4D doesn't see the new hierachy we just quickly created  
  //So we use this to build the caches for the objects we just added and shuffled around  
  doc->ExecutePasses(NULL, TRUE, TRUE, TRUE, BUILDFLAGS_0);

This kind of problem always seems to go away if you can manage to somehow run your object creation code and then do something else for a few hundred milliseconds before attempting to use the new object.
While many times I'm sure this is a threads issue. I'm not 100% convinced that is always the problem.
But whatever you want to call it. This problem shows itself as "I can't use this object yet, please wait just a tiny bit and I'll be able to use it".
In other words. Give me some time to finish making the object before using it.
And that's a big problem in scripts or linear code where the code only executes once.

I've also noticed that inserting objects into the scene is where many times the object isn't ready.
Insertion of the object seems to take a fairly long time. And some objects like materials take longer than others.
It's not an uncommon scenario to create an object and not be able to use it because it has not finished inserting itself into the document. So one common workaround is to create the object, and edit it in memory. Then insert it at the end of your code.
This works. But it's not always the desired workflow.

My notes are filled with kind of thing. Where trying to create and use new objects is not very easy or straight forward.
In fact, I'm still running into new situations requiring unusual workarounds to this very day.
It's a fairly common problem to run into.

I'm sorry if this sounds derrogative. It's not mean to be.
I personally think telling people it works if you know what you're doing is highly condescending. And not helpful or constructive. But as you know Andreas, I try not to take those kinds of comment to seriously.
IMHO object creation in C4D is literally fraught with perils. And you need to be on guard, and ready for problems when doing it. And try to learn as many of the workarounds that you can.
I'm not slamming the program. I'm just telling the OP that this is not weird or unusual.

-ScottA

On 23/05/2017 at 00:06, xxxxxxxx wrote:

Sorry, Scott, but again you failed to tell us, where you execute the above code snippet. I'm not saying "It works, if you do it right". Well, basically I do, but I'm also willing to explain where, when and why. But I can't do this in a general fashion, at least not, if you want us to do some other support work in future. Therefore I asked you to ask specific, detailed questions (favorably in another thread, as this one derailed far enough, if you ask me).
And just because something fails in a similar way, it's neither _always_ nor does it mean, it' due to the same cause.
And I insist to call it a threading issue, as it means you are doing something, where Cinema 4D is doing something else. Most likely you are doing something in a background thread (be it your own custom one or one of C4D's own ones) and for example scene changes are a very, very bad idea to do there.

Insertion of objects by the way, does take virtually no time, it's just a matter of copying a few pointers. The problem is really when and where you do this. So please give us more details, so we can check and explain. Or, if I'm wrong with my assumptions, fix the bug in C4D.
Personally I have never ever run into any problems with object creation.

And for the OPs problem, this really has nothing to do with any timing and/or threading issue. As far as I have understood the underlying code yet, is more about C4D not really being prepared for multiple SDS tags on a single object at once. But I haven't fully understood it yet, so please take this statement with care.

On 23/05/2017 at 07:40, xxxxxxxx wrote:

Originally posted by xxxxxxxx

Sorry, Scott, but again you failed to tell us, where you execute the above code snippet. I'm not saying "It works, if you do it right". Well, basically I do, but I'm also willing to explain where, when and why.

I'm not asking any questions. I'm sharing my experiences with the OP about the struggles involved in creating objects. Because he thought what was going on was "weird".
I'm pointing out examples where you can try to create an object and won't work as expected.
I'm pointing out why someone would easily get tripped up and call it "weird" as the the OP did.
There is no question for you to answer.
The only reply you & Niklas should have written was: "Yup. It can be quite tricky to deal with at times".
Or maybe something like: "Yes. Creating objects in C4D can be tricky. The technical reason has to do with threading issues most of the time."

Originally posted by xxxxxxxx

I insist to call it a threading issue, as it means you are doing something, where Cinema 4D is doing something else.

Ok that's fine. But that's what a time based problem shows up as to me. That's my non technical description of what is happening. You're getting tripped up on the words I'm using.
We are talking about the same thing.
And here's two reasons why I don't like to throw that threads word around like it's a magic word
-1.) I don't know for sure it's always a threading issue because I can't test it on the official API
-2.) Many (if not most) of the people here are not computer science majors.
       They're people who are just trying to do something simple like creating an object then change it in some way. And the program will not let them do it. Spitting terminology like threading at them will not help them figure out how to make it work.

Originally posted by xxxxxxxx

for the OPs problem, this really has nothing to do with any timing and/or threading issue. As far as I have understood the underlying code yet, is more about C4D not really being prepared for multiple SDS tags on a single object at once. But I haven't fully understood it yet, so please take this statement with care.

See what I mean?
You don't really know for sure 100% what is going on in that case. But Niklas labels everything a threading issue without batting an eyelash like it's the magic fix for everything. And you agreed with him on it, even though this might not be the case in this specific scenario.
You just told me you insist on calling them threading issues. Even though you admit this one is not a threading issue.
This is why I don't like to just causally toss around the threading word as if it's magic fix for every situation.

The OP called what was happening to him as "weird".  And I simply told him that this is not "weird"
Then I offered up some of my own personal experiences as examples.
There was no reason for anyone else to jump in and "correct me".
But I've learned my lesson. I'll keep my big mouth shut from now on and keep my experiences and advice to myself from now on.
Sorry for the confusion.

-ScottA

On 23/05/2017 at 08:01, xxxxxxxx wrote:

I really think you must open another thread Scotta because your bug is really not mine.
Because your bug is object creation.
Mine is on object removal.

        new_tag = c4d.BaseTag(c4d.Tsds)
        obj.InsertTag(new_tag)
        tag.Remove()

As you could see I remove the old one and I do nothing else in my script into the new_tag.
And As andreas say it's probably a bug within my script. Because if we think about a tag which don't get the tag TAG_MULTIPLE on his registration. Logicaly make a new tag onto an obj kill the previous one.
So I guess it's why previous_tag.Remove() fail, or least remove something wrong because he must overwrite/delete some pointer. And c4d fail to retrieve the baseTag in the save of the scene. It's jsut a guess but I think is something like that. 
But crash should not happend.

Anyway I really encourage you to post a complete exemple of your bug who crash 100% of the time because I'm very curious and I'm not sure to understand your bug.

And finally thanks everyone for your effort ! :)

On 23/05/2017 at 08:53, xxxxxxxx wrote:

I have no bug or questions gr4ph0s.
I was only pointing out that this is not an unusual problem to deal with in c4d.
I apologize for the confusion. I will keep my advice to myself from now on, because it just confuses people.

-ScottA

On 24/05/2017 at 05:08, xxxxxxxx wrote:

Well, I have the feeling I lost control over this thread...

To recap:

1. gr4ph0s asked a question
2. Scott, you stepped into the thread, saying it is normal to expect weird behavior with our API
3. Niklas and I opposed this opinion
4. I offered help with some of your given examples, assuming (and I still stand on this) you are using the API wrongly in these cases. I thought we could all learn out of these examples.
5. I'm not sure, where this got out of hand, and why you are refusing to take this chance. Even worse, by not showing your examples (in needed detail), you deny us the chance to prove either that our API is not as bad as you state or to prove you're right and we have to investigate such flaws

And yes, sometimes I am picky on technical wording, just because terms, I believe, can make a difference.

For me a timing problem exists, if there is a timing you need to get right. And this is definitely not the case, neither in this specific thread nor anywhere else in our API.

The problem you describe, is something, where you hide the actual problem (which might or might not be a bug in your code or in our API) by changing the timing (and therefore relying on the fact, that certain things running in parallel are finished at some point). Here's where I see this a threading problem, because there's something done in a thread, which is either not allowed to be done there or not properly synchronized. And as soon as the later is fixed, you will see, there's no need for any timing changes anymore. And again, I offer to help investigating such. Either it is a bug we need to fix or it will help you and the community to understand the background better. And hopefully afterwards our API will be much less mysterious or weird.

In any case, we should move this discussion into a new thread, if you are wanting and willing to pull some things, which were said in this thread, straight.

On 24/05/2017 at 05:33, xxxxxxxx wrote:

Back to the original topic:

gr4ph0s you are perfectly right with your assumption about the cause of the issue. Indeed InsertTag() deletes any existing tag, if TAG_MULTIPLE is not set. So your InsertTag(), Remove() sequence removed an already deleted tag (or rather worked on a stale reference). So, I'd indeed say, it's an issue in your script. Nevertheless we will take a look, if we can do something about the fatal consequences of such an error.