What exactly does StopAllThreads do

On 16/11/2016 at 04:14, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   all 
Platform:   Windows  ;   Mac OSX  ; 
Language(s) :     C++  ;

---------
During the time I spent developing various plugins, I often encounter various threading problems. Usually, it is related to accesing document from various threads in multithreaded environment of C4D. The only synchronization mechanism I found within docs is method StopAllThreads, but there is lots of missing details about how it actually works and how shoult it be used.

The documentation says it "Stops all running threads" but that is surely not true - for example if I run some code in C4DThread, it won't be affected. What I think is meant is "Stops all background handlers" (BackgroundHandler callback). That would make sense when StopAllThreads is called from main thread as no background handlers will be reexecuted (as I understand it, background handlers are only executed from main thread). But what happens when I call StopAllThreads from different thread? If it simply stops background handlers, then for how long? Workers could be reexecuted from main thread while I am still working with document from second thread... Or am I supposed to edit document only from main thread and StopAllThreads is useless from other threads? What about reading data from document, am I only allowed to do so from main thread after calling StopAllThreads and background handlers?

And am I even correct in my assumption StopAllThreads only stops background handlers? If so, how does for example drawing in viewport work. When calling StopAllThreads in a loop from second thread (for testing only ofcourse), preview in viewport still kind of works. How is access of drawing thread (which is probably main thread) synchronized with access from any other thread?

Note that I know C4D API has implementation of locks, semaphors, signals etc, but I am asking about situations when other threads working on documents are part of C4D (or other plugins) and therefore I cannot simply use locks to synchronize access (ie I can only control my threads).

On 16/11/2016 at 07:44, xxxxxxxx wrote:

AFAIK StopAllThreads() makes all TestBreak() calls return true. 
See BaseThread, C4DThread and VolumeData. Not 100% sure
though, that's an educated guess. Using StopAllThreads() will
for example stop the Viewport rendering.

If I'm not mistaken the idea is to use StopAllThreads() to stop
threads from acessing the scene, so you can safely modify it
(usually from the main thread).

Still, it would be interesting to get an official reply, and correct
me if I'm wrong. There's not much info about it in the SDK, so
I'm merely talking from experience.

Best,
-Niklas

On 16/11/2016 at 08:30, xxxxxxxx wrote:

Hi Niklas, thanks for reply.

I don't think that is really what's happening (unless there is some other method with similar name). For example when I call StopAllThreads from my CommandData (in main thread) while I also have C4DThread running, it's TestBreak() is still returning false (untill I call End() in other part of plugin ofcourse). I am not sure what happens with C4DThread of other plugins, but I doubt C4D would somehow filter threads termination only for other plugins...

As of Picture Viewer, its rendering (at least for me) is not terminated after calling StopAllThreads() - didn't you meant rendering to viewport? Viewport rendering is terminated and I wouldn't be suprised if it was actually also implemented using background handlers. The fact that you can stop viewport rendering using GeStopBackgroundThreads would also support this (although documentation only says background handlers can be stopped using GeStopBackgroundThreads, it does not says it controls background handlers only :-) ).

I guess it is really safe to handle things from main thread, but I am more interested in handling of other threads...

On 16/11/2016 at 10:13, xxxxxxxx wrote:

Yes I corrected that in an edit just a few seconds after I posted. :) Thanks for talking about your findings, you might be right about background handlers, but as far as I know they are
called in the main thread, so the name "StopAllThreads" wouldn't make much sense.. :c

On 16/11/2016 at 10:36, xxxxxxxx wrote:

Well, according to docs, BackgroundHandler is supposed either to do some really quick work or start new thread that can be terminated by C4D (or plugin using GeStopBackgroundThreads) that calls handler callback with BACKGROUNDHANDLERCOMMAND_STOP command, so the name could mean "stop all threads executed by background handlers" and it would be equivalent to executing GeStopBackgroundThreads with all possible parameters.

Actually thinking of it now, BackgroundHandler callback could be used to solve a lot of my problematic situations :-)

Anyway, let's hope we will get some official insight into StopAllThreads and C4D threading system as this is still just an educated guess :-)

On 17/11/2016 at 02:24, xxxxxxxx wrote:

Hi,

I admit the name "StopAllThreads" is indeed a bit misleading. It should rather be called "StopAllRelevantThreads".
In a way (and in the above meaning of threads) BackgroundHandlers are nothing else but threads. In the end it is a thread, which executes/calls the registered BackgroundHandler callbacks.

So, what does StopAllThreads() do?

It actually stops all threads, that could modify or which need a consistent state of the active scene/document.
This means it stops for example viewport redraw, editor render or rendering of material previews. Additionally it will stop quite a number of background handlers related to these tasks.

What it does NOT do:

It does not influence rendering to Picture Viewer, as this is done on a clone of the scene, there's no need to do so.
And it also does not stop any custom threads a plugin might have started.

Recommended use of StopAllThreads() :

StopAllThreads() is supposed to be used from the main thread (because otherwise there are no means to prevent threads from being restarted directly after the call to StopAllThreads()). It's mainly used in conjunction with asynchronous dialogs, when modifying the scene in reaction to some user input.

Why is this so? Or in other words, why does StopAllThreads() not stop all threads?

Because it's actually enough and more would probably hurt.
First of all, C4D doesn't know, what you are doing in your thread.
Secondly, and more important, you are not supposed (not allowed, actually) to modify the scene from a custom thread. So there's no need to interrupt custom threads, as they are not relevant (in the above context). Actually it wouldn't be desirable to interrupt non-related work.
The general concept is to offload your calculations to another thread (maybe working on cloned objects/documents, whatever is needed), but when it comes down to modifying the active document and applying the results of the calculations, you need to do this from the main thread (using the usual concept of a CoreMessage (SpecialEventAdd()) in conjunction with CoreMessage() function in your dialog or a dedicated MessageData plugin).
So, as long as you do not modify the scene in any irregular way, StopAllThreads() does everything that's needed.

Furthermore there are the usual means to do thread synchronization/serialization manually, when you are in need to: Semaphore, GeSpinLock, GeSignal, ...

And finally I can say, that we already have a threading manual in our pipeline, which will be released in an updated SDK documentation in the not too distant future.

On 17/11/2016 at 06:44, xxxxxxxx wrote:

Hi Andreas,
thanks for confirmations and additional information.

My reasoning why StopAllThreads should not affect all C4DThreads were similar to your points (with addition that any plugin adding C4DThread with not enough fast reaction to change of TestBreak would often block C4D).

I have just a few additional questions:

1. You said StopAllThreads stops "quite a number of background handlers", that indicates not all of those. I understand that C4D can possibly have some background tasks that does not need to be stopped, but does it stop all background handlers registered 3rd party plugins? Or is there some way we can indicate in GeAddBackgroundHandler whether it should be stopped (I mean apart from obvious fact plugin can simply ignore BACKGROUNDHANDLERCOMMAND_STOP)?

2. It seems that EventAdd also stops background workers, for example my registered backround handlers receive BACKGROUNDHANDLERCOMMAND_STOP when I call EventAdd from different place. I would assume this is because background handlers needs to be restarted from the begining to react for possible changes indicated by EventAdd. Is that correct or is there more going on?

PS: Glad to here there are plans for threading manual, it will be highly appreciated :-)

On 20/11/2016 at 06:54, xxxxxxxx wrote:

Hi,

now this is interesting.

So in short this function cold be called StopAllThreadsThatAccessCurrentDocument()  ?

How properly make custom thread that read current document and do some work with it of course without make copy of whole document ?
Probably this is similar to question 1 about.

regards,
Remo

On 21/11/2016 at 10:08, xxxxxxxx wrote:

@Remo: Yep, that would be a nice name for the function. In general it is not a good idea (I mean, not safe) to read the current document from a custom thread. The user (and C4D in the background, executing, redrawing,...) is working with the document. And if you used StopAllThreads() and then started a custom thread from the main thread, you'd need to block in the main thread for your thread to finish its work. Which then again would block C4D all together, which is probably not what you want. That's why rendering to Picture Viewer works with a clone of the document.

See also the BaseDocument manual and especially the first warning in there.

@Firielentul: GeStopBackgroundThreads is called with typeclass 0 and flags BACKGROUNDHANDLERFLAGS_VIEWREDRAW, BACKGROUNDHANDLERFLAGS_EDITORRENDDER, BACKGROUNDHANDLERFLAGS_MATERIALPREVIEW and BACKGROUNDHANDLERFLAGS_PRIVATE_VIEWREDRAW (depending on existence of the handlers). As shown in the docs for BackgroundHandler, you can/need to react on those flags in BACKGROUNDHANDLERCOMMAND_STOP.

And yes, EventAdd() internally does (besides other things) also roughly the same as StopAllThreads().

On 24/11/2016 at 18:23, xxxxxxxx wrote:

Thanks Andreas, that answers all my remaining questions.