Christmas Competition 2016 - ExecuteInMainThread

On 07/12/2016 at 02:16, xxxxxxxx wrote:

Hello everyone,

nothing but glory to win here.
But I thought maybe some of you have fun in a small Christmas "riddle".

There came up a request for a simple to use ExecuteInMainThread function (execute on main thread).

These are the requirements, that have to be fulfilled to win nothing:

• Arbitrary functions should be executed in main thread
• Arbitrary parameter constellations for the called function would be nice
• No constraints on where the function is called (e.g. parallel calls from multiple threads have to work)
• Should be easy to use/integrate in any project
• Registering of plugins to reach the goal is explicitly allowed
• Deadline you need to reach to get absolutely nothing: 2017-01-15
  So, I hope, some of you are eager to win nothing. Please post your solutions in this thread.
  Hopefully you will have fun.
  I'm sure our community will be thankful, but we won't tell you, as you are supposed to get nothing. :wink:

Have a nice Christmas time,
your SDK Team

On 15/12/2016 at 02:20, xxxxxxxx wrote:

Does we must stick to python or it can be a c++ plugin? :)
And moreover even if it's a compition can we talk about our implementation?

On 16/12/2016 at 00:42, xxxxxxxx wrote:

Hello,

talking about and presenting your implementation is the only point of this "competition" :slightly_smiling_face: Please feel free to try your ideas in Python or C++, just as you like.

best wishes,
Sebastian

On 19/12/2016 at 18:11, xxxxxxxx wrote:

My first intuition was to deal with c4d.SpecialEventAdd().

The uggly methold but that will work ! Trust me it's very dirty :p
First you need to register 5 unique id for the SpecialEventAdd. ( I will explain it later why)

Ask for a unique Call ID(all current in use CallID are stored into a container in worldcontainer)
Take any kind of python data. Do a bits representation of if and then convert it in a list of integer.
And at each time you call the SpecialEventAdd() with this bit in argument 1. And the second arguments is an Unique call ID(unique to each script you will understand the use of it after)

In your PluginMessage plugin you catch this message and check if this unique call ID existst if not it initialize new dic with the unique call ID. And after each call data are happend to the current data.

In your script after each byte are passed, you call another specialEventAdd with a new id to tell it's finish for this call ID, and arg2 with a int checksum representing the byte (like that we can confirm we got everything ok in the main thread)

For the variable idea are the same but just a litlle change
You loop into each variable to 2 SpecialEventAdd
first SpecialEventAdd with arg1 = data, arg2=call ID
second arg1=int corresponding of an enum of the datatype, arg2= callID

After the execution MessagePlugin return value overide the container stored in the worldcontainer at ID(callID) with a container contening return value.

Then finaly our script read back the return value and then free the CallID into the container

Pro: easy to setup
Pro: accept any kind of data

Con: Not optimized at all

My second approch is to use the same namespace. In this nothing fancing.
Just create an empty class  put values into this class
Then do a c4d.SpecialEventAdd()

Register any plugin who is in the mainThread.
Catch this call.
Read data from the class and that's it !

Pro: Clean way

Con: Is not portable
Con: Is not easy to use for other developper

Maybe I will talk about some other way ! :)
Got some others idea but still got some stuff to test before. (one usin HyperFile other are using some 3rd-party solution like memcache)

On 23/12/2016 at 15:01, xxxxxxxx wrote:

I'm going to share part of my solution since it's not clear if I can share one of the modules. I have two modules -- One is a simple C4D plugin, the other is a normal Python module.

I called the C4D plugin executeInMainThreadListener.pyp. Here it is:

> import c4d
>
> from c4d import plugins
>
>
>
>
> import c4d_threading
>
>
>
>
> # Plugin IDs 1000001-1000010 are reserved for development. 
>
> # You may use these freely to test scripts, but must request an ID in 
>
> # order to release them.
>
> PLUGIN_ID = 1000001
>
>
>
>
> class ExecuteInMainThreadListener(c4d.plugins.MessageData) :
>
>     '''
>
>     Listens to messages from that should be executed in the main thread. 
>
>     Use executeDeferred or executeInMainThreadWithResult.
>
>     '''
>
>     
>
>     def CoreMessage(self, messageId, bc) :
>
>         '''Override.'''
>
>         
>
>         if messageId == PLUGIN_ID:
>
>             c4d_threading._process_queued()
>
>             
>
>         return True
>
>
>
>
> if __name__ == '__main__':
>
>     plugins.RegisterMessagePlugin(PLUGIN_ID, 'executeInMainThreadListener', 0, ExecuteInMainThreadListener())
>
>
>
The c4d_threading module is where you would call executeInMainThreadWithResult(fn, *args, **kwargs).
There should be a global queue in this module that stores the function to execute, the args, and kwargs. executeInMainThreadWithResult will add these to the queue, call c4d.SpecialEventAdd(PLUGIN_ID), then block using a threading condition. c4d.SpecialEventAdd will cause CoreMessage to run in the main thread. In CoreMessage, fn, args, and kwargs will be dequeued and fn will be called. After the function is executed, it will store the result in a global variable inside of c4d_threading, and notify and release the condition so that executeInMainThreadWithResult can continue running. The rest of executeInMainThreadWithResult will get the result from the global variable and return it. 
For error handling, you can do it the same way you do with the result. Store the exception in a global variable, then raise it at the end of executeInMainThreadWithResult.

_<_img src="https://s30.postimg.org/oybz65ev5/c4d_threading.jpg" height="720" width="1280" border="0" /_>_

There's a big warning you should know about though. If ExecuteInMainThreadListener is not running, calling executeInMainThreadWithResult will freeze Cinema 4D and you will have to force it to close. This is because executeInMainThreadWithResult waits for its condition to be released by the plugin.
I hope this helps anyone who has had threading issues in C4D.

On 26/12/2016 at 02:11, xxxxxxxx wrote:

I've written a number of small helper libraries over the years that I put on GitHub. Just recently I merged
them into one "bigger" Python package. One of the modules is nr.concurrency  which I've always used for
asynchronous jobs that I have to synchronize with the main thread somehow. :)

https://github.com/NiklasRosenstein/py-nr

Unfortunately there's no docs yet, but the code is fairly well documented.  The solution that @erk
proposed is what I would usually be going for if I also needed to wait for the result of the function
running in the main thread. With the nr.concurrency.Job  class, it would look something like this:

*Please make sure you import 3rd party modules safely*

with localimport('res/modules') as _imp:
  _imp.disable(['nr'])
  from nr.concurrency import Job, SynchronizedDeque, synchronized
  
class JobRunner(c4d.plugins.MessageData) :
  
  PluginID = ...
  Jobs = SynchronizedDeque()
  
  def CoreMessage(self, msg, bc) :
    if msg == self.PluginID:
      with synchronized(self.Jobs) :
        jobs = list(self.Jobs)
        self.Jobs.clear()
      for job in jobs:
        job.start(as_thread=False)
    return True
  
  @classmethod
  def PutJob(cls, job) :
    cls.Jobs.append(job)
    c4d.SpecialEventAdd(self.PluginID)
  
  
def RunInMainThread(func, *args, **kwargs) :
  job = Job(lambda: func(*args, **kwargs))
  JobRunner.PutJob(job)
  # Blocks until the Job is finished or re-raises the exception that occurred
  # in the function.
  return job.get() 

Now, usually I don't want to wait for the result of the function executed in the main thread. I could
instead just return the Job  object from RunInMainThread(). But most of the time I use the EventQueue
object to trigger certain actions for example in a Dialog.

with localimport('res/modules') as _imp:
  _imp.disable(['nr'])
  from nr.concurrency import EventQueue
  
  
events = EventQueue()
events.new_event_type('preset-loaded', True)
events.new_event_type('presets-changed', True)
events.new_event_type('tree-changed', True)
events.new_event_type('reloaded', True)
events.new_event_type('show-dialog', False)
events.new_event_type('request-exception', False)
  
  
class Dialog(c4d.gui.GeDialog) :
  
  def CoreMessage(self, msb, bc) :
    if msg == c4d.EVMSG_CHANGE:
      self.ProcessEvents()
    return True
  
  def ProcessEvents(self) :
    for ev in events.pop_events() :
      self.ProcessEvent(ev.type, ev.data)
  
  def ProcessEvent(self, event, data) :
    if event == 'preset-loaded':
      self.grid.Redraw()
    elif event == 'presets-changed':
      self.storage.reload()
      self.UpdateGrid()
    elif event == 'tree-changed':
      self.storage.reload()
      self.UpdateGrid()
      self.tree.Refresh()
    elif event == 'reloaded':
      self.UpdateGrid()
      self.tree.Refresh()
    elif event == 'show-dialog':
      c4d.gui.MessageDialog(str(data))
    elif event == 'request-exception':
      self.storage.online_node.error = "Connection error"
      logger.debug(data)
    else:
      logger.warn("unhandled event: {0!r}".format(evnet))
  
  
# Somewhere from a thread
events.add_event('presets-changed')

Cheers,
Niklas