3. Multi threading in Python - Parallelization

Multi threading in Python - Parallelization

  • N

    Hi all.
    I have read some very old posts about multi threading in python, and I can't understand squat.
    Would it be possible for someone to post a simple example on how to write a script that is multithreaded and does the following:

    1. Get the number of Virtual cores in my system "x" (I have a 12 core Mac pro with 24 threads)
    2. make a List with 100 numbers [1,2,3,4.... 100]
    3. Divide that list into "x" segments
    4. Send each of the segments to each of my cores (threads)
    5. Each thread multiplies each item of the segment by 2 (just to do something)
    6. When all threads are done, assemble a new list with the result of all segments in the correct order.

    Is that even possible?
    Thanks

    0
  • S
    Global Moderator

    Hello an welcome to the PluginCafe.

    Please always add tags to your posts to help the community to organize the forum's content. You can also mark your post as a question using the Q&A system. See Read Before Posting.

    I assume you are using Cinema 4D R20?

    Notice that a script is - well - a script. A script is started, executes some command and then finishes. While doing so, Cinema 4D will be blocked. You can use threads to distribute the work done in the script but the script will keep blocking Cinema while doing so.

    You find all components related to threading in the c4d.threading module.

    The number of threads that should be used can be obtained with GeGetCurrentThreadCount().

    A custom thread is based on C4DThread. Such a custom thread can receive, process and return data. Data can be stored as member variables and accessed using custom functions. The Main() function of the thread contains the code that is executed when the thread is running.

    from c4d import threading

class MyThread(c4d.threading.C4DThread):

    # input data
    data = []
    # output data
    results = []

    # set data
    def SetData(self, arguments):
        self.data = arguments
        
    # process data
    def Main(self):
        
        # multiply each item with 2 and store results
        for item in self.data:
            item = item * 2
            self.results.append(item)
                  
    # return results
    def GetResults(self):
        return self.results

    A new thread is created by creating an instance of the thread class. A thread is started using Start(). You can wait for the thread to finish using Wait().

    # create thread
thread = MyThread()   

# prepare input data
data = []
data.append(1)
data.append(2)
data.append(3)

# set input data
thread.SetData(data)

# start thread and wait
thread.Start()
thread.Wait(False)

# get results
results = thread.GetResults()

    If you want to run multiple thread, you have to start these threads an then wait in a loop for all threads to finish (IsRunning()).

    Creating a list with items and splitting that into segments is just basic programming.

    best wishes,
    Sebastian

    2
  • N

    I have a script that takes 5 hours to do something :-)
    I wouldn't mind it taking 5 hours / 12

    Thank you for the information. I'll put it to good use... and let you know when I fail!

    0