thank you for the update and the simplified code. I assume from the context that these code snippets represent a state which does not crash anymore since the line
Int32 particle = m_masterSystem->AllocParticle(); is not inside the worker loop anymore. And the major problem is basically now that you have to repeat in the second snippet
result = sampler->doSomeCalculations(p.x, p.z);, your expensive call, which you have already done in the first async snippet for the vertices of the object (or whatever
m_padr is for). If this code is still crashing, then the main-thread thing is the cause, and you can jump to the paragraph Main Thread Problem.
As lined out in the previous posting, my first idea would be to invert what you are doing, so that you can also modify existing particles inside your
worker lambda. But the problem seems to be, at least judging by the mock code, that you really cannot predict the number of particles you will need, as the existence of each particle is tied to the outcome of the expensive method
doSomeCalculations(). So, the only route would be then to allocate the maximum number of particles which could be required, which seems to be
m_pcnt in your case. As a mock-algorithm:
m_pcnt tp particles and store their particle ids in an array
- Launch your async lambda
worker and include
particleCollection and the tp master system in its capture.
result = sampler->doSomeCalculations() in the lambda as you do now.
b. The argument
i passed to your lambda should be equivalent to a particle index in
particleCollection, i.e., you can also index
c. Determine with
result if a particle is required or not as you already do in your second snippet.
d. For particles which should not exist, i.e., which should be deallocated, set the life attribute to a negative value as lined out as the preferred method of TP particle deallocation in TP_MasterSystem ::FreeParticle().
e. For all other particles carry out what you did in your second snippet for valid particles (set life, color, pos, etc.).
This would call
doSomeCalculations() only once at the cost of some memory overhead, as you will generate a particle overhead in some or most cases, depending on the likelihood of
1 - sampler->doSomeCalculations().particleValue > 0.
Main Thread Problem
About the secondary problem of not adding scene elements outside of the main thread: There is no easy fix for that.
ObjectData and the specific method you are using are not intended for doing what you want to do, allocating particles. Over the years, developers have worked around these problems, but these workarounds often entail a lot of work.
The principle logic is always:
- I want to add elements to a scene in some
NodeData method which is not on the main thread.
- Collect all data required for that action and store it as an object
task. In your case
task could be a dictionary of the particle data which should be added.
- The next time
NodeData::Message is called and is on the main thread (will likely be the case, but you must still test for it with
GeIsMainThread()), carry out these changes with the help of
task, e.g., allocate and set the particles.
The problem is that there is no dedicated mechanism or message id for that approach. So, you just check every message if there is a
task and then carry it out. Which not only slows down the execution of
Message, which is not good when the slowdown is substantial as everything else is waiting for you, but also comes with no guarantee of time of execution.
Message will inevitably be called at some point after you created a
task, but it could take seconds before the next message comes in. Which mainly happens when a node (object, tag, material, etc.) is not active (selected), as GUI/parameter stuff is the most frequent reason for messages.
For adding a bunch of materials to a scene from some
ObjectData this is all manageable, but for particles the time of allocation matters. I do not see an effective way out of this dilemma. You could allocate the particles ahead of time, i.e., make sure there is always
m_pcnt of invisible particles available for the next call to
ModifyObject. But this then gets all complicated. There is also the problem that tying particle generation to
ModifyObject() in itself is problematic, since there is nothing which prevents this method from being called multiple times per frame.
For completeness: There is
maxon::ExecuteOnMainThread() which allows you to push a lambda to the main thread, which you could invoke in your
MyObjectData::ModifyObject() and sort of solve all these problems. But it effectively just inverts the problem, as then
::ModifyObject() will wait for the
ExecuteOnMainThread() job to be executed on the main thread, i.e., you will bind a method which is for performance reasons not on the main thread,
ModifyObject, to the main thread.