Solved baking material channel and applying result to another material from python tag

jenandesign

Hello, perhaps it is pretty self explanatory from the title what I am seeking to do.

I am avoiding making a plugin at all costs due to the added complexity (both from a developmental standpoint as well as having to ask IT to install a plugin across a massive farm for a single job).

So, I am trying to do everything from a Python Tag which I know carries caveats in terms of what is possible. But in a nutshell, I am trying to render a material channel on one object and then apply the result in an xbitmap shader in another material.

So far I have tried a few things, but with no success because I can't find examples of other people doing this so far.

I have tried:

creating a bake texture tag and using CallButton on the bake button, using a frame checker to make sure it runs only once when the frame is changed. This should work, but then the file that is generated and written to disk tries to overwrite itself which displays a dialog asking "Yes or No" and I have yet to figure out how to "click" one of these buttons to get through this dialog.
creating a new bitmap and using c4d.utils.BakeTexture, but I am running into issues getting this set up as there are no examples running it from a Python Tag (nor do I know if it's even feasible)

Thanks for any and all help / recommendations!

Follow up postings (edited by @ferdinand):

Hello again, checking back in. I have made some progress using an example I found:
http://www.plugincafe.com/forum/forum_posts.asp?TID=13606

My code so far is as follows:

import c4d
import tempfile
import os
#Welcome to the world of Python

def main():

    # Set some variables
    obj = op.GetObject()
    texTags = [obj.GetTag(c4d.Ttexture)]
    texUVWs = [obj.GetTag(c4d.Tuvw)]
    destUVWs = [obj.GetTag(c4d.Tuvw)]

    # Bake texture setup
    bc = c4d.BaseContainer()
    bc[c4d.BAKE_TEX_WIDTH] = 512
    bc[c4d.BAKE_TEX_HEIGHT] = 512
    bc[c4d.BAKE_TEX_CONTINUE_UV] = False
    bc[c4d.BAKE_TEX_SUPERSAMPLING] = 0
    bc[c4d.BAKE_TEX_FILL_COLOR] = c4d.Vector(1)
    bc[c4d.BAKE_TEX_USE_BUMP] = False
    bc[c4d.BAKE_TEX_USE_CAMERA_VECTOR] = False
    bc[c4d.BAKE_TEX_AUTO_SIZE] = False
    bc[c4d.BAKE_TEX_NO_GI] = False
    bc[c4d.BAKE_TEX_GENERATE_UNDO] = False
    bc[c4d.BAKE_TEX_PREVIEW] = False
    bc[c4d.BAKE_TEX_SURFACECOLOR] = True
    bc[c4d.BAKE_TEX_UV_LEFT] = 0.0
    bc[c4d.BAKE_TEX_UV_RIGHT] = 1.0
    bc[c4d.BAKE_TEX_UV_TOP] = 0.0
    bc[c4d.BAKE_TEX_UV_BOTTOM] = 1.0
    bc[c4d.BAKE_TEX_NORMAL_METHOD] = c4d.BAKE_TEX_NORMAL_METHOD_OBJECT

    # Initialize bake texture
    bakeInfo = c4d.utils.InitBakeTexture(doc, texTags, texUVWs, destUVWs, bc, None)

    # Create new bitmap and bake texture
    newBitmap = c4d.bitmaps.MultipassBitmap(512, 512, c4d.COLORMODE_RGB)
    bakeTextureInfo = c4d.utils.BakeTexture(bakeInfo[0], bc, newBitmap, None, None)

    # Store bitmap to file
    tempFilePath = os.path.join(tempfile.gettempdir(), "tempBitmap.png")
    newBitmap.Save(tempFilePath, c4d.FILTER_PNG)
    print(tempFilePath)

    return None

if __name__=='__main__':
    main()

However I am getting crashes immediately after running the InitBakeTexture command. Any idea what I'm doing wrong?

Also for some reason the tag script will run over 300 times upon file save even if the tag is disabled when InitBakeTexture exists in the script. Assuming this has something to do with thread = None and running on the main thread?

R25.010

I tried print(texTags, texUVWs) in the Set some variables section and then got the following error after doing 334 logs for the above issue:

RecursionError: maximum recursion depth exceeded while getting the repr of an object

Seems like when I do InitBakeTexture and save the file, it's opening another thread recursively and running the script again. Not sure why this is happening, but I'm pretty sure this is why I am getting crashes when I actually enable the tag.

EDIT: looks like InitBakeTexture is causing a stack overflow, but I have no idea why.

EDIT2: just realized it's creating a new doc. Ah ha

EDIT3: okay not sure if it's a new doc or what, but it's definitely looking like a stack overflow. Any ideas why this is happening?

Simplified test code (applied to Python Tag):

import c4d
#Welcome to the world of Python
def main():

    # Set some variables
    obj = op.GetObject()
    texTags = [obj.GetTag(c4d.Ttexture)]
    texUVWs = [obj.GetTag(c4d.Tuvw)]
    destUVWs = [obj.GetTag(c4d.Tuvw)]

    print(texTags, texUVWs)

    # Bake texture setup
    bc = c4d.BaseContainer()
    bc[c4d.BAKE_TEX_COLOR] = True
    bc[c4d.BAKE_TEX_COLOR_ILLUM] = True
    bc[c4d.BAKE_TEX_NO_INIT_BITMAP] = True
    bc[c4d.BAKE_TEX_WIDTH] = 512
    bc[c4d.BAKE_TEX_HEIGHT] = 512

    bakeInfo = c4d.utils.InitBakeTexture(doc, texTags, texUVWs, destUVWs, bc, None)

    return None

if __name__=='__main__':
    main()
I believe InitBakeTexture causing a stack overflow

I should mention this only happens when making a change to the Python Tag script and then saving the file.

ferdinand

Hello @jenandesign,

Hi @ferdinand- thanks again for the guidance. Do you have any suggestions about how to avoid the issues you've pointed out?

I unfortunately cannot tell you how to fix this, as this is out of scope of support, since one of our rules is that we cannot support code that is in direct violation of design patterns in our API. This is outlined in our Forum Guidelines under the section "Scope of Support" where even this specific case, the threading restrictions of the API, is given as a concrete example. You are basically asking me when you can cross the street even though the stopp-light is red. And while you factually might get away with it from time to time, the only correct answer to that is "you should not, and if you do, do it at your own risk".

Your fixed version is also still calling EventAdd() which is also forbidden.

However the problem I am having now is that the material does not update with the texture as it changes. I tried everything from BaseMaterial.GetPreview() to BaseMaterial.Message(c4d.MSG_UPDATE) to BaseMaterial.Update() at no avail.

You cannot change the state of the scene in a non main-thread context. Most GUI related functions will either exit right away or will only be reflected in the app once the program reaches the GUI updates in the main loop again.

With that all being said, I have hinted at the solution already in my previous posting. You can offload stuff to message() by piggybacking on another message id and. You should also check with c4d.threading.GeIsMainThread() if you are indeed on the main thread in message(), as there is technically no guarantee that message() will always be executed on the main thread. Once you have established that you are on the main-thread in a call to message(), you can modify the scene graph, add events and update the GUI to you hearts content. So, you first would have to set an indicator in main() that you want to carry out action x and when then message() is being called, you can carry it out based on the existence of the indicator. But you can face many problems with an approach like that, which is why this is out of scope of support.

You are misusing our API, tags are not intended to be used in the way you are using them.

Cheers,
Ferdinand

ferdinand

Hello @jenandesign,

thank you for reaching out to us. We would ask you to take a look at our Forum Guidelines regarding the structure of a question and how to tag your posting with the necessary information. I have added a version tag for you and did consolidate your postings into one. Please make use of the edit feature yourself in the future so that all users can enjoy clear topic structures.

Your Task

About your question: There are a few things which are still unclear to me. So, to summarize what I do understand:

You want to bake some channel cA in some material node mA into a texture.
This texture should then be linked as Xbitmap shader in a channel cB in a material node mB.
You do not want to do this a plugin but as some sort of scripting solution.

What is unclear to me:

Why does it have to be a Python Programming Tag? Why not just some Script Manger script with GeDialog for example? Which would lend itself much better for this.
I assume the implied requirement is that this whole thing should somehow run automatically on the render farm you do mention.

Your Problems with our API

There are three problems with what you are trying to do.

A Python programming tag has no interpreter context guard ifnamemain, you are not supposed to execute main() yourself on module initialization in a Python Programming tag. Cinema 4D will call main() for you (multiple times).
The main() function of a Python Programming Tag is subject to the Threading Restrictions of Cinema 4D. Which renders tasks like creating new materials and texture tags illegal from within main(). Also, texture baking and file operations should not be done there.
The unclear execution order / flow diagram of your program. c4d.plugins.TagData.Execute() and by extension the main() function of a Python Programming tag is being called a lot. For each frame, parameter modification, and other events main() will be called. There is also no guarantee that main() will only be called once for each event, sometimes it is called multiple times in a row. Which does not mesh very well with what you are trying to do, as you would have your code run multiple times.

So, you should clarify what you want to do exactly and why you apparently want to execute the texture baking multiple times. There are technically ways to do this in a Python Programming tag by offloading the work to message() which most of the time runs on the main thread. The common approach would be here to listen for a button click in the user data interface of your Python Programming tag, but that does not seem to be what you want to do.

You can technically hook into some messages for other contexts in message() as a sort of hack, but that would be outside of scope of support due to the hacky nature of this approach. You can pick any message for that that is being sent sufficiently often, but you then have to make sure that you evaluate yourself if any further actions should be taken when the message comes in. The evaluation depends on your project. I could expand here on the message system, but this is all getting very speculative, so I would ask you to clarify what your solution is intended to achieve.

Cheers,
Ferdinand

jenandesign

Hello @ferdinand! Thank you for your incredibly well thought out and detailed response! I really appreciate it. The amount of information you have come back with from my hodgepodge posting is illuminating to say the least.

Why does it have to be a Python Programming Tag? Why not just some Script Manger script with GeDialog for example? Which would lend itself much better for this.

The reason for a Python Tag is threefold:

My task requires that the operation updates on every frame so that other systems can be built upon it. This rules out a Script Manager script which would need to be executed manually and does not work at render time. Specifically in my case, I am trying to achieve particle emission via texture luminance that is baked from lights / surface color.
The setup is to be shared across a team of remote workers, and requiring other team members to manage/update a plugin or script file along with a scene file adds undue complexity to an otherwise simple tool. It is much easier to have every bit of kit live in the scene file alone.
As mentioned earlier, this will be rendered on a farm [remotely], and this is another reason to keep everything contained within the scene file (as opposed to a plugin which would require the farm admin to manage installations and updates).

At the recommendation of a very smart and dear friend, I am using a bit of user data to store a baking state/flag. With this, I was able to hack my way through the recursion issue. Things are now working as intended.

import c4d
import tempfile
import os
#Welcome to the world of Python

def main():
    baking = op[c4d.ID_USERDATA,4] # a boolean flag
    if baking == True:
        return

    # Set some variables
    texTags = [op[c4d.ID_USERDATA,1]]    # source texture tag 
    texUVWs = [op[c4d.ID_USERDATA,2]]
    destUVWs = [op[c4d.ID_USERDATA,3]]
    destMat = op[c4d.ID_USERDATA,5]      # destination material

    # Bake texture setup
    bc = c4d.BaseContainer()
    bc[c4d.BAKE_TEX_SURFACECOLOR] = True
    bc[c4d.BAKE_TEX_PIXELBORDER] = 3
    bc[c4d.BAKE_TEX_WIDTH] = 1024
    bc[c4d.BAKE_TEX_HEIGHT] = 1024

    # Init bake texture, prevent recursion
    op[c4d.ID_USERDATA,4] = baking = True
    bakeInfo = c4d.utils.InitBakeTexture(doc, texTags, texUVWs, destUVWs, bc, None)
    op[c4d.ID_USERDATA,4] = baking = False

    # Create new bitmap and bake texture
    newBitmap = c4d.bitmaps.MultipassBitmap(1024, 1024, c4d.COLORMODE_RGBw)
    bakeTextureInfo = c4d.utils.BakeTexture(bakeInfo[0], bc, newBitmap, None, None)

    # Store bitmap to file
    tempFilePath = os.path.join(tempfile.gettempdir(), "tempBitmap.png")
    newBitmap.Save(tempFilePath, c4d.FILTER_PNG)

    # Update the destination material
    shader = c4d.BaseList2D(c4d.Xbitmap)
    shader[c4d.BITMAPSHADER_FILENAME] = tempFilePath
    destMat.InsertShader(shader)
    destMat[c4d.MATERIAL_LUMINANCE_SHADER] = shader
    destMat.Message(c4d.MSG_UPDATE)
    destMat.Update(True, True)
    c4d.EventAdd()

    return None

if __name__=='__main__':
    main()

jenandesign

One thing I still have yet to figure out is how to get the destination material to update its texture in the viewport/editor.

How to run c4d.BITMAPSHADER_RELOADIMAGE?

https://developers.maxon.net/docs/Cinema4DPythonSDK/html/classic_resource/shader/xbitmap.html

Thank you,
Leah

ferdinand

Hello @jenandesign,

I am glad that is working for you, a few points about your final script though, I have put them down as comments below. You are also not allowed to perform GUI operations from main(), like for example updating a material preview and propagate these changes to the GUI.

Cheers,
Ferdinand

import c4d
import tempfile
import os
#Welcome to the world of Python

def main():
    ### Everything fine below this point.

    baking = op[c4d.ID_USERDATA,4] # a boolean flag
    if baking == True:
        return

    # Set some variables
    texTags = [op[c4d.ID_USERDATA,1]]    # source texture tag 
    texUVWs = [op[c4d.ID_USERDATA,2]]
    destUVWs = [op[c4d.ID_USERDATA,3]]
    destMat = op[c4d.ID_USERDATA,5]      # destination material

    # Bake texture setup
    bc = c4d.BaseContainer()
    bc[c4d.BAKE_TEX_SURFACECOLOR] = True
    bc[c4d.BAKE_TEX_PIXELBORDER] = 3
    bc[c4d.BAKE_TEX_WIDTH] = 1024
    bc[c4d.BAKE_TEX_HEIGHT] = 1024

    # Init bake texture, prevent recursion
    op[c4d.ID_USERDATA,4] = baking = True
    bakeInfo = c4d.utils.InitBakeTexture(doc, texTags, texUVWs, destUVWs, bc, None)
    op[c4d.ID_USERDATA,4] = baking = False

    # Create new bitmap and bake texture
    newBitmap = c4d.bitmaps.MultipassBitmap(1024, 1024, c4d.COLORMODE_RGBw)
    bakeTextureInfo = c4d.utils.BakeTexture(bakeInfo[0], bc, newBitmap, None, None)

    # Store bitmap to file

    ### tempfile is nice, but you could run into serious problems on a server which
    ### which has its access rights pinned down
    tempFilePath = os.path.join(tempfile.gettempdir(), "tempBitmap.png")
    ### sort of okay outside of the main thread but dangerous 
    newBitmap.Save(tempFilePath, c4d.FILTER_PNG)

    # Update the destination material
    shader = c4d.BaseList2D(c4d.Xbitmap)
    shader[c4d.BITMAPSHADER_FILENAME] = tempFilePath
    
    ### Everything down form here is basically illegal. You insert a node into the
    ### scene graph (a shader) and invoke an event, all things that are not allowed 
    ### in main() because it is not on the main thread. The problem with these threading
    ### restrictions is that people are sometimes not aware of them or figure out that
    ### they can ignore them without anything bad happening immediately. But they are
    ### there for a reason, and your script not crashing in one scene does not mean
    ### it won't crash in another scene.
    ###
    ### If this works for you, this is great, but please be warned, you are treading
    ### here on shaky ground. If anything at the point you are making these modifications
    ### is relying on some node pointer which is then garbage after your modifications,
    ### this will crash Cinema 4D. The chance is relatively low, but these restrictions are
    ### there for a reason. You have been warned ;)
    destMat.InsertShader(shader)
    destMat[c4d.MATERIAL_LUMINANCE_SHADER] = shader
    destMat.Message(c4d.MSG_UPDATE)
    destMat.Update(True, True)
    c4d.EventAdd()

    return None

### The context guard is not only not required in a Python Programming tag, but 
### should not be done at all.
# if __name__=='__main__':
#    main()

jenandesign

Hi @ferdinand- thanks again for the guidance. Do you have any suggestions about how to avoid the issues you've pointed out?

jenandesign

Hello @ferdinand. I realized there is no need to create a new shader in the material, simply updating/maintaining the Xbitmap file path should suffice.

Thankfully tempfile is giving me no issues with the farm.

However the problem I am having now is that the material does not update with the texture as it changes. I tried everything from BaseMaterial.GetPreview() to BaseMaterial.Message(c4d.MSG_UPDATE) to BaseMaterial.Update() at no avail. If you have any recommendations how I can force a reload of the Xbitmap's file, I am all ears.

import c4d
import tempfile
import os
#Welcome to the world of Python

def main():
    baking = op[c4d.ID_USERDATA,4] # a boolean flag
    if baking == True:
        return

    # Set some variables
    texTags = [op[c4d.ID_USERDATA,1]]    # source texture tag
    texUVWs = [op[c4d.ID_USERDATA,2]]
    destUVWs = [op[c4d.ID_USERDATA,3]]
    destMat = op[c4d.ID_USERDATA,5]      # destination material

    # Bake texture setup
    bc = c4d.BaseContainer()
    bc[c4d.BAKE_TEX_SURFACECOLOR] = True
    bc[c4d.BAKE_TEX_PIXELBORDER] = 3
    bc[c4d.BAKE_TEX_WIDTH] = 1024
    bc[c4d.BAKE_TEX_HEIGHT] = 1024

    # Init bake texture, prevent recursion
    op[c4d.ID_USERDATA,4] = baking = True
    bakeInfo = c4d.utils.InitBakeTexture(doc, texTags, texUVWs, destUVWs, bc, None)
    op[c4d.ID_USERDATA,4] = baking = False

    # Create new bitmap and bake texture
    newBitmap = c4d.bitmaps.MultipassBitmap(1024, 1024, c4d.COLORMODE_RGBw)
    bakeTextureInfo = c4d.utils.BakeTexture(bakeInfo[0], bc, newBitmap, None, None)

    # Store bitmap to file
    tempFilePath = os.path.join(tempfile.gettempdir(), "tempBitmap.png")
    newBitmap.Save(tempFilePath, c4d.FILTER_PNG)

    # Update the destination material
    shader = destMat[c4d.MATERIAL_LUMINANCE_SHADER]
    shader[c4d.BITMAPSHADER_FILENAME] = tempFilePath
    # Nothing seems to be working after this to update the material
    destMat.GetPreview()
    destMat.Message(c4d.MSG_UPDATE)
    destMat.Update(True, True)
    c4d.EventAdd()

    return None

ferdinand

Hello @jenandesign,

Hi @ferdinand- thanks again for the guidance. Do you have any suggestions about how to avoid the issues you've pointed out?

I unfortunately cannot tell you how to fix this, as this is out of scope of support, since one of our rules is that we cannot support code that is in direct violation of design patterns in our API. This is outlined in our Forum Guidelines under the section "Scope of Support" where even this specific case, the threading restrictions of the API, is given as a concrete example. You are basically asking me when you can cross the street even though the stopp-light is red. And while you factually might get away with it from time to time, the only correct answer to that is "you should not, and if you do, do it at your own risk".

Your fixed version is also still calling EventAdd() which is also forbidden.

However the problem I am having now is that the material does not update with the texture as it changes. I tried everything from BaseMaterial.GetPreview() to BaseMaterial.Message(c4d.MSG_UPDATE) to BaseMaterial.Update() at no avail.

You cannot change the state of the scene in a non main-thread context. Most GUI related functions will either exit right away or will only be reflected in the app once the program reaches the GUI updates in the main loop again.

With that all being said, I have hinted at the solution already in my previous posting. You can offload stuff to message() by piggybacking on another message id and. You should also check with c4d.threading.GeIsMainThread() if you are indeed on the main thread in message(), as there is technically no guarantee that message() will always be executed on the main thread. Once you have established that you are on the main-thread in a call to message(), you can modify the scene graph, add events and update the GUI to you hearts content. So, you first would have to set an indicator in main() that you want to carry out action x and when then message() is being called, you can carry it out based on the existence of the indicator. But you can face many problems with an approach like that, which is why this is out of scope of support.

You are misusing our API, tags are not intended to be used in the way you are using them.

Cheers,
Ferdinand

ferdinand

Hello @jenandesign,

without any further questions we will consider this topic as solved by Friday, December the 17th.

Thank you for your understanding,
Ferdinand