Custom MaterialData UV & Parameter Bugs [SOLVED]

On 30/10/2014 at 09:45, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   15 
Platform:   Windows  ;   
Language(s) :     C++  ;

---------
Hi PluginCafe,

===== Download for complete sample that illustrates the described bugs ======
Test scene including a texture and some geometry to showcase the bugs:
https://dl.dropboxusercontent.com/u/15216944/C4DForum/MyMaterialBugsTestScene.zip

Visual Studio 2012 Project with "MyMaterial" including all resources. I kept the Plugin.cdl64 in it if you simply want to test the plugin without recompiling it. Note that it was compiled for R15.
https://dl.dropboxusercontent.com/u/15216944/C4DForum/MyMaterialBugProject.zip

===== Problem 1 - UV mapping of parametric sphere with parent in editor viewport ======
I got the rendering of my material working in the editor by using InitGlImage(). 
It seems to work for:
* Everything when OpenGL is disabled in the Preferences (so software rendering is used)
* All polygonized/editable objects (at least I have not encountered a problem here)
* All parametric objects except spheres
* Spheres if they are not in any hierarchy

It does not work for:
* A sphere that is a child of another object where the material is assigned to one of its parents

To verify this bug please download and open the test scene. Simply turn OpenGL on and off and watch how the uv mapping of the sphere is broken when OpenGL is on:

_<_img src="https://dl.dropboxusercontent.com/u/15216944/c4dforum/mymaterial_sphere_uv_mapping.png" height="249" width="901" border="0" /_>_

===== Problem 2 - UV mapping of parametric spheres  when rendering ========
I also got the rendering working by implementing CalcSurface etc.
It seems to work for all polygonzed/editable objects as well as for all parametric object, except:
* Spheres

To verify this bug, please download and open the test scene and simply hit render. Below you can see that the two parametric spheres have wrong uv-mapping where the right one, being a polygon object, shows the correct uv-mapping.
<_<_img src="https://dl.dropboxusercontent.com/u/15216944/c4dforum/standardrenderer_sphere_wrong_uv_mapping.png" height="358" width="843" border="0" /_>_" />

===== Problem 3 - Parameter updates when OpenGL is enabled ========
This problem was already explained here:
https://plugincafe.maxon.net/topic/8104/10550_materialdata-fetchsample-texture-solved

Nobody seems to know an answer and Andreas Block told me that this is a little bit more complicated, so I decided to reformulate the problem here including a whole project to replicate the bug.

In the next image you can see that all parameter updates are performed "one step behind". This only happens when OpenGL is enabled. If the diffuse color is, for example, green and I want to change it to blue, then the update is only performed in the preview but not visible in the editor. As soon as I change any other parameter or the diffuse color, the changes can be seen in the editor, but always on update step behind. The "dark purple" in the last image should be blue, I had to recreate the image, sorry for that ;-)
Note that I added an EventAdd() to solve the same issue when OpenGL was disabled. This unfortunately didn't solve the problem for OpenGL though.

_r="0" />

===== Outro ========
If you have any problems compiling/opening any of the attached files, please tell me.

This post cost me quite some time, so I would really appreciate help and answers from the support team. Some of the bugs seem to require one of the developers, I believe.

If we could solve all the problems in this post, it seems to be a great addition to the sdk-samples, because it illustrates how to emulate the C4D materials in a custom way.

Thank you all!
FrozenTarzan

On 30/10/2014 at 13:53, xxxxxxxx wrote:

Try turning the Render perfect option off for the spheres. And see if the problem goes away.
I wrote an identical version of this same plugin in R13. And it works well.
But I also get strange results from spheres when their render perfect option is enabled.
It only happens when rendering.
The preview texture on the object in the scene editor shows up properly for me even with RP enabled. So it's probably something in the volume data that needs setting up.

There's probably some code in the standard materials written by Maxon that handles the render perfect scenario.
I never bothered to ask them what they did in their code to make that work properly. But I'd like to know myself.

-ScottA

On 30/10/2014 at 14:54, xxxxxxxx wrote:

Hi Scott,

you are right. Turning "render perfect" off fixes the UV-mapping bug for the editor as well as for the rendering:
_<_img src="https://dl.dropboxusercontent.com/u/15216944/c4dforum/mymaterial_sphere_uv_mapping_renderperfectoff.png" height="97" width="400" border="0" /_>_

* So can this be verified as a bug by MAXON support?
* How are we supposed to fix this or is there a workaround, since I do not want to force my users to turn the "render perfect" option off by hand. A script, doing this, seems like a hack. The default material obviously found a way to cope with the "render perfect" option.

[EDIT]: The "render perfect" option is turned off by my own plugin (not the stripped example above) because I had trouble with them when they were converted to RayObjects. Anyone interested, here the corresponding thread:
https://plugincafe.maxon.net/topic/8226/10722_relations-baseobjectrayobjectopolygon-solved

So in some sense I already use a hack to cope with the perfect spheres. But my script turns the options off when the renderer is started and notifies the user about what has happend. Like: "5 perfect spheres found, turing off 'render perfect'". But in the editor I would have to check when a sphere is created, check for the option, turn it off and notifiy the user. All this seems to be such a mess.

On 30/10/2014 at 22:40, xxxxxxxx wrote:

I think the best approach would be from the material tag!!, tags can modify object data, so you can force turning off render perfect from the material tag, and put this in your documentation

On 31/10/2014 at 07:30, xxxxxxxx wrote:

Hello,

thanks for your extensive post. As it seems there are some issues unclear I will write a simple material example in the next weeks trying to verify your problems or to show solutions.

I also took a quick look at your code. It seems there are may questions open. For example, in MyMaterial::InitRender you ignore the given document and work on the active document.

When Cinema renders a document it will (almost) always create a copy of that document and renders that copy. This way you can edit your document (or any other document) while the render process is happening in the background.
The active document is the document that is currently open in Cinema; this document may not be related to a current render task at all. In some cases (like rendering using Team Render) there may be no active document at all.
When Cinema creates that copy of a document it must create a copy of all elements in that document. This is when CopyTo() is called. Using GetActiveDocument() instead of the actual document related to that render process could lead to any weird behavior.

For any specific question I suggest to create one thread per question so everyone can keep track on the progress on this question.

Best wishes,
Sebastian

On 31/10/2014 at 07:42, xxxxxxxx wrote:

I can send you my version if you want Sebastein.
Mine is much, much simpler (one .cpp file, no STL stuff, etc..) than FT's version. And it works perfectly except for sphere's with the RP option enabled.
But it's using the older R13 SDK code.

There is definitely an issue with handling Render Perfect sphere's that we need to be told how to deal with.
This definitely needs to be documented in the SDK.

-ScottA

On 31/10/2014 at 10:09, xxxxxxxx wrote:

Hey Sebastian,

I'm looking forward to your sample :-)

For example, in MyMaterial::InitRender you ignore the given document and work on the active document.

I commented the line in code, but I guess not that clear^^ I changed irs.doc to GetActiveDocument() because the GenerateTexturePath failed when the RenderDocument() call was made for the preview. The path targeted a strange subfolder of module in some C4D directory, so I was not able to find my textures. The GenerateTexturePath part of my code is also commented and weird, as the documentation about this call is plain wrong!

On 03/11/2014 at 01:56, xxxxxxxx wrote:

Hello,

can you give an example of how you have problems with GenerateTexturePath()?

Also, I must discourage you from using GetActiveDocument() in any render context because the active document may not be related to the render process at all.

Best wishes,
Sebastian

On 03/11/2014 at 03:03, xxxxxxxx wrote:

I guess you know that the docu about GenerateTexturePath is wrong? There is already a thread about it here in the forum. Search for "TEXTURE (description element)") in the online R15 docu. 
This code is from MyMaterial::loadDiffuseTexture, that can be found in the attached project.

// Instead of this:
GenerateTexturePath(pDoc->GetDocumentPath() + pDoc->GetDocumentName(), Filename(diffuseTextureString), Filename(), &texturePath)
// This has to be used:
GenerateTexturePath(pDoc->GetDocumentPath(), diffuseTextureString, Filename(), &texturePath)

Since I have fixed this code, I don't have any problems, except that the docu should be corrected.

The real problem I'm having is, that the GenerateTexturePath returns a completely strange path when it is called for the material preview. You can simply debug this:
1. In MyMaterial::InitRender() change the GetActiveDocument() back to irs.doc (which is the "correct, active, current, used" document for the rendering)
2. Add this line just before GenerateTexturePath is called in loadDiffuseTexture:
Filename documentPath = pDoc->GetDocumentPath();
3. Set a breakpoint in loadDiffuseTexture at the newly created line
4. Hit F5 in visual studio and C4D will start, because I have configured the debugger like so.
5. Navigate to the MyMaterialBug.c4d scene that I have attached and open it (all from C4D, so that the debugger is used!)
6. The breakpoint triggers, hit F10 (step over) and you see that the documentPath is indeed the correct one (E.g. C:\Users\FrozenTarzan\Desktop\MyMaterialBug). So just hit F5 (continue) or step over the rest to verify that everything works fine. This call was made to generate the OpenGL image (at least I guess so)
7. Double click the MyMaterial.
8. The breakpoint triggers again, hit F10 (step over) and see that the documentPath is something like this:
C:\Data\Tools\Cinema4d\resource\modules\xtensions\preview_scenes
This path cannot be used to find the textures with GenerateTexturePath because there are none! So my only solution was to change the line irs.doc to GetActiveDocument because then I can retrieve the correct paths for my textures.

Please use the instructions above to verify the problem.

On 05/11/2014 at 06:10, xxxxxxxx wrote:

Hello,

when you double click on your material Cinema will create an new preview image. This preview image is created by rendering a simple scene. This scene is stored in the folder you mention above (resource\modules\xtensions\preview_scenes). To find out if the current render process is a preview rendering simply check the flag INITRENDERFLAG_PREVIEWRENDER.

The path to the original scene is stored in the given document's BaseContainer with the ID DOCUMENT_SECONDARYPATH.

  
  if(irs.flags & INITRENDERFLAG_PREVIEWRENDER)  
  {  
      if(irs.doc && irs.doc->GetDataInstance())  
      {  
          const Filename originalPath = irs.doc->GetDataInstance()->GetFilename(DOCUMENT_SECONDARYPATH);  
          GePrint(originalPath.GetString());  
      }  
  }  

Best wishes,
Sebastian

On 06/11/2014 at 01:18, xxxxxxxx wrote:

Hey Sebastian,

sorry for the late reply!

Your code snippet works great. My textures are found now :-)

Open issues:
* Perfect sphere options and workaround for correct UV-mapping when OpenGL is enabled and when scene is rendered. As we discussed earlier there are hacks to accomplish it by turning the "perfect sphere" off, but I would like to see an official statement.

* Parameter-Update bug when OpenGL is enabled

Thanks,
Andreas

On 07/11/2014 at 07:06, xxxxxxxx wrote:

Hello,

to support perfect spheres and also different projection types set in the Texture Tag you have to set the VOLUMEINFO_EVALUATEPROJECTION flag in GetRenderInfo().

When you use InitGLImage() you have to set the filled bitmap dirty using SetDirty(). This will trigger the update.

We are currently working on a new MaterialData example for the SDK and want to publish is soon.

Best wishes,
Sebastian

On 07/11/2014 at 07:45, xxxxxxxx wrote:

Can I make a suggestion about your example? Or is it too late?

The current Materials examples don't show us how to make a material with only a couple of channels.
They either have no channels. Or all of standard the channels.
And it took me a while to figure out how to make a material with only a Color and Alpha channel because of that.

I also discovered that the checkboxes to enable/disable the channels are just ordinary generic gizmos declared in the .res file. Which I was surprised to discover.
Things like this are not currently explained very clearly, or at all, in the current MaterialData examples.

-ScottA

On 07/11/2014 at 11:26, xxxxxxxx wrote:

I would encourage too a very single addition "if possible" to the example, as I'm creating my renderer, I got 1 little thing to know about "material preview" , how they are rendered

On 08/11/2014 at 02:10, xxxxxxxx wrote:

Hey Sebastian,

to support perfect spheres and also different projection types set in the Texture Tag you have to set the VOLUMEINFO_EVALUATEPROJECTION flag in GetRenderInfo().

You are right, that this solves the uv-mapping of the parametric spheres without turning the "render perfect" off. But it breaks my material preview. When I use the VOLUMEINFO_EVALUATEPROJECTION flag my material preview still shows my diffuse color, but as soon as I load a texture, the preview becomes completely black, when I remove my texture, it shows the diffuse color again and seems to work. 
EDIT: I debugged it and this call in CalcSurface() :

m_pDiffuseTexture->GetPixel(x, y, &r, &g, &b);

always returns 0, 0, 0 for r, g, b when the render call is made for the preview.

When you use InitGLImage() you have to set the filled bitmap dirty using SetDirty(). This will trigger the update.

NICE! It works! For everyone else:
In: Bool MyMaterial::InitGLImage(BaseMaterial* mat, BaseDocument* doc, BaseThread* th, BaseBitmap* bmp, Int32 doccolorspace, Bool linearworkflow)

when you have set your pixels (in my case fetch a texture and multiply a color value) simply call:
bmp->SetDirty();
in addition to the call, already in my sample:
EventAdd();

I will close the other related thread and set it to solved. In this one we still have some issues with the preview and the projection evalutation.

@MohamedSakr:
My sample project available at the top shows how to create the preview. I copied it from one of the examples. As you might have noticed, there are still bugs in it. ;-)

Bye,
Andreas

On 08/11/2014 at 11:16, xxxxxxxx wrote:

In my R13 version. I didn't use GetRenderInfo().
If I add that to my code. And set the VOLUMEINFO_EVALUATEPROJECTION flag. It does fix the render perfect problem. And displays the image correctly in the editor window even when rendering.

However. It also breaks the material preview image.
It turns it into a bright glowing white color.

Looking forward to seeing the final example.

-ScottA

On 10/11/2014 at 02:30, xxxxxxxx wrote:

Hello,

The BaseChannel system is just a utility tool to organize a material. It is only used by the standard material so GetChannel() will only work with the standard material.

If you use the metaphor of "channels" in your material is up to you. In the resource file of a material plugin you can define different groups that allow you to organize your parameters. Take a look at the "Cheen" material to see how each group controls a completely different aspect.

Using and customizing the material preview is a large topic. For the most simple case just add "INCLUDE Mpreview;" to your res file and the preview itself will handle the rest. If you have any specific questions please open a new thread so the topics don't get mixed up.

Best wishes,
Sebastian

On 10/11/2014 at 03:01, xxxxxxxx wrote:

Hi Sebastian,

Using and customizing the material preview is a large topic.

That should not be a large topic in the API... it's just a small image of a single render pass^^

Have you tried to use the VOLUMEINFO_EVALUATEPROJECTION flag in my sample project? Have you tested it with spheres? Can you verify that the preview is broken because the GetPixel() returns 0, 0, 0 for the fetched color (like I described above)?

I'm really interested how those interactions can happen in the API. The evaluation of a projection of UV-coordinates has nothing to do with a bitmap. Why in the world is there a difference WHEN and WHERE I fetch texels from a bitmap. This makes absolutely no sense and should be tackled!

Thanks,
Andreas

On 11/11/2014 at 00:55, xxxxxxxx wrote:

Hello,

if the given x/y coordinates are outside the dimensions of the given BaseBitmap, GetPixel() won't do anything and will not change the given r,g,b values. You should check if the x/y coordinates based on the UV-settings are valid.

I uploaded the simple material example. It may be included in the SDK.

MaterialData example (ZIP)

Best wishes,
Sebastian

On 11/11/2014 at 09:10, xxxxxxxx wrote:

Thanks for the sample! You are right that the uv coordinates of my objects where greater than 1.0. I really don't know why... this again makes no sense at all because the uvs should be the same independent of the "evaluate projection flag". The mapping looks exactly the same in the preview... but whatever, using this code in CalcSurface() works now:

float integral = 0.0f;
float u = modf(vd->uvw.x, &integral);
float v = modf(vd->uvw.y, &integral);		
  
Int32 x = static_cast<Int32>(m_pDiffuseTexture->GetBw() * u); 
Int32 y = static_cast<Int32>(m_pDiffuseTexture->GetBh() * v); 
  
UInt16 r = 0;
UInt16 g = 0;
UInt16 b = 0;
m_pDiffuseTexture->GetPixel(x, y, &r, &g, &b);

There is only one singe bug in here:
1. Enable OpenGL
2. Create a Null-Object
3. Create a parametric sphere (do not make it editable!)
4. Make the sphere a child of the null object
5. Create MyMaterial (download still up there in the post) and select a texture for the diffuse/color channel
6. Assign it to the Null-Object (the parent of the sphere!)
7. Select the sphere (child!) and move it

The result is, that the uv-mapping changes depending on the position of the sphere relative to the parent (null-object). It seems that the mapping is only correct, when the sphere is exactly at the origin/center of the parent.

See here:
_<_img src="https://dl.dropboxusercontent.com/u/15216944/c4dforum/materialbug_opengl_renderperfectsphere_parent_movefromcenter.png" height="239" width="997" border="0" /_>_