Shader tree/node implementation

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 09/02/2005 at 13:47, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   8.012 
Platform:   Windows  ; Mac  ;  Mac OSX  ; 
Language(s) :     C++  ;

---------
I'm currently at an impass on how to go about adding a Shader tree/node system to the Materials of Cinema4D. I know that Fusion allows mixing of two channels, but this is very limited (and have heard of the complexities of 'simulating' trees with it).

The initial problem is how deep the system will need to go to be useful: from nodes to render, nodes to material, nodes to channel, or some other level of implementation.

Do I go the DarkTree/Simbiont route and use the nodes to create a final texture bitmap (but all internal to C4D) or do I keep the nodes pure and procedural until render (or creating the material display)?

I'd appreciate some direction from which to start so that I'm not coding for weeks and find that it was a bad path to follow... ;)

Thanks,

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 09/02/2005 at 18:44, xxxxxxxx wrote:

Just write a volume shader that is your shading system.

It really is pretty simple. Inputs / Outputs...

Do not make it more complex than it needs to be.

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 09/02/2005 at 21:01, xxxxxxxx wrote:

Do I really need 'volume' shading for mostly regular materials though (surface shaders and textures)? Volume incurs that it has to work on a solid 3D space form. I didn't say 'volume', I said 'procedural'. :)

And if it's so simple, why isn't there a mega shader-node system already for C4D? There isn't. There's Fusion (a whole two channels combined) and Simbiont (which does all of the nodal stuff in a $520 external application and passes you a bitmap).

Hmmmm....

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 10/02/2005 at 07:27, xxxxxxxx wrote:

Because we chose not to release it. 9-)

I wrote it for v7, then v8. Hmm, we released the expression hub for free ( been around a long time ). Maxon chose to develop Xpresso. We hear rumours they are working on their node based material system. It will most likely suffer the same issues Xpresso does but no one is going to lay out a couple of hundred for something they can pretty much do internally.

By volume shader I meant a MaterialData plugin. Old lingo.

If you show us yours we will show ours. 9-)

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 10/02/2005 at 09:16, xxxxxxxx wrote:

Smart alek, huh? ;)

Right. It will be a Material plugin (thanks for clarification there). But do I write it, as I suspect, with the end result being a texture shader? I haven't done any Material or shader plugins yet (only Menu and Tag).

My feeling is that this will be a shader with a dialog/AM in which to display and connect nodes that sends its results to the Material's Channel. Right path?

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 10/02/2005 at 10:22, xxxxxxxx wrote:

I prefer turse to smart ass but everybody has an opinion... :-)

The reason to create a system like this is to create a workflow. The workflow in channel shader system is fairly limited ( not being able to see mutliple trees, no aliasing support, limited global scene access ). You are writing a single self contained shader as a hub. A starting place to retrieve data and a place to set data.

The AM is a bad place to create a node layout. Just create a button to launch the editor. All of your editing will take place in the editor ( similiar to Xpresso editor ).

Not sound offensive, on the whole it is not difficult. It is just alot of details ( node classes, GUI, etc... ).

Probably the most difficult part for me was optimization ( when do nodes need to be evaluated, when do you use the cache, how much to expose to the user, etc... ).

Start by creating the node classes. The parent node structure that will be inherited by all of your nodes is very important. If this is solid the GUI is much simpler. Special cases are bad and can usually be avoided with a little forethought or rewriting ( alot of rewriting ). When I did mine there was no parameter descriptions in C4D. We wrote our own, now it is much simpler. I guess that was what I was trying to get at. Alot of the work is already done comparatively.

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 15/02/2005 at 13:28, xxxxxxxx wrote:

But it depends on how you are looking at this. After doing some 'studying-up' and looking at the shader examples in the SDK, I don't think that I want a MaterialData plugin. This means implementing all of the available C4D channels and their internal controls over again. Why? I don't want to bypass C4D's Material system, I want to enhance it.

I want my plugin to attach to a Material's channel (using C4D's existing Material and Channel system), implementing the nodal system into that. That sounds like a ShaderData plugin to me. ;) Each Material will have its own tree, but I want to keep the current C4D Material and its Channels as the end result.

Not difficult for "Mr. Material Shader Guy" maybe (I know about you). But to "Mr. 3D Guy", no shader experience, this is all new to me. I'm awaiting "Textures and Modeling", but it will not be here for another week or so. There is precious little info on this type of material system (and I've scoured books and net hard).

P.S.: Are you a game programmer, by chance? ;)

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 15/02/2005 at 15:23, xxxxxxxx wrote:

As far as the shaders go you only learn by doing. I wish you a good journey.

Know what about me? SLA is just one of many tools I have helped develop. Currently we have a complete app in development. We write whatever we need. Not really limited to plugins anymore.

Game programmer!? Did your homework did ya? One of my many talents. 9-)

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 15/02/2005 at 16:28, xxxxxxxx wrote:

That's what I intend on doing. And it will be a bumpy ride, I suspect. ;)

Yes, that's what I mean. I haven't touched C4D materials (except to set them) or materials in general. Done plenty of 3D CG programming some time ago (and more recently in C4D's SDK), but not at the shader level. But you have lots of background expertise in this area. In my case, I'm only programming (again) for the set of plugins being created. No intentions on going back to software development unless there is a lucrative offer. :) Actually, I'd rather be creating CG animations.

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 16/02/2005 at 12:58, xxxxxxxx wrote:

My real job is creative at www.coreaudiovisual.com. Basically I am a proceduralist whose creative capacity is inventing the techniques we use. My development efforts are 100% focussed on creating tools to use. That is why FM took a back seat, it is viable for us now and we have no real need to make it something it is not. I am sure Maxon will deliver something that will keep the fanboy base appeased.

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 16/02/2005 at 14:28, xxxxxxxx wrote:

In my case, I'm my business. :) Development (C,C++,Java,HTML,COFFEE,Python,...), 3D CG, and AV editing/authoring.

Now, see, I like the idea that Maxon is working on a tree-node based material system. But, mine is to support Poser 5's tree-node system (purchased and adapted from Pixels3D Shadermaker) in C4D 8.0+. So, no matter how hard Maxon is working (and I'm sure they are), it wouldn't be any benefit to me. ;) This will be an add-on to my current commercial plugin, but might sell it separately if the end result is as hoped.

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 16/02/2005 at 23:44, xxxxxxxx wrote:

Quote: Originally posted by kuroyume0161 on 16 February 2005
>
> * * *
>
> In my case, I'm my business. :) Development (C,C++,Java,HTML,COFFEE,Python,...), 3D CG, and AV editing/authoring.
>
> * * *

Same for me :-) I started web scripting (Perl and PHP) 5 years ago, then got to Cinema 4D and started with COFFEE then got into C++ as my brother (long time programmer) forced my to switch. Now plugin dev for c4d (had some offers from big companies to write for 3DS, XSI and InHouse but as a student, no way). When my studies are finsihed I hope I can make my way as an artist combined with sort of programming. (c4d will surely stay my main app for a long time :)

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 18/02/2005 at 09:21, xxxxxxxx wrote:

Darf, do you have any references useful for creating a multi-linked node GUI? I see that this will all have to take place in a GeUserArea within a GeDialog since the SDK widgets won't work in this situation. This somewhat explains "Xpresso". ;)

Katachi, simultaneously thrilling and scary to be running any type of computer home-business these days. Costs go up and up and up while income goes down and down. If they raise my monthly mortgage payments one more time, my next address will be a cardboard box. :)

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 18/02/2005 at 09:44, xxxxxxxx wrote:

We didn't really use any references. Bt was completely conveived and executed ground up the way we wanted to use it. We also had a GeUserArea and a node class and went from there. All nodes are actually Command Plugins that register themselves with the Node Hub. All interaction is in the node itself ( event handling ). It is actually fairly simple concept. Your nodes will need to be able to respond to events you create such as new frame, new sample, etc... This will allow the nodes to define themselves as executable or dummies ( always start at the top of the stack and work your way down ). It is very important to be able to define node caches and use them whenever possible. The node class is your foundation, spend the time required on it.

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 18/02/2005 at 10:05, xxxxxxxx wrote:

Hi darf,

I was afraid you'd say this. ;) So far, I have rudimentary Node and NodeInput classes (which allows for variable inputs). Still debating on whether to use the Node class like this (allocating input arrays) or create the many types of node into derivations of this (allocating nodes only). Either way, the overall structure will remain the same, just means that inheritance must be carefully propagated.

So, you had two levels of event handling - one for the HUB and one for the nodes? Is that right? Interesting approach using nodes as CommandPlugins. How did you avoid the problem of unique ID registration or did you just use one Plugin ID and another unique ID associated with the plugin?

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 18/02/2005 at 13:23, xxxxxxxx wrote:

Two levels of event handling? Event hits userarea, depending on event it is passed to the appropriate node or global handler. Using command plugins is the obvious choice as it makes it simple to add nodes. Unique ID registration? Plugin ID defines type. There is no other id required ( in our system ).

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 05/03/2005 at 22:06, xxxxxxxx wrote:

Myabe we're looking at this from two different vantage points.

My outlook is that there is a ShaderTree which contains the ShaderNodes. One ShaderTree per Material, ShaderNodes being nothing more than data storage and algorithms for the particular type. These are linked to the Material by shaders - so, for example, you have nodes connected to the Color, Diffusion, Transparency channels and therefore there is a shader created in each of these channels to act as a conduit for the end result during render or matpreview (of course copying any shader/texture displaced in that channel for later restoration if needed).

My only difficulty at this stage is where to store the ShaderTree itself. It is not derived from C4D classes, so it needs to be stored somewhere - not in the shader. Closest so far is the Material's UserData, but that creates a problem of retrieving it for the shaders.

If you would be so kind as to explain an alternative approach in this specific situation in more detail, I'd be very grateful. To my disadvantage maybe, I find the structure of C4D's SDK to be very confusing - sort of like the way that German professors spew long-winded sentences that finally end on the verb (no disrespect there, just an example). It seems the SDK is built on this stack of information premise with the final resolution at the end requiring a recurse back through the sentence to find the understanding. ;0)

As an instance of the confusion: NodeData for Node Plugins. Okay. When you're done making all of the resources and strings and classes for the NodeData, how do you allocate it? There is no PluginNode. In this case, do you just create your derived class and gNew it? How could anyone but the developers possibly know since there is not one single explanation anywhere (not online, not in the Docs, not in the examples, not in the Resource). This SDK needs about 10 times the examples and twice the documentation for those of us who don't think exactly like the developers.

ETA: I'm not alone in this. I've seen some of the best C4D plugin developers with similar problems and questions in this forum - most of the questions revolve around confusion over Descriptions, BaseContainers, Nodes. One deficiency (and it is glaring) is that there seems to be intentional inability to extend the existing 'objects'. UserData gets you so far, but it is sufficiently limited to make it useless - in my case anyway.

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 05/03/2005 at 22:20, xxxxxxxx wrote:

You are hitting what took us years of planning and a year of execution. It is a very simple concept. What we learned was to abstract. You are not creating a shader system, you are creating a node based data evaluator ( go ahead, make fun ). After using MAYA and Shake for the last few months I have to say I do not believe C4D will catch up in that regard. It is a guessing game. Sterner helps as much as he can but I am not even sure he is full time. Maxon does not seem to have the resources. So, remember, important questions require code examples, post on PluginCafe and give it a week before flippin ( which I have done alot when depending on this stuff for money ). Watch your rants here, Sterner has always helped us when we really needed it.

darf

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 05/03/2005 at 23:46, xxxxxxxx wrote:

I'm not ranting as much as venting frustration. And I have nothing but respect for Mikael. It is a shame though that the other vast experiences involved in this forum will not share any of it. There are no other resources (unlike Maya and 3DSMax who have books and videos and courses and limitless online resources) but here, PluginCafe. There might be more under the .de moniker - my Deutsch isn't good enough (nor will it ever be).

1. A ShaderPlugin is required (absolutely and irrevocably) to get the ShaderNode data into each Material's Channel. That's easy and no way around it. I'm not recreating C4D's Material system with its vast amounts of built-in usefulness and then try to keep up with it as it progresses.

2. Since each ShaderTree represents ShaderNodes connected to one Material (the ShaderNodes can connect to more than one Channel in the Material), there is a need to store the ShaderTree someplace outside of the ShaderNode (or, the worst that any programmer could ever conceive: have the ShaderNodes maintain separate but identical ShaderTrees, always keeping them identical - that is both memory consuming and perposterous - I won't even consider such a hair-brained strategy). Make sense?

3. There is no place to store the ShaderTree. UserData (i.e.: CustomeDataType) is too limited. I cannot put the ShaderNode editing dialog in the AM for the UserData (again, as you stated yourself, perposterous), so the editor dialog has to be called upon with a button or similar. But UserData doesn't support buttons (at least not without creating a CustomGUI just for a button). And accessing a non-C4D something class from the UserData is an exercise in futility. A CommandPlugin will only be relevant when it is launched and then would need to store ALL the ShaderTrees for all materials that contain them for all documents. There should be no need to manage all of the ShaderTrees in one place since they are isolated by Material - the only communication may come in cut/copy/paste.

See, you spoke of a 'hub' as if you were controlling all of your nodes for everything in one place or on a PluginMaterial basis. I need to control nodes on a Material by Material basis (thus the extreme focus on attaching it to the Material) - C4D "rigid as steel" Materials with limited extensibility. Yes, I could make a CommandPlugin that lets me work on every material of each Document one at a time, but how does that work during programmatic construction of ShaderTrees for existing Materials (not loading documents but reconstruction of another shader node system)?

So, what was your 'hub'? How does one orchestrate several plugins across more than one document?

I ask because it is difficult and there isn't any second set of eyes to provide support or expertise. I can think abstractly, but not in the same way that the SDK works - at least not without an abstract explanation of the overall architecture. Thus my references to many getting lost in this abstraction unexplained. There are many strange designs to it. Just the way that plugins are described (using NodeData) but created (using some Plugin inteface - sometimes nearly inaccessible) requiring the data attached to the plugin which you created to be passed using messaging. Yikes - I believe in direct and simple. This is obfuscation at its ultimate. You should really check out the amazing simplicity of Java and Python. Much, much, much less work for much, much, much more progress (in an unlimited programming interface).

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 06/03/2005 at 07:52, xxxxxxxx wrote:

Forest, trees, they are the same yet different.

You are jumping way beyond where you should start. Create an object that has a few nodes that you can create, edit, move around ( even if they are hardcoded into the one plugin ). Then take on more. This is a simple yet large project.

Yes, I used messaging at that time to register the command plugins as nodes. I added functions to the class decaration to do what they needed. Thus, Command Plugin becomes node ( to every hub it applies to ). A hub is merely a Tag, Object, Shader, or VP Filter that then calls a node tree. A bit more complicated than that but not impossible.

darf