Weird filenames when using make project [SOLVED]

On 27/03/2014 at 07:55, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   R15 
Platform:   Windows  ;   
Language(s) :     C++  ;

---------
Hi,

I have an object which uses external files and adds them to the asset tracking when a MSG_GETALLASSETS message is received. Unfortunately when copying this object (e.g. with copy/paste) I get weird asset file names when exporting with "make project" although the filenames passed to AssetData::Add() are correct (in this case identical).
For example if the original asset filename is "name.xml.gz" and I have one copy of the object then two identical files will be exported with the following names: "name.xml.gz" and "name.gz"
If I create a second copy of the original object an additional file "name.xml_100.gz" will be added and each additional copy will add another file with an incremented number (101, 102, etc.).
Is this a bug? Is there a way to avoid this behavior?

Best regards,
Satara

On 17/04/2014 at 15:12, xxxxxxxx wrote:

Hi Satara, I'll forward this question to the developers. Could you please elaborate on the "Make
Project" part? I can't find a command in Cinema 4D that is called like that.

Best,
-Niklas

On 18/04/2014 at 01:31, xxxxxxxx wrote:

Hi Niklas, thanks for forwarding the question. What I mean is the "Save project with Assets" functionality reachable from the file menu. In the docs it's still often mentioned as "Make project". See c4d_basedocument::SaveProject().

On 24/09/2014 at 02:51, xxxxxxxx wrote:

Hi Niklas,
this problem still exists, could you please forward it to the developers? If you need more input just let me know.

On 05/11/2014 at 10:58, xxxxxxxx wrote:

Hi Satara,

Acknowledging that I'm going to actively research this problem and get back to you in this thread if I have an answer or any questions.

Thanks,

Joey Gaspe
SDK Support Engineer

On 07/11/2014 at 11:31, xxxxxxxx wrote:

Hi Satara,

No, this is not a bug.  Your object gets a notification that the assets were renamed: see MSG_RENAMETEXTURES.  You could rename them back after all assets are collected and renamed, but you can't avoid this behavior.

You need this message (MSG_RENAMETEXTURES) if you provide Cinema 4D with file names in MSG_GETALLASSETS.  If any file names collided and have been renamed (eg. after the user selected "Save Project with Assets"), you will know with this message.

You should also support MSG_MULTI_CLEARSUGGESTEDFOLDER.  You use it like this:

  
if (type == MSG_MULTI_CLEARSUGGESTEDFOLDER)  
{  
  
  BaseContainer *ctrdata = ((BaseObject* )node)->GetDataInstance();  
  Filename texname = ctrdata->GetFilename(YOUR_FILENAME_ID);  
  ctrdata->SetFilename(YOUR_FILENAME_ID, texname.GetFileString());  
  return True;  
}  

When the user saves the project with assets, everything will be moved to the new target directory.  In this case all absolute paths must be converted to just file names (because the files are next to the document and the absolute paths are not needed anymore).

I hope that helps!

Joey Gaspe
SDK Support Engineer

On 11/11/2014 at 02:22, xxxxxxxx wrote:

Thank you for clarifying this process, Joey.

The main reason for my question was that I want to reduce the project size by avoiding duplicate assets. I now keep track of which assets have been added already by using a set and just ignore duplicate assets (don't add them again).
The asset is a static variable of my class which is derived from ObjectData. Obviously I have to clear it after every saving process. I use MSG_DOCUMENTINFO_TYPE_SAVEPROJECT_BEFORE for this purpose now. Is it save? I mean are there cases where assets get registred, but this message is not sent (maybe Team Render)?

On 13/11/2014 at 19:52, xxxxxxxx wrote:

Hi Satara,

I carefully combed through C4D's source and found that your hunch seems to be correct:

The MSG_DOCUMENTINFO_TYPE_SAVEPROJECT_BEFORE message is sent as long as SAVEPROJECT_DONTCOPYFILES flag is not set when SaveProject() is called.  The only time that flag is set by C4D is when NetRenderService::StartRendering() is called, and there's no way to override it.

I'll leave it to you to determine if that is safe or not for your code.

Joey Gaspe
SDK Support Engineer

On 25/11/2014 at 18:27, xxxxxxxx wrote:

Hi Satara,

I was wondering if you had a chance to work on this problem, and if you had any new questions?

Thanks,

Joey Gaspe
SDK Support Engineer

On 10/02/2015 at 02:01, xxxxxxxx wrote:

Hi j_gapse,
sorry for not letting you hear from me. I thought this issue would be resolved, but now I noticed that this approach does not work when using Team Render.
As explained before, I only add assets once by storing the names in a set and then check if an asset has already been registered.
When I start a render job using Team Render MSG_GETALLASSETS is received multiple times in a node. If I only add the assets at the first time and skip them next time I get the following error (dialog-messagebox) : "Saving failed - Details: Could not create render job because all needed assets could not be extracted". Can you please explain me what exactly happens inside the core so I can try to find a workaround?
And regarding NetRenderService::StartRendering(), is it also used for Team Render or only Net Render?
Best,
Burak

On 11/02/2015 at 20:27, xxxxxxxx wrote:

Hi Satara,

Perhaps what you want to do is not possible beyond any shortcuts and optimizations you may already have implemented.  I looked at the processing of MSG_GETALLASSETS in practically all cases in C4D.  It's a message being sent around with asset data following along, collecting more at each stop that isn't a simple rebroadcast, building up in size as it passes through various areas of C4D (materials, textures, lights, sound, etc.).  It's not very complicated in terms of the message broadcasting and processing.  It makes no attempts are reducing duplication, as it's goal is to not miss anything, so it passes through all lists of relevant data.  In some cases, which I didn't analyze deeply, the rules to determine what to add, if anything, can get complicated, but the essence of what it's doing is what I'm describing.

Attempting to reduce duplication within C4D would probably result in lower performance than if you do it at the end, as you would only be checking once per asset name from the full list (as the case is if you're using a set as you mentioned in a previous post), rather than repeatedly as the list is being built up, which goes up exponentially in processing requirements.

As for NetRenderService::StartRendering(), it is used for Team Render, as I found it implemented in its code.

Joey Gaspe
SDK Support Engineer

On 12/02/2015 at 04:04, xxxxxxxx wrote:

Hi Joey,

I wanted to chime in on this (I work with Satara). It feels like there is a misunderstanding here. For all I'm saying now I'm making the (potentially wrong) assumption, that the full functionality of this mechanism is exposed to the SDK and that there is no magic going on behind the scenes that are not accessible from the outside.

You are saying that it seems like what Satara is trying to do is impossible and talk about your observation that C4D doesn't seem to make any attempt on avoiding duplication. I am not sure if I can follow, because clearly when more than one object or shader reference the texture multiple times or an xref us used multiple times, it is still only written once. As we obvserved further up in this discussion, given the way MSG_GETALLASSETS works, this is only possible if some care is taken to not duplicate assets, otherwise they will show up multiple times. So I'm wondering what makes you think now that Cinema 4D doesn't avoid duplicates. The only way this seems plausible to me would indeed be the case, that there is something going on under the hood that's not exposed through the SDK.

Assuming we're still d'accord about our objects having to do some bookkeeping about which assets have already been reported to MSG_GETALLASSETS, the problem Satara is having really is that Team render seems to create a duplicate of the document which then get's another run of MSG_GETALLASSETS (after the original document already had its run). And that's the case that's causing trouble.

So it would be cool to know why a duplicate is created and how this duplicate should be properly handled. This is particularly interesting as the cloned document seems to complain if assets are not reported. Why is it doing that? Is is comparing the asset lists?

Thanks for your great support, Joey!

Best
Timm

On 18/02/2015 at 07:08, xxxxxxxx wrote:

Wé are tracking the assets on per document basis now, which seems to do the job.

On 18/02/2015 at 16:02, xxxxxxxx wrote:

Hi Timm,

Sorry about the delayed response, I was busy with other matters lately.

Yes, there's a misunderstanding here, as I find it's a complicated situation:

- I assume you see C4D as a black box when I get broad situational questions that lack example code: "Can you please explain me what exactly happens inside the core ... ?".  I figured that meant 'message system', so I searched for code that handles MSG_GETALLASSETS, as that's the closest to code I was supplied.  Otherwise, there's no way for me to know where to look across millions of lines of code for 'core' functionality.

- When I report back the inner workings, I assume it shifts your view and may answer the real question you might have.  Otherwise, I expect a response with clarifications and questions that reveal the exact issue needing to be solved, and within a reasonable delay to keep up the pace to resolution.  Clarification only arguably occurred with your recent response.

- From my direction, it appeared to be the same question as back in November, just asking for more details.  Without clear information, it seems I gave answers to the wrong questions.  From now on, I'll ask for clarification at the beginning instead of assuming too much.

As for the confusion about duplication: Sorry, I didn't specify clearly what I meant.  The code I looked at does not immediately imply any duplication avoidance is attempted, even if I theoretically could have found some.  There could be duplication reduction happening either before (by not sending the message down certain paths) or after it was done collecting the assets (I didn't explore that possibility).  However, if you still find at least some duplicates, it means there isn't an aggressive attempt at reducing duplication.  Plus, to try to figure out all cases of duplication reduction would mean spending an inordinate amount of time analyzing the code, while unlikely to provide you with any useful information.

That leads me to answering your new (or at least clarified) question, why the Team Render duplicates the document:  When rendering is launched, all the data is duplicated so that you may continue working on the original data while rendering executes.  That Satara states tracking assets on a per document basis seems to do the job makes sense.  I actually don't know more specifics about the whole process (ie if any data is left out, etc.).

Please let me know if it works out for you or not, I'm glad to help, even if the process does not always go smoothly.

Joey Gaspe
SDK Support Engineer

On 20/02/2015 at 10:08, xxxxxxxx wrote:

Hi,

I guess your problem is solved, as per your statement above and in the other topic related to this problem (Team Render and MSG_MULTI_CLEARSUGGESTEDFOLDER), so I'll re-close this topic as solved, but leave it open to additional postings if ever you have more questions.

Thanks,

Joey Gaspe
SDK Support Engineer