THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED
On 31/10/2012 at 17:51, xxxxxxxx wrote:
Ok, here's what happens:
Cinema 4D will abruptly crash, without the usual dialog indicating that C4D has crashed and that a crash report has been written, while our plugin is performing a task.
Ok, here's what happens:
Cinema 4D will abruptly crash, without the usual dialog indicating that C4D has crashed and that a crash report has been written, while our plugin is performing a task.
What the problem seems to be:
The problem seems to be a rare heap corruption caused by the way the C4D API keeps track of if memory was allocated using C4D's memory management (e.g. C4DOS.Ge->Alloc and consorts) or by the default (e.g. malloc). It occurs in C4D R10 (possibly previous versions, too, but we don't support these) and R11, as well as possibly R11.5 and newer versions but I haven't seen it happen there.
What's happening exactly:
Looking at c4d_memory.cpp/.h, you can see that C4D overrides new and delete also has a way of keeping track of when C4D's memory management is available and what was allocated with it. This whole part revolves around (this is for anyone else interested. If you are familiar with how it works, you can skip this part) :
#define STDLIB_MEM_MAGIC -1
static Bool stdlib_mem_used = FALSE; // changed to TRUE if static constructors have allocated memory before c4d's memory management was available
t_C4DOS, which is NULL as long as C4D's mem management isn't available and
void *AlienMem(size_t s, Bool clear)
{
if (s<1) s=1;
void *p = malloc(s+sizeof(VLONG));
if (!p) return NULL;
if (clear) memset(p,0,s+sizeof(VLONG));
*(VLONG * ) p = STDLIB_MEM_MAGIC;
p = (void * ) ((UCHAR * )p + sizeof(VLONG));
stdlib_mem_used = TRUE; // static constructor has allocated memory
return p;
}
As you can see stdlib_mem_used is set to TRUE "if static constructors have allocated memory before c4d's memory management was available" (which is, when you are using the stl, pretty much unavoidable).
AlienMem() is called by the overridden new and delete operators for as long as t_C4DOS is NULL. If a block of memory has been allocated with with AlienMem(), it extends the requested ammount of memory by sizeof(VLONG). The first few extra bytes are set to STDLIB_MEM_MAGIC so these blocks of memory can be identified and it then returns the pointer to after the leading extra VLONG.
Why this causes a heap corruption (very, very rarely) :
After quite some time spent on debugging this we have found that sometimes, for unknown reasons (as far as we could tell, randomly), free() is called for when deleting something (usually inside an std::vector::reserve call).
Calling free() on memory allocated by C4D's memory management is pretty bad and causes a heap corruption.
Before we come to the why, here's the interesting bit of a log file where we logged most memory allocation and deallocations:
[] Allocated with C4DOS.Ge->Alloc() : 37bfaca0
[] Freeing with C4DOS.Ge->Free() : 37a0fef0
[] Allocated with C4DOS.Ge->Alloc() : 37e9ad30
[] Freeing with C4DOS.Ge->Free() : 37b7ee80
Allocated with C4DOS.Ge->Alloc() : 38143cb0
Freeing with C4DOS.Ge->Free() : 381242c0
[] Allocated with C4DOS.Ge->Alloc() : 37d43680
[] Freeing with C4DOS.Ge->Free() : 37bfaca0
[] Allocated with C4DOS.Ge->Alloc() : 37ef9820
[] Freeing with C4DOS.Ge->Free() : 37e9ad30
Allocated with C4DOS.Ge->Alloc() : 3820bce0
Freeing with free() : 38143cb0
(crash when free is called)
Note that even though the block with the adress 38143cb0 was allocated by C4DOS.Ge->Alloc() it is being freed with free().
For real now, why?:
We think that this is caused by the way the identifying of AlienMem blocks is working:
void operator delete(void *p)
{
if (p)
{
void *temp = p;
if (stdlib_mem_used) // memory allocated by static constructors?
{
if (((VLONG* )p)[-1]==STDLIB_MEM_MAGIC) // is this a block for the stdlib?
{
free((void* )((UCHAR* )p - sizeof(VLONG)));
return;
}
}
C4DOS.Ge->Free(temp);
}
}
if (((VLONG* )p)[-1]==STDLIB_MEM_MAGIC)
detects AlienMem blocks by checking if the preceding VLONG is equal to STDLIB_MEM_MAGIC. If this is the case it correctly calls free() instead of C4DOS.Ge->Free().
But what if the preceding VLONG happens to be equal to STDLIB_MEM_MAGIC by pure chance? free() is called when it should not be called and causes the heap corruption. In the session the above log exerpt is from, it took about 2.5 million deallocations to happen.
The bug is relatively reproducible when you do a lot of small memory allocations of ramdom size - or during some regular mesh processing in a very complex scene (which is usually where it happens in our case).
Our workaround:
We are currently working around this in a similar way as the C4D API keeps track of AlienMem blocks. We allocate one extra VLONG for every allocation (the actual allocation is passed on to C4DOS.Ge->Alloc(), etc.) and set this extra preceding VLONG to 0, to make sure it can never be STDLIB_MEM_MAGIC by chance.
Here is the code:
void* ourC4DOSAlloc(size_t s, int line,const CHAR *file)
{
// Alloc an extra VLONG with the mem
size_t use_size = s + sizeof(VLONG);
VLONG* mem = (VLONG* )C4DOS.Ge->Alloc(use_size, line, file);
// Write a zero to the start of the allocated mem. The important thing is that the value we write is != STDLIB_MEM_MAGIC
mem[0] = 0;
// Return the address of element one as the actual mem
return mem + 1;
}
void* ourC4DOSAllocNC(size_t s, int line,const CHAR *file)
{
// Alloc an extra VLONG with the mem
size_t use_size = s + sizeof(VLONG);
VLONG* mem = (VLONG* )C4DOS.Ge->AllocNC(use_size, line, file);
// Write a zero to the start of the allocated mem. The important thing is that the value we write is != STDLIB_MEM_MAGIC
mem[0] = 0;
// Return the address of element one as the actual mem
return mem + 1;
}
void ourC4DOSFree(void *p)
{
VLONG* actual_mem_start = (VLONG* )p - 1;
C4DOS.Ge->Free(actual_mem_start);
}
All calls to C4DOS.Ge->Alloc(), C4DOS.Ge->AllocNC() and C4DOS.Ge->Free in c4d_memory.cpp are replaced with calls to our wrappers.
This fixes the problem for us.
Cheers!
Yves