Does Python Always Handle Memory Correctly?



  • On 29/05/2013 at 20:34, xxxxxxxx wrote:

    Hi,
    I have a question about how Python handles memory in our plugins.

    I have two plugins that do the same task:
    PoseLibrary     - Written In C++
    PyPoseLibrary  -Written in Python

    These plugins connect a GeDialog with a special tag plugin that stores images in it. Using the Read(), Write() & CopyTo() methods for storing them in the document's hyperfile.

    Both versions work. As far as the user would know.
    But the C++ has memory leaks. And the Read, Write() & CopyTo() system in C++ hates it when I do an Insert(new_image) to update one of the BitmapButtons images array with a different one. And throws a debugging error in V.S.
    I'm having an absolutely horrendous time getting the C++ version to work without leaking memory and throwing other debugging errors.
    And like I said. As a user just using the plugin normally. I would never even know that these errors were happening in the C++ version if I wasn't using it in debugging mode.

    So I'm feeling very paranoid about my Python version. And I'm wondering if it too is leaking memory.
    When we build plugins in Python. Can I really trust it 100% to handle the memory for me correctly. And in all circumstances?

    -ScottA



  • On 29/05/2013 at 23:08, xxxxxxxx wrote:

    Memory leaks can always be prevented. What does the tracer say? For sure, you have forgotten to free your allocated memory at some point.



  • On 30/05/2013 at 01:49, xxxxxxxx wrote:

    You simply don't care if there are memory leaks or not in Python, because if there are, it is not your fault
    (except in cases where it could be your fault, like messing with ctypes and dlls).

    It is however always possible to avoid memory leaks in C++, I have always managed to figure them out
    and no plugin of mine has (as far as I can tell from testing) having memory leaks.

    -Nik



  • On 30/05/2013 at 08:29, xxxxxxxx wrote:

    But I do care if I have memory leaks in my python plugins.
    I care a lot. Even if it's not my fault.
    I need to know if I can rely 100% on Python to handle this correctly for me when I use things that are in the SDK.

    @Frank,
    In my C++ version. I'm getting memory leaks from the tag plugin.
    If I add a BaseBitmap::Free(the specific bitmap here) at the end of the Copy(), and Read() methods. I can fix those leaks. But then it makes the plugin not work correctly.
    Specifically. It doesn't save the images to the saved document's hyperfile. So when the .c4d file is opened again. Those images are gone when the GedDialog tries to use them. And I also can't select things with the mouse because it messes up the CopyTo() method.

    The other error I get is when I do an insert(render,i) to the array of images in the tag.
    I do this remotely from the GeDialog.
    The reason for this code is to update an existing Bitmap Button's image by changing a specific image in the tag's image array.
    I do this by deleting a specific image in the array(Remove(i)). And inserting a new one in the same array index position(Insert(render,i)). So my GeDailog Buttons always stays in sync with the array of images in the tag.
    In simpler words. I find the specific image in the array I want to change. Then I delete it and replace it with a different one. Using the images array index position to do this.
    But those three methods Read(), Write(), and CopyTo() don't like it when I do this "Insert()" thing.
     
    The VS output error is:

    First-chance exception at 0x0000000140d2ea21 in CINEMA 4D 64 Bit.exe: 0xC0000005: Access violation reading location 0xffffffffffffffff.  
    Unhandled exception at 0x0000000140d2ea21 in CINEMA 4D 64 Bit.exe: 0xC0000005: Access violation reading location 0xffffffffffffffff.
    

    And VS also opens a page pointing to this:

    void BaseBitmap::Free(BaseBitmap *&bm)  
    {  
      C4DOS.Bm->Free(bm);  
      bm=NULL;  
    }
    

    I think what's happening is that when I replace an image in the array by using Remove() then Insert(). The Read(),Write, CopyTo() methods are not updating properly to see that change.

    If interested. The plugins can be downloaded from my site. With all of the source code: https://sites.google.com/site/scottayersmedia/plugins

    The thing that's making my memory so hard to manage it those frack'n three methods.
    Read(), Write, & CopyTo(). I hate them.😠

    The reason I made a Python version was because I couldn't fix the debugging errors in the C++ version. But now I'm worried about my Python version too.
    I just want to make 100% sure that the Python is truly working properly. And not leaking memory too.

    -ScottA



  • On 30/05/2013 at 10:00, xxxxxxxx wrote:

    with __del__ you can kind of create memory leaks in python. read the gc module in the 
    python docs. there are also quite some tutorials about the python garbage collector.



  • On 30/05/2013 at 10:41, xxxxxxxx wrote:

    Originally posted by xxxxxxxx

    But I do care if I have memory leaks in my python plugins.
    I care a lot. Even if it's not my fault.
    I need to know if I can rely 100% on Python to handle this correctly for me when I use things that are in the SDK.

    Nobody would be using a language that does not allow to manage memory on your own but
    leaks it. Python itself is pretty much memory leak free and if memory leaks appear in you Cinema
    plugin, then it is most likely because of a bug in the Python SDK. And those memory leaks are very
    rare and can be fixed with a service upgrade without you having to change your code.

    -Niklas



  • On 30/05/2013 at 11:16, xxxxxxxx wrote:

    I normally take it for granted that Python just handles the memory properly.
    But since I have two plugin using the same methods Read(), Write(), CopyTo(). And getting horrendous fits from the C++ version in the debugger. I'm now a bit concerned about it.

    Like I said. You would never know there was problems in the C++ version if you didn't use a debugger on  it.
    And since we don't debug our Python plugins. I'm feeling a bit concerned that I'm creating potentially harmful plugins that leak memory.

    -ScottA



  • On 30/05/2013 at 11:39, xxxxxxxx wrote:

    You see memory leaks in the console. If there's a memory leak in your Python plugin, you'll see it in the
    console just like a memory leak from a C++ plugin.

    -Nik



  • On 30/05/2013 at 12:15, xxxxxxxx wrote:

    Oh?
    I've never seen that before. Do you happen to know when it will show up?

    What I mean by that is in C++ the leaks don't show up in the console until you close C4D. They don't show up when the program is running.
    And if there is a leak. Instead of the debugger automatically stopping. The console stays open until you manually stop the debugger.
    Do you know if it works the same way in Python?

    -ScottA



  • On 30/05/2013 at 14:17, xxxxxxxx wrote:

    The debugger doesn't stop when a memory leak appears, it stops when you do something bad like
    accessing an unallocated memory address or memory that you do not have access to. Memory leaks are
    only memory leaks at the point the program ends.
    Any memory that is not freed at the point the application has teared down is considered a memory leak
    and no-one but the programmer (should) know(s) when a memory block *will become* a memory leak at some point
    unless you're doing proper reference counting like Python does.

    -Nik



  • On 30/05/2013 at 16:46, xxxxxxxx wrote:

    Is there a way I can create a Python memory leak test script, or plugin, to see this console behavior?

    No offense. But I need to see it to believe it. 🙂

    -ScottA

    Edit- I also just remembered that I cannot make Python go into debug mode using the c4d_py_debug.txt file. Which might be why I'm not seeing any messages about memory leaks in my python version.

    No matter where I put that text file. Nothing happens in the console that tells me it's in debug mode.
    I've tried putting the file in the same folder as the plugin, The User folder, the AppData folder, The Maxon folder in the AppData folder, the python folder in the AppData folder...etc..
    Lets just say. I've put the thing everywhere I can think of. And the console never goes into any sort of debugging mode.



  • On 02/06/2013 at 11:55, xxxxxxxx wrote:

    Can anyone tell me how to use debugging in Python?
    I've put  "c4d_py_debug.txt"  &  "c4d_py_verbose.txt"  files into my user folder. But nothing new is happening in the console.

    What do I do next?

    -ScottA


Log in to reply