Symbol stripping on OS X

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 31/08/2012 at 06:06, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   r14 
Platform:   Windows  ; Mac  ;  Mac OSX  ; 
Language(s) :     C++  ;

---------

Edit: Post updated to contain the latest findings

Symbol stripping on OS X

Hi folks,

we recently discovered a problem on OS X in a 3rd party plugin that could create a crash at startup in r14, r13, ...

The crash is quite hard to track down, as it depends on timing, memory content and is more likely to happen on Apple's latest OS X version (2/3rd of the customers running into this problem are using OS X 10.8.1).

This is what happens:
- The OS allocates memory (e.g. for a window) via new. This is is handled by the OS default implementation.
- A plugin exports symbols for new & delete. As soon as it is loaded its new & delete operators replace the OS default implementation.
- The OS is releasing memory (e.g. when closing a window). Instead of calling the OS default implementation of delete that suddenly will call the plugin's delete function.

Depending on the content of the memory block (that should be freed), crashes are possible; it just depends on the content of the memory block if this happens or not.

Why did this happen?

Here 's a selection of what might cause the trouble:

- XCode project & target settings might be conflicting, creating a mess in the build phase.

- Symbol stripping for the release compile might be disabled (check project and target settings!)

- Many of you used the R12 SDK to build plugins running on R12, R13 & R14. When we developed Cinema R12 we used XCode 3.1 and the project settings were adjusted to that. If you switched to XCode 3.2.6 and used that for build plugins based on the R12 SDK, Apple introduced an unpleasant surprise: The global symbol stripping doesn't work anymore and you don't get any warning or error about it. You have to look deep into the log files to find this remark:

"[…] nmedit: removing global symbols from a final linked no longer supported. […]"

Note: this problem might have been introduced with earlier versions of XCode 3.2.x already; we verified this behaviour in XCode 3.2.6.

How can I verify that symbol stripping worked?

Use this:

nm <path of your dylib> | c++filt -p -i 

command in the terminal to verify that no new/delete symbols are exported.

Solutions

- Use the R13.061 SDK to build plugins for R13 & R14, if you don't use R14 specific functionality (or rely on things that changed with R14).

With the R13 SDK we re-wrote the projects from the ground up for XCode 4 (which in the end we didn't use, as XCode 4.0 wasn't production-ready) and then had to add several workarounds to the project to make XCode 3.2.x work again; luckily those workarounds circumvented the stripping problem Apple introduced in XCode 3.2 too (as we know now).

The R13 SDK can be used with XCode 3.2.6 - this is what we used for building R13 - or XCode 4.x. Note: when using XCode versions > 4.2, you might have to adapt the compiler settings, as Apple removed gcc support.

- Use the R14.025 SDK with XCode 4.2 or newer to build plugins utilizing R14 functionality (or relying on functions that have been changed with R14)

- Use XCode 3.1.x only if you want to create plugins for R12 and with the R12 SDK. Check the compiler/linker log for messages indicating that stripping doesn't work anymore and use "nm" to verify that new/delete isn't exported. Don't use XCode 3.2.x or newer to create plugins with the R12 SDK.

Settings in XCode

There is a plentora of settings in XCode and I'll only list the most important ones here; in doubt (if you're not using the cinema4dsdk project as base for your plugin) please have a look at the R13.061 SDK or R14.025 SDK projects and their settings (we tripple-checked these versions)

For the release build make sure that:
- "Deployment Post Processing" is enabled (otherwise the whole stripping process isn't initiated)
- "Strip Debug Symbols During Copy" is enabled
- "Strip Linked Product" is enabled
- "Strip Style" is set to "Non-Global Symbols"
- "Exported Symbol File" is pointing to the "export.txt" containing the "_c4d_main" symbol in your plugin (path usually is "../../resource/_api_lib/export.txt")

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 08/09/2012 at 17:23, xxxxxxxx wrote:

There has to be more to this.  All of my plugins have symbol stripping enabled and the proper export.txt referenced.  I installed one of my plugins in my R14 that a user says is crashing C4D R14 and it worked without incident.  Could it just be that they need to be recompiled if using a later version of Xcode and hope for the best?

Thanks,

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 09/09/2012 at 00:32, xxxxxxxx wrote:

Originally posted by xxxxxxxx

There has to be more to this.  All of my plugins have symbol stripping enabled and the proper export.txt referenced.  I installed one of my plugins in my R14 that a user says is crashing C4D R14 and it worked without incident.  Could it just be that they need to be recompiled if using a later version of Xcode and hope for the best?Thanks,

Robert, I've downloaded version 1.1 of DropToFloor from your page. Using this nm command (path of course specific to my machine) :

  
nm /Users/myaccount/DropToFloor/DropToFloor.dylib | c++filt -p -i   

to the downloaded dylib reports 1360 demangled C++ symbols. Starting with:

  
0000000000022ee6 s stub helpers   
0000000000001570 t global constructors keyed to _Z19RegisterDescriptioniRK6StringP13LocalResource   
00000000000015d0 t global constructors keyed to _ZNK9DescLeveleqERKS_   
000000000000dae0 t FindPlugin   
0000000000016180 t GeDebugOut   
0000000000016100 t GeDebugOut   
0000000000015140 t GeOpenHTML   
0000000000016130 t GeUpdateUI   
0000000000015b40 t GetTagName   
0000000000015b90 t GetTagType   
0000000000005330 t IsAlienMem   
00000000000152b0 t ShowBitmap   
0000000000015290 t ShowBitmap   
0000000000015f00 t AddShortcut   
00000000000166e0 t CallCommand   
0000000000010f20 t GeFCopyFile   
00000000000150a0 t GeOutString   
0000000000015250 t GePrintNoCR   
0000000000015010 t GeShowMouse   
0000000000009630 t GenerateUVW   
0000000000015ed0 t GetShortcut   
0000000000008010 t GetToolData   
0000000000002410 t PluginStart   
0000000000015600 t ReadRegInfo   
00000000000152f0 t StatusClear   
0000000000009550 t Triangulate   
0000000000005cc0 t UpdateMenus   

- in between more than 1000 symbols - and ending with:

  
00000000000055d0 t operator delete[]   
0000000000005370 t operator delete[]   
00000000000053b0 t operator delete[]   
00000000000053f0 t operator delete[]   
0000000000005610 t operator delete   
0000000000005350 t operator delete   
0000000000005390 t operator delete   
00000000000053d0 t operator delete   
0000000000005ab0 t operator new[]   
0000000000005a00 t operator new[]   
00000000000057d0 t operator new[]   
0000000000005700 t operator new[]   
0000000000005650 t operator new   
0000000000005520 t operator new   
0000000000005950 t operator new   
0000000000005870 t operator new   
00000000000129d0 t operator+   
                 U ___cxa_atexit   
                 U ___cxa_pure_virtual   
                 U ___stack_chk_fail   
                 U ___stack_chk_guard   
0000000000022e50 t ___tcf_0   
0000000000000000 t __mh_dylib_header   
0000000000007480 T _c4d_main   
0000000000034640 s _fallbacklevel   
                 U _free   
0000000000034600 d _lib_brushbase   
00000000000345f8 d _lib_ca   
00000000000345e8 d _lib_ngonbase   
                 U _malloc   
                 U _memcpy   
                 U _memset   
0000000000034638 s _path_storage   
0000000000034620 s _resource   
00000000000345f0 d _t_C4DOS   
                 U _vsnprintf   
                 U dyld_stub_binder   
00000000000015e4 t dyld_stub_binding_helper   

Either there is a bug in the XCode version you're using (can't rule that out, as we just checked it with 4.4 and 3.2.6, as these are the versions we used in r13 & r14), or you haven't enabled symbol stripping.

Best regards,

Wilfried

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 09/09/2012 at 06:22, xxxxxxxx wrote:

Here are some screen shots of my Xcode (3.2.6) configuration for DropToFloor.  Note that the plugin was originally built in an earlier version of Xcode.  But SolidChamfer 2.1 was just built in this version and it has the same configuration (and is purported to crash - which I did not experience).  I am on Mac OS 10.7.4, for the record.

If you see what needs to be changed, please let me know.

http://www.kuroyumes-developmentzone.com/wp-content/uploads/2012/09/ScreenShot2012-09-09_1.jpg

http://www.kuroyumes-developmentzone.com/wp-content/uploads/2012/09/ScreenShot2012-09-09_2.jpg

http://www.kuroyumes-developmentzone.com/wp-content/uploads/2012/09/ScreenShot2012-09-09_3.jpg

http://www.kuroyumes-developmentzone.com/wp-content/uploads/2012/09/ScreenShot2012-09-09_4.jpg

http://www.kuroyumes-developmentzone.com/wp-content/uploads/2012/09/ScreenShot2012-09-09_5.jpg

http://www.kuroyumes-developmentzone.com/wp-content/uploads/2012/09/ScreenShot2012-09-09_6.jpg

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 09/09/2012 at 08:08, xxxxxxxx wrote:

Originally posted by xxxxxxxx

Here are some screen shots of my Xcode (3.2.6) configuration for DropToFloor.  Note that the plugin was originally built in an earlier version of Xcode.  But SolidChamfer 2.1 was just built in this version and it has the same configuration (and is purported to crash - which I did not experience).  I am on Mac OS 10.7.4, for the record.If you see what needs to be changed, please let me know. [...]

The strip style is wrong. If you click on the strip style entry in the Xcode 3.2.6 build settings, it displays these explanations about the strip style (I'll write down the options, as this forum hosts no images) in the info part of the window (there 's a round "point" at the bottom of the build window that you can grab and open the info display, if you haven't yet) :

  
Defines the level of symbol stripping to be performed on the linked product of the build. The default value is defined by the target's product type. [STRIP_STYLE]   
  
All Symbols - Completely strips the binary, removing the symbol table and relocation information. [all, -s]   
Non-Global Symbols - Strips non-global symbols, but saves external symbols. [non-global, -x]   
Debugging Symbols - Strips debugging symbols, but saves local and global symbols. [debugging, -S]   

With your setting "Debugging Symbols" you only strip the debugging symbols, but you leave ALL other symbols - esp. new/delete - in place.

Choose "Non-Global Symbols" and then use the "nm" command (I showed you) to verify that no new/delete gets exported.

The crash is quite hard to reproduce and understand (that 's why we couldn't track in down when we saw it in r13). It really just depends on timing and memory content, if the code crashes or not; e.g. one case I had on my machine: It works (using the same archive) on system A & system B, but when you start the program hosted on system A via network from machine B, it crashes - binaries are id., it's just the different timing and memory content in these cases that made the difference.

Best regards,

Wilfried

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 10/09/2012 at 02:31, xxxxxxxx wrote:

Very good to know that, thanks Wilfried!

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 10/09/2012 at 15:33, xxxxxxxx wrote:

I think I followed your instructions correctly, but the plugin is still hanging in R14. I'm still using Xcode 3 - do I have to update to Xcode 4? Will the plugin still work in R12 if I compile in XCode 4 using the R14 sdk project?

nm outputs this:

  
/Applications/MAXON/CINEMA 4D R12/plugins/PlaneSmart/PlaneSmart.dylib(single module) :   
        U std::nothrow   
        U ___cxa_atexit   
        U ___cxa_pure_virtual   
        U ___stack_chk_fail   
        U ___stack_chk_guard   
0000b260 T _c4d_main   
        U _floor   
        U _fmod   
        U _free   
        U _malloc   
        U _memcpy   
        U _memmove   
        U _memset   
        U _tan   
        U _vsnprintf   
        U dyld_stub_binder   

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 10/09/2012 at 15:57, xxxxxxxx wrote:

Originally posted by xxxxxxxx

I think I followed your instructions correctly, but the plugin is still hanging in R14. I'm still using Xcode 3 - do I have to update to Xcode 4? Will the plugin still work in R12 if I compile in XCode 4 using the R14 sdk project?

nm outputs this:
[...]

That looks pretty fine. Which Xcode 3 version did you use?

Moving to XCode 4 and using the r14 sdk project makes the plugin run on r14 only - XCode 4.4 doesn't support OS X 10.5 (BaseSDK) anymore and it doesn't support PPC chips either.

Best regards,

Wilfried

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 00:02, xxxxxxxx wrote:

I have read some hacky stuff on the internet about copying the 10.5 SDK folder from earlier Xcode versions to the Xcode 4.4 folder. Will that work or is it an unstable/unreliable solution?

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 00:12, xxxxxxxx wrote:

TBH, for me the easiest solution was to install 3.2.6 and 4.4 on the same machine (running Lion) and then use 3.2.6 for all builds except the R14 one, when I use 4.4. It means that for each build I can use the cinema4dsdk project which is correct for each version of Cinema as the basis, ensuring that the correct settings are in place for each one.

It's a bit less convenient since it means I now have to use two development environments on the Mac but it avoids missing important settings for one or other build.

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 00:43, xxxxxxxx wrote:

I have a quite complete Plugin Collection and tested each single Plugin. In Total 372 Plugins.
I tested on a OSX 10.8.1 System with R14.025.
These are Plugins that make Cinema freeze or Crash on Startup:

Free Plugins (out of 238)
NoiseDeformer
SplineNoiseDeformer
UVDeformer
AdvancedAtomArray
ChamferMaker
GearBuilder
Reeper X 1.1
Sidewalk
smartoutline
UberTracer
wornEdges
MultiImporterV0.1b
TrueSymmetry
AlignToVector
AMa_1D_Snap
copypastepolys
PointCollapse
CD_AlignToCam
DropToFloor
DublicateObject
ground
HomePSR
SwitchObject
TransformTool
AIT
CD_HPBView
Oscillator
smarttween
TrainDriver
Random Select Tool
SplitSelection
AltRender_v0.37b
ColourTable
ManualKerning
Override Child Selection
RestartCinema
SceneDocumentor
smartsave
solo 1.2.0
Atmosphere
CamShader
CellularShader
Color Changer
PlaneSmart
RandomMaterial
SmartTexPath1_0
Starfield
ThingsOnSurfaceShader
TimeShiftShader

Komerzielle (out of 59) :

GeoTools2011
Greebler
ToothedWheelV2
Harvester
CD_Symmetry1.0
SolidChamfer2.0
SPLURF
DropIt Pro
SplineSPREAD
SurfaceSPREAD
enDOFin
Storm Tracer
camgriptoolsV2
CD_SpringyKeys1.0
Control4DPRO_v1.1.63
Unfurl
Cinapsis
stacx

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 02:15, xxxxxxxx wrote:

It might be these are all plugins that have been compiled using a project file derived from the R12 cinema4dsdk.xcodeproj.

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 04:15, xxxxxxxx wrote:

just to add:
we fixed our Blackstar plugins, so
SPLURF
enDOFin
Cinapsis
and all others now work fine with R14

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 05:44, xxxxxxxx wrote:

Unfurl is build for R13 using the R13 cinema4dsdk project so it must be the default (either Maxon's project or Xcode's) symbol stripping type.  This entire thing is a bit unnerving since it would rarely show up (now it is epidemic) and one cannot really test for this.  So, I'll have to change the strip type setting, run that command, and hope that the crashes magically vanish (for 15 plugins).  Not my favorite way to spend my weekend.

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 05:48, xxxxxxxx wrote:

Originally posted by xxxxxxxx

I think I followed your instructions correctly, but the plugin is still hanging in R14. I'm still using Xcode 3 - do I have to update to Xcode 4? Will the plugin still work in R12 if I compile in XCode 4 using the R14 sdk project?

[...]

Follow-up FYI. Turns out a few things went wrong (mixed-up project & target settings in Xcode 3). Although your target settings modified the stripping (should have happened in the project settings) and nm did report just a few symbols, the size of your dylib (680 kb) was sus**cious - and there had been a few symbols used that are related to debugging (e.g. the stack_chk_guard symbol).

Dragging the dylib on a hexeditor shows, that the symbols & debug symbols are still there. Might be caused by the mixed-up project/target settings or by XCode 3.2 - we discovered today that removing global symbol from a final linked dylib based on the r12 sdk didn't work anymore in XCode 3.2.6 (while it did with XCode 3.1 that we used for building r12). Xcode doesn't even warn you about that; you've to look into the log output to discover it. More on that later - we're preparing a document about it...

I've set up your plugin project based on the r13 sdk and the size shrinked down to 111 kb, symbols are stripped and the crash on 10.8.1 is gone.

Best regards,

Wilfried

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 07:26, xxxxxxxx wrote:

Howdy,

I noticed that my plugin CD Symmetry Tools is in the list of plugins causing a crash, but the deployment settings in my plugin's project show that I am using the "Non-Global Symbols" setting in the release version:

Since I don't really know much about the specific project settings, I simply copied the cinema4dsdk project, deleted the source, added my own source code and then changed the Product Name to my plugin's name.

Do I need to change something else?

Adios,
Cactus Dan
P.S. My plugins were compiled with the first R14.014 demo on a 32bit machine running Snow Leopard 10.6.8 and Xcode 4.2

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 07:50, xxxxxxxx wrote:

Hi Dan,

This is confusing me, too. Because you've got Strip Style set to non-global symbols for release, so they should be stripped, but under strip linked product for release it says 'no'. Does that mean the symbols are being stripped or not? Beats me, I sure don't know.

And it's worrying because I also used the sdk project as the basis for the compile on R14 (though it is the 14.025 version if that makes any difference).

Steve

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 07:58, xxxxxxxx wrote:

Originally posted by xxxxxxxx

Howdy,
I noticed that my plugin CD Symmetry Tools is in the list of plugins causing a crash, but the deployment settings in my plugin's project show that I am using the "Non-Global Symbols" setting in the release version:

Since I don't really know much about the specific project settings, I simply copied the cinema4dsdk project, deleted the source, added my own source code and then changed the Product Name to my plugin's name.
Do I need to change something else?
Adios,
Cactus Dan
P.S. My plugins were compiled with the first R14.014 demo on a 32bit machine running Snow Leopard 10.6.8 and Xcode 4.2

For the release build make sure stripping is enabled ("strip linked product -> Release: Yes"). Please check project and target settings for that.

Use the "nm <path> | c++filt -p -i" command to verify that new & delete symbols aren't exported.

Best regards,

Wilfried

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 08:02, xxxxxxxx wrote:

Howdy,

OK, I've just discovered something. Since I have several machines and always prefer to install updates on a separate machine first, I have the first R14.014 demo, the release version R14.014 and the updated R14.025 demo available and there are different settings for the "Strip Linked Product" parameter in the cinema4dsdk project:
R14.014 demo:
Debug = Yes
Release = No

R14.014 release:
Debug = No
Release = No

R14.025 demo:
Debug = Yes
Release = Yes

So, maybe the latest settings are correct?

Adios,
Cactus Dan

THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED

On 11/09/2012 at 08:08, xxxxxxxx wrote:

Howdy,

Originally posted by xxxxxxxx

...For the release build make sure stripping is enabled ("strip linked product -> Release: Yes"). Please check project and target settings for that....

AHA! So my assumption was correct that the latest version's settings are the ones to use. Thanks. 

Edit:
Or should the Debug be set to No?

Adios,
Cactus Dan