Actually, I don't see any big issue in your code and our role is not to develop for you.
The only suggestion I can tell is instead of calling
SetPoint for each point, maybe it's worth calling directly SetAllPoints.
Moreover, you could also do a proper profiling by splitting your ModifyObject into separate functions in order to know which part is taking longer and after it's up to you to rework your algorithm. Niklas wrote a great article about that.
And finally switching to C++ will definitely give you performance improvement. Even with a similar code base.