THE POST BELOW IS MORE THAN 5 YEARS OLD. RELATED SUPPORT INFORMATION MIGHT BE OUTDATED OR DEPRECATED
On 06/08/2011 at 21:56, xxxxxxxx wrote:
Well that didn't give me the results I was hoping.
Here's what I am doing.
First I find the center of the polygon.. this is named ctr.
then I develop a rotation matrix of that polygon by doing this.
//Polygon Rotation Matrix
//=====================================//
Vector p = points[a]; //polygon point
Vector n = CalcFaceNormal(points, polys[i]); //Normal
Vector scale = Vector(Len(objPoly->GetMg().v1), Len(objPoly->GetMg().v2), Len(objPoly->GetMg().v3)); //Get Scale
//Contruct Matrix
rm.off = ctr; //The base of the matrix
rm.v1 = !((p - ctr) % n); //X axis points toward the second point
rm.v2 = !(rm.v1 % n); //Y Axis is perpendicular to the X axis
rm.v3 = !(n); //Z Axis is along the normal
//Scale the Matrix
rm.v1 = !(rm.v1 * scale.x);
rm.v2 = !(rm.v2 * scale.y);
rm.v3 = !(rm.v3 * scale.z);
//=====================================//
Then I find the mouse position in world space with...
//Cursor Position in World Coordinates
Vector msWorld = bd->SW(mx, my, ctr);
//Distance from the mouse to the polygon
Vector dis = msWorld - ctr;
so now what I want to do is to alter the rotation matrix that I found using the directional vector calculated by dis. this vector gives me the distance between the polygon center and the mouse cursor in world space. So I know I will need to alter the points of the polygon based on the "dis" and the rotation matrix. I am just not sure how to do this. I tried the steps you mentioned above nux and I got some very weird results. points went all over. 
Any thoughts on how to rotate the polygon so that the normal of the polygon is pointing in the same direction as a line drawn between ctr & msWorld?
Thanks,
Shawn