SOLVED CAD Normal Tag flipped after Polyon merge (join) and axis repositioning - how realign?

Dear Developers,

so after some great help from Zipit as allways I got my combine script working.

But the Matrix Manipulation does not repect the "Cached" NormalTag from my CAD import and flips it somehow. I Cannot "correct" the Normal Tag afterwards in Cinema because its limited functionality (read only?). Only Deleting it makes the Polygon object look normal but that is not an option for me - I need the sensible data (curvature shading) of the Normal Tag

I fear its exactly this problem but this is defenitly to much for my coding experience 😞
Is this NormalTag manipulation the right direction or can I do it simpler ?

Here is a screenshot to show you whats the problem. (red is wrong light from above still top is shaded dark).

2020-11-11 14_13_44-Window.png

kind regards
mogh

Hi @mogh,

sorry for taking my sweet time here, but I had to untangle some NormalTag stuff first. Your normals are not flipped, but since you do transform your geometry, you also have to transform the normals by the same transform. Below you will find a script which does that and is based on your script. I have also posted a version which avoids this whole slightly overcomplicated setup here.

Cheers,
Ferdinand

"""Demonstrates how to manipulate NormalTags.

Your normals are not flipped, but actually have to be transformed by the same
transform we did apply to the vertices (i.e. geometry). Because normals are
related to the local point/vertex space of a PolygonObject and when we shuffle
there things around, the normals won't be *right* anymore.

My code is all in the function adjust_point_node() and will handle both the 
vertex as well as normal transforms. Which can be quite computationally heavy 
due to all the shoving around of data. The advantage here is that this code 
will have nicer axis.
"""

# Version 1.4 Striped version
# This Script needs Null-Objects Called "Axis" with polygon objects to run

import c4d
import array


def GetNextObject(op):
    if not op:
        return
    if op.GetDown():
        return op.GetDown()
    while op.GetUp() and not op.GetNext():
        op = op.GetUp()
    return op.GetNext()


def get_all_objects(op):
    allachsen_list = list()
    all_objects_list = list()
    while op:
        if op.GetName() == 'Achsen-Objekt' or op.GetName() == 'Axis':
            allachsen_list.append(op)
        all_objects_list.append(op)
        op = GetNextObject(op)
    return all_objects_list, allachsen_list


def JoinCommand(doc, op):
    res = c4d.utils.SendModelingCommand(command=c4d.MCOMMAND_JOIN,
                                        list=[op],
                                        mode=c4d.MODELINGCOMMANDMODE_ALL,
                                        doc=doc)

    # Checks if the command didn't failed
    if res is False:
        raise TypeError("return value of Join command is not valid")
    elif res is True:
        print("Command successful. But no object.")
    elif isinstance(res, list):
        if c4d.GetC4DVersion() < 21000:
            res[0].SetAbsPos(c4d.Vector())
        op.Remove()
        # Returns the first item containing the object of the list.
        return res[0]


def adjust_point_node(node, transform):
    """Corrects points and normals on a point object after running the join
    command in the scenario specific to PC13004.

    Args:
        node (c4d.PointObject): The PointObject to transform.
        transform (c4d.Matrix): The transform to apply.

    Raises:
        TypeError: When either `node` or `transform` are not of expected type.
    """
    def read_normal_tag(tag):
        """`Reads a `c4d.NormalTag` to a list of c4d.Vector.

        Args:
            tag (c4d.NormalTag): The tag to read the normals from.

        Returns:
            list[c4d.Vector]: The red in normals. There are polygon_count * 4 
             normals, i.e. each vertex has a normal for each polygon it is
             attached to. The normals are stored in polygon groups.

        Raises:
            RuntimeError: When the memory of the tag cannot be red.
            TypeError: When `tag` is not a `c4d.NormalTag`.
        """
        if not (isinstance(tag, c4d.BaseTag) and tag.CheckType(c4d.Tnormal)):
            msg = f"Expected normal tag, received: {tag}."
            raise TypeError(tag)

        buffer = tag.GetLowlevelDataAddressR()
        if buffer is None:
            msg = "Failed to retrieve memory buffer for VariableTag."
            raise RuntimeError(msg)

        data = array.array('h')
        data.frombytes(buffer)
        # Convert the int16 representation of the normals to c4d.Vector.
        return [c4d.Vector(data[i-3] / 32000.0,
                           data[i-2] / 32000.0,
                           data[i-1] / 32000.0)
                for i in range(3, len(data) + 3, 3)]

    def write_normal_tag(tag, normals, do_normalize=True):
        """`Writes a list of c4d.Vector to a `c4d.NormalTag`.

        Args:
            tag (c4d.NormalTag): The tag to write the normals to.
            normals (list[c4d.Vector]): The normals to write.
            do_normalize (bool, optional): If to normalize the normals.
             Default to `True`.

        Note:
            Does not ensure that `normals` is only composed of c4d.Vector.

        Raises:
            IndexError: When `normals` does not match the size of `tag`.
            RuntimeError: When the memory of the tag cannot be red.
            TypeError: When `tag` is not a `c4d.NormalTag`.
        """
        if not (isinstance(tag, c4d.BaseTag) and tag.CheckType(c4d.Tnormal)):
            msg = f"Expected normal tag, received: {tag}."
            raise TypeError(tag)

        buffer = tag.GetLowlevelDataAddressW()
        if buffer is None:
            msg = "Failed to retrieve memory buffer for VariableTag."
            raise RuntimeError(msg)

        if do_normalize:
            normals = [n.GetNormalized() for n in normals]

        # Convert c4d.Vector normals to integer representation.
        raw_normals = [int(component * 32000.0)
                       for n in normals for component in (n.x, n.y, n.z)]

        # Catch input data of invalid length.
        count = tag.GetDataCount()
        if count * 12 != len(raw_normals):
            msg = (f"Invalid data size. Expected length of {count}. "
                   f"Received: {len(raw_normals)}")
            raise IndexError(msg)

        # Write the data back.
        data = array.array('h')
        data.fromlist(raw_normals)
        data = data.tobytes()
        buffer[:len(data)] = data

    # --- Start of outer function --------------------------------------------
    if (not isinstance(node, c4d.PointObject) or
            not isinstance(transform, c4d.Matrix)):
        msg = f"Illegal argument types: {type(node)}{type(transform)}"
        raise TypeError(msg)

    # Transform the points of the node.
    points = [p * ~transform for p in node.GetAllPoints()]
    node.SetAllPoints(points)
    node.Message(c4d.MSG_UPDATE)
    node.SetMg(transform)
    # Transform its baked normals.
    tag = node.GetTag(c4d.Tnormal)
    if isinstance(tag, c4d.NormalTag):
        # We just have to apply the same transform here (without the 
        # translation).
        normals = [~transform.MulV(n) for n in read_normal_tag(tag)]
        write_normal_tag(tag, normals)
    else:
        msg = f"Could not find normal data to adjust for: {node}."
        print(f"Warning - {msg}")


def joinmanagment(n):
    # "Axis" null will be not alive in a few steps get everything we need
    # from it
    if n.GetUp():
        parent = n.GetUp()
    else:
        print("No Parent To Axis Null. Probably not save to run this script.")
        c4d.StatusClear()
        c4d.StatusSetText(
            'No Parent found! - Probably mo CAD import Doc. Script Stopped.')
        exit()
        return False

    parentmg = n.GetMg()
    newobject = JoinCommand(doc, n)  # combine the poly objects

    if not newobject.IsAlive():
        raise TypeError("Object is not alive.")
        return False

    newobject.SetName(str(parent.GetName()))
    newobject.InsertUnder(parent)
    newobject.SetMg(newobject.GetMl())
    adjust_point_node(newobject, parentmg)


def main():
    c4d.CallCommand(13957)  # Konsole löschen
    op = doc.GetFirstObject()

    c4d.StatusSetSpin()
    all_objects, allachsen = get_all_objects(op)  # get two lists
    null_counter = len(allachsen)

    if null_counter == 0:  # check if found something to do.
        c4d.StatusClear()
        c4d.StatusSetText('No Axis Objects found, nothing to do here.')
        print("No Axis Objects found, nothing to do here.")
        exit()

    counter = len(all_objects)
    secondcounter = 0
    c4d.StatusSetText('%s Objects are processed.' % (null_counter))

    for n in allachsen:
        secondcounter += 1
        c4d.StatusSetBar(100*secondcounter/counter)  # statusbar
        if joinmanagment(n) == False:
            break

    c4d.StatusClear()
    c4d.EventAdd()  # update cinema 4d
    print('END OF SCRIPT')
    return True


if __name__ == '__main__':
    main()

Hi,

we'll have to take a look at it tomorrow. But I think I already have a pretty good idea on how to do it. However, it would be beneficial to have an example of the actual data you are working on. By that I mean a scene on which we can run your previous script, which also contains said normal tags. If you are not at liberty to share your data publicly, you can also send an email to sdk_support(at)maxon.net. We will treat your data confidentially.

Cheers,
Ferdinand

@zipit I did send an email with some CAD test data.

Thank you for all your help.
mogh

Hello @mogh,

just as a heads up, this request has not been forgotten, I have been working on it. There are just more hoops to jump through, than I originally did anticipate. You will get your answer here soon.

Cheers,
Ferdinand

Hi @mogh,

sorry for taking my sweet time here, but I had to untangle some NormalTag stuff first. Your normals are not flipped, but since you do transform your geometry, you also have to transform the normals by the same transform. Below you will find a script which does that and is based on your script. I have also posted a version which avoids this whole slightly overcomplicated setup here.

Cheers,
Ferdinand

"""Demonstrates how to manipulate NormalTags.

Your normals are not flipped, but actually have to be transformed by the same
transform we did apply to the vertices (i.e. geometry). Because normals are
related to the local point/vertex space of a PolygonObject and when we shuffle
there things around, the normals won't be *right* anymore.

My code is all in the function adjust_point_node() and will handle both the 
vertex as well as normal transforms. Which can be quite computationally heavy 
due to all the shoving around of data. The advantage here is that this code 
will have nicer axis.
"""

# Version 1.4 Striped version
# This Script needs Null-Objects Called "Axis" with polygon objects to run

import c4d
import array


def GetNextObject(op):
    if not op:
        return
    if op.GetDown():
        return op.GetDown()
    while op.GetUp() and not op.GetNext():
        op = op.GetUp()
    return op.GetNext()


def get_all_objects(op):
    allachsen_list = list()
    all_objects_list = list()
    while op:
        if op.GetName() == 'Achsen-Objekt' or op.GetName() == 'Axis':
            allachsen_list.append(op)
        all_objects_list.append(op)
        op = GetNextObject(op)
    return all_objects_list, allachsen_list


def JoinCommand(doc, op):
    res = c4d.utils.SendModelingCommand(command=c4d.MCOMMAND_JOIN,
                                        list=[op],
                                        mode=c4d.MODELINGCOMMANDMODE_ALL,
                                        doc=doc)

    # Checks if the command didn't failed
    if res is False:
        raise TypeError("return value of Join command is not valid")
    elif res is True:
        print("Command successful. But no object.")
    elif isinstance(res, list):
        if c4d.GetC4DVersion() < 21000:
            res[0].SetAbsPos(c4d.Vector())
        op.Remove()
        # Returns the first item containing the object of the list.
        return res[0]


def adjust_point_node(node, transform):
    """Corrects points and normals on a point object after running the join
    command in the scenario specific to PC13004.

    Args:
        node (c4d.PointObject): The PointObject to transform.
        transform (c4d.Matrix): The transform to apply.

    Raises:
        TypeError: When either `node` or `transform` are not of expected type.
    """
    def read_normal_tag(tag):
        """`Reads a `c4d.NormalTag` to a list of c4d.Vector.

        Args:
            tag (c4d.NormalTag): The tag to read the normals from.

        Returns:
            list[c4d.Vector]: The red in normals. There are polygon_count * 4 
             normals, i.e. each vertex has a normal for each polygon it is
             attached to. The normals are stored in polygon groups.

        Raises:
            RuntimeError: When the memory of the tag cannot be red.
            TypeError: When `tag` is not a `c4d.NormalTag`.
        """
        if not (isinstance(tag, c4d.BaseTag) and tag.CheckType(c4d.Tnormal)):
            msg = f"Expected normal tag, received: {tag}."
            raise TypeError(tag)

        buffer = tag.GetLowlevelDataAddressR()
        if buffer is None:
            msg = "Failed to retrieve memory buffer for VariableTag."
            raise RuntimeError(msg)

        data = array.array('h')
        data.frombytes(buffer)
        # Convert the int16 representation of the normals to c4d.Vector.
        return [c4d.Vector(data[i-3] / 32000.0,
                           data[i-2] / 32000.0,
                           data[i-1] / 32000.0)
                for i in range(3, len(data) + 3, 3)]

    def write_normal_tag(tag, normals, do_normalize=True):
        """`Writes a list of c4d.Vector to a `c4d.NormalTag`.

        Args:
            tag (c4d.NormalTag): The tag to write the normals to.
            normals (list[c4d.Vector]): The normals to write.
            do_normalize (bool, optional): If to normalize the normals.
             Default to `True`.

        Note:
            Does not ensure that `normals` is only composed of c4d.Vector.

        Raises:
            IndexError: When `normals` does not match the size of `tag`.
            RuntimeError: When the memory of the tag cannot be red.
            TypeError: When `tag` is not a `c4d.NormalTag`.
        """
        if not (isinstance(tag, c4d.BaseTag) and tag.CheckType(c4d.Tnormal)):
            msg = f"Expected normal tag, received: {tag}."
            raise TypeError(tag)

        buffer = tag.GetLowlevelDataAddressW()
        if buffer is None:
            msg = "Failed to retrieve memory buffer for VariableTag."
            raise RuntimeError(msg)

        if do_normalize:
            normals = [n.GetNormalized() for n in normals]

        # Convert c4d.Vector normals to integer representation.
        raw_normals = [int(component * 32000.0)
                       for n in normals for component in (n.x, n.y, n.z)]

        # Catch input data of invalid length.
        count = tag.GetDataCount()
        if count * 12 != len(raw_normals):
            msg = (f"Invalid data size. Expected length of {count}. "
                   f"Received: {len(raw_normals)}")
            raise IndexError(msg)

        # Write the data back.
        data = array.array('h')
        data.fromlist(raw_normals)
        data = data.tobytes()
        buffer[:len(data)] = data

    # --- Start of outer function --------------------------------------------
    if (not isinstance(node, c4d.PointObject) or
            not isinstance(transform, c4d.Matrix)):
        msg = f"Illegal argument types: {type(node)}{type(transform)}"
        raise TypeError(msg)

    # Transform the points of the node.
    points = [p * ~transform for p in node.GetAllPoints()]
    node.SetAllPoints(points)
    node.Message(c4d.MSG_UPDATE)
    node.SetMg(transform)
    # Transform its baked normals.
    tag = node.GetTag(c4d.Tnormal)
    if isinstance(tag, c4d.NormalTag):
        # We just have to apply the same transform here (without the 
        # translation).
        normals = [~transform.MulV(n) for n in read_normal_tag(tag)]
        write_normal_tag(tag, normals)
    else:
        msg = f"Could not find normal data to adjust for: {node}."
        print(f"Warning - {msg}")


def joinmanagment(n):
    # "Axis" null will be not alive in a few steps get everything we need
    # from it
    if n.GetUp():
        parent = n.GetUp()
    else:
        print("No Parent To Axis Null. Probably not save to run this script.")
        c4d.StatusClear()
        c4d.StatusSetText(
            'No Parent found! - Probably mo CAD import Doc. Script Stopped.')
        exit()
        return False

    parentmg = n.GetMg()
    newobject = JoinCommand(doc, n)  # combine the poly objects

    if not newobject.IsAlive():
        raise TypeError("Object is not alive.")
        return False

    newobject.SetName(str(parent.GetName()))
    newobject.InsertUnder(parent)
    newobject.SetMg(newobject.GetMl())
    adjust_point_node(newobject, parentmg)


def main():
    c4d.CallCommand(13957)  # Konsole löschen
    op = doc.GetFirstObject()

    c4d.StatusSetSpin()
    all_objects, allachsen = get_all_objects(op)  # get two lists
    null_counter = len(allachsen)

    if null_counter == 0:  # check if found something to do.
        c4d.StatusClear()
        c4d.StatusSetText('No Axis Objects found, nothing to do here.')
        print("No Axis Objects found, nothing to do here.")
        exit()

    counter = len(all_objects)
    secondcounter = 0
    c4d.StatusSetText('%s Objects are processed.' % (null_counter))

    for n in allachsen:
        secondcounter += 1
        c4d.StatusSetBar(100*secondcounter/counter)  # statusbar
        if joinmanagment(n) == False:
            break

    c4d.StatusClear()
    c4d.EventAdd()  # update cinema 4d
    print('END OF SCRIPT')
    return True


if __name__ == '__main__':
    main()

thank you zipit,

as I am totally fine with a less heavy version (from the other thread) I would mark this as solved.

I still want to Thank you for all the other people which might need the actuall NormalTag manipulation in the future.

kind regards
mogh