VP camera clipping with non-perspective projection

On 05/03/2016 at 08:34, xxxxxxxx wrote:

User Information:
Cinema 4D Version:   R14-17 
Platform:   Windows  ; Mac  ;  
Language(s) :     C++  ;

---------
Hello,

My Video Post plugin renders the objects in a scene in a specific way. It uses VolumeData::GetRay()
and VolumeData::TraceGeometryEnhanced(), then use the SurfaceData::p as input for computing
the color of the current pixel.

Everything works fine with "Perspective" camera projection, but any other mode introduces clipping.

Perspective Render

Isometric Render , similar results in all other projection modes

Code Sample

    for (LONG x = 0; x < info.xres; ++x) {
      vd->GetRay(x, y, &ray);
  
      BaseTag* tag = nullptr;
      Bool hit = false;
      RayHitID lhit;
      hit = vd->TraceGeometryEnhanced(&ray, MAXREALr, lhit, 0, RAYBIT_0, nullptr, &si);
      if (!hit || !si.op || !si.op->link) {
        hit = false;
        continue;
      }
  
      // ....
  
      BaseObject* op = si.op->link;
  
      // ....
  
      Vector p = pns::invert_matrix(op->GetMg()) * si.p - op->GetMp();
      Vector col = pns::compute_color(p, pns::equi_rad(op->GetRad()));

Where could the clipping come from? I was hoping this is just me missing some important setting
or so. If it is not so simple, I'll be able to write a full example that reproduces the problem on
another day.

Thanks a lot in advance!
Niklas

On 05/03/2016 at 09:24, xxxxxxxx wrote:

Apparently the problem also doesn't exist in "parallel" projection. Another problem I noticed and which
might be related, is that there are some rare cases (special and exact camera angle to an edge of the
geometry, it seems) where the ray goes straight through the object although it should not. Best example
is a Cube at the world center and the default viewport camera. I have only experienced this with the
perspective mapping yet.

Perspective Camera , notice the black pixels in the center of the image

Parallel Camera

On 07/03/2016 at 09:30, xxxxxxxx wrote:

Hello,

I not sure why this clipping appears. The only thing I found was this pattern to correct the ray position in this case:

  
const RayCamera* rayCamera = vps->vd->GetRayCamera();  
const Int32      camType = rayCamera->type;  
  
//...  
  
vps->vd->GetRay(x, y, &ray);  
  
// Adjust Starting Point  
if ((camType == CAMERA_PARALLEL) || (camType == CAMERA_AXONOMETRIC))  
{  
  ray.p -= ray.v * 1000000.0;  
}  

best wishes,
Sebastian

On 07/03/2016 at 12:45, xxxxxxxx wrote:

Hey Sebastian, thanks that seems to work for now. I kinda doubt that's the right way to solve it, but for
now I'm happy with a workaround!

Do you have any idea where the black pixels in the middle could be coming from? Interestingly with the
workaround from your reply, the artefact strength increases by a lot and it now also appears in parallel
projection mode (I used your method from above for all projection methods in this test).

On 08/03/2016 at 09:35, xxxxxxxx wrote:

Hello,

I cannot reproduce such black pixels. Can you check what triggers these black pixels? Does TraceGemoetry() fail for these pixels or does your "compute_color" function returns black for these intersection points?

Best wishes,
Sebastian

On 08/03/2016 at 19:28, xxxxxxxx wrote:

I don't know what may be wrong, but this line looks dangerous:
Vector p = pns::invert_matrix(op->GetMg()) * si.p - op->GetMp();
I would use si.p with the op bounding box (min,max interpolation) to determine rgb color rather than using matrices.

On 09/03/2016 at 02:08, xxxxxxxx wrote:

That line brings si.p (which is a surface position) into local space and calculates the center delta to be used in the compte color function. Apparently for coordinates (or whatever property) in the equilateral triangle so a correctly interpolated color can be computed. Just guessing here but this wouldn't work with a simple BBox side interpolation.

On 09/03/2016 at 02:26, xxxxxxxx wrote:

Concerning the black line. I would assume this is an issue in compute_color. It appears as if it is dependant on the incident ray angle.
In the two images above, the top one shows a vanishing line the steeper the angle (downwards) while the parallel one, hitting the edge almost straight, is continously solid.

Just an observation (as I don't know what compute_color is actually doing) but that would be the first place I would look for (zero multiplication, out of scope or similar..).

Maybe you can find out what the two passed values are when this happens and in that case add a correctional term to the passed values to make sure this doesn't screw up the color (of course if you have access to the function's source it would even be better to handle this in there directly).

On 09/03/2016 at 03:31, xxxxxxxx wrote:

Hey folks, thanks for chiming in!

@MohamedSakr: Katachi is right, it computes the local position of the surface intersection. The 
compute_color() function just range maps the intersection point in the bounding box into the range
[0..1] and returns that as the color.

  inline Vector compute_color(Vector const& p, Vector const& r)
  {
    return Vector(
      pns::clamp01(r.x > 1.0e-7 ? ((p.x / r.x + 1.0) * 0.5) : 0.0),
      pns::clamp01(r.y > 1.0e-7 ? ((p.y / r.y + 1.0) * 0.5) : 0.0),
      pns::clamp01(r.z > 1.0e-7 ? ((p.z / r.z + 1.0) * 0.5) : 0.0));
  }

@Katachi: I wish you were right, but it appears the problem really is TraceGeometry(). To verify this, I
set the color to plain red when the function returns false (ie. it didn't hit anything).

        if (!hit || !si.op || !si.op->link) {
          hit = false;
          color_line[x * cpp + 0] = 255;
          color_line[x * cpp + 1] = 0;
          color_line[x * cpp + 2] = 0;
          continue;
        }

And the result is this:

If the problem were in compute_color(), the line would still be black. To clarify, if I move the camera
a tiny little bit, the line disappears (second image).

Thanks for helping,
Niklas

On 09/03/2016 at 03:52, xxxxxxxx wrote:

Ah I see, of course if it misses the hitpoint in the first place then it's indeed the TraceGeometry call itself. And I just saw that you even wrote that before! Sorry, me again blinded by the light. :)

Hmm, okay, at least your code looks fine (not sure if setting the RAYBIT_CUSTOM flag will help but I never understood what this flag tells C4D at all so this is wild speculation).

Have you tried using TraceGeometry() instead? Does this show the same problem?

On 09/03/2016 at 04:03, xxxxxxxx wrote:

Btw. what is the value of vd->ray->v (and its other properties) in case of the missed surface hit? Maybe you can perturb the direction slightly to workaround the missed hit in such a case?

On 09/03/2016 at 04:46, xxxxxxxx wrote:

Actually I wrote about TraceGeometry() already, but then the Plugincafe server was unreachable for a few
seconds and my original reply was lost. :angry: So yeah, unfortunately I get the same behaviour with TraceGeometry().
And using RAYBIT_CUSTOM also doesn't change a thing. :/

About vd->ray, I don't know exactly where this value is derived from as it seems to change, but maybe
it's just not properly copied to the new VolumeData (the renderer is multithreaded).  This is an excerpt
for the output of black pixels that should actually have been hit.

No hit at 415, 190
    vd->ray:  Ray{p: (600, 300, -600), v: (-0.206, -0.564, 0.8)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.679, -0.201, 0.706)}
No hit at 420, 191
    vd->ray:  Ray{p: (600, 300, -600), v: (-0.206, -0.564, 0.8)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.675, -0.203, 0.71)}
No hit at 465, 200
    vd->ray:  Ray{p: (600, 0, 0), v: (0, 0, 0)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.632, -0.213, 0.745)}
No hit at 425, 192
    vd->ray:  Ray{p: (600, 300, -600), v: (-0.206, -0.564, 0.8)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.67, -0.204, 0.714)}
No hit at 400, 193
    vd->ray:  Ray{p: (600, 300, -600), v: (-0.206, -0.564, 0.8)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.692, -0.205, 0.692)}
No hit at 515, 210
    vd->ray:  Ray{p: (600, 300, -600), v: (-0.204, -0.563, 0.801)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.582, -0.223, 0.782)}
No hit at 531, 220
    vd->ray:  Ray{p: (600, 300, -600), v: (-0.201, -0.563, 0.802)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.564, -0.234, 0.792)}
No hit at 530, 230
    vd->ray:  Ray{p: (600, 300, -600), v: (-0.21, -0.565, 0.798)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.563, -0.246, 0.789)}
No hit at 400, 201
    vd->ray:  Ray{p: (600, 0, 0), v: (0, 0, 0)}
    GetRay() : Ray{p: (600, 300, -600), v: (-0.691, -0.215, 0.691)}

I tried setting *vd->ray = ray; after using GetRay(), but that didn't change anything except that in the
output you can see that the two rays are actually equal. :')

This is what I used

      if (!hit) {
        if (prev_hit) {
          info.lock.Lock();
          GePrint(pns::fmt("No hit at %s, %s", x, y));
          GePrint(pns::fmt("    vd->ray:  %s", *vd->ray));
          GePrint(pns::fmt("    GetRay() : %s", ray));
          info.lock.Unlock();
        }
        prev_hit = false;
        continue;
      }

Thanks!
Niklas

On 09/03/2016 at 05:08, xxxxxxxx wrote:

Thanks for the list (output is indeed due to threading). Hmm, there's nothing indicating anything "wrong" or which one would assume causing any trouble. Changes in the direction seem to fit the line and column changes of the ray creation.

And sorry, I was a bit unclear: I meant to change the direction of the ray you pass to TraceGeometry(). But it doesn't seem detectable from the values so one cannot simply add a perturbation value.

Well, I am out of ideas for now I'm afraid. :-/ But would also be interested to know the outcome of this and if it is indeed a bug or at least what is causing this.

On 09/03/2016 at 05:45, xxxxxxxx wrote:

Thank you anyway, Katachi. :) Oh and about your idea of sligthly  changing the direction of the ray, which
I forgot to address in my previous reply: Unfortunately I don't think that will be an option as the output
has to be very accurate. And it sounds very hack :D But I can give it a try at least.

On 09/03/2016 at 08:27, xxxxxxxx wrote:

Hello,

I still can't reproduce the issue here. Can you provide some simplified VideoPost code that reproduces the issue (without any third party functions etc.)?

Best wishes,
Sebastian

On 09/03/2016 at 13:02, xxxxxxxx wrote:

Hey Sebastian,

Yes, you can find a minimal example below. This is the result:

/**
 * Copyright (C) 2016  Niklas Rosenstein
 * All rights reserved.
 *
 * Demo plugin to reproduce second issue about the black pixels
 * in this PluginCafe Thread:
 *
 * https://plugincafe.maxon.net/topic/9383/12545_vp-camera-clipping-with-nonperspective-projection
 *
 * @file main.cpp
 * @created 2016/03/09
 * @lastmodified 2016/03/09
 */
  
#include <c4d.h>
#include <res/c4d_symbols.h>
  
class MyVideoPost : public VideoPostData {
public:
  
  static NodeData* Alloc() { return NewObjClear(MyVideoPost); }
  
  /**
   * Called to allocate additional buffers required by this videopost.
   * We allocate one new buffer for our rendering.
   */
  void AllocateBuffers(BaseVideoPost*, Render* render, BaseDocument* ) override {
    this->buffer_id = render->AllocateBufferFX(
      VPBUFFER_POSTEFFECT, "MyVideoPost", 32, true);
  }
  
  /**
   * Invoke our #DoRender() function after the actual rendering.
   */
  RENDERRESULT Execute(BaseVideoPost*, VideoPostStruct* vps) override {
    switch (vps->vp) {
    case VIDEOPOSTCALL_RENDER:
      if (!vps->open) this->DoRender(vps);
      break;
    }
    return RENDERRESULT_OK;
  }
  
private:
  
  /**
   * Called from #Execute() to render into the buffer allocated in
   * #AllocateBuffers() of which we saved the ID in #buffer_id.
   */
  void DoRender(VideoPostStruct* vps) {
    VPBuffer* buffer = vps->render->GetBuffer(VPBUFFER_POSTEFFECT, this->buffer_id);
    if (!buffer) return;
  
    Int32 const cpp = buffer->GetCpp();
    Int32 const line_width = buffer->GetBw() * cpp;
    AutoGeFree<Float32> line(NewMemClear(Float32, line_width));
    if (!line) return;
  
    Bool hit;
    Ray ray;
    RayHitID lhit;
    SurfaceIntersection si;
  
    for (Int32 y = 0; y < buffer->GetBh(); ++y) {
      ClearMemType<Float32>(line, line_width);
      for (Int32 x = 0; x < buffer->GetBw(); ++x) {
        vps->vd->GetRay(Float(x), Float(y), &ray);
        hit = vps->vd->TraceGeometry(&ray, LIMIT<Float>::Max(), lhit, &si);
        if (!hit) continue;
        line[x * cpp + 0] = 1.0;
        line[x * cpp + 1] = 1.0;
        line[x * cpp + 2] = 1.0;
      }
      buffer->SetLine(0, y, buffer->GetBw(), line, 32, true);
    }
  }
  
  /**
   * ID of the buffer that we allocate in #AllocateBuffers().
   */
  Int32 buffer_id = 0;
  
};
  
  
Bool PluginStart() {
  return RegisterVideoPostPlugin(
    234324 /* TEST ID! */, "MyVideoPost", 0, MyVideoPost::Alloc, "", 0, 0);
}
  
  
void PluginEnd() {
}
  
  
Bool PluginMessage(Int32 msg, void* pdata) {
  return true;
}

Thanks for looking into it.

Niklas

On 10/03/2016 at 09:22, xxxxxxxx wrote:

Hello,

with your code I can now reproduce the issue. But if I add the above snippet to adjust the start point the faulty pixels disappear. Can you confirm that?

  
// Adjust Starting Point  
if ((camType == CAMERA_PARALLEL) || (camType == CAMERA_AXONOMETRIC))  
{  
 ray.p    -= ray.v * 1000000.0;  
}  
  
hit = vps->vd->TraceGeometry(&ray, LIMIT<Float>::Max(), lhit, &si);  

Best wishes,
Sebastian

On 10/03/2016 at 10:26, xxxxxxxx wrote:

Hello,

sorry I can not confirm that the faulty pixels disappear. The effect becomes stronger as described above.
I added the line directly without a check of the camera type beforehand:

        vps->vd->GetRay(Float(x), Float(y), &ray);
        ray.p -= ray.v * 100000.0;

Using your code instead doesn't seem to change the output, which makes sense since the perspective
projection is neither a parallel nor an axonometric projection, thus the ray position is not modified.

        Int32 camType = vps->vd->GetRayCamera()->type;
        if ((camType == CAMERA_PARALLEL) || (camType == CAMERA_AXONOMETRIC))
        {
            ray.p    -= ray.v * 1000000.0;
        }

Best,
Niklas

On 11/03/2016 at 05:39, xxxxxxxx wrote:

Hello,

what kind of system do you use to build and test the code?

Best wishes,
Sebastian

On 11/03/2016 at 06:35, xxxxxxxx wrote:

Hi Sebastian,

I use my own build system. But I have just pasted the code into a 100% fresh copy of the cinema4dsdk
Visual Studio project of Cinema 4D R16.050 and compiled it with VS2013 x64 (18.00.40418) and still
get the exact same result.

Regards,
Niklas