The light-field of a certain scene is the set of all light rays that travel through the volume of that scene at a specific point of time. A light-field camera for example captures and stores a subset of the lightfield of a scene.

I've got an unstructured subsampling of such a scene (a few billion rays, each having a direction and light intensity information).

What I wish to do now is to create an approximation of the original scene that created this lightfield, with the aproximation consisting of 3 arbitrarily positioned (alpha-)textured 2D planes (in 3D space) where each point on the surface radiates light otwards uniformly in all directions based on the pixel color at this position.

So I guess this is like finding regions in the volume where similarly 'colored' rays intersect, such that the planes maximize the number of intersections they can cover.

So, the available data is the few billions of rays, the desired output is the parameters(position, normal and size) of the three planes plus one RGBA texture for each.

I'm asking here about experiences and opinions: Is this problem rather well-suited for a machine learning approach or rather not?


A classical algorithm I could think of to solve this would be to voxelize the volume and use pathtracing to add a color sample for each ray to all cells along its way, then give each cell some value based on how similar all its contained samples are and then search for planar surfaces that intersect as many high rated cells as possible.

But maybe machine learning is better suited for such a problem?

  • $\begingroup$ Do you have more data than just direction and intensity? To define the trajectory inside a volume you also need some reference location that the ray is projecting to or from - e.g. its intersection point with the scene/volume. $\endgroup$ – Neil Slater Jan 7 '20 at 12:00
  • 1
    $\begingroup$ Oh, of course! I do have the point of impact of each ray (NOT the point of origin! think as if the whole scene was an image sensor. So, I know from what direction the lightrays come from and where they impact as well as the color/intensity of the source, but not how far away this source is along the ray $\endgroup$ – matthias_buehlmann Jan 7 '20 at 13:36
  • $\begingroup$ also some of the lightrays originate outside of the volume for which I have impact points $\endgroup$ – matthias_buehlmann Jan 7 '20 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.