Photoshop Lightfield: A dream what photography could be

... The only reason that we cannot do this problem of superconductivity is that we haven’t got enough imagination.”
-Richard Feynman-

 

In 2012 the first Lytro camera shipped to the consumer market place. The image quality was poor, but the functionality it promised made me dizzy. In my mind I imagined the potential of a professional light-field camera and assumed that it, or a similar technology would be the future of imaging. Instead nearly four years later we still have basically the same mirrorless and digital cameras as we did then. Since then the race to hit 100MP is on, maybe this year a consumer non-medium format camera will be the first one to hit the goal. Do not get me wrong, I often daydream about what it would look like to be able to print wall sized prints with the clarity of a contact print. In reality my 24MP is more than enough for anything I am likely to print. Instead of giga-pixel imaging I want to see the next step in evolution. There is such huge potential for light-fieldesque imaging, that I think we are having difficulty imagining what it could do. We are so used to thinking of 3D imaging technologies as gimmicks, it is difficult to see the potential solutions to problems in imaging that we have yet to realize are in front of us. Our lack of imagination is driving us to demand faster horses. 

 

Lacking a professional light-field camera, I have begun to take inventory of what is possible with what I have, a camera, tripod, and computer. With this setup I can relatively easily use techniques such as focus stacking, stereo imaging, panoramic/360 image stitching, and some others that fail to come to mind. With all this I can collect a good amount of information to throw at my computer, so this just leaves me with the problem of what to do with this information. Lacking the equivalent of a Photoshop Light-Field edition I am looking to assemble my own suite of programs to do what I can only imagine.  Here are just a few of the abilities I would like when editing images:

  • Automatic Depth Map construction (would already useful in the current version of Photoshop)
  • Removal of obstructing objects based on their distance from the camera (I imagine being able to shoot through branches, remove them based on their distance from the camera, and have the information of what was behind those branches)
  • Create a virtual field camera, using the depth information with two technically correct focus stacked stereo images. With that information it wouldn't be difficult to for a decent programmer using currently available libraries to create an infinitely adjustable layer of focus in the captured scene. Flat planes of focus would no longer be the only game in town.
  • A basic 3D model of the scene could be able to be created without too much difficulty. Many programs now create point clouds, and complex mesh forms to compose the scene. As a photographer a simpler mesh scene made up of simple flat objects would be all I would want.
  • I am not particularly interested in video, but the idea of slow panning movements through a scene to capture the viewers attention seems to have a lot of unexploited potential. As many of the methods I have seen produce amateurish results, or are very labor intensive.  

 

The limits of the above techniques would be if the subject was still enough to capture all the needed images. The problem is that I have not been able to find any single piece of software that gives me these abilities. I could assemble a whole suite of programs that could do this in small pieces, but nothing I know of currently like a Photoshop Light-Field. Photoshop does do 3D, but as far as I can find modeling a scene from a pair of 2D images is not part of it's functionality. Most of the 3D modeling programs out there generally are for creating scenes, not capturing scenes. So my plan is to first brush up on my meager computer programming skills, and try to write some simple scripts that will allow me to automate the creation of a depth map from 2 images. From what I have read there is some Python libraries that should be able to do this relatively easily using OpenCV. With my skill set, the relatively easy will be very difficult in regards to programming. But, if successful, this alone will give me some interesting possibilities in photoshop using the lens blur module. Next, I want to find a way to create a 3D scene of an image, and do photoshop like edits to it in a program such as Blender. Finally, if all that succeeds, be astonished that one of my crazy ideas worked. 

If anyone has any thoughts or resources it would be greatly appreciated. My searches only found some windows depth map creation programs, and since I work on a Mac I am looking for other solutions. I will let you know what comes of my experiments. 

“I am sure the next step will be the electronic image, and I hope I shall live to see it. I trust that the creative eye will continue to function, whatever technological innovations may develop.” – Ansel Adams