The downsides, which, of course, this press release doesn't mention:
- Greatly, greatly reduced image resolution. Great big dedicated-camera sized lens and image sensor, cellphone-camera sized pictures. 1680×1050, at most. (1.76MP)
- Color aberration. The microlenses have to be small, of course, so they're going to be made of single physical elements, rather than doublets.[1]
- Various amusing aliasing problems. (note the fine horizontal lines on some of the demo shots)
- Low FPS. Each image requires lots of processing, which means the CPU will have to chew on data for a while before you can take another image.
- Proprietary toolchain for the dynamic images. Sure, cameras all have their particular RAW sensor formats, but this is also going to have its own output image format. No looking at thumbnails in file browsers. Photoshop won't have any idea what to do with it. Can't print it, of course.
- - You can just produce a composite image that's sharp all over, but why not use a conventional camera with stopped-down[2] lens, then?
- It's going to be really thrillingly expensive. This is a given, of course, with new camera technology.
Proprietary toolchain for the dynamic images – you could embed the full dynamic information in a JPEG extension and use the JPEG thumbnail as a thumbnail and the JPEG image as your selected representation. Then you could use your nonstandard code to regenerate the JPEG image data from your full data when you wished.
You can just produce a composite image that's sharp all over… – but what fun is that? I'm having a grand time with SynthCam on my iPhone to act like a large lens camera to regain distance dependent focus even though I have a tiny lens.
It's going to be really thrillingly expensive. – It doesn't need to be. Tiny micro lenses on the image sensor might be much cheaper than large chunks of precision glass. Think "inkjet-like print head squirting one of the resins used for plastic eyeglasses into etched depressions relying on surface tension to form the lens". Just guessing there. Maybe placing precision sized beads in each depression and then heating to reflow into a surface tension defined lens would work better.
You can just produce a composite image that's sharp
all over… – but what fun is that?
With DSLRs that have huge lenses with large aperture sizes, the depth of field varies a lot and those lenses also have a sweetspot in focal-length / aperture for which images produced are the sharpest. Lowering the aperture size increases the depth of field, but then you've got another problem as the shutter speed also has to be adjusted and you end up using a tripod.
Getting a picture that has everything in focus and is tack sharp can be seriously challenging.
To tell you the truth, while these photos with adjustable focus seem cool, focus is not my pain - what I want is to be able to take great photos (the kind that 35mm cameras can do) for reasonable prices, preferably with something that fits in my pocket.
And to expand on the point above - focus is not painful when the camera has enough focus points. I played with a Nikon D3s that has a whooping 51 auto-focus points; let me tell that it's freaking awesome, as it can track your subject as it moves. The problem is that consumer-level DX DSLRs only have like 11 focus points, which is still cool, but point&shoots suck badly in this area, most of them focusing only in the center of the image.
Another problem with this project that I can see - people don't like playing with their images on the computer. When you take 500 photos in a single day, and another 600 photos the next day (like when going on a trip), it's really painful to carefully adjust each image, not to mention that the RAW formats are huge and seriously cuts in the number of photos you can take ... yeah, making adjustments is great, but I prefer making more photos, that's why I shoot in JPG and don't regret it.
Focusing with point and shoots should not be a problem. They all have a huge depth of field due to their small sensors, so you have to try really hard to put something out of focus. Smudged pictures on point and shoots are usually not a result of lack of focus, but something else (e.g. shaky camera).
Even with DSLRs you usually only need one focus point. You can focus at the center and re-frame. It is very simple to do, and is much simpler than choosing a focus point (and much safer than trusting the camera to automatically choose a focus point for you). Multiple focus points can be very useful in certain rare circumstances: when shooting something really fast off of center without being able to pre-focus or when your camera is bolted down on a stand. Even then I cannot possibly imagine one anyone would need 51 points. This is obvious feature creep.
But yeah, I am really not sure who the intended market for this camera is. Focusing is just not a pain point, in my opinion. This camera could be used by artists and professional photographers to play around with the depth of field to get a great artistic shot, without making their subjects wait. But with the micro-lens design, will it have enough image quality for professionals? I guess we will see.
Focusing with the center AF sensor only and recomposing will generally put the plane of focus behind the intended subject (assuming the lens does not have an extraordinary degree of field curvature). The geometry is easy to see if you remember that your focus distance is the distance from the sensor/film plane to the subject, not the distance from the lens to the subject. The problem is more noticeable with a wider lens, a larger aperture setting and a closer subject, but it's always there. That's why Hasselblad (who only offer one AF point in the H4D camera system) has incorporated positional sensors and focus correction in their latest models.
If your camera offers the option, choose a focusing composition that will put your subject as close as possible to its final position in the picture and use the focus point closest to that position.
You cannot re-frame when shooting moving subjects - kids playing, sports, birds, cars, motorcycles, boats - there's all kinds of instances in which your subject doesn't stand still and even has unpredictable moving patterns, so keeping your subject in the center of the frame and/or re-framing is in many instances not feasible.
Nikon DSLRs have this 3D tracking feature in which you select an object to keep in focus and it refocuses based on its movement inside the frame when it hits the focus points. And when the subject exists the frame and re-enters, auto-focus comes back. 51 focus points may seem like feature creep, but as I said, it's freaking awesome when shooting moving targets like birds.
Even for subjects that are still, like for portraits, you have a lot more freedom for composition as you just select the person's eyes and then you can move around while the eyes are kept in focus.
Of course, you can do a good job with a single focus point, but professionals and amateurs need predictable results, because good moments for taking photos are rare and you don't want to screw up because your camera wasn't properly focused.
That's why I can see partly the utility of this technology here, but on the other hand I can see serious problems with it too, the biggest one being that for most people quantity of photos trumps quality. Another problem that I can see is the one I mentioned above; precise and predictable focus is not that much of a problem with modern cameras. And yet another: mega-pixels and quality of optics count a lot. Well, maybe once passed a certain threshold, the there's less ROI from a higher MP, but still, under 6 MP a camera is only usable for publishing on Facebook.
My consumer DLSR has 4 FPS and I don't worry about focus as I just continuously shoot like 20-30 pictures in a row to make sure one of them is good, and usually one of them is.
> under 6MP a camera is only usable for publishing on Facebook.
2560x1600 pixels is 4.1MP. At 1:1 that will completely fill the most monstrous computer monitors one can get for under about $10k. At 300 pixels per linear inch (a very reasonable resolution for photos; about the same as the iPhone 4 "retina" display) it will give you a picture with 10" diagonal.
Unless you're making posters or big prints for photographic competitions or something, even at 4MP raw pixel count is not going to be your problem. It's completely untrue that below 6MP your pictures will be useless for anything beyond Facebook.
(Of course pixel count starts to matter more if, e.g., you're taking pictures of distant birds or distant celebrities with a not-especially-long lens and you need to crop heavily. Most photographers, most of the time, are not doing that.)
Don't forget about pixel quality (it's understandable because the industry has been encouraging it for years).
A 6MP sensor would produce a fantastic 10"-diagonal print if its pixels weren't affected by noise and it was combined with a high quality lens. Unfortunately sensors of that resolution are typically small (=noisy pixels) and placed in point-and-shoot cameras (=cramped, low-quality optics).
Or you could buy a top of the range pro DSLR from 5years ago with this resolution, with fantastic lenses and bullet proof build for the price of a modern entry level camera
I find the higher mega-pixel really helps on cropping. Especially when photographic wildlife like a dragonfly its nice to be able to crop a photo to just focus on that while still maintaining resolution. For me 10MP minimum is for this purpose rather than large prints. 2c this looks like a really cool technology, I would love to see it built into phones.
"people don't like playing with their images on the computer."
...until they have a great shot ruined by improper focus. Or worse, an entire afternoon of great shots at a memorable event ruined by the autofocus switch set to "off". Us geeks may notice in time, but most consumers/users won't.
It's a piece of first-generation consumer electronics, made in small production runs by a new company that's already spent $30 million in R&D. While each camera will not cost much, they will be priced high.
Not necessarily. If the company is looking to recoup its investment soon, sure. But if the company is looking to get mass-market appeal then they'll price it more aggressively. The latter may be a good idea given the declining interest in dedicated cameras - make it cheap enough, while it's still novel, and people will actually buy the thing.
Not necessarily. If you're using a plenoptic camera purely for producing 2d images, you can reduce pixel size further than you would normally. This is because each pixel in the output image is the average of many primary pixels in the sensor. The primary pixels can be smaller and noisier while still getting a smooth output image. In rough terms the resolution loss is a cosine term over the pixel areas involved, so it's modest.
The technology also allows a tradeoff between resolution and sharpness that is novel, including the possibility of realizing resolutions beyond the diffraction limit.
The cameras used in research projects suffered reduced resolution because they were a frankenstein modification of an off the shelf DSLR. A camera designed to be plenoptic from the beginning has different constraints.
"color aberration and aliasing"
I believe these can be addressed in the processing stage. I don't see anything about either of these issues that's insurmountable.
"Low FPS. Each image requires lots of processing"
The processing can be done at any time after the fact. The primary image capture is just that, a raw image, same as any other camera.
The processing is also relatively simple convolution which can be done via FFT. Overall it's comparable to common video and image compression algorithms, not something new that requires a supercomputer in your camera.
"Proprietary toolchain"
There's no reason we can't standardize raw plenoptic images. Also, once the plenoptic raw has been processed it can be saved as any available 2d or 3d format, from png to psd to jpeg. These files will be fine in Photoshop or on the web.
"You can just produce a composite image that's sharp all over, but why not use a conventional camera with stopped-down[2] lens, then"
This allows depth of field independent of aperture size, which is a new capability. This will greatly aid low light photography, particularly dim landscapes.
But secondly, the images contain depth information, with all that implies. This is a very different tool from a stopped down lens. It's capable of realizing images that are physically imposible with a traditional lens and sensor.
"thrillingly expensive"
Citation needed. AFAIK each piece of technology involved is well understood and readily manufactured.
"The processing can be done at any time after the fact. The primary image capture is just that, a raw image, same as any other camera."
I suppose that's the advantage. I could see myself changing the settings so that you preview the image right after the shot (fashion, portrait) or have no preview at all and keep processing power focused on shooting as many photos as possible (sports, wildlife, candid).
"The picture resolution, he added, was indistinguishable from that of his other point-and-shoots, a Canon and a Nikon. Eliminating any loss of resolution in a camera like Lytro’s, which is capturing light data from many angles, is a real advance, said Shree Nayar, a professor at Columbia University and an expert in computer vision."
There are other techniques for computational cameras that produce "flexible depth of field" -- images with objects both near and far in focus [1]. That technique works a little differently: the image detection plane is shifted during image capture, and then you apply digital signal processing. Very cool stuff.
I realize that this type of discourse is the raison d'être of this site, but I think it's pretty funny that I came to this discussion immediately after reading today's xkcd (http://xkcd.com/915/)
"- Low FPS. Each image requires lots of processing, which means the CPU will have to chew on data for a while before you can take another image."
This will improve and processor speed improves eventually making it a nonissue.
"- Proprietary toolchain for the dynamic images. Sure, cameras all have their particular RAW sensor formats, but this is also going to have its own output image format. No looking at thumbnails in file browsers. Photoshop won't have any idea what to do with it. Can't print it, of course."
Sure, the raw image needs to be processed but you can export the focused product and do what you like, can't you? I can invasion a raw pic and a "best guess on what you'd like to be in focus" jpeg along with it. You can then improve on the jpeg by using the raw image to change what's in focus and recreate the jpeg.
Sure, the raw image needs to be processed but you can export the
focused product and do what you like, can't you?
Certainly, and that will be a cool feature. I don't think it's worth all the other tradeoffs inherent in a light-field camera.
This will improve and processor speed improves eventually making it a nonissue.
We are not talking about the future. We are talking about what this particular first-generation product is likely to do, and I, Samuel Bierwagen, will bet you $20 that this camera won't do better than three seconds per photo.
Good point about referring to first-gen model though I'd be looking forward to it being able to take 1 good picture fast then 3 so so pictures. Either way, I can't wait to see how well it and future versions work in real world environment.
This is the biggest advancement in photography I've ever seen. All radical new technologies have some caveats like this initially like how LCD displays were worse than plasma for a long time. Wait a few generations for the tech to mature and I bet there will be no going back.
- Greatly, greatly reduced image resolution. Great big dedicated-camera sized lens and image sensor, cellphone-camera sized pictures. 1680×1050, at most. (1.76MP)
- Color aberration. The microlenses have to be small, of course, so they're going to be made of single physical elements, rather than doublets.[1]
- Various amusing aliasing problems. (note the fine horizontal lines on some of the demo shots)
- Low FPS. Each image requires lots of processing, which means the CPU will have to chew on data for a while before you can take another image.
- Proprietary toolchain for the dynamic images. Sure, cameras all have their particular RAW sensor formats, but this is also going to have its own output image format. No looking at thumbnails in file browsers. Photoshop won't have any idea what to do with it. Can't print it, of course.
- - You can just produce a composite image that's sharp all over, but why not use a conventional camera with stopped-down[2] lens, then?
- It's going to be really thrillingly expensive. This is a given, of course, with new camera technology.
[1]: http://en.wikipedia.org/wiki/Doublet_(lens) [2]: http://en.wikipedia.org/wiki/F/stop#Effects_on_image_quality