The Future of 3D at the SMPTE Stereo Conference
July 20, 2010 – 2:53 pm
Presentations at SMPTE’s first-ever conference focused exclusively on stereo 3D, held in New York last week, ran the gamut, dealing with everything from camera rigs for stereo acquisition to the effects of video compression and the challenges of transmitting a finished 3D signal. Last week, I ran down the basic content of presentations by Panasonic, which described its new 3D camcorder, and Sony, which outlined a potential suite of post-production tools that may one day address all the imperfections of run-and-gun stereo acquisition. This week, I’ll take a brief look at a few other papers that provided hints (and wishes) for the future of 3D image capture.
Attending the two-day conference meant getting an awful lot of information to process. Some of it may even have seemed contradictory, or at least counterintuitive. But information is what’s needed right now, and in large quantities. One audience member succinctly summed the current quality-control dilemma facing stereoscopic acquisition in one of the day’s Q&A sessions. “It used to be, as soon as the image came on screen, you knew there was a problem,” the attendee said. “Now, it takes two hours before someone gets a headache.” This conference was all about figuring out how to head off those headaches at the pass.
The conference opened with Alexander Melkumov, director of the Department of 3D Digital Cinema of the Russian Cinema and Photo Research Institute and president of Stereokino, doing his best to change conventional thinking about stereo acquisition. He called the use of two-camera rigs for capturing 3D “archaic,” comparing it to the old three-strip film process for capturing Technicolor movies.
Instead, Melkumov described the Russian “Stereo 70″ system, which he said has been used to shoot more than 30 3D films since 1966. The camera works by exposing both the left-eye and right-eye images on a single piece of 65mm film, yielding two stereo frames that, at 18mm by 26mm, are each close to the size of a standard 35mm frame. Special “optical blocks” were developed to make the system work, and the design was updated last year for use with digital cinema cameras, he said.
Melkumov showed images of a Phantom 65 high-speed camera outfitted with two different kinds of 3D lenses designed to capture left-eye and right-eye with a single sensor, including the just-developed “new-generation” ZEPAR lens from MKBK, the Moscow Cinema Design Bureau. “More than 60 years have passed since Eisenstein saw the first 3D Soviet film, and we are still not ready for it,” he lamented. “We do not have cameras to produce 3D movies that would match the level of traditional movie cameras.”
His final argument was for an increase in digital sensor resolution that would make it possible to capture stereo images side by side, in a fashion similar to the Stereo 70 approach. “We hope that the companies developing digital cameras will design with an 8K sensor, the size that will be equal to the 65mm cinema standard for the production of high-grade and quality 3D movies,” he said.
Bernard Mendiburu, a digital cinema consultant, spoke about the characteristics that will be necessary for tools now in development tools that seek to fix 3D images in real time. Amusingly, he characterized the struggle for quality stereo imagery as “Captain Roundness vs. the Evil Cardboard Puppeteer.” “We need to find a way to have 2D crews shooting perfect 3D,” he said. “In a movie production, what’s expensive is time. You can’t have 100 people on the set waiting for your 3D camera to be fixed.”
The problems facing 3D shooters are staggering — lens imperfections that will never be noticed in a flat image suddenly become deal-breakers when images captured by that glass are paired with those of another lens that has its own singular characteristics. For example, he noted that problems are introduced because 3D cameras are likely to use zoom lenses rather than primes for the simple reason that it’s more difficult to swap out lenses on a mirror rig. “But a zoom lens’s progression is not regular,” he said. “If you go all the way, the optical axis will move one way or another. You fix this by putting your camera on a three-axis control. When one zoom is going up, you push the camera down.” Once you compute all the different corrections your lenses will require, Mendiburu suggested, you can build a look-up table that could be loaded into a motion-control computer that would use it to automatically compensate during the shoot.
“You want to fake the perfect lens for that scene for that camera and for that sensor. For all those mechanical faults in your system, you have a perfect image,” he said. “And you want to do this in real time. 3D TV will love it.” He said companies currently working on the technology include Sony and 3ality, but cautioned that nobody is yet doing it in real time. Different elements of the technology should become available from vendors over the next 10 years, he said.
Compensating for all the unique features of a given piece of glass would require pixel-level image alignment, including warping the picture to correct discrepancies in geometry between the two images, searching the image for tracking points, computing positive and negative parallax and creating warnings for excessive convergence and divergence, vertical disparities, and deviations from the planned depth budget. As an example, Mendiburu showed a viewfinder image that represented depth in the picture through graphic overlays — blue triangles designated areas of the picture that would seem to be inside the viewer’s room, while red triangles show parts of the scene that would appear behind the screen.
One of the flashiest presentations belonged to Blair Barbour, CEO and chief scientist at Illumin-X, who demonstrated “Photon-X” technology for using a single camera with a single lens for “3D volumetric image capture.” The results were striking — Barbour showed how an apparently 2D image taken with a single camera could be effectively rotated in 3D space. A man’s portrait, for instance, suddenly becomes a three-dimensional wire-frame — a mask-like object that can be viewed realistically straight-on, from an angle, or even in profile.
Here’s how Barbour explained the technology: Ordinary CCD or CMOS sensors measure the intensity of light to provide color images. The Photon-X system involves modifications to the sensors that allows them to read the magnitude and phase of light. “Magnitude is color information and phase is shape information,” he said. “Both color and shape are thus acquired simultaneously from a single camera.”
He wasn’t claiming that the system was ready for deployment in the entertainment industry — so far, the cameras are generally being used for biometric security applications — but he was clearly testing the waters. One of the more tantalizing promises he made is that Photon-X can capture video of a human face and then use an “expression lookup table” to calculate which emotions are being expressed based on which muscles are being moved underneath the skin. “If you’re smiling, I can tell you whether that’s a real smile or a fake smile,” Barbour said. “Cameras will become so smart in the future that you’ll get automatic feedback from a marketing perspective.”
A superficially similar yet fundamentally different technology was in the offing from Advanced Scientific Concepts, which showed its own five-year-old technology for a solid-state “Flash LIDAR Camera.” It uses a laser pulse with a width between five and 10 nanoseconds to make a quick 3D scan of an environment. As a real-time demo, VP of Business Development Thomas Laux used the company’s TigerEye camera model to make a low-res 3D version of the SMPTE conference room.
The current camera only has a 128×128 pixel imager, but the company is confident that a 720p version could be created if there’s a market for it. Today, the camera is used in industrial, space, and military applications. Laux said that automotive companies are also interested in using its depth-mapping capabilities to create fully autonomous vehicles. An eventual broadcast application, he said, might combine 3D and 2D data in real time, for example, to create 360-degree views of a football field.
There was lots more on offer at the conference, but some provocative tidbits came out of a presentation by Daniele Siragusano, supervisor and stereographer of digital post-production at CinePostproduction in Munich, who sought to explore what a target screen size for a stereo film might be. Siragusano dropped way too much math and stereo science to do justice to in this blog post, but, in summary, he never quite answered his own question. He said there is a range of appropriate sizes for stereo exhibition, beginning at about seven meters of screen width, which he said is a minimum for judging depth perception in a 3D film.
Beyond that, he concluded that the problems with 3D exhibition tend to come not from different screen sizes, but from different viewing distances. In general, he said, the first third of a movie theater is always going to be a “problem zone” for viewers, suggesting that long-standing presumptions about cinema architecture may need to be challenged if new theaters are to be designed with stereo viewing in mind. (The “sweet spot” for viewing 2K content in a typical cinema is almost at the back of the theater, he said.)
And finally, he challenged the idea that 2K projection is adequate, noting that viewers can generally perceive stereo divergence on a screen at a level that 2K images can’t resolve. “2K resolution does not take full advantage of the depth-perception detail that the human eye is capable of,” he said. “Stereoscopic cinema needs more horizontal spatial resolution.”