Session 4: Keynote Speaker: Mark Schubin, Technology Consultant

What is 3D?

Jan 14, ITU-R SG6 defined 3 generations of 3D; first generation is plano- stereoscopic (with 4 sub-levels, 1 = anaglyph), second generation is multiview, and third generation is object-wave profile (holography), which is 15-20 yrs away.  RabbitHole is delivering film-based holography (we have examples in the ETC Consumer 3D Experience Lab).  It is limited to 1280 frames.

Less than full stereoscopic is enhanced chromostereopsis and pulfrich effect.  Chromostereopsis works as long as you control the colors in the scene.  View-shifting works by jiggling, and microstereopsis (Trioscopics) doesn’t cause vision problems but doesn’t work well either.

The term ‘3D graphics’ is used for both CGI and stereoscopic 3D, which can make conversations confusing.

POOT is plain old ordinary TV.  If you close one eye, you might sense depth.

He reviewed the visual cues mentioned by Pete Lude in a previous talk.  Papers by S. Nagata, 1991 and J. Cutting & P. Vishton, 1995 contain graphs illustrating the influence of the various cues.

Ames rooms confuse perspective vs. size cues.  (Ames rooms are the distorted, optical-illusion rooms that make the same object appear huge in one corner and tiny in the other.)  Five Ames rooms were used in Lord of the Rings to make Gandalf appear much taller than the Hobbits.  If Lord of the Rings was shot in 3D, the effect would not work.

Why does 3D matter?

Things to consider when creating  3D:

– The placement of the convergence point when shooting

– Pupillary distance can be 40-80 mm for children

– Screen size

– Viewing distance (if your pupillary distance matches the negative parallax on the screen, the object will come out 3’ if you are 6’ away, and 25’ if you are 50’ away, which is less credible.)

The movie theatre is not the home.  In a theatre, the audience is most likely entirely in the zone of comfort.  At home, the zone of comfort is only a small portion of the available seating locations around the TV.

Conflict /             Effect

– perspective-size /             incorrect relative size

– occlusion – stereopsis /             graphics difficult to view

– vergence-accommodation /             possible discomfort

– stereovisual – vestibular /             possible discomfort

– stereopsis – vergence /             incorrect depth

– impairment, choice  /             might training help the muscular areas? We don’t know.

Why are so many TVs coming out with active shutter technology?  Because polarization is hard to implement.  With active shutter, you don’t have to do anything to the screen.  You just add the emitter, so it ends up about the same price.  This is good for the CE manufacturer, because they are not necessarily concerned about the cost of the glasses.  Also, the battery life of the battery in the glasses is not their concern.

While people who are sensitive to 3D can watch polarized in 2D by putting the same lens in both eyes, isolating one image for both eyes in active shutter glasses may produce flicker.

BBC R&D White Paper 180, posted on the web, discusses how to synthesize 3D.

Steve Scklair said 3D requires fewer camera positions.”  The same was said at the birth of HDTV.  Audience expectations evolved, so expect the gradual adoption of faster cuts and shorter scenes.

Terms that are often used in marketing and press releases but rarely credible include: “the first”, “successful”, and “good enough.”

A paper by I. Howard & B. Rogers, 1996, discussed microstereopsis.  Human perception of stereo is greater than human perception of luminance.

Digital Optical Technology Systems, The Netherlands, is a glasses-free display that uses a slit iris to induce color fringes that creates a 3D effect.  The patents for this technology have expired.  It works with one ordinary lens per camera position, as long as it is outfitted with a slit iris.  This solution, which has a nearly zero interocular distance, has no ghosting, but it is incapable of a WOW effect.  It is a ‘kinder gentler 3D.’

(See Gary Shapiro’s editorial on 3D in the current issue of Vision magazine.)

Q&A

What types of content are particularly suited for 3D?  From a physiological standpoint, talking heads and children’s shows are well suited for 3D because of narrow depth range and comfort.  Getting people to buy things, though, will take sports and movies.

How can you shoot the Grand Canyon in 3D?  Make the camera spacing very large.  Hyperstereo reduces the 3D at a distance and heightens it up close.

Do you want to make any points about Avatar?  We need to interview everyone who had a problem viewing Avatar.  That is the epitome of 3D, so we need to understand why those people had problems.

Why is this rebirth of 3D different?  We have the digital technology, management of the convergence plains (keystoning), multiple camera sizes are available, and other factors, so from a technical perspective we are in a new place.  I cannot speak to the market question of whether this is the future.

Session 3: 3D Conversion

Moderator(s):

Brad Collar, Vice President, Technology, Warner Bros.

Panelist(s):

Barry Sandrew, Ph.D, Founder and President/COO, Legend3-D, A Legend Films Company

Chris Bond, President – View D, Prime Focus

Chris Yewdall, Executive Director and Chief Executive Officer, DDD USA, Inc.

Warren Littlefield, President, The Littlefield Company

Brad Collar

There are often two controversies when the entertainment industry looks at a new technology; should we do it, and what is the best way to do it?  The list of options in 3D keeps growing.  This panel will focus on 2D-to-3D conversion.

Warren Littlefield

Home 3D is the next breakthrough.  In 1996, he did a 3D episode of 3^rd Rock from the Sun.  It doubled their audience, and helped them win the May sweep.  The lesson learned was that the audience was looking for something new.

There are three categories of 3D content to the home; filmed entertainment, live content, and games.

Four business models (Warren’s term):

– Home video: over 600 episodes of StarTrek are available for a 3D sale.  He thinks 3D conversion will be the killer app

– 3D networks – expect more this year

– Worldwide program distribution

– Digital distribution, including iTunes, Amazon, etc.

The number of sets that will be sold is unknown.  With great cost comes great risk.  The revenue models will depend on how many 3D sets are sold.  There is no format war, just a need for content.  This is an opportunity to up-sell the library.  The box office for 3D indicates a sustained appetite, dispelling the idea of ‘stunt’ appeal.  Building the market will be about volume – volume of set sales and volume of content (live events, movies, and series) together (the chicken plus the egg).

Chris Yewdall

(Chris calls himself the self-proclaimed minister of propaganda for 3D)

Conversion can be done at one of three quality levels: good (embedded in 3D device, automated 3D conversion), better (3D conversion with automated depth recover, manual focal point/depth effect decisions), and best (original 3D content creation, or 3D conversion with manual / semi automated depth recover).  The cost goes up accordingly.

Carl Franklin wrote a great book, “Why Technology Fails.”  Any 3D conversion must pass the “so what” test.

Since 1993 DDD has been developing a process called Depth-Image Based Rendering (DIBR).  A single 2D image is depth mapped using monocular queues to create the conversion.   They have been working to make the manual aspects more efficient by continually improving the computational rules.  Cost went from $100k/minute in 1999 to $1,500/min in 2007 by relying on the automation more and the human intervention much less.

3D conversion consists of two stages: depth recovery from 2D content, and 3D scene reconstruction from depth and source image.  Auto 3D conversion will not deliver the “better” category (first paragraph).  Human intervention is required, particularly for 3D focal point.  At the end of the day, the source content is still 2D, but filming in 2D with the scene set-ups designed for 3D improves the results.

Barry Sandrew, PhD

(He invented the first auto-colorization process, and he contributed to the Alice in Wonderland conversion)

(He thanked Pierre de Lespinois (from the previous panel.  Pierre was very critical of conversion.) for the kind words. (sarcasm))  70% of the over $80M of Alice opening weekend was from 3D screens.  Rather than go through his slides, he showed recent work to present a more accurate sense of the current state of conversion.

Christopher Bond

Christopher oversaw conversion of Clash of the Titans, and presented the chronology of the process.  Prime Focus received the call in mid-January to review the film.  They had the kick-off meeting with the Director, Editorial, and VFX supervisors, who were skeptical that it could be done in 8 weeks, in London on Jan. 19^th.  The delivery date for the final version of the conversion was fixed at March 19^th.  The details of what they were to be given when work started included:

– not a locked cut / expect editorial changes / scene additions

– 90-100 minutes run time, 1000-200 shots

– No graded material, begin work on raw scenes ASAP

– Less than 1/8 of the VFX shots were final at the start of conversion

Having to deal with working with an unlocked, ungraded cut meant redefining their work process:

– Focus efforts on the “most locked” reels

– Work with raw scans (so they were prepped when the graded material arrived) / 8 frame handles head and tail

– 16 frames – 1935 shots = 21.5 additional minutes

– Conceive and develop tools to reprocess graded material once delivered

– Automate as much of the pipeline as possible

– Everything went into a database, which was created for this process

The creative elements came into play by this process:

– Define “Keystone” shots throughout the film

– Bring “Keystone” shots to final ASAP (approximately 5-10 days) and show them to WB

– Apply notes / feedback from WB and propagate to shots within the same sequence

– The dailies went through many rapid-fire iterations.  Dailies were being worked on in 3 rooms, 1 Dolby and 2 RealD

– Because of the quantity of dailies, they came up with a grading system – A (perfect), B (some artifacts), C (first-pass)

– Twice weekly formal client reviews, progressing from scattered shots, to sequences, to reels, to, near the end, the entire movie

– Daily reviews in the last few days covered 15-17 minutes of reviews each day

– Toward the very end they applied convergence and watched the shots in ‘cut’

Lessons learned

– Do not underestimate editorial and ‘conform’ needs and reviews.  This is a massive amount of data.

– Working with an unlocked cut meant that lots of content ended up on the cutting room floor.

– Clients change their minds, even on fundamental issues like how much stereo

– We can convert a movie in 8-10 weeks.  Just stay calm.

Q&A

(Brad) Why is there so much controversy about conversion? (Warren) Consumers will vote with their wallets.  (Barry) Something this new will attract both criticism and praise, but it will make money if done right.  (Chris Bond) Sound, Panavision, TV all met the same criticisms, which faded as they improved and gained market acceptance.

(Brad) Will costs come down?  (Chris Yewdall) TVs can do the ‘good.’  Avatar shows the ‘best.’  We will fill in the middle over time, and costs will come down.

(Brad) Do you see a head-end broadcast converter box?  (Warren) Live conversion will not be as good as human intervention in the near term.  The automated process in the TV cuts out the content maker from the revenue and control, so it is critical that they get involved in the process somehow.

(Brad) Chris (Yewdall), you brought up good, better, best.  Will consumers see the difference?   Should they be branded differently?  (Chris Yewdall) No, they should not be branded differently.  There are already variations; 720p v 1080p v 1080i.  Over time, the conversion technologies will improve to the point where consumers won’t be able to tell the difference because there may not be a difference.

(Brad) What are the technical advantages of converting a new feature versus an old feature.  (Barry)  The degree of creative input is much less for older features.  There is a great deal of creative input for new films, which adds time an cost. (Warren) We’ll have a new generation of filmmakers shooting in 2D, knowing it will be converted.  Their shooting process will be changed by that knowledge, and the viewer will benefit from that.  It is just starting to happen.