Documente Academic
Documente Profesional
Documente Cultură
Opinions on The
State of the industry
by < chris@chrisgrayson.com >
Contents:
Where VR went wrong,
why this matters to AR
and how to set it right. Private Launch Party VR / AR Style Augmented World Expo
Optically transparent Christopher Grayson The largest Augmented Reality
displays will be speaking. and Virtual Reality event of any
kind, in the world.
Smartglasses
general purpose
AR interface
Other notable
developments
Consumer face recognition
& Mesh-networks
Monday, Feb. 27th Wednesday, May 31st
Events
Request an invite WeWork Civic Center, to Friday, June 2nd
CONCLUSIONS San Francisco Santa Clara Convention Center
2
Oculus Rift
ships to the consumer
Consumer Stereoscopic
(class of 2011)
For CES 2011, stereoscopic cameras were a thing. Many were introduced but now no longer available.
The trend peaked prematurely. The problem in 2011 was that there were no good 3D content consumption
devices, VR headsets were still a few years away. You could take stereoscopic 3D images and videos, but
there were very few ways to view them. The personal viewers that existed were little more than glorified
Victorian era stereoscopes.
Where VR content went off course 4
The focus shifted to gaming where 360° content is the norm. With the gaming
industry in the driver’s seat, and with 360° the standard approach, it was
taken as a given that 360° was best for all things (and even treated as more
important than depth).
Stereoscopic cameras were off the market just before headsets arrived.
Where VR content went off course 5
“Camera-Through” AR The stereoscopic camera in a phone was the right direction, but the lenses need to
be placed at the proper pupillary-distance, aprox. 60mm, to match human scale.
Vintage examples:
Vuzix camera-through AR
To go forward, AR needs to take one step back. This turns any phone-based
VR headset into an AR headset via camera-through, just as AR was performed in
pre-2011 AR & VR headsets.
With the size of the iPhone’s hardware market, if Apple adopted this approach, it
would both flood the market with UGC VR content, as well as create a transitional
Camera-through attachments shown on
NVIS & Sensics VR headsets in years past. stage from smartphone to AR smartglasses.
Where VR content went off course 7
Course correction
Lucid is principally a VR video
editing software company.
CEO Han Jin says they
introduced their stereoscopic
camera, LucidCam, because If a player steps up to shows some leadership, this can still happen.
there were no good
consumer stereoscopic Apple should have done it with the introduction of the iPhone 7. I’ve lost
cameras on the market. confidence that Apple is going to show industry leadership. Given the
LucidCam is a reference
model. Han also agrees with
introduction of both the Surface Studio and HoloLens, I’m most inclined to view
Greg Madison of Unity, and Microsoft as the innovation leader among large tech companies today. As a Mac
this author, that 360º video is, user since the 80s, I don’t say that casually and would love to be proven wrong.
in most cases, a distraction.
I write with unwavering conviction: If just one major handset maker stepped up
and introduced a smartphone with a stereoscopic camera*, capable of
UGC VR video and camera-through AR, it would do more to propel both
the VR & AR industries forward than another dozen me-too headsets, or
anything happening in the AR smartglasses space.
* it has to happen at scale, so it has to be from a major handset maker — a startup simply cannot do it —in fact, Apple or Samsung may be the only two players
with enough market share to impact the market — though in their absence, another handset maker could make a big splash in the market by moving fast.
Where VR content went off course 8
Sources:
1 Sony Corporation’s final calendar quarter is their Q3 fiscal quarter, hence quarters are reported here as holiday 2016 quarter.
2 Apple reported their Q4 2016 iPhone sales.
3 iDC Worldwide Quarterly Mobile Phone Tracker, Feb. 1 2017
Volumetric broadcast is the holy grail. 9
Optically transparent
displays
Optical see-through displays 11
Waveguides
Today, there are only three companies that matter in waveguides.
Why? Because there are only three who have shown they can go all the way to manufacturing.
Hardware is hard. Near-eye optics are very hard. Waveguides are harder still.
Going from an engineering concept to a functional prototype is exceptionally
difficult. Taking that IP from prototype to a scalable, manufacturable product is
much more difficult than filing patents and issuing press releases (It’s clearly
more difficult than raising money).
I’m also watching TruLife Optics and Akonia Holographics to see if they can take their designs to manufacturing. My skepticism of Magic Leap
does not stem from their recent PR problems, but from their ability to take their ambitious designs to manufacture (or even to prototype?).
Kayvan Mirza of Optivent recently wrote a Magic Leap Analysis that is recommended reading.
Optical see-through displays 12
Waveguides
Addendum
Journey Technologies
In the course of writing this report, I was contacted
by the founder of Journey Technologies of Beijing.
She boasted that they have a design “like Lumus,”
with a 36° FOV and 1280x720 resolution.
She claims they can manufacture 500 units per
month, and can “easily” ramp up to 1000 per month
with their existing manufacturing facility.
She included a photo of their optical unit that
indeed resembled Lumus’ design.
The Chinese are masters of reverse engineering
and the Lumus design is very basic, making them
vulnerable to commoditization.
Optical see-through displays 13
Others
There are other near-eye-optic see-through display systems besides
waveguides. Meta is notable for their exceptionally wide, 90° FOV, the widest
on the market. It is a beautiful display. I compare the Meta 2 display to the
top-of-the-line graphic work station displays of the late 90s. Even as flat-panel
displays came onto the market in the early oughts, a nice CRT still had better
color and higher resolution, and a 21” Viewsonic was a larger screen than
anything available in an LCD … but clearly
the writing was on the wall. META 2
One notable exception to the waveguide trend in low profile displays is ODG (Osterhout Design Group), whose R9 has a
magnified OLED micro-display projecting into an outward beam-splitter, onto a reflector/combiner, all in a slim (-ish) form
factor, featuring both a higher resolution, and a 50° FOV that rivals that of current waveguides. It is also worth mentioning
that they don’t suffer the display artifacts of current generation waveguides including halo/glow and rainbow-ing image
distortion. Just as with the inferior image quality of early generation flat-panels, these problems will be solved in time.*
* You can read a good counter argument against waveguides at Karl Guttag’s blog, Magic Leap & Hololens: Waveguide Ego Trip?
14
Smartglasses
general purpose
AR interface
15
Smartglasses
general purpose
AR interface
Waveguides +
Eye-Tracking
Depth Sensors
Computer vision
electroencephalogram (EEG)
Smartglasses: a general purpose AR interface 16
EYe-Tracking
Eye-Tracking alone can substantially improve image quality in AR & VR by enabling foveated rendering —
rendering in highest resolution only that portion of the display directly where the user is looking — putting less
burden on the GPU and allowing for higher frame rates.
Coupled with depth sensors and computer vision, things get much more interesting. If the device knows both where
the user is looking, and has an understanding of the environment (i.e.: knows what the user is looking at), we only
need to add intent (user command). It is in this context where I will make the case for Eye-Tracking + EEG.
51% 39% 6% 1%
Based on a survey of 500 consumers in the United States Source: Business Insider / Creative Strategies Creative Commons cc BY-ND
Waveguides + Eye-Tracking +
Sensors + Cv + EEG for A | B
Smartglasses: a general purpose AR interface 17
Voice is for
language translation See Real-time Skype Translation
by Microsoft Research.
International
Language
translation
Waveguides + Eye-Tracking +
Sensors + Cv + EEG for A | B + Speaker + Microphone
Smartglasses: a general purpose AR interface 18
electroencephalogram (EEG)
“EEG based brain computer interface has
demonstrated its capability to control a AB UI Analogy:
An A | B selection via EEG is
device such as controlling the fly of a model derivative of a two button mouse
helicopter. The EEG based binary input and the user’s eyes as the cursor.
device shall become available in the near
Notables in the consumer EEG device space
future depending on the market needs.”
include interXon (maker of Muse), Personal
—Dr. Bin He, University of Minnesota
Neuro (maker of Spark), and Emotiv (maker
EEG AS Command of Epoc+, and Insight). All currently marketed as
EEG still has a substantially slower wellness products, with various meditative features, Emotiv has
response time than that of a human finger an SDK for third-party developers, some of whom are already
on a mouse. On the following page experimenting with using EEG for command.
Dr. Geoff Mackellar, CEO, Emotiv gives a
Smith Lowdown Focus MUSE by interXon Personal Neuro
more nuance and cautious analysis. by Spark
NOTABLE:
Safilo’s SMITH brand of
eye-frames have partnered Emotiv Epoc+ Emotiv Insight
with interXon to introduce
the Smith Lowdown Focus,
Mpowered by Muse. The
form factor is proven
possible, though the
functionality is, at this
Waveguides + Eye-Tracking + time, still limited to Muse’s
meditation app.
Sensors + Cv + EEG for A | B
Smartglasses: a general purpose AR interface 19
electroencephalogram (EEG)
EEG AS Command … with Caveat
EEG still has a substantially slower There are several issues in calculating latency for any response. The Sensory
Detection Time is the basic brain response time - typically around 210-260ms
response time than that of a human finger depending on age, for an unfamiliar but expected task. This is the point at which
on a mouse. Dr. Geoff Mackellar, the brain has decided to act and starts to initiate the motor signal in response,
which takes around 100ms to execute. Sensory Detection time and the motor
CEO, Emotiv gives a more
delay are both reduced for highly trained tasks, so for example athletes and
nuanced and cautious analysis. very experienced gamers can reduce the overall reaction time to specific kinds
of events as a result of habituation.
While EEG-based response may not
Emotiv offers a direct mental command system based
be fast enough to compete with motor on a user-trained set of reproducible mental patterns
signals “in general,” it is still the opinion which are related by a machine-learning classifier system as belonging to each
of the commands. In this case the classification is made at 250ms intervals,
of this author that the trade-offs in privacy based on analysis of the most recent second of data. Latency in this case
for voice command and social acceptance depends both on the computational side, with an expected delay of around
250ms, and the user’s ability to form the mental state, which depends on the
for gestural interfaces will make EEG the
user’s experience level and ranges from 50ms to several seconds. These delays
winning UI in most consumer user cases. occur end-on with the ordinary Sensory Detection Time, where the subject must
form the intention to act before starting the process.
Additional latencies much also be taken into account for EEG systems. Firstly,
the signal processing chain usually includes finite impulse response filters
to remove signal artefacts such as the 50/60Hz line hum present almost
everywhere. This is picked up from the electrical mains supply and contributes
very large artefacts into the data stream. Typical filter have an inherent latency
of 50-60 milliseconds. Computational effects also introduce latency, where
a series of calculations must be made and these are generally applied to a
retrospective signal, often in the range of 0.25-5 seconds.
In general, EEG-based is not fast enough to compete with direct motor signals
except in cases where the subject’s motor system is compromised.
Waveguides + Eye-Tracking +
Sensors + Cv + EEG for A | B — Dr. Geoff Mackellar, CEO, Emotiv
20
Other things
worth mentioning
Worth mentioning exhibit A 21
Face recognition
Why Mesh-networks matter to AR
Consumer Face Recognition 25
29
30
TALK:
BLIND SPOT
fashion eye-frames
waveguides &
low power processors
t h e c o m i n g b a t t l e to ow n co n s u m e r aR s m a r t g l a s s e s .
Speaking
Christopher grayson
San Francisco, I am in you.
C i v i c C e n t e r , S a n F r a n c i s co
Monday
Feb. 27th
BUY TICKETS
April 31 - June 2, 2017
31
Conclusions
While the media treats VR as the new gaming console, Represented by
I see both VR and AR as globally transformative
technologies — something as big as the invention For custom Reports By
Christopher grayson
of the telephone, on par with the invention of the
internet itself — VR and AR are the technologies COntact
Jay Shiel
the internet has been waiting for. +1 212-984-8500
jshiel@glgroup.com
Virtual Reality, the ability to place two or more people
from anywhere in the world into the same virtual space
— potentially infinite in size, with a sense of presence —
will sweep away national borders, challenge the
global political landscape, and possibly even change
the way we think of humanity itself.
This report has been a collection of observations,
ideas, and opinions, loosely connected by the
common theme of virtual and augmented reality.
Entrepreneur
If you like what I’ve had to say, I’d like to hear from you, Creative Director
Christopher Grayson
< chris@chrisgrayson.com >
33
All product names, logos, and brands are property of their respective owners.