Augmented Reality Law, Privacy, and Ethics: Law, Society, and Emerging AR Technologies

Augmented Reality Law, Privacy, and Ethics

Law, Society, and Emerging AR Technologies

Brian D. Wassom

Table of Contents

Cover

Title page

Copyright

Dedication

Endorsements

Author Biography

Technical Editor Biography

Acknowledgments

Part A: Understanding the landscape

Chapter 1: What is Augmented Reality Law, and Why Should I Care?

Abstract

What is augmented reality law?

Why study AR law?

Conclusion

Chapter 2: A Summary of AR Technology

Abstract

Introduction

Defining our terms

A technology for all senses

Supporting, or augmented world, technologies

Levels of adoption

Part B: AR & the Law

Chapter 3: Privacy

Abstract

Introduction

Sources of privacy law

Privacy concerns raised by AR

Using AR to enhance privacy

Chapter 4: Advertising, Marketing, and eCommerce

Abstract

Introduction

AR’s use in advertising and marketing

False advertising and unfair competition

Advertising disclosures

Conducting commerce

Conclusion

Chapter 5: Intellectual Property

Abstract

Introduction

Patents

Trademarks

Copyright

The right of publicity

Chapter 6: Real Property Rights

Abstract

Introduction

The basic rights at issue

AR: where property rights and free speech collide

Scarcity in augmented real estate

Other intersections between property rights and AR

Chapter 7: Torts and Personal Injury

Abstract

Introduction

Intentional torts

Negligence

Products liability

Automotive

Conclusion

Chapter 8: Criminal Law

Abstract

Introduction

Unintentional run-ins with the law through AR

Intentional criminal activity

Law enforcement usage

Turning the cameras backwards: wearables as a means to monitor law enforcement

Chapter 9: Civil Rights

Abstract

Introduction

The current requirements for accommodating the disabled in digital media

How AR can meaningfully improve the lives of disabled persons

Chapter 10: Litigation Procedure

Abstract

Introduction

Gathering evidence for use in legal proceedings

V-Discovery

Assisting lawyers with legal research

Augmented reality in the courtroom

Personal jurisdiction

Part C: AR & Society

Chapter 11: Politics and Civil Society

Abstract

Introduction

AR as a means of mobilizing people for social good

AR and the erosion of civil society

Hope remains

Chapter 12: Personal Ethics

Abstract

Introduction

Will augmented world technologies erode our ability to make ethical decisions?

Will augmented world technologies corrupt our ethical decisions?

Will augmented world technologies lead us to form bad habits?

Chapter 13: Addiction and Pornography

Abstract

Introduction

AR Addiction

Pornographic and prurient Content

Conclusion

Index

Copyright

Acquiring Editor: Chris Katsaropoulos

Editorial Project Manager: Benjamin Rearick

Project Manager: Surya Narayanan Jayachandran

Designer: Matthew Limbert

Syngress is an imprint of Elsevier

225 Wyman Street, Waltham, MA 02451, USA

Copyright © 2015 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

ISBN: 978-0-12-800208-7

For information on all Syngress publications visit our website at http://store.elsevier.com/

Dedication

To the two mentors who had the most meaningful influence on the first 15 years of my professional career. The Honorable Alice M. Batchelder personifies integrity and excellence, and taught me to respect the legal system. Herschel P. Fink, Esq. taught me to love the law I practice, and to practice the law I love. Both gave me amazing opportunities to serve in ways that fundamentally shaped my career. I hope to pay forward to others all that I can never repay to them.

Endorsements

Any techie that follows Augmented Reality knows that AR continues to surge under Moore’s law. Brian Wassom is the indisputable, top legal expert in the realm of Augmented Reality. His perspective and legal lens continues to focus on AR and its journey to revolutionize technology. This book is a must read for any person looking to delve into the augmented world and absorb the rapidly changing legal and ethical landscape of cutting-edge high technology and its influence on society. This book is an unmatched AR resource that yields a powerful comprehension of an evolving mass medium.

–Joseph Rampolla – Cyber-crime expert/Augmented Reality Dirt Podcast creator/Co-author of Augmented Reality: An Emerging Technologies Guide to AR book

"Brian D. Wassom is my go-to resource on anything having to do with how Augmented Reality and emerging technology relates to legal issues. His writing is clear, impactful, and highly accessible regarding complex legal and technical issues. His book Augmented Reality Law, Privacy, and Ethics provides compelling evidence as to why Augmented Reality will drastically change culture over the next few years and how people need to prepare for what lies ahead. Brian’s wisdom, humor, and insights make Augmented Reality Law, Privacy, and Ethics a pleasure to read, and a must-have resource for anyone wishing to understand how our vision of the future will be perceived through the lens of Augmented Reality."

–John C. Havens, Contributing writer for Mashable, Slate, and author of Hacking H(app)iness – Why Our Personal Data Counts and How Tracking it Can Change the World.

We’re at the precipice of the next, visual era, with smart glasses that will forever change how we look at the world. More than just a comprehensive look at the related legal, social, and ethical issues, this book will get you thinking about the full impact of what’s to come.

–Dave Lorenzini, CEO of Arc

As the mass-media industries adapt to the newest mass medium, Augmented Reality, the combined abilities of digital, mobile, social, and virtual, all produce a quagmire of challenges and threats – as well as opportunities. Brian’s groundbreaking book is an invaluable guide to the treacherous ground that media owners, content creators, talent, news organizations, and others will face as they rush to stake their claims in the AR world. A must-read and invaluable resource for the next ten years.

–Tomi T. Ahonen, author of 12 books including Mobile as 7th of the Mass Media.

Brian Wassom is the world’s leading expert on augmented reality (AR) law. Wassom’s research pioneered the field of AR law and currently defines the way it is understood by developers. His writing points out the heart of the salient issues facing the rapidly growing field of AR. Wassom’s texts are required reading for my Mobile AR graduate course at NYU.

– Mark Skwarek, a full-time faculty member at New York University (NYU) and the director of the Mobile AR Lab at NYU.

Brian Wassom brilliantly illuminates some of the tricky issues of privacy, law, and ethics that will determine whether Augmented Reality results in an enhanced or degraded future for humanity.

–Tish Shute, Head of Product Experience at Syntertainment and cofounder and chief content officer of Augmented World Expo.

Wassom thoroughly highlights most of the key issues facing AR today while establishing a clear path for analysis in the future. From advertising, to smart cars, to the augmented criminal organisations of the future, Augmented Reality Law, Privacy, and Ethics is must read for anyone looking to become deeply involved in AR over the next decade.

–Brendan Scully, Senior Business Development Manager, Metaio, Inc.

Author Biography

Brian D. Wassom litigates disputes and counsels clients from Fortune 50 companies to startups concerning copyright, trademark, publicity rights, privacy, and related intellectual property and advertising issues. He is a partner in the law firm of Honigman Miller Schwartz and Cohn LLP, and chairs the firm’s Social, Mobile and Emerging Media Practice Group. Brian authors a popular blog on emerging media at Wassom.com that features the section Augmented Legality®, the first regular publication devoted to the law governing augmented reality. Brian presents regularly to industry groups, legal education seminars, and conferences across the country on intellectual property, digital media, and related topics.

Technical Editor Biography

Allison Bishop is a highly experienced criminal reviewer with 10 years experience reviewing cases, criminal research, and criminal activity. Allison also works as a paralegal performing tasks such as briefing cases and reviewing case law and the working within the legal process. As an editor, Allison enjoys reviewing and editing works based on criminal justice, legal studies, security topics, and cybercrime. When Allison is not entrenched in studies, she loves to exercise, cook, and travel.

Acknowledgments

I am indebted to those who assisted in editing this manuscript, including Brian Coe and Karen Larson. My thanks as well to all those whose input shaped my outlook on, and understanding of, the augmented reality industry and all of the potential it holds, including Brian Mullins, Gaia Dempsey, Jay Wright, Joseph Rampolla, Ori Inbar, Tish Shute, Will Wright, Dave Lorenzini, John C. Havens, Trak Lord, Brendan Scully, Daniel Suarez, Dr. Mark Billinghurst, Dr. Steve Mann, Keiichi Matsuda, Mark Skwarek, Bruno Uzzan, Eric Mizufuka, Dima Kislovskiy, Cecilia Abadie, Brian Selzer, Robert Rice, Pete Wassell, Steven Petersen, Brad Waid, Drew Minock, the members of the AR Detroit Meetup Group, and so many more.

All thoughts, speculation, and analysis in this book – and especially any errors in fact or logic – are mine alone. This work is not intended as legal advice or a solicitation for legal representation, and does not reflect the views of my law firm or its clients. Moreover, although this book mentions several actual products and companies in the course of explaining augmented reality and related technologies, nothing herein is intended as a criticism of, or suggestion of wrongdoing involving, any mentioned product or company. To the contrary, this book is meant as a celebration of the technological advances that these pioneers represent, and to prepare society to better understand the world that such developments will usher in.

Part A

Understanding the landscape

What is Augmented Reality Law, and Why Should I Care?

A Summary of AR Technology

Chapter 1

What is Augmented Reality Law, and Why Should I Care?

Abstract

This chapter explains that the book is a horizontal survey of a number of different areas of law that are likely to be significant to people developing and using augmented reality (AR) technology, rather than a deep dive into any one particular legal doctrine. It also seeks to persuade the reader that AR law is an important subject to study because AR technology is poised to be an important aspect of the economy in the near future. Mass adoption of AR technology is inevitable for three primary reasons: convenience (meaning the unique way in which AR allows people to interact with digital data in the same way they would intuitively react to physical objects), creativity (meaning the unique space for creative expression that AR provides), and capability (meaning that digital and mobile technologies have reached sufficient critical mass to allow for the mass adoption of AR). Additional evidence of AR’s inevitability is found in the amount of money that investors have already put into the industry, and the amount of revenue it is projected to generate.

Keywords

Horizontal analysis

law of the horse

inevitable

Iron Man

billion-dollar industry

convenience

creativity

capability

Information in this chapter

• The horizontal nature of studying augmented reality law

• The inevitability of augmented reality technology

• The economic significance of augmented reality technology

What is augmented reality law?

One of the joys of writing the first book on a topic is having the freedom to frame the discussion however seems best to me. The topics of discussion in the following chapters are the ones that I find the most important to explore based on my own experience practicing law and spending time with members of the augmented reality (or AR) industry.

But there are also downsides to a project like this. Among those is the need to justify the book’s existence before convincing anyone to read it. In the case of AR law, I am often required to explain to listeners what AR even is before I can broach the subject of why the law governing it is distinct and significant enough to require its own book.

That is the function of this chapter and the next. Here, I will attempt to persuade you that the AR industry is one to take seriously, and that it will be important to understand (and to help shape) the law governing the use of AR technology. Assuming that you remain sufficiently open to these conclusions to follow me to Chapter 2, I will explain in greater detail the nature of AR and its related technologies. From that foundation, the rest of the book will survey a number of different legal and ethical topics that are likely to be, or are already being, implicated by AR.

I hope you will stick with me to the end, and agree that it was worth the ride.

A horizontal study

If you are a student, then you are likely accustomed to studying one concept – such as contract law, chemistry, or grammar – at a time. Even in professional settings, individuals and entire companies often find themselves, consciously or unconsciously, thinking and operating within defined tasks, categories, or industrial segments, to the exclusion of all other subjects. We frequently refer to these areas of concentration as silos or verticals, implying that the people inside them may build up quite a bit of knowledge of, or experience in, the given topic, but have relatively little idea how that topic relates to anything else. For example, an automotive engineer may spend years, even an entire career, immersed in the inner workings of a particular subsystem of a car, with no understanding or concern as to how that subsystem relates to or affects the rest of the vehicle. Similarly, many legal and medical professionals develop highly specialized (and expensive) skills in a niche practice area, but would not have the first clue how to help a random client who walks in off the street with a basic, everyday problem.

This is not such a study. "Instead of a deep ‘vertical’ look at one legal doctrine, this [book] will survey several disparate topics ‘horizontally.’¹ In the current professional vernacular, it cuts across several verticals. Put another way, this book takes as its starting point one particular industry – the companies and innovators developing AR and related technologies – and surveys the various legal issues that members of that industry are likely to encounter. This approach has the advantage of being enormously more useful for the members of that industry and the professionals (like me) who would serve them, but it can be a bit disorienting (at least at first) for students accustomed to more abstract analysis.

That is not to say, by any means, that vertical studies of legal principles do not have their place in academia, or that students should avoid reading a book like this one. To the contrary, courses in basic legal doctrines provide the building blocks necessary for applying the law to complex problems. Horizontal exercises like this one can be ideal vehicles for transitioning from book learning to the ability to counsel clients in real-life situations. That is one reason why horizontal studies like this one are not uncommon during the third year of law school.

Perhaps the most direct audience for this book, however, is the growing ranks of those business people and technological dreamers who are out there, even now, literally building a new world around us all by means of what we currently call AR or augmented world technology. I have been privileged to meet and interact with scores of these innovators who are rapidly forming an industry out of concepts that were pure science fiction mere months earlier. They have the foresight to recognize just how much our world will change when we finally master the art of interweaving our digital and physical means of experiencing the world.

When I speak at AR conferences and events or counsel clients in this industry – usually after the audience has already heard from several entrepreneurs who cast grandiose visions of what can be done with the technology – I sometimes joke that it is my job as the lawyer in the group to crush their dreams and bring them back down to earth. Yet my actual intention (both there and here) is quite the opposite. These innovators’ dreams are so inspiring because they actually have a chance at being realized. But if AR entrepreneurs are going to successfully bring their visions to fruition, they need informed guidance from advisors who understand the realities and requirements of the legal and business worlds. These advisors must shepherd the innovators through the tricky landscapes and potential pitfalls of regulatory checklists, investment deals, IP protection, and all of the minutiae on which visionaries ought not spend too much of their time. I want more members of this industry to recognize their need for such guidance for the legal services industry to be better prepared to provide it.

This leads to two more currents that are important for me to mention at the outset. First, this book cannot, and does not attempt to, provide legal advice. Consult a lawyer directly before making business decisions. Second, the laws discussed herein are almost exclusively those of the United States. Although the AR community is truly worldwide and many legal and ethical principles cross national boundaries, it is the American legal system in which I practice and that forms the context for my analysis.

The law of the horse

Today, there is almost no one who could honestly be called an AR lawyer. This will remain true for some time, even as the industry begins to mature. One reason for this is that AR law is a concept much like a term I learned in my law school days: the law of the horse. This phrase illustrates the difference between vertical and horizontal legal studies. The idea behind it is that there is no such thing as horse law. Rather, if I own a horse and have a problem with the jockey, for example, I would seek counsel from an employment lawyer. If my shipment of hay doesn’t arrive, I should consult a commercial transactions attorney. And if my neighbor complains about the smell of horse ranch, I might consult an attorney experienced in nuisance law.

Each of these lawyers would be practicing some aspect of horse law, in some colloquial sense, but you would not call any of them a horse lawyer, because lawyers do not usually hold themselves out in that manner. Lawyers typically market their services according to particular categories of legal doctrine or practice. Historically, relatively few lawyers have packaged their services according to the needs of a particular industry, even though it might be more efficient for our hypothetical horse owner to find a lawyer or law firm specializing in horse law than to seek counsel from different specialists on each issue.

I first heard this law of the horse analogy applied to Internet law, to make the point that there was no such thing. Rather, the Internet and its use implicate virtually every legal vertical, depending on the context. Internet law is a horizontal subject (and thus not worthy of study in a law school, or so was the implication when I first heard the term used).

In the same way, AR law is also like the law of the horse. Defined literally, AR law encompasses all of those fields of legal practice that AR companies will encounter – including corporate, tax, intellectual property, real estate, litigation, and personal injury, among others. Indeed, if AR reaches even half of its potential, it is poised to revolutionize society at least as much as the Internet itself has done. It is inevitable, therefore, that such a sea change in how we conduct ourselves on a daily basis would also influence the laws governing that behavior and how they are applied. Yet, even today, we still see relatively few lawyers marketing their expertise in Internet law, and virtually none have yet grasped the significance of AR as such.

Today, more lawyers and law firms recognize the value of organizing their services according to clients’ needs rather than by traditional categories of practice. This, in part, is why many law firms have assembled industry teams focused on the needs of particular types of companies and comprising a number of specialists from relevant legal disciplines. Practice groups like these are one way in which legal professionals can more comprehensively and efficiently serve the needs of horse owners or any other given industry. Working within a general practice firm composed of lawyers working in dozens of different focus areas is another.

As only one such example, I help to lead my firm’s Social, Mobile and Emerging Media Practice Group, so named so as to encompass both the social media that presents today’s most pressing digital media issues and tomorrow’s emerging media such as AR. For several years now, I and other members of this team have gotten to know professionals within the AR industry and – together with other members of our general practice firm – helped them solve the issues they encounter across a broad spectrum of legal disciplines. It gratifies me to say that I am not personally aware of any other legal practice group as focused on the AR sector. As the inevitability of AR becomes more apparent, however, I expect that we’ll see more such teams intended to serve this important industry.

Why study AR law?

If you are not already as enamored of the AR industry as I am, you may not yet be convinced that AR law is worth your time to study. In that case, allow me to recount some of the reasoning that led me to conclude that this field will be so important.

Inevitability

In this chapter I have already used the word inevitable to describe the increasing prominence and impending ubiquity of AR. That is because I see AR not so much as a brand-new concept that will someday suddenly emerge onto the scene, but rather as a medium that has existed for decades and that is beginning to manifest itself with increasing speed as we finally see the development of the technology that can make it happen.

There are dozens of factors fueling the inevitability of widespread AR. Since consonance makes things more memorable, however, I will summarize them as the three C’s of convenience, creativity, and capability.

Convenience

Not many years ago, humanity’s best means of reading and recording data was on two-dimensional pieces of paper, which we stitched together and stored in books. When that data began to migrate onto computers, we displayed it on monitor screens, and books became files and folders. Over time, the screens became incrementally more aesthetically pleasing – flatter, higher-resolution, and more mobile – and even displayed some digital images that had the illusion of three-dimensionality. But the context in which these displays have appeared – the computer screen – has always been a two-dimensional rectangle.

AR is a unique step forward in the way we experience digital data, because it liberates that data from its two-dimensional box to make it truly appear (as far as human senses can perceive) to be three-dimensional. Granted, there will almost always be some medium (such as eyewear, a window, or a mobile device) through which we experience the display, and those media will remain two-dimensional for the foreseeable future. But AR creates the illusion that the display is present among, and even interacting with, our physical surroundings. Perception is reality, as the saying goes, and it is the perception of this illusion that we call AR.

One fundamental reason that there will always be an impetus to experience data in this format is that physicality is intuitive to us. As children we have to learn to read and write, but playing with physical objects comes naturally. The less work our brains need to do in order to translate and process data, the more readily our minds will embrace it.

Take, for example, the yellow line of scrimmage and the blue first-down line that appear in most televised football broadcasts these days (Fig. 1.1). The technology to create this illusion is actually one of the earliest forms of AR in mass media. Today it is even more sophisticated, with all manner of game statistics appearing as if they were on the football field itself. And the images themselves are so high-resolution and rendered so fluidly that the illusion of physicality is complete. The result has been to make it significantly easier for viewers not schooled in the rules of the game to comprehend the action. It’s one thing to say the offense needs to carry the ball 15 more yards to the 30-yard line; it’s another thing entirely to say they need to reach that blue line. One statement takes significantly less mental processing to understand, which, for some viewers, is the difference between enjoying the broadcast and changing the channel. Indeed, I have heard from several people who attended their first live football game and were disappointed by the experience of trying to follow the game without the digital overlays on the field. For some children who have never watched a game on television without those overlays, the effect is jarring; they had never considered the fact that the lines weren’t actually there!

Figure 1.1   NFL broadcasts contained some of the earliest examples of mass-market AR.

For the same reasons, there is a certain level of understanding about a thing that we as humans cannot reach unless we experience the thing physically. In my line of work, when young litigation attorneys are arguing a case involving a specific place or product, they learn the value of actually visiting the place or holding the product in their hands. That experience does not always reveal more quantifiable data about the thing, but there is a qualitative level of understanding that the attorney gains. They feel as if they understand the thing better, and are therefore often better able to form and express arguments about it.

The Iron Man movies offer another example of the same truth. In each of the four films in which Robert Downey, Jr.’s version of the Tony Stark/Iron Man character has appeared to date, we see him use AR to design complex machinery, architecture, or landscapes (Fig. 1.2).² Whatever it is that he’s studying, Stark views digital renderings that are projected into the space in front of him. By means of poorly explained but fantastically acute holographic and motion-sensing equipment, he physically grasps, manipulates, and alters the data as easily as he would a physical object. (Actually, it’s even easier, since a real physical object would offer resistance and could not hang motionless in empty space.) When Stark needs to study an object more closely, he sweeps his arms in broad gestures to expand the display to hundreds of times its original size. If Stark needs to walk among the digital objects as if they were surrounding him on all sides, he can do that.³ Each such cinematic sequence comes at a point in the plot in which Stark needs to overcome a design problem or gain new insight that he could not grasp merely by reading lines of code or digital images on a computer screen. And each time, it works.

Figure 1.2   The Iron Man films are among the most popular depictions of AR.

Despite all of the entertaining, fast-paced action and gee-whiz effects of the Iron Man movies, these AR design sequences have so stirred viewers’ imaginations that they remain some of the most memorable scenes in the films. Perhaps that is because this way of interacting with data just feels so natural to so many people – and also so tantalizingly plausible that we wonder why we don’t already have such devices in our own offices and living rooms.

No less than Elon Musk feels the same way. Musk is the billionaire entrepreneur behind Tesla Motors, SpaceX, and the proposed Hyperloop train that could carry passengers from Los Angeles to New York in half an hour. As such, he is already the closest thing that our actual reality has to Tony Stark. He cemented that parallel on August 23, 2013, when he tweeted: Will post video next week of designing a rocket part with hand gestures & then immediately printing it in titanium. Iron Man director Jon Favreau responded, Like in Iron Man? Musk replied, Yup. We saw it in the movie and made it real. Good job! (See Fig. 1.3.)

Figure 1.3   Elon Musk acknowledged Iron Man as the inspiration for his own AR system.

The next week Musk followed through, demonstrating on YouTube how SpaceX engineers were combing such devices as the Leap Motion gesture sensor, the Oculus Rift virtual reality headset, and a 3D projector to design rocket parts more or less exactly the way that Tony Stark would.

The point of this exercise was not merely to emulate Iron Man (or any of the other Hollywood films that depict AR being used in such a utilitarian manner, such as Terminator, Serenity, Mission: Impossible – Ghost Protocol, and G.I. Joe: The Rise of Cobra, just to name a few). Rather, Musk explained in his YouTube video that designing three-dimensional objects using a variety of 2-D tools … doesn’t feel natural. It doesn’t feel normal, the way you should do things.⁴ Interacting with digital objects that appear to be real, on the other hand, only requires a designer to understand[] the fundamentals of how the thing should work, as opposed to figuring out how to make the computer make it work.⁵ Then, Musk said, you can you can achieve a lot more in a lot shorter period of time.⁶ In the terminology of this chapter, the AR experience becomes a more convenient way to interact with the data.

Notice the importance of feelings in Musk’s explanation of the technology. His premise is that if an interaction feels normal and natural on an intuitive level, it will be a more efficient and effective interaction. And that is a difficult premise with which to argue. The fact that interacting with data in this manner just feels right is one reason that humanity will inevitably design its technology to function in precisely that manner.

Creativity

Another fundamental characteristic of human nature is the need to express ourselves as individuals. The unique potential of AR to fuel such creative expression also contributes to the technology’s inevitability.

When a medium of expression is more convenient and intuitive to use – in other words, when we don’t have to think about how to use it, but can focus more on what we want to do with it – the medium will be an effective means of expressing ourselves. At the same time, the depth of what we can express is also limited in many ways by our chosen medium. For example, coloring, pointing, screaming, and grunting all come to young children more naturally than actual words. But one reason kids soon turn to language is because they quickly reach the limits of how much they can express with these other forms of communication. On the other end of the spectrum, I have a good friend who is a master violinist. His instrument gives him a voice that can express emotion to a depth that mere words cannot reach. But only through years of rigorous training did that means of expression become natural enough for him that he could use the violin to express actual music, as opposed to the painful shrieks the instrument would emit if I tried to use it.

In a similar way, digital imagery has become a rich medium for creative expression. And although two-dimensional rendering still requires a significant amount of training and skill to do well, the means to create it is becoming cheaper and easier to use all the time. As we add more dimensions to those images, the potential for creative expression goes up, but so do the practical barriers to entry. High-quality three-dimensional imagery is still difficult to do well; just witness the difficulties that movie companies faced getting audiences to accept 3D movies, despite the constant pressure to make them commercially viable. Taking those three-dimensional images and making them appear to be physical objects that persist and adapt to human interaction over time – what some in the AR industry refer to as 4D – remains an even tougher nut to crack.

The cornucopia of creative expression that awaits when the public at large is able to experience AR is a big part of what keeps innovators working on the technology. To illustrate the qualitative difference between creative expression in standard 2D versus 3D or 4D, picture (the original) General Zod and his Kryptonian cohorts taking bodily form again as they escaped their two-dimensional Phantom Zone prison in the 1980 movie Superman II (Fig. 1.4). Or, even more aptly, consider the Space Liberation Manifesto advocated by science fiction author and Wired columnist Bruce Sterling at the 2011 Augmented Reality Event in Santa Clara, California. There, as part of his keynote address, Sterling arranged for a group of rebels dressed in faux-futuristic jumpsuits to hijack the speech to spread flyers advocating a populist agenda for this new blended reality. The manifesto – which Sterling promptly published in his Wired blog – read, in part:

Figure 1.4   The two-dimensional Phantom Zone prison in Superman II.

The physical space we live in has been divided, partitioned and sold to the highest bidder, leaving precious little that is truly a public commons. The privatization of physical space brings with it deep social, cultural, legal and ethical implications. Private ownership of physical space creates zones of access and trespass, participation and exclusion. Private use of physical space becomes an appropriation of our visual space, through architecture, so-called landscape design, and ubiquitous advertising whose goal is to be seen well beyond the boundaries of privately owned property. Simultaneously, private space becomes the preferred canvas for street artists, graffiti writers and other cultural insurgents whose works seek the reclaimation [sic] of our visual space, the repurposing of private political and commercial space for their alternative cultural messages.

The nature of SPACE is changing. In the past, space primarily meant physical space – the three dimensional cartesian world of people and places and things. Networked digital computing brought us the notion of cyberspace – an ephemeral consensual hallucination that nonetheless appeared to have an almost physical sense of place, a separate and parallel universe alongside the physical world. Today, as computing and connectivity become pervasive and embedded into the world and digital information infuses nearly every aspect of the physical environment, space has become an enmeshed combination of physical and digital – a ‘blended reality’. Cyberspace has everted; reality is enspirited.

This new physical+digital SPACE brings new characteristics, new affordances, new implications for culture. Its physical dimensions are finite, measurable, subject to ownership and control, but its digital dimensions are essentially infinite, subjective, and resistant to centralized control or governance. The new SPACE opens tremendous opportunities for access, expression and participation, but also for commercialism, propaganda, and crushing banality.⁷

Prolixity aside, this passage does a good job of foretelling the tremendous opportunities for creative expression in a world where the digital and physical can be combined in a meaningful, perceptible way. The manifesto’s example of graffiti illustrates the point well. When people are limited to physical means to express themselves, one person’s artistic appropriation of a given object (such as a brick wall) necessarily conflicts with the interests of others who would use that object for different purposes (such as the landowner). With AR, a potentially infinite number of people could superimpose their own expression on the same physical wall without changing anything about the wall in real space. As Sterling notes, this explosion of creative democracy will, over time, have profound implications not only for our art but for our culture as well.

Capability

Before society at large can experience the medium of AR, it first needs the technological capability to do so. The fact that we are now beginning to cross that practical threshold is what makes the future potential of AR an important consideration for the present. Sterling’s manifesto is right to note that computing and connectivity [have] become pervasive and embedded into the world,⁸ because it is that development that will lay the groundwork for ubiquitous AR. The sheer amount of computational ability that we all carry with us each day has reached a critical mass that enables some truly amazing experiences. And by application of Moore’s law, which holds that processing power doubles roughly every 2 years, we can expect that potential to grow exponentially.

We are at the point where each step forward in computational ability promises an entire new layer of digital–physical interaction. Brian Mullins, president and cofounder of the industry-leading company Daqri, has often noted in his public presentations that it was the addition of a compass, accelerometer, and enhanced processing power to the iPhone 4 that allowed AR apps to make the jump from simply detecting QR codes and other 2D markers to recognizing three-dimensional objects and overlaying data onto them in four dimensions.

That device hit the market in June 2010 – 3 short years before this writing. Now Apple considers the iPhone 4 too antiquated to sell any longer. Virtually all of the devices the Elon Musk used in his Iron Man-esque YouTube video – e.g., the Leap Motion sensor and the Oculus Rift headset – have been introduced in the interim. If we have gone from relatively simple iPhone apps to gesture-controlled rocket design in 3 years, what will be possible in another year? In 3? Ten?

Follow the money

The progression of digital technology to date and the multiple visions of our augmented future from people who understand the technology are persuasive evidence of AR’s imminence. But these are not the only indicators. Investors and market watchers are also increasingly placing their bets on AR.

SmarTech Markets Publishing’s revenue forecasts

In 2013, SmarTech Markets Publishing released a report called Opportunities for Augmented Reality: 2013–2020. Despite identifying a number of practical hurdles that must still be overcome, SmarTech believes that there is enough in this analysis to suggest a strong and profitable future for AR.⁹ Some of the reasons SmarTech offered for this conclusion include that:

• The mobile industry is already huge and sophisticated.

• AR is already out there as a deployed technology to some extent.

• It also fits in well with other important trends such as the rise of tagging/RFID, NFC, location-based services, image recognition, and visual search.

• Strong business cases can be made for AR using today’s technology.

• Many of today’s backers are firms with deep pockets.¹⁰

The report lists several well-known companies in the AR industry that had already received recent venture capital investments of between $1 and 14 million, including Layar, Tonchidot, Total Immersion, Ogmento, Ditto, Wallit, Flutter, GoldRun, CrowdOptic, Blippar, and Wikitude.¹¹ In June 2013, shortly after the release of SmarTech’s report, Daqri announced $15 million in private investment to support its own AR platform.¹²

These funds are called investments for a reason; the people making the investments expect a return on their money. The SmarTech report gives good reason to expect one. It forecasts revenue in the AR industry to exceed $2 billion by 2020, and to surpass $5 billion just 2 years later.¹³

Tomi Ahonen’s predictions on AR usage

SmarTech is not the only prognosticator to talk in numbers of this magnitude. Tomi Ahonen is an oft-quoted consultant and author of 12 books on the mobile industry. He characterizes AR as the Eighth Mass Medium of human expression, following print, recordings, cinema, radio, television, Internet, and mobile.¹⁴ He has followed the growth rates of these technologies, and come up with his own forecasts for the rate at which society will adopt AR. Ahonen predicts 1 billion users of AR across the globe by 2020, with that number climbing to 2.5 billion by 2023 (Fig. 1.5). Translated into revenue, these figures are more optimistic than SmarTech’s prediction.