University Of Portsmouth

School of Creative Technologies

Final year Project undertaken in partial fulfilment of the requirements for the BSc (Honours) in Entertainment Technology Virtual Replay in television and broadcasting: are broadcasters reusing old technology?
by Chris Bennett 619311 Supervisor: Gary Bown Project Unit: CT6CTPRO March 2014 Project Type: Study

Abstract: This dissertation explores the recent technological advances in live television replays. Two companies Replay Technology in America and national broadcaster NHK in Japan - are trialling systems that create slow motion replays in a style reminiscent of the Bullet Time sequences used in the 1999 movie, The Matrix. These virtual replays are set to enter the home television market in the near future and will offer the user a more unique viewing experience. Documenting the history of Bullet Time and sports instant replays and utilising interviews with industry professionals, alongside secondary research, shows the development of these systems, and allows this study to conclude that Bullet Time is still very much in use, but has evolved from its original form. This study also demonstrates that the technologies are still changing to meet the demands of a new generation of users, who grow ever more sophisticated in their home viewing expectations. Recommendations for a successful implementation to the home market have also been made.
1

I.

Acknowledgements I would like to thank

Farley Brian, Tim Kilgour, Diego Prilusky, Oren Haimovitch Yogev, Nick Trumble, Joe Leveson, George Cook, Gary Bown, Kensuke Ikeya, Don Kennedy, Yuichi Iwadate and James Mackie for their contributions to this study. I would also like to thank my parents, David and Julie Bennett for their support.

II. I. II. 1.0 Acknowledgements Table of Contents Introduction 1.1 1.2 1.3 2.0 2.1 What is Bullet Time?

Table of Contents

Page Number 2 3 4 4 7 8 8 9 9 11 12 13 14 18 19 20 21 23 24 27 27 29 33 36 37 38 40 48

1.1.1 Classing the Visual Effect Scope and Objectives Hypothesis History of Bullet Time 2.1.1 The First Uses of Time Slice as a Visual Effect 2.2 2.3 2.1.2 Similar Techniques to Time Slice From Time Slice to Bullet Time: the 20th Century and a move to Virtual Cinematography The Significance of Time Slice within the Slow Motion Field 2.3.1 Moving Forward: the next step into Live Television 2.3.2 The Importance of The Matrix 2.3.3 The Demise of Time Slice 2.4 2.5 3.0 Next Generation Time Slice: 3D Modelling Marketing the Virtual Replay

Literature Review

2.5.1 Development for Commercial Enterprise Methodology - Sources and Methods 3.1 3.2 Authenticating and Sourcing Information Planning the Project

4.0 5.0 6.0 7.0 8.0

Findings Conclusions 5.1 Recommendations Evaluation Reference List 7.1 List of Figures Appendices 8.1 8.2 Email Communications Interview Extracts

49
49 57

1.0

Introduction

The first live television bullet time replay was created in 2001; the 'EyeVision' system (Carnegie Mellon goes to the Super Bowl, 2001, para 2) used more than thirty cameras to create a Bullet Time replay of highlights within Superbowl XXXV. Over a decade later, companies in America and Japan are releasing systems for live broadcast that are very similar to EyeVision. Will this cause an increase in television manufacturing and on-demand service sales, similar to those experienced with the introduction of High Definition television? Similar initiatives, such as 3D stereoscopic television, have seen a fall in popularity (Anthony, 2013, para 2), and there is a danger that the introduction of new slow motion Replay Technology will be nothing more than a novelty, grabbing the attention of the user for a short period, before becoming an expected standard to existing on-demand service packs. The complex nature of these systems means there are few companies that manufacture live television Bullet Time replay systems, and due to the commercial sensitivity of the work involved, these companies do not tend to share information. There are also few official publications or research papers available in this area. Therefore, research has been conducted in the form of interviews, which have provided first hand insight into the industry that both creates and utilises these systems. 1.1 What is Bullet Time? This dissertation will look at the development of a video camera technique known as Bullet Time, also commonly known as frozen time, flow motion or time slice (Argy, 2001, para 1). The effect relies upon one simple technique - using multiple cameras to create a smooth, slow motion replay of an object. Figure 1 shows a simple break down of a subject and the cameras. As each camera takes a shot in sequential order (1 -7 in this example) the perspective of the viewpoint changes, giving an alternating view of the subject. Cutting from one camera to the next in a split second creates a dynamic movement of this viewpoint, and adjusting the interval of time that each camera takes a shot, allows the filmmaker to change the speed of the sequence. A smaller interval time creates the slow motion effect now widely known as Bullet Time.
4

Figure 1. A simple example detailing the arrangement of cameras in a bullet time setup. This differs from a typical camera setup where a camera will record movement of a subject from one fixed viewpoint. Bullet Time rigs are more advanced and allow for direct manipulation of both time and space within the same sequence and the end result is a fluid 'pan' around a slow moving or stationary subject. A pan is a basic video move and can be described as the "sweeping movement of a camera across a scene," (Rouse, 2007, para 1). The audience would be treated to a full view around the object as opposed to the traditional fixed plane view. Pans are limited by the speed at which a single camera can be moved, whereas Bullet Time has many cameras already in the desired position which gives the director a way to slow down or 'stop'

time, creating a continuous fluid shot tracking around any given subject, and tying the footage together into one seamless dynamic shot. As the Bullet Time effect only features this slow motion panning view it is not always necessary to use a bank of cameras synchronised together. There are other ways in which the Bullet Time effect can be reproduced; Rober (Rober, 2013) re-created the concept in the manner shown in figures 2 and 3. He moved one camera around the image at a high speed, as opposed to the image moving between the multiple static cameras. He created a small setup that used a ceiling fan motor to spin a single camera and its background around a stationary subject. When played back, the background and camera appear to be fixed, and it is the subject that gives the illusion of movement. This technique is inexpensive and replicates the effects of Bullet Time although the effect is very rough and limited in its outcome, only able to capture footage of small subjects that would fit between the background and camera.

Figure 2. Screenshots from Rober's video, showing the spinning motion of the setup.

Figure 3. A simple illustration detailing Rober's experiment in more detail. 1.1.1 Classing the Visual Effect The Bullet Time style can be broken down and sub divided. There is the original format - an analogue camera system that utilises a bank of cameras to shoot a subject, which is known as Time Slice (Time Slice, n.d., para 1). The frames are then stitched together creating a short, rough sequence of a subject. However, as no cameras are exactly alike, both digital sensors and film cameras produce slightly differing colours and quality even if they are exactly the same make, and these discrepancies are exaggerated the longer the sequence. Further problems are caused by the gap between each camera when mounted on a rig, as due to their size, cameras may be positioned a few centimetres from one another and are difficult to align perfectly. When the frames are added together, there is a noticeable jump from one frame to the next; this look can be utilised to create stylistic art films but is not regarded as suitable for large budget productions due to its rough nature. In order to maintain continuity throughout this dissertation, this analogue technique shall be referred to as 'Time Slice.' The second format Bullet Time, uses the traditional camera setup of Time Slice but incorporates computer graphics and modelling to create a smoother, more polished sequence. Unfortunately, the added integration of computer graphics significantly increases both the
7

overall project cost and production time, limiting the technique to large budget companies. This blend of Time Slice and computer graphics was first used in the Warner Brothers Matrix films (Wachowski, Wachowski, 1999) and the term Bullet Time became an instant success. From this point on, Bullet Time will only be used in reference to the Warner Brothers, Matrix films technique, although many sources and quotes will refer to it more generally. 1.2 Scope and Objectives This dissertation will focus on live television and the future of Bullet Time within the entertainment industry. Two companies are currently making interesting breakthroughs with instant replays incorporating Bullet Time techniques: American company Replay Technology with their freeD system (Technology, n.d.), and Japans NHK (Nippon Hs Kykai, otherwise known as the Japanese Broadcasting Company) who have released a multi camera system (Press release, 2013, para 1). Both developments have come to fruition within the last twelve months and are currently being trialled in live television sports scenarios. This dissertation will explore and analyse the technologies that are in use today, showing how their development has been heavily influenced by historic technologies and novel viewing experiences. 1.3 Hypothesis 'Manufacturers are looking for specific technologies to promote; older techniques such as Bullet Time would be an appropriate technology for television and with further development could see an integration into the next generation of television sets.'

2.0 2.1

Literature Review History of Bullet Time

Although not known by this term, the first true experiment with Time Slice was instigated by Eadweard Muybridge in 1878 with his study 'The Horse in Motion' (Horse in Motion, n.d.). As the human eye cannot perceive fast action events and analyse the smallest of details, a popular debate at the time was whether all four hooves left the ground when a horse was galloping. Muybridge used twelve cameras and an elaborate series of electrical mechanisms to trigger the shutters in sequence, with thousandths of a second intervals between each. An image taken from Muybridge's sequence can be seen in Figure 4.

Figure 4. Muybridge's first Horse in Motion work. Fortunately the set of cameras worked and the importance of the achievement was instantly realised, reported the next day in an article that described Muybridges discovery as : Second only, among the marvels of the age, to the wonderful discoveries of the telephone and phonograph (How a trotting horse, 1878, p. 1).

Muybridge's high speed photographs demonstrated a new actuality: photography could represent aspects of the world in motion that were beyond the capacity of human visible perception (Braun, 2012, p. 143.). Muybridge changed perceptions of reality with these series of twelve photographs, capturing time in exposures thousandths of a second long. Time could now be recorded and manipulated to describe events, though at first only in the form of a simple moving image. In 1879 he invented the Zoopraxiscope (Motion picture pioneer, 2011, para 5), a spinning disk with small animated images which gave the illusion of movement. It was the first time people could see animations come to 'life' and paved the way for greater developments that would eventually lead to the start of basic motion pictures. In 1888, in America to give lectures on 'animals in motion,' Muybridge met Thomas Edison and the two decided to work together. This event was the inspiration for Edison to improve on the system developed by Muybridge, inventing what has become the modern day movie in the form of a filmstrip. (Eadweard Muybridge, n.d., para 4) Muybridge's initial experiments were effectively the birthplace of the motion picture, and his meeting with Edison, a significant milestone in its development. It was a natural progression to develop the method and create the movie pictures of today. In 1940, another form of camera technique was developed - super slow motion. Although it was not a direct form of Time Slice, it did help to develop high speed photography and motion cameras. Harold Edgerton (1903 - 1990), an esteemed professor and engineer, utilised many different techniques to slow down time. He even created slow motion footage of bullets using Xenon strobes to flash, capturing the bullet in flight. This technique worked well, as the flash of a strobe exposes a piece of film creating a snapshot of the subject, as opposed to a blurry exposure (Harold Edgerton, 1999, para 3). This era marked an important step; the possibility to use Time Slice to slow down and analyse what was really happening in the everyday world, rather than making inexact visual observations. Both Muybridge's and Edgerton's work was important as they used new techniques to gain an understanding of how things worked around them. Time Slice helped to prove or disprove many theories of the time and Edgerton's slow motion work was initially used to conduct scientific experiments on extraordinarily fast objects, such as bullets and explosions.
10

However, it also marked a change in the history of Time Slice and led to the start of visual effects. 2.1.1 The first uses of Time Slice as a Visual Effect In 1967, a Japanese anime cartoon called 'Speed Racer' (Yoshida, 1967) was released and the opening credits featured a style that had previously been unseen in cartoons - two seconds of the Time Slice effect (Figure 5).

Figure 5. Shots of the Speed Racer Bullet sequence in effect, less than one second in duration.

It was a very rough cut that only featured a few frames panning around a car and the lead character but this was the first use of Time Slice in the entertainment industry (Rolli, n.d., para 5). This small clip helped to establish the technique in the Japanese comic industry and inspired future development of the visual effect within the world of film. Traditional animation gave the look from one fixed point of view, but this short sequence proved that it was possible for this hypothetical camera to move too. Another important step was realised in 1980, when Tim Macmillan (Time Slice, n.d.) at the University of Bath created basic experiments that he would later come to call 'Time Slice.' Macmillan constructed a 360 degree loop that featured hundreds of pinhole cameras. When

11

the film in each camera was exposed to the light from a flash strobe, it captured a slice of time (BBC Tomorrows World 1993, 2010). This is the process technically known as 'Time Slice.' This was the first time cameras would be used to create a 360 degree still shot capturing real life subjects. At this stage the result was rough, and there was no stability within the shot; colours and slight movement in each frame limited the initial Time Slice work to the art world, with video producers unwilling to use an effect that would not suit their films. Although an important moment for the technique, there was still some development needed, and this came in the form of 'View Morphing' (Seitz & Dyer, n.d., para 3). Used heavily by Michel Gondry (Michel Gondry, n.d., para 3), the method uses shots from two cameras distanced slightly apart. Computer software then generates a predicted sequence of shots of the space between the two cameras - these added shots are known as 'interpolated' frames. Gondry used this interpolation method to create a bizarre new effect called 'View-Morphing. It was very popular and Gondry became well known for his creative work in the entertainment and advertising industry. Renowned visual effects supervisor, John Gaeta, referenced Gondry's work for his inspiration on The Matrix film in 1999 (Wachowski, Wachowski, 1999): For artistic inspiration for bullet time, I would credit... along with director Michel Gondry. His music videos experimented with a different type of technique called view-morphing and it was just part of the beginning of uncovering the creative approaches toward using still cameras for special effects. (200 Things That, 2006, p.136) All of these experiments and techniques had a profound effect on what would become the entertainment industry: each was a stepping stone to the next, and without one, the next in line might not have been developed. 2.1.2 Similar Techniques to Time Slice There are no theoretical limits to the Time Slice effect, where it is possible to orchestrate any camera move imaginable in a full 360 degree field, traversing through both time and space. However, in real world applications there are restrictions, primarily the cost of the system and the creative vision of the director. The cost of a Time Slice rig increases exponentially with the length of sequences and the complexity of moves envisioned by the director.
12

There are many cases where the Time Slice effect has been used in production, although varying methods were used to create a related effect. Television manufacturer Phillips used a similar technique in 2009 to create an advert called 'Carousel,' used to market a new television (Berg, 2009). Using a continuous shot, one motion controlled camera on a track made its way through a hospital complex. Instead of many cameras taking photographs at split second intervals, trained stuntmen and actors simply held their poses, creating the illusion of paused time. The two techniques are quite different but do create similar results. However it is still a very expensive process, utilising cutting edge stabilised camera rigs. It would stand to reason that only large budget film houses could afford the use of Time Slice, but in recent years there has been an influx of independent film makers and individuals developing their own ways to build a rig that mimics the Time Slice technique, thereby bringing costs down. Time Slice has also featured in hundreds of films, adverts and games, all with differing results. In 2010 the surf brand Rip Curl filmed a marketing campaign featuring the Time Slice technique, using a mix of DSLR and small GoPro cameras (Masefield, 2010). 2.2 From Time Slice to Bullet Time: the 20th century and a move to Virtual Cinematography Many individuals had experimented with the Time Slice effect but before 1999 the technique was not well known. It had made a few appearances in television shows and in the cinema such as Kill and Kill (Hall, 1981) which featured a low budget slow motion sequence, but it was with the release of the film, The Matrix (Wachowski, Wachowski, 1999) that the new form of Time Slice - Bullet Time - was cast into the limelight, the effect being heavily used by the marketing department to advertise the film. The visual effects supervisor, John Gaeta, originally coined the expression 'Bullet Time,' as sequences featuring bullets held in slow motion were a highlight of the film. Eight years later the term 'Bullet Time' was registered by Warner Brothers, U.S Trademark Registration number: 3349252 (2007) and from then on the effect would usually be known this way. The Warner Brothers method featured a bank of cameras synchronised together, each camera taking a photo in series, generally with a fraction of a second gap between each shot. The time gap between each camera in The Matrix film, which used over

13

100 cameras, generated sequences up to 12,000 frames per second (Godoski, 2011). There was however, a slight problem with the camera positioning - even with the cameras packed as closely as possible there was a noticeable gap between each frame within the sequence producing a jumpy, stop motion effect. To overcome this, Michel Gondry's interpolation method filled the gap between each camera with computer generated shots. These extra interpolated frames helped to create super smooth, slow motion instead of a jittery rough cut sequence (Wilkinson, 2010, para 1), essentially using Tim Macmillan's original technique in a neater and more precise way. This was a new technique and would have been very time consuming, and therefore only feasible for a large production house with a multimillion dollar budget to spend on special effects. This was the start of 'Virtual Cinematography,' a move into a virtual world, where a combination of real and virtual objects would be seamlessly merged. It incorporated real life actors in front of green screens and the world of 3D modelling to create a virtual world with real life attributes, people moving and interacting as they would in real life. 2.3 The Significance of Time Slice within the Slow Motion Field

Slow motion is the direct relative of Time Slice, with the two techniques arguably created within the same 19th Century Muybridge experiment. Although very similar in nature, with both providing a replayed sequence that manipulates time, there is one key difference; with Slow Motion the movement of the camera is generally limited to a fixed plane of view, whilst Time Slice sequences are free to move in any direction and orientation imaginable. In 1878, it was common belief that an animal would always have "one hoof in contact with the ground" (Padgett, n.d., para 1). With the multi camera setup Muybridge had derived, it was obvious that this was not the case, as a still shot provided proof that at full gallop a horse would be completely off the ground. As well as disproving a common theory, the technique was important because it slowed down the modern world and allowed people to see the intricate movements of a fast moving object. Muybridge's work "ushered in a new kind of photography, one which captured the invisible and showed us what we cannot see with the naked eye" (Edwards, 2010, para 1). His work altered the perception of reality, as he had essentially created the first slow motion replay - a significant moment in history because, from this point
14

on, people would be able to see subjects moving far more slowly, allowing the human eye to see the previously unseen. Scientific observations could now be made on the course of a moving object. It is a fair assessment to conclude that in the late 19th century the Muybridge experiment came out of necessity and a functional need to slow down the world led to a breakthrough. This work continued, with scientific experiments throughout the 1900s, with notable work in the military. In 1950, the Magazine 'Popular Mechanics' featured an extract detailing the American military's slow motion, explosive video testing. Their camera was able to create photographs at a rate of "100 Million a second" (Windsor, 1950, p. 158). There was not much more information retrievable from the extract but it does provide another insight into the capabilities of slow motion and its ability to study the world around it. Cameras have continued to progress in the last century and large digital sensors can record, in vast resolutions, the minute details of fast action subjects. These developments have formed important breakthroughs with life-saving applications; for example, the automotive industry uses slow motion replays extensively, conducting crash tests which undoubtedly help to drive improvements - "Video was shot in high definition super-slow motion, allowing us to see every detail of a vehicle's passive safety mechanisms as they are employed" (High-Def, 2013, para 3). Referring to the system devices such as air bags in cars, this source gives added credit to the significance of Muybridge's work, as it would be very difficult to monitor these automotive systems without his technique. Slow motion has become a very important technology and is now regularly used, in every possible setting imaginable, from the entertainment industry to scientific recording. Slow motion has improved our understanding of the modern world. During the 1950s and throughout the development of slow motion cameras and practices, there was a change in principles. Similar to the theories of modern Architecture, when ideology changed with the idea that 'Form follows Function,' a term coined by Louis Sullivan (1896, p. 408), the entertainment industry started to use Slow Motion for visual effects. It appeared as a stunning aesthetic form on camera and the key moments of a film could be elongated which would help build suspense. There are many unsupported articles of information that try to

15

place the origins of Slow Motion visual effects, but according to Stephen Prince the short, slow motion scene featured in the 1954 film, The Seven Samurai (Kurosawa, 1954), "Decisively changed modern cinema" (1991, p.348); a few seconds of slow motion occur when a warrior is murdered and his body falls slowly to the ground. Modern television and filmmaking has adopted the slow motion visual effect and it is frequently shown on a variety of different media sources, from the high tension boxing sequences in Raging Bull (Scorsese, 1980), to the complex action scenes of 300 (Snyder, 2006). Live sports games frequently use slow motion to replay highlights that are significant to the game. The slow motion draws out the length of the sequence allowing the audience to see what is really happening in more detail. The internet provides an example of how slow motion video is adored by its audience, with websites and online blogs setup to demonstrate amazing forms displayed at a fraction of their real time speed. It can be seen that Slow Motion is a powerful visual effect that will continue to be used in the film and television industries: From a transhumanist perspective, slow motion videography is a technology that allows us to extend our senses beyond their natural capacities. It allows the revelation of data hidden in the folds of time, just as a microscope allows us to appreciate the wonders of a fly's wing or a microbe's choreography. (Eagleman, 2009, para 5) David Eagleman, a neuroscientist and writer with a background in the biological studies of visual illusions and time perception (Time Perception, n.d.), provides a credible reason for the admiration Slow Motion video receives, showing why it has maintained its popularity and continued use. Analysing the features of the modern world is the key to the popularity of Slow Motion; providing a dissection of the smallest details gives a hidden view that a normal eye would never be able to see. There is a great significance to slowing down time in motion pictures and it can be used to achieve a variety of effects. As Jones (2005, p. 149) points out, varying frame rate and shutter speed in the camera can affect the apparent speed - and enhance the excitement - of an action scene, which is something that directors may want to ultimately control and manipulate. Sanders (2014, para 5) agrees with this theory when he states that:
16

Slow motion can also be used to raise the dramatic tension in a scene. If its played back at slow motion, you keep the audience at the edge of their seats longer, as they crave to know what happens next". Directors rely on tension to keep the viewers interested - it helps to create an interesting film or sequence. The Time Slice effect used in today's industry is a very close relation of Slow Motion, with the added dynamic movements of Time Slice giving it the edge in the entertainment industry. The following extract suggests the importance of slowing down sequences to draw out tension in a scene or to reveal something visually obscure: Camera movement is relatively restricted, if not completely static, during conventional slowmotion sequences. In bullet time, on the other hand, it can be accelerated considerably. The description of the technique should perhaps be completed by adding that bullet time is frequently employed in order to visualize processes (e.g. a figure dodging bullets) that would be hardly visible, if at all, in real time. (Kindt & Muller, 2003, p.37). This quote exemplifies the importance of Bullet Time, as it highlights the reason Time Slice is significant within the slow motion criteria. It suggests that slow motion is necessary to properly view these high speed scenes that would normally be missed. Kindt and Muller also provide a notion that Time Slice techniques help to create the dynamic sequence flow that regular slow motion would not be able to generate, when they refer to typical slow motion as 'restricted' and 'static'. This additional movement of the camera is important because where a traditional camera would be fixed in place, the audience is now provided a more detailed view. The pan of the Time Slice camera reveals far more of the scene than other traditional techniques, enhancing the story and providing an arguably more entertaining shot for the audience. With the two technologies - Time Slice and Slow Motion - so closely linked, it is crucial to bear in mind the fundamental differences. With both techniques developing together from essentially the same source, each has its own specialised and applicable field in the modern era. Slow motion is very functional, it has a wide range of uses but is not as stimulating as the more visually appealing sequences generated by Time Slice.
17

2.3.1 Moving Forward: the next step into Live Television As the visual effects industry moved forward so did the world of television. Live action instant replays were developed in 1955 and helped to change the live television sports industry (Saunders, 2013, para 16). Instant replay was devised to re-show a significant moment, allowing the audience to go back in time and review the importance of something that had just happened. The first trial use of this technique was December 7th, 1963: Tony Verna, a hotshot 29-year-old director for CBS Sports, brought a 1,200-pound videotape machine to Philadelphia's Municipal Stadium. After several failed attempts early in the game, Verna was finally able to get the machine to work... However, the black-and-white replay was indistinguishable from live action, and there were no on-screen graphics informing fans what they were seeing. (Saunders, 2013, para 10). Despite this faltering start, instant replays have become an everyday technique used heavily within the sports broadcasting market. It was very significant and allowed the audience to reevaluate something that may have happened too quickly for the eye to properly review. It was the first time the audience at home could start to make up their own minds and see what was really happening. It was not for another 38 years that the instant replay and Time Slice methods were brought together - at the 2001 Superbowl XXXV (Carnegie Mellon goes to the Super Bowl, 2001, para 3). Inventor Takeo Kanade devised a setup using more than thirty cameras, creating a rough but functional system, coined by CBS (Columbia Broadcasting System) as 'EyeVision' (Carnegie Mellon goes to the Super Bowl, 2001, para 2). A selection of shots shown in the broadcast can be seen in figure 6.

18

Figure 6. A selection of shots demonstrating the Bullet Time sequence of 'EyeVision,' lasting less than a second in duration. The system had taken inspiration from The Matrix film (Wachowski, Wachowski, 1999) and a research budget of approximately 2.5 million dollars (Martzke, 2001, para 9) was put aside to trial a live version. When analysing live television from a theoretical perspective it would seem that incorporating Time Slice instant replays would begin to create a logistical nightmare, realistically requiring a large amount of computing power and co-ordination. The original large multi camera rigs first developed by companies such as Warner Bros are not practical for live broadcast replays. These rigs were huge, fixed in place and focused to one small subject area; to equip a stadium with this kind of coverage would be a hugely expensive and very laborious process. Although the 'EyeVision' system worked well it did not re-appear, as the cost and set up time prohibited further expansion. It was another decade before Time Slice would reappear on live television in any form.
2.3.2

The Importance of The Matrix

The first Matrix film (Wachowski, Wachowski, 1999) was important because it was the first time special effects were used to further the narrative of the film as opposed to just being a great effect in itself - something that was rarely achieved previously (Gills, 2005, p114). The films
19

premise is that the world around is not all that it seems and seeks to help the characters open their eyes to the reality of the situation. According to Lisa Purse: Bullet-time is, in its very dynamics, an explicit expression of the need to see everything, to see the whole. It is an expression both of the films mastery over the visual its ability to show everything but also the spectators mastery of the visual his or her ability to see everything. (Geoff, K, 2005, P159) Purse highlights the relationship between the narrative and effect. This is significant because the narrative and effect would merge; the audience would not just see a stunning visual effect but the flow of the narrative would be maintained. The effect had a serious impact on the special effects industry and actually helped to sell the film and its subsequent sequels, not just create an appealing on-screen effect. Farley Brian, a freelance special effects producer, highlights how new effects can really influence the popularity of films: In regards to The Matrix, the story of the techniques used actually sold the film. It is possible the film would have sold nowhere near as well without the seed that is Bullet Time. We had a project which didn't take well, only hitting a small audience, but it had new technology which the media picked up on and it marketed itself! (F.Brian, personal communication, November 21, 2013). With any major new technology there is an associated hype. If the effect is very noticeable, the audience may endeavour to find out more - how was it created etc, little details that enhance their understanding and satisfy their curiosity. The associated press will typically help to market a film, writing reviews and articles that explain more about what is really happening. 2.3.3 The Demise of Time Slice Almost as suddenly as it sprang on the public scene, however, bullet time seemed to wear out its welcome... Put to scullery work as a sportcasting aid in the 2001 CBS Superbowl, parodied in Scary Movie (2000), Shrek (2001), and The Simpsons, bullet time became first a clich, then a joke. (Rehak, 2007, p28). The popularity of Time Slice took off between 1999 and 2003, featuring in practically every media field possible. Cartoons, games, advertisements and television all rushed to feature it, and with its overnight success came its quick decline. According to John G. Cawelti:
20

One can almost make out a life cycle characteristic of genres as they move from an initial period of articulation and discovery, through a phase of conscious selfawareness on the part of both creators and audiences, to a time when the generic patterns have become so well-known that people become tired of their predictability. (Rehak, 2007, p29) When an effect becomes so widely used and predictable it ceases to be 'special.' With few directors wanting to use it again, Time Slice quietly 'died', in a similar fashion to the overused 'Morph' effect used in the early 1990s (Rehak, 2007, P29): Morphing digitally transforming one image into another was a flashy, obvious and very popular technique. But when it began to seem dated, no one wanted to feature the effect. In reality, morphing never went away; it just was used more subtly. We now use morph as a way to smooth out elements, but no ones aware of it. (Argy, 2001, para 5). The creators of 'EyeVision' knew that it was important to not overdo the effect: "We want to use it when it makes sense, not cram it in or overuse it." (Martzke, 2001, para 12). Even with this limited playback it was clear that Time Slice had to succumb to its own success. Its overuse and constant appearances in adverts and cartoons such as 'The Simpsons' saw a huge drop in popularity, with no-one looking to employ a technique that had saturated in the broadcast and entertainment field. 2.4 Next Generation Time Slice: 3D Modelling

Point cloud systems are an example of how Time Slice and 3D modelling can work together in unison. Typically using a 3D scanner to analyse a large number of points on the surface of an object, the relative distance of the points from the scanner allows for a precise map of the object to be established. This technology is having a profound effect in the many different fields in which it is employed, allowing designers to quickly map an object or place with high precision. Time Slice can be used alongside this point cloud method, with multiple cameras providing a 360 view of a subject. Compositing the images with point cloud software generates a fully three dimensional model of the subject. Tim Kilgour believes Time Slice and Point Cloud generation have become inextricably linked:

21

When used in film production, this system [Point Cloud technique] creates highly detailed composites of a subject which could be modelled quickly; it is less time consuming and costly than a visual artist building a 3D model from photographs. When asked if the Time Slice method was no longer being utilised: No I don't think so, we use it in different ways. Its evolved and mixed with other technology (T. Kilgour, personal communication, November 21, 2013). The implications of this technology are vast, allowing for high precision and fast three dimensional rendered models. This is an improvement on previous techniques as typical models would require a long time to process and would be created by a developer from photographs and measurements. Point clouds have generally been limited to infrared light or laser based systems which are used to create points of data. An emitter sends out a beam of light which bounces off an object, is received and subsequently analysed. The measured points are then modelled giving a detailed three dimensional surface. Replay Technology in America has developed a very similar method for their 'freeD' system but the company utilises ordinary camera sensors. Technology officer Matteo Shapira spoke about the system: We use synchronized feeds of high resolution video. The video then gets processed using our algorithms to create a 3D database of voxels. After the freeD database is created, we had built an interactive real-time rendering engine that allows viewing of the captured scene from any desired angle (Carroll, 2013, para 5). Multiple cameras are used to map 'voxels' in space and create a full composite (Bringing 'The Matrix' to the NFL, 2013). A voxel is a measurable unit that defines a point in space; similar to a 2 dimensional pixel, it holds an extra Z co-ordinate which is necessary to translate and map a point in three dimensional space. (Rouse, 2007). The system is very quick, only taking up to a minute to create a composite, allowing for use in live television broadcasts. The render can be panned around or slowed down making for much more interesting replays. According to Oren Haimovitch-Yogev, CEO of Replay Technology: You dont have to have a camera right in front of the scene. So we can have multiple angles and endless choices, and we can run the same scene in different ways. As a result, freeD also offers a unique way for fans to view controversial calls, creating a 3D effect and depth of field on a 2D plane. (The next big thing?, 2013, para 9)
22

This is important because it gives the broadcast operator much more control over the scene which can be actively manipulated and reviewed. This fully interactive system is very similar to the playback system of NHK but is more powerful. Having created a smooth slow motion replay of the scene it harks back to the original Warner Bros (Wachowski, Wachowski, 1999) Bullet Time rigs. Although the two systems are loosely related, it is obvious how much technology has helped to improve both systems during the previous decade, with fewer cameras, quicker processing times and more flexibility. Comparing Takeo Kanades 2001 EyeVision (Carnegie Mellon, 2001) technique against Replay Technology's (Technology, n.d.) work provides a good way of charting the progress made in the field. EyeVision was rough and jumped from cut to cut, with many cameras slightly out of alignment. However this was new ground for live television replays so was to be expected from a new technology. Although little has happened in the last decade that built on the previous success of EyeVision, without this first trial system the more promising freeD system may never have come to fruition. 2.5 Marketing the Virtual Replay

Technological trends come and go and those that can make money are heavily marketed. They are typically trialled in the cinema and, if successful and suitable, are integrated with television sets. It is worth analysing the recent push by manufacturers and service providers for the reintroduction for 3D stereoscopic film. This saw the mass production of new stereoscopic movies and television shows, even a new device for the home - the '3D television'. Marketed as the future of TV, 3D sets were sold to the public and although at some initial cost, seemed popular. However, according to Mark Harrison of BBC North, "3D is dead" (Svetlik, 2013, para1). Whilst live replay technology is visually appealing, is it powerful enough to warrant investment from television manufacturers and the services industry? These industries are constantly looking to forward new technology that will encourage home users to replace their existing devices. The developers at Replay Technology are keen to introduce a freeD system to the home user and it would seem that freeD may be marketed in a similar fashion to 3D, something that Matteo Shapira, the Chief Technology Officer of Replay Technology, believes will happen:

23

Our next step in our development pipeline is to allow the home viewers themselves to move around freely within the captured scene. What is now reserved for an event's producers and directors will become the domain of every sports fan, which is allowing full interactive viewing around the scene. (Carroll, 2013, para 8). This would mark the start of another revolution in television and broadcasting, allowing the consumer to take control and become fully interactive with their television. Although the reintroduction of stereoscopic television may not have been as successful as manufacturers would have hoped, it would seem that freeD has potential. Although Shapira may describe it as a 'next step' it seems more likely that this is still a great distance away in terms of reality. Even with the pause and rewind features of today's modern televisions, it seems a stretch that a full replay system with interactive cameras is just around the corner. However, there are no obvious drawbacks such as the need for stereoscopic polarising glasses and the headaches often caused by long viewing times, commonly associated with stereoscopic television use. If the freeD system could be formulated for market, it is quite likely that it would take off in popularity. 2.5.1 Development for Commercial Enterprise

A purely revenue based approach suggests that Time Slice models are a good prospective application for broadcasters and manufacturers. An uptake in this system could quite possibly create a new era in live television and broadcasting with added interactivity for the user and a more fulfilling experience all-round. The audience interacting with their content is a popular 21st century idea - merging people and the technology around them. With the invention of the internet and the ever expanding world of online content, developers have come to realise that the products they create have to meet a target audience. Known as Interaction Design (Joshuajnoble, 2011, para 2), it is the process by which designers try to fit the needs of the people that will be using the product or software. Constant development has seen the rise of interactive games and applications as the developer attempts to create a complete immersion of the user. With the recent invention of 'Smart TVs,' the audience can connect to the web and do more than just watch standard television formats, giving the smart TV a unique selling point

24

(USP). According to Dan Brilot, Media Consulting Director for the market research agency YouGov: The smart part of a Smart TV is not the main reason many people are buying them at the moment; its more about future-proofing their TV set in the same way that lots of people did when they bought HD TVs. (Brilot, 2012, page 2) Brilot demonstrates a powerful point, that people are merely buying new televisions to future proof. With consumers not necessarily worrying about the technological aspects, it would seem that manufacturers simply need to create a series of television sets that would convince a consumer it was worthwhile investing. These USPs help to sell products and generate added revenue for manufacturers, who in turn advance the technology and improve the products, generating new profit streams and revenue, ultimately following the standard product life cycle developed by Raymond Vernon in 1966 (Vernon, 1966) see figure 7.

Figure 7. The product life cycle devised by Vernon (1966)

25

As markets start to become saturated with a certain product it falls into decline. In the case of the black and white analogue television, the market had become saturated, there was no reason for members of the public to replace their current television sets with out of date black and white TVs and so manufacturers let the technology fall out of popularity. All products and technologies follow this model, and although manufacturers are able draw out the maturity phase with upgrades and other promotions, eventually the product will fall into decline and another will need to be developed to take its place. As soon as a technology comes into saturation and starts to decline, it is up to the producer to promote and advance another system, either completely new or a spin off model. "This is an experimental field, a lot of the experimental guys are out in the cold as no one cares about creativity, it's all done to generate profit and hit a target audience" (F. Brian, personal communication, November 21, 2013). This is why Time Slice-style interactive replays for television are a potential revenue stream for broadcasters. Rarely seen before and able to add extra versatility and interactivity, it is a sensible conclusion that the system will be adopted. Whether it becomes an add-on piece for current standard sets or is manufactured as a new television set, remains to be seen.

26

3.0 3.1

Methodology - Sources and Methods Authenticating and Sourcing Information

This study utilises information from a variety of sources, all of which can be broken down into two sub classes - primary and secondary research. Both classes are very significant and it is important to realise both the importance and limitations of each. Primary research is a form of first hand data collection, from interviews, surveys and experiments. It is more time consuming and difficult to retrieve, however the researcher conducting the research is able to identify any problems with errors and is aware of any bias, something that is not readily published in secondary research reports. Secondary research is quick and easy to obtain, simply defined as the collection of data from pre-existing research such as internet sources and books. Although vital to providing background context, is it much harder to authenticate results and check the credibility of sources. Secondary research initially played a large part in this dissertation; readily available on the internet and from press releases, it provided a good starting point to set the foundations of the project. Creating a context and assessing each technology in detail provides a rich review that aids insight into this industry. Historical information was documented in abundance; the university library is a good source for books and research papers which were complimented by use of the internet. However, it is important to be aware of inaccuracies and media bias that exists on the internet; retrieving the original sources of information was necessary in order to validate any information. Due to the specialised nature of the subject matter, there are only a few people in the television and film industry who would know about the developments of virtual replays in any significant detail, which compounded by the commercial sensitivity surrounding new product developments, meant few publications or research papers exist on the subject of Time Slice technologies. The next step was to locate and contact industry professionals in order to obtain sources for primary research. This was done via networking, contacting companies and through the use of
27

social media such as LinkedIn. Interviewing sources would provide a rich source of data from which to extrapolate key points. Unfortunately, as the interviewees were located in the UK, US and Japan, the project was limited to just two face to face interviews. After a number of months networking, two participants were contacted who were willing to be interviewed. This initial primary research would create establishing points on which to further explore the studies hypothesis. The two industry professionals in question were; Farley Brian, a Senior Digital Producer (F. Brian, personal communication, November 21, 2013) and Tim Kilgour, a Senior Designer (T. Kilgour. personal communication, November 21, 2013). Both have dealt with bullet time techniques and were aware of the industry field in great detail. Their previous experience provided a rich source of primary data that lends credibility to the overall nature of the project. Further research involved email correspondence with authors and developers from around the world: Bob Rehak, professor and chair at Swarthmore College, author of 'The Migration of Forms: Bullet time as Microgenre' Joe Leveson, Technical Director for Prime Focus Film Kensuke Ikeya, developer of the NHK multi camera system Oren Haimovitch Yogev, CEO of Replay Technology Diego Prilusky, Creative director of Replay Technology Don Kennedy, Media Producer DigInfo News.

The formulation of interviews was very important as they provided a further source of qualitative research. There are two types of research: qualitative, a way of attempting to interpret the things that people say, and quantitative, which uses statistical data from questionnaires to quantify meaning.

The open ended questions associated with qualitative research can be used to allow the interviewee to answer in their own words. "Although time consuming to conduct, qualitative research tends to offer forth a wealth of varied information on a small case or set of cases over a broad set of data" (Jasonhopper, 2011, para 4). This is important because the interviewee is
28

not limited in their response, allowing for a more honest opinion and answer, and occasionally something is said that the interviewer did not think to ask. This is why qualitative research based interviews were the best method of obtaining results for this subject. Within a specialised field it is necessary to ask the people that use the technology how it works and transforms the world around it. A member of the general public would not usually know how Bullet Time effects are created and would add little useful insight to the investigation. This makes surveying the general public redundant with respect to this dissertation. To improve reliability, time was taken to design each question asked in order to avoid imparting biased questions and answers: Respondents give inaccurate answers for a number of different reasons. They give inaccurate answers both consciously for reasons of their own, and also without any conscious realization that the information they are giving is inaccurate. (Brace, 2008, p 195.) It was important to maintain the credibility of this investigation as, with such limited information available, it was necessary to objectively assess all sources. A biased or leading question could have affected the outcome of the investigation very easily. Avoiding sources from possibly biased websites, such as Wikipedia, helped to eliminate possible unreliable sources. Due to the distances involved with certain contributors, some of the interviews were conducted via email, a medium which does not typically provide a strong source of qualitative feedback. Very often the responses received are closed and only answer the question asked in a very basic form. However, the email exchanges associated with this dissertation have been both insightful and beneficial. 3.2 Planning the Project

In order to gain the information necessary to complete this project, it was important to start planning from an early date. Initially starting in July 2013, after the first articles about NHK's multi camera work (Press release, 2013, para 1) were released to the media in early June, tracking down credible sources was key to the project. Derek Swetnam, the author of a popular
29

book covering dissertations, provided points to bear in mind during planning, suggesting that work should be written up "as soon as the material is ready" (Swetnam, 2000, 3rd ed, p18). This is important for a project of this type because any research that is left may be forgotten and excluded. With so many technologies involved it is key to detailing all of their influences and effects within the industry. Creating a basic structure for the project was undertaken with the use of Gantt chart, a common way to plan and track the progress of projects: "One of the most popular and useful ways of showing activities (tasks or events) displayed against time." (What is a Gantt Chart?, 2012, para 1) Essentially a Gantt chart is a versatile and visual tool to effectively communicate timescales and deadlines. Working backwards from two known deadlines submission and binding dates, the project was broken into three logical phases: Research identifying sources, interviews and analysing feedback Content developing the basis of the dissertation, documenting the historic and future developments Finishing completing the conclusion, abstract, references etc. This phase also included the binding and submission of the paper. For each defined phase, it was necessary to estimate how long it would take to complete each sub-component. Although this produced a very rough time schedule, it also provided a visible representation of the overall project and allowed a method to track project progress. It was important to stick to deadlines and update the charts when necessary, otherwise it would have been easy to lose track and fall behind.

30

Figure 8. Gantt chart 1, created July 2013

Figure 9. Gantt chart 2, created January 2014

31

The first chart (figure 8) was completed in July 2013 and shows an estimated 3 month period for research, with much of the writing taking place over the subsequent months. However, this is not an accurate time scale for what really occurred, as much of the research took far longer than originally planned. Researching a topic of this type with little previous knowledge is very time consuming; as one technology is revealed there is another one very closely related that also needs to be researched and understood. This continued development took far longer than anticipated and even in the last few months there was always new information being gathered. For this reason it was necessary to create another chart. Completed in January 2014, the second chart (figure 9) shows how long the research period had taken even in the last stages of the project. With a due date in March, there was still a lot to document and improve upon. Contacting various developers in order to gather more specific details on their technologies took a lot longer than expected. The second chart is more developed, with additional tasks such as binding and reference checking allocated in order to more accurately forecast the project deadlines.

32

4.

Findings

The face to face interviews with Brian and Kilgour provided strong positive correlation in support of broadcasters building on existing technologies to develop new products and services. FreeD and NHK's multi camera setup provides proof that technology is being reused but as an evolution of old technology rather than the development of a new idea. During the study's interview stage, when asked if Bullet Time was dying off, Kilgour answered: "No I don't think so, we use it in different ways. Its evolved and mixed with other tech" (T.Kilgour, personal communication, November 21, 2013). This was supported further by Brian: "Technologies follow cycles and can be profoundly hit and miss, it's a bit shocking when you look back and find these seeds," (F. Brian, personal communication, November 21, 2013). This quote supports the question 'are broadcasters reusing old technology.' Brian points out that technologies tend to reappear and may fall in and out of popularity, and goes on to refer to 'seeds' suggesting that technologies such as Bullet Time will be reused in another form or variation as a launch pad for something else. Many of the email interviewees went on to say that the technology is not just a re-emergence but an evolution. Tim Kilgour believes: The old rigid forms of Bullet Time have been developed; we now use these multi camera methods to create complex 3D models. Technology develops over time and the best pieces are cotton picked and used in new places. (T. Kilgour, personal communication, November 21, 2013). Kilgour gives further credit to the theory that technology is being reincorporated but goes on to say that it is developed further, describing it as an "evolution of sorts." The two interviews drew attention to the evolution of these techniques as opposed to simply reusing the technique developed from 2001. Both interviews produced credible evidence for a reuse of technology. At first glance there did appear to be one piece of contradictory evidence; Joe Leveson, who does not deal directly with live television, asked members of his team for thoughts on the subject: I did ask a couple of my supervisors about bullet time and according to them it's not used so much nowadays. They use what is called a Digi Double, which is a fully
33

rigged and textured model of the character. The reason for this is because the levels of texturing has moved forward quite a bit since bullet time became used in the late 90s. It's also cheaper and more freedom when it comes to planning shots. (J. Leveson, personal communication, October 27, 2013) Although Leveson is right to suggest that stand alone Bullet Time techniques have become less popular within the industry, there are examples of where it has become integrated with other methods. Replay Technology and NHK systems are evidence of this but the reference to the 'Digi Double,' a 3D model of a subject, is an indication of the move into virtual cinematography. Multi camera rigs and 3D modelling are used side by side to create one amalgamation of coexisting technologies. All of the evidence collected supports the hypothesis that companies are redeveloping and refining existing technologies and combining them with other solutions and techniques to make composite solutions Point Clouds being a typical example. This technology has helped to influence a new way of creating and viewing different dimensions with time based content. Its been re-developed from film production to live capabilities as there is a lucrative market for televised sporting events, but as technology/software engineering seems to be progressing so quickly in this field, it will soon be superseded by a much simpler, cheaper, and more versatile solution for such purposes, usually the second wave of development makes the real money. (F. Brian, personal communication, March 6, 2014) Brian predicts that despite the ongoing development of these replay systems that have only recently been configured for live television purposes, they may soon be replaced by a next generation system designed to maximise profit for companies. Brian's quote points out the underlying fact driving this development - the profitability of the system. However according to Diego Prilusky (personal communication, March 19, 2014): I think that combination of tools, fast computing, digital image, low prices of cameras with very big sensor and a very talented and versatile team working together a combination of all of those, made freeD to exist now and not a decade ago.

34

Prilusky is a developer of the freeD system and points out that development of the system was driven through technological change as opposed to the company building a product specifically for commercial enterprise. It may seem that Brian's initial interview evidence contradicts this: "This is an experimental field, a lot of the experimental guys are out in the cold as no one cares about creativity, it's all done to generate profit and hit a target audience" (F. Brian, personal communication, November 21, 2013). However freeD has only recently come into the public eye and it is important to remember that being a new system, there is some way to go before the team at Replay Technology will have a marketable product suitable for the home. It remains to be seen if manufacturers will try to implement the virtual replay within their televisions. This is the stage where the potential for commercial success will drive further development if the product is found to be suitable.

35

5.0

Conclusions

It is important to take note that although the original Time Slice method has evolved, from the original simple 19th century experiments of Muybridge and his multi camera film setup to the 3D renders of Replay Technology, it is possible to trace the process right back to its origins. With the first use of the Time Slice effect to study horses in 1878, it would suggest that the technology has progressed very slowly over the last two centuries. Up until 2012, Bullet Time was still shot in exactly the same way as the original camera setup, with multiple cameras taking one still, which was later combined to create the sequence. It is interesting that with the original work of Muybridge leading to the creation of the first motion picture, that it would take another 200 years for the method to be used again, and then only to add effects to said motion pictures. This dissertation answers the question 'Virtual Replay in television and broadcasting: are broadcasters reusing old technology.' The evidence behind the development of Time Slice replays provides support for the idea that technology in the industry is being reused but in an evolved form. There is both primary and secondary evidence to support the idea that this progression is not being driven because of breakthroughs in technology but as a natural progression of ideas and techniques. This information can be seen from both the face to face interviews and email communications. A revolution in live television and broadcasting has seen 'interactivity' becoming a significant key. Both designers and manufacturers will be looking to involve and immerse the audience as much as possible, so it stands to reason that consumers will see either NHK's bullet time system or Rapid Technology's freeD system in homes within the foreseeable future. The playback offered by these two companies is ultimately controllable and enhances the standard viewing that is currently available. It is possible that these Bullet Time techniques are another trend following a cyclical pattern, emerging for a few years before disappearing again but the promotional work done by these companies will stand them in good stead. There have been many trends in the last half century, relying on the fact that manufacturers need customers to want to replace their television sets
36

in order to drive turnover. Recent years have seen a number of advances in technology generating a drive in demand for new televisions sets: Quality of the image has substantially improved; Size of the devices has reduced in depth, but increased in viewing size; Reduction in cost; More efficient television sets.

In addition, the home user is subject to a continued marketing campaign to subscribe to new broadcasting services movies, sports, comedy etc. In 2010 there was a push by manufacturers to create three dimensional stereoscopic televisions in order to replace high definition televisions (HDTVs), but the campaign did not go well, with many broadcasters dropping 3D segments from their televised packages and overall sales of television sets falling. After the recent dip in popularity of 3DTVs it would seem that virtual replays would make a worthy replacement. As an additive package similar to a sky box or an internal system that would see a whole new range of television sets, the freeD system seems to be a likely forerunner. This could just be another technology that spends its time in the limelight, only to be dropped again when a newer more potentially profitable system comes along, but for now it seems an appropriate choice to improve the customers viewing experience. 5.1 Recommendations

It is in the best interests of the developers that have devised freeD and multi camera technologies to prolong the life of Bullet Time style virtual replays. With the popularity drop in 2003, and the loss of similar technologies such as the morph, it should be advised that these new systems are not overused. Heavy marketing and promotion would see a saturation of the market and this in turn would likely lead to a drop in public popularity. Due to the great costs of the system, broadcasters would no longer choose to feature unpopular technology and any hope of integrating the system with television sets would be lost. It is therefore imperative to retain strong patents on the system, and attempt to prevent competitors from releasing similar technologies whilst refraining from over-marketing the system. Referring back to the product

37

life cycle (Raymond, 1966) this would lengthen the introduction and growth phase. Establishing the product within the market is key, but preventing this saturation, whereby the public can readily see it demonstrated in all aspects of everyday life, is of the utmost importance. If consumers see a stunning effect every day it loses its appeal and becomes boring to watch. Further studies are necessary to gauge public perception of the virtual replay, which if tracked would help companies gather information as to whether marketing strategies are really helping to sell their products. Whilst the use of replay technology is visually stunning, it may be difficult to see how this technology, in a domestic environment, would lead to wide scale development of new television sets or specialised replay subscription offerings. Taking lessons from the failings of 'Eyevision' in 2001 and the significant drop in popularity of the Bullet Time effect in 2003, a period where broadcasters would no longer use the technique, it is important to limit the use of the freeD system in order to prevent a recurrence and subsequent loss of popularity. It may be the case that Bullet Time style replays will follow a similar course; a temporary growth phase, followed by saturation and a quick decline. Whilst it is generally accepted that the consumer is purchasing new televisions for the home, replay technology may not be seen as a must have and as such may prevent a successful drive in sales. In addition, would a broadcaster like the BBC or SKY readily invest in creating a specialist subscription channel, particularly considering the recently perceived failure of 3DTV. This is an area that needs to be studied further, as it is impossible to predict the potential success rate of a television system when it is at least a year away from release. Following the recommendations suggested in this dissertation could see that a domestic system is more than an overnight success but the overall public perception remains to be seen. 6.0 Evaluation

This dissertation has thoroughly explored the background context and leading developments of the 3D virtual replay. Due to the ever changing nature of the industry this project, in a similar fashion, has also evolved. In its earliest stages, this dissertation analysed the likelihood of independent filmmakers being able to readily produce Time Slice style sequences on a small
38

budget. However the freeD system and its potential for change within the industry was deemed to be more significant. With this large change in direction the early initial findings did not have such a strong relevance to the project. During certain interviews, such as those with Kensuke Ikeya (Appendix 9), developer of NHK, and the initial meetings with Farley Brian (Appendix 16) and Tim Kilgour (Appendix 15) questions were asked that were more relevant to the initial direction. The interviews conducted later were more appropriate but did have an impact on the time scale, with some research received as little as a week before the project deadline. In hindsight, there was not much that could have been done to alter the course earlier in the project but if this dissertation was conducted again it would be important to schedule early interviews with more specific questions targeted at the 3D virtual replay.

39

7.0

Reference List

Agisoft Photoscan Professional Edition. (n.d.). Retrieved from Agisoft company website: http://www.agisoft.ru/products/photoscan/professional/ Anthony, S. (2013, January 9). 3D tv is dead. Retrieved from: http://www.extremetech.com/extreme/145168-3d-tv-is-dead Argy, S. (2001). Frozen f/x still in action. Retrieved from: http://variety.com/2001/film/news/frozen-f-x-still-in-action-1117792382/ BBC Tomorrows World 1993: Time Slice Films Feature [Webcast]. (2010, January 19). Retrieved from: http://www.youtube.com/watch?v=fIpmUi8HI1k Berg, A. (Director). (2009). Carousel [Advertisement]. Netherlands: Tribal DDB. Box office / business for the Matrix (1999). (2013). Retrieved from: http://www.imdb.com/title/tt0133093/business?ref_=tt_dt_bus Brace, I. (2008). Questionnaire Design : How to Plan, Structure and Write Survey Material for Effective Market Research (2nd ed.). London, GB: Kogan Page Ltd. Braun, M. (2010, September 15). Eadweard Muybridge. London: Reaktion Books. Braun, M. (2012). Eadweard Muybridge. London, GBR: Reaktion Books Brilot, D. (2012, April 30). TV on demand. Retrieved from: http://cdn.yougov.com/cumulus_uploads/document/j8hi9ko29n/Smart%20TV%20%20%20Consumer%20PR%20%28FINAL%29.pdf Bringing The Matrix to the NFL. (2013, September 9). Retrieved from: http://www.tvtechnology.com/prntarticle.aspx?articleid=221089 Brown, S. (2013). From novelty to normal: 3DTV as a special effect. Critical Studies in Television; Autumn 2013, 8(3). Retrieved from: http://ehis.ebscohost.com/eds/detail?sid=f4b18717-d70a43f3-b34540

782a4f1cc771%40sessionmgr110&vid=3&hid=109&bdata=JnNpdGU9ZWRzLWxpdmU%3d#db=f 3h&AN=92009808 Carnegie Mellon Goes to the Super bowl. (2001, January). Retrieved from: http://www.ri.cmu.edu/events/sb35/tksuperbowl.html Carreker, D. (2012). Game Developer's Dictionary : A Multidisciplinary Lexicon for Professionals and Students. Boston: Cengage Learning. Carroll, J. (2013, September 10). Innovator's take: 'Free' Dimensional video and the future of the moving image medium. Retrieved from: http://www.visionsystems.com/articles/2013/09/innovator-s-take-free-dimensional-video-and-the-future-of-themoving-image-medium.html Charles, A. (2012). Interactivity: New media, Politics and Society. Retrieved from: http://ehis.ebscohost.com/eds/detail?sid=170cc298-dde9-4954-87b0a23b5ab146d3@sessionmgr115&vid=3#db=nlebk&AN=542470 Costa, B. (2013, October 11).The next big thing? How Replay Technologies' freeD system is taking sports tv by storm. Retrieved from: http://sportsvideo.org/main/blog/2013/10/the-nextbig-thing-how-replay-technologies-freed-system-is-taking-sports-tv-bystorm/#sthash.GZHF78Ar.dpuf Debevec, P. (1996). Modelling and Rendering Architecture from Photographs. Retrieved from: http://people.ict.usc.edu/~debevec/Thesis/debevec-phdthesis-1996.pdf Debevec, P. (2006, August). Virtual Cinematography: Relighting through Computation. Retrieved from: http://graphics.stanford.edu/papers/lfphoto/comp-photoarticles/Data/DEBEVEC.pdf Dominques, E., Michalska, A. (n.d.). 15-463: Computational Photography - Final Project: View Morphing. Retrieved from: http://www.cs.cmu.edu/afs/andrew/scs/cs/15463/f07/proj_final/www/amichals/

41

Eadweard Muybridge. (n.d.) Retrieved from: http://www.wildfilmhistory.org/person/180/Eadweard+Muybridge.html Eagleman, D. (2009, February 21). Why do we love slow motion video? Retrieved from: http://www.sentientdevelopments.com/2009/02/why-do-we-love-slow-motion-video.html Ebay demo [webcast]. (2013, September). Retrieved from: http://vimeo.com/73813340 Edwards, S. (2010, September 23). The photgraphic Reality of Eadweard Muybridge. Retrieved from: http://www.currentintelligence.net/cameraobscura/2010/9/23/the-photographic-realityof-eadweard-muybridge.html Fenlon, W. (2012). 48 FPS and Beyond: How High Frame Rate Films Affect Perception. Retrieved from: http://www.tested.com/art/movies/452387-48-fps-and-beyond-how-high-frame-ratesaffect-perception/ Generating 3D content from multi- viewpoint images. (n.d.). Retrieved from: http://www.nhk.or.jp/strl/english/aboutstrl1/r1-2-2.htm Gills, S. (2005). The Matrix trilogy: Cyberpunk reloaded. 6a Middleton Place, Langham Street, London W1W 7TE: Wallflower Press. Godoski, A. (2011). Under the influence: The Matrix. Retrieved from: http://www.screened.com/news/under-the-influence-the-matrix/2218/ Goldstein, M. (2008), DSLR Camera Prices Falling. Retrieved from: http://www.photographyblog.com/news/dslr_camera_prices_falling/ GoPro. (2013). HERO3 Cameras. Retrieved from: http://gopro.com/hd-hero3-cameras Hall, I. (1981, May). Kill and Kill again [Motion Picture]. South Africa/ United States: APC/ Again Productions/Kavalier Films Harold Edgerton. (1999, February). Retrieved from: http://web.mit.edu/invent/iow/edgerton.html
42

High Def Slow motion crash testing shows art of safety. (2013, August 28). Retrieved from: http://www.autonet.ca/en/2013/08/25/high-def-slow-motion-crash-testing-shows-art-ofsafety High Speed Camera. (n.d.). Retrieved from: http://edgerton-digitalcollections.org/techniques/high-speed-photography Hipstomp. (2011). Rip Curl & Time Slice Camera Array Collabaration Lets You Percieve Surfing as Never Before. Retrieved from: http://www.core77.com/blog/technology/rip_curl_timeslice_camera_array_collaboration_lets_you_perceive_surfing_as_never_before_20925.asp Horse in Motion. (n.d.). Retrieved from: http://www.tate.org.uk/whats-on/tatebritain/exhibition/eadweard-muybridge/exhibition-guide/horse-motion How a trotting horse appears when at a 2:20 Gait. (1878, June 18). Sacramento Daily Union, p. 1. Hruska, J. (2011, July 5). Low 3-D uptake worries TV manufacturers. Retrieved from: http://hothardware.com/News/Low-3D-Uptake-Worries-TV-Manufacturers/ Hsu, W., Zhang, Y., Ma, K. (n.d.). A Multi-Criteria Approach to Camera Motion Design for Volume Data Animation. Retrieved from: http://vis.cs.ucdavis.edu/~ybzhang/papers/vis13hsu.pdf Jasonhopper. (2011, February 16) Why do qualitative research?. Retrieved from: http://methodlogical.wordpress.com/2011/02/16/why-do-qualitative-research/ Jones, G.E. (2005). 24P digital video: make your DV movies look like Hollywood. Boston: Cengage Learning. Joshuajnoble. (2011, June 21). What is interaction design? Retrieved from: http://thefactoryfactory.com/wordpress/?p=823 Johnson, S. (2014, January 6). What is a point cloud? Retrieved from: http://www.wisegeek.com/what-is-a-point-cloud.htm

43

Kindt, T., Muller, H.-H. (2003). Narratologia: What is Narratology?: Questions and Answers Regarding the Status of a Theory. Berlin: Walter de Gruyter. Kinect for Xbox 360. (n.d.). Retrieved from Microsoft website: http://www.xbox.com/enGB/Kinect Kurosawa, A. (Director). (1954). Seven Samurai [Motion Picture]. Japan: Toho Studios. Laird, J. (2013, October 12). Step aside 3D: 4K actually delivers in movies and games. Retrieved from: http://www.techradar.com/news/computing/step-aside-3d-4k-actually-delivers-inmovies-and-games-1181885 Lister, M., Dovey, J. & Giddings, S. (2008). New Media: A Critical Introduction. Florence, KY: Routledge. Martin, B., Hanington, B. (2012). Universal Methods of Design : 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions. Retrieved from: http://ehis.ebscohost.com/eds/detail?sid=5bab1156-6c29-4c35-96c52adfab4dc6b2@sessionmgr198&vid=1#db=nlebk&AN=576491 Martzke, R. (2001, January 23). Instant replay gets 'Matrix'-like view of key plays. Retrieved from: http://usatoday30.usatoday.com/sports/2001-01-23-focus-matrix.htm Masefield, R. (2010, October 20). Timeslice goes bullet-time with Rip Curl Mirage promo. Retrieved from: http://www.masey.com.au/2010/10/timeslice-goes-bullet-time-with-rip-curlmirage-promo/ Michel Gondry. (n.d.) Retrieved from the Internet movie database website: http://www.imdb.com/name/nm0327273/ Motion picture pioneer: Eadweard Muybridge and the zoopraxiscope. (2011, November 5). Retrieved from: http://www.lomography.com/magazine/lifestyle/2011/11/05/100424 Padgett, K. (n.d.). Eadweard Muybridge - the bet that lead to moving pictures. Retrieved from: http://agilewriter.com/Biography/Muybridge.htm
44

Patra, S., Bhowmick, B., Banerjee, S., Kalra, P. (n.d.). High resolution Point Cloud Generation from Kinect and HD Cameras using graph cut. Retrieved from: http://www.cse.iitd.ac.in/~mcs103515/VISAPP_2012_246_CR.pdf Point Clouds. (n.d.) Retrieved from Pixar website: http://renderman.pixar.com/view/pointclouds Press release. (2013, May 9). Retrieved from: http://www.nhk.or.jp/pr/english/press/pdf/20130509-001.pdf Prince, S. (1991). The Warrior's Camera. Chichester, West Sussex. Princeton University Press. Purse, L. (2005). The New Spatial Dynamics of the BulletTime Effect. In G. King (ed.) The Spectacle of the Real: From Hollywood to Reality TV & Beyond, pp.151-160. Bristol: Intellect Books. Rehak, B. (2007). The Migration of Forms: Bullet Time as Microgenre. Film Criticism, 32 (1), pp.26-48. Rober, Mark. (2013, May 9). EPIC Matrix effect w/ a ceiling fan & 1 GoPro (ghetto bullet time) [Webcast]. Retrieved from: http://www.youtube.com/watch?v=OEd5lTmeAH8#t=103 Rockwell, K. (2012). Nikon D1. Retrieved April 21st, 2013 from: http://kenrockwell.com/nikon/d1.htm Rolli, G., Sande, P. P., Rodriguez, F. D., Ewequiel, W. S. L. (n.d.). Bullet Time. Retrieved from: http://sabia.tic.udc.es/gc/Contenidos%20adicionales/trabajos/Peliculas/FX/ Rouse, M. (2007, October). Definition pan (video). Retrieved from: http://whatis.techtarget.com/definition/pan-video Rouse, M. ( 2007, May). Definition - Voxel. Retrieved from: http://whatis.techtarget.com/definition/voxel

45

Saunders, M., Lewis, P., Thornhill, A. (2012). Research methods for business students. Harlow: Pearson Education Limited. Saunders, P. (2013). How instant replay has revolutioned sports on tv. Retrieved from: http://www.denverpost.com/sports/ci_24539959/how-instant-replay-has-revolutioned-sportstv Sanders, S. (2014) Using Slow Motion in Action Movies. Retrieved from: http://www.stevesdigicams.com/knowledge-center/how-tos/filmmaking-tips/using-slow-motion-in-actionmovies.html Scorsese, M. (Director). (1980). Raging Bull [Motion picture]. United States: United Artists. Seitz, S., Dyer, C. (n.d.). View Morphing. Retrieved from: http://www.cs.cmu.edu/~seitz/papers/sigg96.pdf Smed, J., Niinisalo, H., Hakonen, H. (2005). Realizing the bullet time effect in multiplayer games with local perception filters. Computer Networks, 49 (1), pp.27-37. Snyder, Z. (Director). (2006). 300 [Motion Picture]. United States: Warner Bros. Stroboscopic. (n.d.). Retrieved from: http://edgerton-digitalcollections.org/techniques/stroboscope Sullivan, L. (1896). The tall office building artistically considered. Lippincott's magazine. 403409. Svetlik, J. ( 2013, January 17). 3D TV is dead, replaced by 4K, says BBC controller. Retrieved from: http://www.techradar.com/news/television/hdtv/3d-tv-is-dead-replaced-by-4k-says-bbccontroller-1125721 Swetnam, Derek. (2000, April 1). Writing your dissertation: how to plan, prepare and present successful work. Oxford: How to Books.

46

Technology. (n.d.). Retrieved from Replay Technologies company website: http://replaytechnologies.com/technology/ The Eye network. (2001, January 28). Retrieved from: http://sportsillustrated.cnn.com/football/nfl/2001/playoffs/news/2001/01/28/superbowl_tv_a p/ The Matrix. (1999). Retrieved from the Internet Movie Database website: http://www.imdb.com/title/tt0133093/ Time Slice. (n.d.). Retrieved from: http://www.timeslicefilms.com/people-2/tim-macmillan Vernon, R. (1966). The Product Life Cycle. Retrieved from: http://db.lib.uidaho.edu/ereserve/courses/b/business/380_01/life.pdf Wachowski, A., Wachowski, L. (Directors). (1999). The Matrix [Motion Picture]. United States, Australia: Warner Brothers Pictures/ Village Roadshow Pictures/ Silver Pictures/ Groucho II Film Partnership. Walford, A. (2007). What is Photogrammetry? Retrieved from: http://www.photogrammetry.com/index.htm Warner Bros. Entertainment Inc. (2007). U.S Trademark Registration number: 3349252. Washington, DC: U.S. Patent and Trademark Office. Wilkinson, S. (2010, December 31). Is Frame Interpolation a Godsend or the Devil's Spawn? Retrieved from: http://www.soundandvision.com/content/frame-interpolation-godsend-ordevils-spawn What is a Gantt chart?. (n.d.). Retrieved from: http://www.gantt.com/index.htm Yoshida, T. (Producer). (1967). Speed Racer [Television Series]. Tokyo: Fuji TV. 200 Things That Rocked Our World: Bullet Time. (2006, February). Empire, 200, 136.

47

7.1 Figure 1

List of Figures Page 5

A simple example detailing the arrangement of cameras in a bullet time setup [Primary source].

Figure 2

Screenshots from Rober's video, showing the spinning motion of the setup. Rober, M. (2013, May 9). Epic Matrix effect w/ a ceiling fan & 1 GoPro (ghetto bullet time). Retrieved from: https://www.youtube.com/watch?v=OEd5lTmeAH8

Figure 3

A simple illustration detailing Rober's experiment in more detail. [Primary source].

Figure 4

Muybridge's first Horse in Motion work. Muybridge, E. (1878). The Horse in motion. Retrieved from: http://cosmicdiary.org/mshowalter/files/2013/02/Figure-1.jpg

Figure 5

Shots of the Speed racer Bullet sequence in effect, less than one second in duration. Superherocartoonsite. (2009, January 13). Speed Racer 1967 Cartoon intro. Retrieved from: https://www.youtube.com/watch?v=suCm1w_KTiY

11

Figure 6

A selection of shots demonstrating the Bullet Time sequence of 'EyeVision', lasting less than a second in duration. Best of EyeVision (pregame show and examples. (2001, January 28). Retrieved from: http://www.ri.cmu.edu/events/sb35/eyevision_best_of.mpg

19

Figure 7 Figure 8 Figure 9

The product life cycle devised by Vernon (1966) [Primary Source]. Gantt chart 1, created July 2013. [Primary source]. Gantt chart 2, created January 2014. [Primary source].

25 31 31

48

8.0 8.1

Appendices

Email Communications

Appendix 1 - Bob Rehak, author of 'The migration of Forms: Bullet Time as Microgenre February 16, 2014 Question #2, about the resurgence in popularity of bullet time, I'd address by suggesting that enough time has now passed (about a decade since the release of the two Matrix sequels) to erode the parodic connotations of the effect; in essence, the joke stopped being funny and then was forgotten entirely, rendering BT again spreadable across media. I noticed a variation of of bullet time, for example, in Dredd (2012) played absolutely seriously. Hope this is helpful, and again, sorry I'm not equipped to answer your more technical questions.

Appendix 2 - Bob Rehak, author of 'The migration of Forms: Bullet Time as Microgenre March 3, 2014 Hi Chris. I think this and your previous question about the future of bullet time both go back to my initial response that BT constitutes a powerful new mode of visualization -- a new phrase in the cinematic/spatial lexicon, if you will -- that is now in resurgence following the expiration of the "punchline" effect associated with the Matrix and its knockoffs & parodies. In my article I argue that technologies underlying bullet time don't matter -- the visual effect is reproducible using any number of modalities -- and that it's only the signature visual that travels. As for ideology, for me this is linked to the way BT is interpreted in relation to this or that parent text; in the Matrix it meant one thing, but with the waning of that association, BT is free to signify differently. It's basically like a copyright that has expired, putting the visual effect in the public domain, where it will surely continue to mutate and evolve. Appendix 3 - Farley Brian, Senior Digital Producer - January 30, 2014 So, with point clouds, i do remember the radiohead project as the first time the process went viral, but my guess is that the technology was developed for military/oil exploration etc (which may be why theres no definitive articles on such things as they are both really quite secretive)... also it uses the mathematical basis of Delaunay triangulation (1934) so i guess the theory was there and it became really useful when the technology evolved (computation ability combined with laser/IR development) A good article here on the radiohead project and related links http://www.graphics.com/article-old/building-radiohead%E2%80%99s-house-cards
49

And this motions the lidar sensor (uses lasers instead of ir as it can scan much larger areas) and in their blurb it mentions designed for obstacle detection and navigation of autonomous ground vehicles and marine vessels With the replay Tech, they really dont give much away on their site. The only thing i can think of is: each camera knows its position in relation to the other cameras/forming a 3d space just in relation to each other, then (at a guess) all of the cameras are configured to point at/work with one individual focus point (which then becomes the user control point for operating the system as a whole). Mapping the 360 background to this focal point is the bit i cant decide on, the golf clip they use really does seem to suggest that maybe an additional 360 degree camera is used as well for mapping/mixing into the focus of the footage into a consistent 360 degree 3D space, whats a bit of a giveaway is when you look at the background spectators, they operate just like cardboard cut outs as the camera moves, theres no change to any angles as it moves from left to right, so maybe its two systems combined but the 360 camera fills the background. In terms of how it comps together i have seen so many impressive software based image processing algorithms (especially when you look at augmented reality apps that dont rely on anything but the live camera to map to) i would hazard a guess that it uses a software approach as opposed to point clouds, but with a fair amount of pre configuration with regard to every camera position/integration of singular 360 capture for the background. Appendix 4 - Farley Brian, Senior Digital Producer - March 6, 2014 Overall, I think this technology has helped to influence a new way of creating and viewing different dimensions with time based content. Its been re-developed from film production to live capabilities as there is a lucrative market for televised sporting events, but as technology/software engineering seems to be progressing so quickly in this field, it will soon be superseded by a much simpler, cheaper, and more versatile solution for such purposes, usually the second wave of development makes the real money. I would say that its real value lies in its influence, as it's an early representation of the future possibilities of time-based media, but not just tv and film but for new emerging formats. Project tango and similar developments online really point to where the potential expansion of such techniques lie, i.e : http://www.youtube.com/watch?v=Qe10ExwzCqk Its still at early stages but eventually i think we will see a similar effect thats even more versatile through this area/medium of development.

50

Appendix 5 - Farley Brian, Senior Digital Producer - March 20, 2014 Interesting to see that a fair amount of post production needed to go into the matrix to get the effect correct. And with FreeD, really reminds me of http://photosynth.net/about.aspx which has been around for some time, i do remember initially they sold it as a combination of geo tagged shots submitted by various users who took photos at the same location, eventually all angles would make up a 360 degree representation. Overall, i do think we are in the middle of a huge transition thats going to outmode what film/traditional moving media is at present, and as they say the increase in computational power/software capabilities is really whats driving these new ways of doing things and how their system came about, and now such effects and movie making overall are now not just in the hands of huge studios, but are slowly going to form a set of tools anyone can use for whatever purpose, thus expanding what the medium can do. Their electricity analogy certainly does point at how one unintentional thing can lead to / influence another and adding to this, the invention of the cathode ray tube / tv came about in the Victorian era when they were actually looking for evidence of the spiritual world... Appendix 6 - Joe Leveson, Technical Director Prime Focus- October 27, 2013 Sorry for the delayed reply. It's been a very busy week. I did ask a couple of my supervisors about bullet time and according to them it's not used so much now days. They use what is called a digi double, which is a fully rigged and textured model of the character. The reason for this is because the levels of texturing has moved forward quite a bit since bullet time became used in the late 90's. It's also cheaper and more freedom when it comes to planning shots. If I were you I'd look to go down this route. I hope this information is helpful.

Appendix 7 - Joe Leveson, Technical Director Prime Focus- October 31, 2013 A digi double is a 3d character that has been modelled built and textured to look exactly like a real life character. It is then rigged (meaning a skeleton is put in side ) so that it can be animated. Here is an article of the man of steel: http://www.fxguide.com/featured/man-of-steel-vfx-milestones/ If you scroll down you'll see an article a section on digi doubles. To get something like this done you will can contact and number of visual effects companies in soho. Which one will depend on your budget. The more you spend to the better it will look though a smaller company will do a good job and would be suited to take it on. A company that would do commericals and music videos is your best bet.
51

Appendix 8 - Joe Leveson, Technical Director Prime Focus- October 31, 2013 no worries. This is a digi double head: http://www.youtube.com/watch?v=jAP59_FzKIo I recommend going down this route as its much easier to find a company that does this and more flexible than bullet time. Appendix 9 - Kensuke Ikeya, developer of NHK multi camera system - October 30, 2013 Sorry for may late replay. I answer to your question. >1. Can I ask where you got the idea for this project, was it from previous Bullet Time rigs used in film/ television or person developing a similar system? I am inspired by "Eye vision" which was developed by prof Takeo Kanade. http://www.ri.cmu.edu/events/sb35/tksuperbowl.html Some improvements are added to my system to make my system easier to use at actual TV program production site. >2. What do you think the future holds for the technology, in your opinion will it be used in mainstream television and if so how many months/years would you predict it being integrated into common footage? Practical use in Japanese TV program has already begun. I used my system for replay of gymnastics sports scene in June and will also use for replay of figure skating in November. In the future ( sorry, I can't predict), I want to use it for a free view point video and a integral 3D TV. >3. What do you think of the multi-camera companies that use computer animation to create 3D footage such as; http://replay-technologies.com/ I think that movies replay-technologies company creates are high quality and very cool ! In the future, I want to create such high quality movies on real time from multi-viewpoint videos captured my multiple robotic camera. >4. How many cameras can be used in the setup? Now, I use 9 robotic cameras in the set up.
52

It is possible to increase number of robotic cameras infinitely. Appendix 10 - Don Kennedy -Media Producer, October 21, 2013 We wrote the article based on an interview we conducted with a representative of NHK STRL, as well as this (rather sparse) press release: http://www.nhk.or.jp/pr/english/press/pdf/20130509-001.pdf Also, someone commented on the YouTube page that a similar system was used during a Super Bowl broadcast in 2001. You can find out more information about that 'EyeVision' system as well as sample videos here: http://www.ri.cmu.edu/events/sb35/tksuperbowl.html If you have any other questions please don't hesitate to get in touch. Appendix 11 - Don Kennedy -Media Producer, October 28, 2013 You could possibly get in touch with Takeo Kanade, the computer vision researcher at Carnegie Mellon who developed the eye vision system. Also, this company: http://replay-technologies.com/ are doing similar stuff but generating 3D environments using the footage and metadata from the camera.

Appendix 12 - Oren Haimovitch Yogev, CEO of Replay Technology - January 28, 2013 See these clips to better understand the technology. We are working on passive elements without I/R depth emitters. http://vimeo.com/82561034 http://vimeo.com/82561032 http://vimeo.com/80774790 We are coming to the household in Late 2014-start of 2015. Appendix 13 - Oren Haimovitch Yogev, CEO of Replay Technology - October 19, 2013 Take a look at this one , interview with our CTO and another 2 pieces below which will give you some data. http://www.vision-systems.com/articles/2013/09/innovator-s-take-free-dimensional-videoand-the-future-of-the-moving-image-medium.html
53

http://www.tvtechnology.com/prntarticle.aspx?articleid=221089 http://sportsvideo.org/main/blog/2013/10/the-next-big-thing-how-replay-technologies-freedsystem-is-taking-sports-tv-by-storm/

Appendix 14 - Diego Prilusky, Creative director for Replay Technology - March 19, 2014 Your questions are reasonable and they are really tackling the basic of the existence of our product and why after a decade of the matrix effect, the bullet time, we still find enough interest around the world for our technology. The answer really is in within the questions, mainly because the questions are referring to the bullet effect as his own, or more to the end result. And looking at the end result, the possibility of moving around a frozen moment in time can be achieved in many ways. As you might already investigated in the bullet time effects was developed for few years before it was finally used in The Matrix movie in 1999. The effect permitted to film a subject and create a movie, a sequence of images played back in a timeline of a timeless moment. The effect is quite amazing by the actual possibility to cheat our timeline existence by capturing a timeless event and perceiving it in our timeline perception. Capturing a moment in time, was already invented two centuries ago by exposing a mixture of paper and chemicals to the light for a given moment, thus storing this light reflection into a finite capsule which we can always look at and re-experience this reflected light a picture. Eadweard Muybridge continued this experiments of encapsulating a fraction of light by doing so sequentially. He created a rig of multiple cameras exposing each at a continues ordered time steps sampling the reflected light for any given moment. Thus the video was created, you can always replay this sequence of pictures and re-experience the recorded moment. Very similar to what and how Muybridge invented the moving picture in a way, the bullet time was created. Instead of timing each camera to a sequential time interval, the cameras are all together simultaneously expose to light, taking a picture at the same moment of time. Now, although all cameras perceived the same moment, each one of them is positioned in a different position capturing a different angle of the subject. When placing enough cameras one near the other, creating a sequential path, the images can be played sequentially, in the same order the cameras source is positioned. By doing so we get a single moment in time (freeze bullet time) while the point of view change, which creates a sort of camera motion.

54

This same technique was used by EyeVision in 2001 for sport. The reason the video was very shaky and non-smooth was due to the fact that this technique lacks of single focus point. The cameras are aimed at a defined region, by the time EyeVision used robotic arms to dynamically aim all the cameras at once to a specific subject. This works good enough to get the freeze moment with motion once the images captured are played sequentially. But the shakiness comes from having each camera with a slightly different focal lens, and the subject not always at the center of the image. So by playing this in a timeline the centered subject changes position from picture to picture, even if its just slightly it will be enough to get a shaky result. It is equivalent to moving the camera all the time while filming a subject. In The Matrix, special effects department had to front the same challenge. The reason that it looks so smooth in the final result was due to the amount of cameras used, having a high number of cameras placed one just next the other simulates moving a single camera in that path. So lets say each camera is 20cm from each other, placing 25 cameras next to each other will take a sum of 500cm. When playing this images sequentially at a frame rate of 25 frames per second, this setup will be equivalent of a camera motion moving 5 meters per second. So by placing the cameras as close as possible to each other is possible to get a final result equivalent to a slower camera motion. Secondly the images captured in The Matrix where post processed one by one and realigned digitally so all the focus points where the same, making sure the main subject is always positioned in the center frame. This removed the shaky effect of moving subject. EyeVision had a low camera number without a constant focus point. Also this shaky result can be perceived in all similar camera rig setup done by hobbies posted in any global video source like YouTube. An additional limitation the above technique has, is the camera motion. The final camera motion of the frozen moment will be defined by the way the capturing cameras all placed next to each other. In order to change the camera motion, there is need to move physically the cameras in the rig by the new order wanted. freeD is different from this techniques. freeD is a video format that encodes the images captures into a volumetric database. The similarity from the above is that for each given moment in time a pictured is captured simultaneously from different angles, same as the techniques above. But the instead of playing the captured images sequentially in a timeline, the pictures are analyzed and encoded to a single volumetric image. By having the three dimensional location of each camera used, each image has a known source position. Then by triangulating the same pixel from all the pictures we can get the 3d position of that pixel in a volumetric space, defined by 3 axis. Once the volumetric database is achieved, we get a special image that can be rendered from any position with a virtual camera, what we call the freeD
55

camera. This technique needs thousands more computation cycle than the previous used a decade ago. But the advantages are huge: There is no need to have a very large amount of cameras one next to each other to get a slower and smooth camera motion, with a 10% of cameras is enough to build a volumetric database that can be seeing from any angle. This will be one of the main reasons we get the polished, 3D fluidity. Instead of playing each image one after the other, we combine all the information from multiple cameras to a single volumetric image that can be seeing from any angle. Because of the large amount of information we get captures, in order to reduce computing time we rescan each time only moving objects, while static objects get a volumetric database once. So you can say its a type of photogrammetry and previously composed 3D models. Its very similar in a way to how a standard video codecs work, for example MPG, it will update in the video only the changing pixels. freeD is doing the same, but generating a special video format. A video format with enough information to see it from any angle. So whats new after a decade? I think that the answer is in the increase of GigaHertz being able to be calculated per second. The computational power today open doors to new techniques that before were too slow or just non-reasonable to do in near real-time. Also the movement of video from film to digital has created a huge amount possibilities to manipulate information never possible before. Could freeD exist a decade ago? Maybe, but its the same on asking why the electricity was invented when it was invented? What did they had at that time, that wasnt before, that enabled the invention of electricity? There is not a straight answer. I think that combination of tools, fast computing, digital image, low prices of cameras with very big sensor and a very talented and versatile team working together a combination of all of those, made freeD to exist now and not a decade ago.

56

8.2

Interview Extracts

Appendix 15 - Interview Transcript Extracts - Tim Kilgour, November 21, 2013 "I've worked for Fieldtrip for a while [a visual effects company] and I've spent a lot of time in the animation and visual effects industry." "The work that Replay Tech is producing is pretty cool, looking at this video, there is what looks to be point cloud generation." "Point clouds were developed to study terrain maps and things like that." Kilgour goes on to demonstrate examples of Point Clouds. "These videos show the Kinect, a Microsoft camera and guys at home can create an interactive environment of their own home." "When used in film production, this system [Point Cloud technique] creates highly detailed composites of a subject which could be modelled quickly, it is less time consuming and costly than a visual artist building a 3D model from photographs." When asked if the Time Slice method was no longer being utilised: "No I don't think so, we use it in different ways. Its evolved and mixed with other tech." When asked how this would relate to Bullet Time; "Bullet Time has come a long way, we used this on a job recently and you can see here [demonstrating a model of a 3d head] that the image we get is actually pretty good. A couple of cameras arranged round the model take an instant snapshot, point clouds interpret the data and assemble the 3D model. Saves a lot of time building models from photographs!" "The old rigid forms of Bullet Time have been developed; we now use these multi camera methods to create complex 3D models. Technology develops over time and the best pieces are cotton picked and used in new places." When asked if Bullet Time was dying off; "No I don't think so, we use it in different ways. Its evolved and mixed with other tech." Appendix 16 - Interview Transcript Extracts - Farley Brian, November 21, 2013 "Well, my names Farley and I've worked in every possible vfx job you can imagine. I'm now working for Pioneer creating web videos for their DJ equipment."

57

"I'm pretty sure a guy called Tim Macmillan was the first to use Bullet Time and then that was picked up by Warner. I remember the old QuickTime VR software which we used to hack and replicate this 360degree bullet time effect. It's pretty interesting how much it's come on. " "I've had a look at these companies [Replay Technology and NHK] and I'm pretty impressed. It does look like they've [Replay Technology] used point clouds to shoot the subject which is framed inside a 3D environment." "It's definitely true that the industry is changing, it's all about being interactive and everything working together. The new Bob Dylan music video is an awesome example, you can change the channel within the video and it's all synched together!" Brian demonstrates the video. "On the flip side, this is an experimental field, a lot of the experimental guys are out in the cold as no one cares about creativity, it's all done to generate profit and hit a target audience" "Technologies follow cycles and can be profoundly hit and miss, it's a bit shocking when you look back and find these seeds" "In regards to The Matrix, the story of the techniques used actually sold the film. It is possible the film would have sold nowhere near as well without the seed that is Bullet Time. We had a project which didn't take well, only hitting a small audience, but it had new technology which the media picked up on and it marketed itself!"

58