Surface Computing

1. Introduction
1.1 Surface Computing:
Surface Computing is a brand new category in computing that offers a unique user experience. Its the way of working with computers with no more mouse and keyboard. The Microsoft has developed the surface table called Microsoft Surface. It turns an

ordinary tabletop into a vibrant, interactive surface. The product provides effortless interaction with digital content through natural gestures, touch and physical objects. Microsoft Surface is a 30-inch display in a table-like form factor thats easy for individuals or small groups to interact with in a way that feels familiar, just like in the real world. The multi-touch surface computing technology has the ability to recognize objects by distinguishing between the differences in pressure and Temperature of what is placed on the surface. Depending on the size and applications installed in the surface, two or more people can be doing different or independent applications on the device. In essence, its a surface that comes to life for exploring, learning, sharing, creating, buying and much more. Everything can be done with our finger tips.

1.2 Natural User Interface (NUI):

Natural User Interface is a new trend in User Interface that allows people to interact with digital content in the same way they have interacted with everyday items such as photos, music and games with their hands, with gestures and by putting realworld objects on the surface. Its a direct interaction with computer using Gestures. Surface computing prototypes have employed gestures created by system designers. Gestures are appropriate for early investigations; they are not necessarily reflective of user behaviour. Designing tabletop gestures that relies on eliciting gestures from nontechnical users by first portraying the effect of a gesture, and then asking users to perform its cause. NUI provides multi-touch, multi-user, real world object interaction, and 360 degree UI.

Dept of CSE, NCET, Bangalore

2010-11

Page 1

Surface Computing

2. Complete System
2.1 Hardware Design:

Fig 2.1: Hardware parts (1)Screen: A diffuser turns the Surface's acrylic tabletop into a large horizontal "multitouch" screen, capable of processing multiple inputs from multiple users. The Surface can also recognize objects by their shapes or by reading coded tags. (2) Infrared camera: Surface's "machine vision" operates in the near-infrared spectrum, using an 850-nanometer-wavelength LED light source aimed at the screen. When objects touch the tabletop, the light reflects back and is picked up by multiple infrared cameras with a net resolution of 1280 x 960. (3) CPU: Surface uses many of the same components found in everyday desktop computers a Core 2 Duo processor, 2GB of RAM and a 256MB graphics card. Wireless communication with devices on the surface is handled using Wi-Fi and Bluetooth antennas (future versions may incorporate RFID or Near Field

Communications). The underlying operating system is a modified version of Microsoft Vista. (4) Projector: Microsoft's Surface uses the same DLP light engine found in many rearprojection HDTVs. The footprint of the visible light screen, at 1024 x 768 pixels, is actually smaller than the invisible overlapping infrared projection to allow for better recognition at the edges of the screen.

Dept of CSE, NCET, Bangalore

2010-11

Page 2

Surface Computing

2.2 Working:
Microsoft Surface takes massive multi-touch and amplifies it like a stadium loudspeaker. Inside a Surface unit, five infrared cameras peer up at the screen while an adjacent projector bathes the tabletop with infrared light. Once fingers or hands touch the tabletop, the infrared cameras see the reflection and tell a Windows Vista-based computer running Surface what those hands or fingers are doing, and how the application should respond with the corresponding content. This vision-based technology allows Microsoft Surface to respond dozens of different touches. So several people can individually or collaboratively work on a single Microsoft Surface. While doing so, they can open documents or images, zoom them to larger or smaller sizes, rotate them, copy them from one user to another or close them all with a tap or the flick of a finger. Microsoft Surfaces vision system also allows it to identify objects placed atop it, so it can communicate with a smart phone or tagged card that might contain digital records.

2.3 Surface Features:

There are four features of a surface Computing

1. Direct Interaction: Users can actually grab digital information with their hands
and interact with content through touch and gesture, without the use of a mouse or keyboard.

2. Multi-Touch:

Surface

computing

recognizes

many

points

of

contact

simultaneously, not just from one finger as with a typical touch screen, but up to dozens and dozens of items at once.

3. Multi User: The horizontal form factor makes it easy for several people to gather
around the Surface, providing a collaborative, face-to-face computing experience.

4. Object

Recognition: Users can place physical objects on the display to trigger

different types of digital responses. There should be an barcode for the nonelectronic objects.E.g. Surface technology using an object with a tag- wineglass placed on surface to bring up other information about that particular wine.

Dept of CSE, NCET, Bangalore

2010-11

Page 3

Surface Computing

3. Technology behind the Surface Computing

3.1 Camera:

Fig 3.1.a Camera

Fig 3.1.b Infrared LED

Microsoft Surface uses cameras to sense objects, hand gestures and touch. The user input is then processed and displayed using rear projection. Specifically Microsoft Surface uses a rear projection system which displays an image onto the underside of a thin diffuser. Objects such as fingers are visible through the diffuser by series of infrared sensitive cameras, positioned underneath the display. An image processing system processes the camera images to detect fingers, custom tags and other objects such as paint brushes when touching the display. The objects recognized with this system are reported to applications running in the computer so that they can react to object shapes, 2D tags, movement and touch. Fig 3.1.a shows the camera placed inside the surface table. Unlike most touch screens, Surface does not use heat or pressure sensors to indicate when someone has touched the screen. Instead, five tiny cameras take snapshots of the surface many times a second. This allows Surface to capture many simultaneous touches and makes it easier to track movement, although the disadvantage is that the system cannot (at the moment) sense pressure. Fig 3.1.b shows the infrared LED placed under the diffuser of surface. Five cameras mounted beneath the table read objects and touches on the acrylic surface above, which is flooded with near-infrared light to make such touches easier to pick out. The cameras can read a nearly infinite number of simultaneous touches and are limited only by processing power. Right now, Surface is optimized for 52 touches, or enough for four people to use all 10 fingers at once and still have 12 objects sitting on the table.

Dept of CSE, NCET, Bangalore

2010-11

Page 4

Surface Computing

3.2 Tagged objects and tag visualizations:

Microsoft Surface applications can recognize special tags besides fingers and objects. These tags are similar to bar codes in concept and can store a particular value which can be retrieved by Surfaces vision system. Tags are a pattern of white dots (infrared reflective) in black background (infrared absorbing) and are normally printed on a card or are printed and stuck to the plain surface of an object.

Microsoft Surface supports two types of tags.

3.2.1 Byte Tags

These tags store 8 bits of data (1 byte), so there are 256 unique tag values. Each byte tag that the Microsoft Surface Vision System identifies has an orientation and an 8bit tag value. The Fig 3.2.1.a illustrates the basic tag that all other tags are based on.

Fig 3.2.1.a: Byte Tag 1: Infrared-absorbing background 2: One infrared-reflecting circle (0.125-inch radius) in the center of the tag. This circle locates the tag on the Microsoft Surface screen. 3: Three infrared-reflecting circles (0.08 inch radius) located 0.28 inches from the center of the tag in each direction (left, right, and down). These "guide" circles determine the tag orientation. In addition, each tag contains from zero to eight data bits that define the tag value. These data bits are white circles (0.075-inch radius) that are centred in one of the following locations: - 0.3058 inches horizontally and 0.2038 inches vertically (bits 1, 2, 5, and 6) from the center of the tag. - 0.2038 inches horizontally and 0.3058 inches vertically (bits 0, 3, 4, and 7) from the center of the tag.

Fig 3.2.1.b: infrared-reflection circle positions

Dept of CSE, NCET, Bangalore

2010-11

Page 5

Surface Computing In the Fig 3.2.1.b the highest order bit (bit 7) is at the 1 o'clock position when you look at the printed side of the tag. Less significant bits are then read counter-clockwise from the 12 o'clock position. A circle represents 1. The absence of a circle represents 0. Examples of bytes tags:

Tag 0x00 (00000000)

Tag 0xFF (11111111)

Tag 0xC6 (11000110)

3.2.2 Identity Tags:

These tags store 128 bits of data (two 64-bit values), so there is an effectively unlimited range of unique tag values. The operating principle is the same as for bytes tags. As for tags bytes, an identity tag has circles of position.

Fig 3.2.2 : position of infrared circle Tag Visualizer is a content control that automatically displays visualization objects when a tag is placed on the grid. The definition of the tags has been already written using TagVisualizationDefinition. Each definition specifies what kind of tag (Byte or Identity) we are using, its value, and the source file for Tag Visualization. Finally we have a TextBlock to display the values of tag object. Fig 3.2.2 shows how infrared circle are positioned on the tag.

Dept of CSE, NCET, Bangalore

2010-11

Page 6

Surface Computing

4. Applications
4.1 Basic Applications:
4.1.1 Paint Touch

Fig 4.1.1 paint application Paint Touch is an application for Microsoft Surface created to engage children in painting and drawing with both their hands and real-world objects like paint brushes. The application is created to allow companies to easily integrate their branded imagery into a playful activity that children can email so they don't lose their creation, as well as being able to view it on a dedicated and branded website. Fig 4.1.1 shows the snapshot paint application. There are three draw modes that can be toggled by touching an icon on the bottom of the toolbar: brush, paint, and reveal, the last of which is kind of a negative brush that shows a background bitmap underneath. The brush mode is a bit spotty and tends to skip, but the paint mode is smooth and fun. You can draw using one finger, all your fingers at once (good for drawing hair), the palm of your hand, or using any natural object such as a regular paintbrush. Using the program is like having a flashback to finger painting back in kindergarten (minus the mess), and certainly children will have tons of fun with this kind of application.

Dept of CSE, NCET, Bangalore

2010-11

Page 7

Surface Computing

4.1.2 Music Touch:

Fig 4.1.2 Music application The Music application works like a virtual jukebox, displaying music arranged by album and allowing the user to flip over albums, select songs, and drag them to the "Now Playing" section. Fig 4.1.2 shows the snapshot of the music application. The album browser works a bit like Apples Cover Flow, although many albums are visible at once without scrolling. In addition to playing music that is already stored on the unit's hard drive, Music can also transfer songs from portable music players. Mark Bolger demonstrated this by placing two Zunes (portable media devices) on top of the Surface and using the wireless connection to drag and drop songs between the units, the song list, and the Now Playing section. I mentioned to the team that this was the first time I had ever seen even one Zune "in the wild," and they joked that Microsoft headquarters didnt really count as being in the wild. We can note that sharing songs in this manner would be "subject to DRM restrictions, of course.

Dept of CSE, NCET, Bangalore

2010-11

Page 8

Surface Computing

4.1.2 Photo Sharing:

Fig 4.1.3 Photo sharing application Transfer of data or content has never been easier, than with a Microsoft Surface table in hand. This gesture-based interface enables to easily drop data from any devices on to it, and similarly enables any device to drag data from it. A Smartphone owner can move their content freely between two devices by simply dragging content off their phone onto a Microsoft Surface Table and back onto another device instantly. Photos are arranged into albums that look like piles (stack). Tapping the pile once spreads it around the screen and from there you can drag, rotate, and resize the images to your hearts content. Since Surface can detect many touches at the same time, multiple people can sort and resize pictures. The program can also apparently sort photos into stacks and even whole stack can be shared. Just place your Smartphone on to the Microsoft Surface table and drag the required contents (data, documents, music, and photos) out of your Smartphone and drop them on to the surface table, and then simply put the device onto which data needs to be copied and just drag the required data on to it. It also allows you to preview your data instantly. Fig 4.1.3 shows the snapshot of user interface provided by photo sharing application. It relies on a parallel TCP and UDP Socket connections to get the screen syncing with real time, as close as possible. Any devices simply need to join the open Wi-Fi network created by the Surface and then launch the apps.

Dept of CSE, NCET, Bangalore

2010-11

Page 9

Surface Computing

4.2 Business Applications: 4.2.1 Resto Touch:

Fig 4.2.1 Restaurant application Imagine ordering food at a restaurant, or drinks at a bar, by putting your hands on the table and making your selections. No more asking the waiter for a paper and pen, or pointing at menu items and then negating the onion listing, or speaking your choices and hoping the waiter understood enough to bring back the right dishes. Fig 4.2.1 shows the view of restaurant application. Surface allows up to 4 persons to directly order from the table, displaying photographs of all meals, appetisers and wines in an entertaining manner. On trying to make systems more networked, when we order a drink at a bar, and later use the touch screen applications to store information, it will remember the profile and offer the same drink again that time or next time. Surface also has the ability to read bar codes on items to provide further information to the user. This means that when a customer simply sets a wine glass on the surface of a table, a restaurant could provide them with information about the wine theyre ordering, The experience could become completely immersive, letting users access information on the wine-growing region and even look at recommended hotels and plan a trip without leaving the table. Microsoft also notes that the transfer of digital content is also possible with Surface.

Dept of CSE, NCET, Bangalore

2010-11

Page 10

Surface Computing

4.2.2 Retail Touch:

Fig 4.2.2 Retail application Surface provides its retail store visitors with unique opportunities to learn about the growing universe of mobile applications and devices. Designed to give shoppers a chance to experience the best of the mobile they select. The interactive touch displays, which will work without a traditional mouse or keyboard, will allow customers to do the following:

Learn: Customers can review features of a particular mobile device by simply placing it on the display. Surface will recognize the device and provide a graphical overview of its capabilities. Customers will be able to place two devices side by side on the unit and easily compare their feature

Explore: Customers view interactive coverage maps at the national, state, local or street level, using simple touch and hand movement to scale and move the maps, determining their coverage area.

Customize: In the future, customers will be able to drag and drop ring tones, graphics, video and more by grabbing content with their hands from a menu on the display and dropping it into the phone. The idea behind this technology is again the tag object concept, the mobiles are

tagged with the tag behind and once the mobile is place on the surface it recognizes the tag and display the features of it, the features are before stored in the system. Fig 4.2.2 shows the specifications of the 2 mobiles placed on the surface.

Dept of CSE, NCET, Bangalore

2010-11

Page 11

Surface Computing

4.2.3 Money Touch:

Fig 4.2.3 Banking application In this application user just places the credit card on the surface and do the banking procedures he/she wants to do. The application will recognize the credit card that placed on the surface and provides the user interface. If the two credit cards are placed on the surface the money transfer from one to another can be made, the update to the account is made as soon the transaction has made because the surface table is connected to the bank database server. Fig 4.2.3 shows the snapshot of banking application. This application is also used in RestoTouch, that when its time to pay the bill in the restaurant as we know 4 heads can order from the table, each heads can place their credit cards on the surface and can share the bill as each one paying certain percentage of bill. Of course, this cuts down on interacting with the waiter, but for the 2.0 crowd. This makes both the guests and even the hotel management to have a well defined system for bill payment. The identification of the credit card is with the identity tag present backside of the card, simple principle is again is tag object recognition. The account details and the back ground process which is required for the payment of bill is one by the application.

Dept of CSE, NCET, Bangalore

2010-11

Page 12

Surface Computing

4.2.4 MEDHOSTs application (Healthcare a Digital Touch)

Fig 4.2.4 Healthcare application To make the patient-doctor consultation more comfortable, collaborative and productive is the goal of an application developed by Texas Health Resources and Microsoft partner Infusion Development. Using Microsoft Surface, the solution uses digital health records and images along with video and diagrams to make the doctorpatient consultation more productive, more understandable and less apt to result in mistakes. This prototype also include an application designed to help patient scheduling and improve overall hospital efficiency, especially when experiencing a rush of patients such as in the aftermath of a natural disaster or construction accident. Patients have a choice where they receive care, and healthcare systems are looking for innovative ways to differentiate themselves around patient experience. Microsoft Surface helps healthcare providers do that. Fig 4.2.4 shows the healthcare interface provided for the doctor and the patient. The goal is to ensure that patients learn more, understand their diagnosis better, and have a sense that the physician is using the best technology available. Its pretty well known that when a patient is better educated during their visit with a doctor, they have a better experience and better satisfaction. This application really helps them make more of the visit by really collaborating with their doctor, rather than just being talked at them.

Dept of CSE, NCET, Bangalore

2010-11

Page 13

Surface Computing

5. Surface in future
These are just a glimpse of what is possible with surface computing.

5.1 Surface Computing to the library:

A student walks into digital library media center, heads over to the reference desk and places his palm on the surface to access his network pass. The system, recognizing the student, then displays an assignment from his teacher, along with the pathfinder you created for the project. Another student lays his/her handheld on the desk and seeing his/her assignment appear, reaches out and drags it onto her device. Suddenly an announcement on the far edge of the desk catches her attention. The student touches the flyer, bringing it to the top of the pile. When she spreads her hand, the document magically expands so she can take a closer look. As information moves into a digital realm, it can become less personal and more distant. Without physical interaction, information on computers can become very abstract. Surface computing changes this; it makes working with digital information much more personal and physical.

5.2 Surface to the home:

Integrating the system with the application that controls the house electric appliances with Wi-Fi technology will give the user a control of his home appliances with singe touch and multiple operations. The application should be password protected so that unknown user cannot operate the application. It should be protected from childrens so that no damage of system will occur.

5.3 Interactive Teamwork Table:

The multi-touch surface computers will encourage the small group of team work on the same topic with multi-touches and gives the optimum results without no more external communication like one explaining about the changes as others can see and go accordance with that. This will help in the project designing work. The group of IT professionals with 3 to 4 members can design Software Engineering process on a single surface.

Dept of CSE, NCET, Bangalore

2010-11

Page 14

Surface Computing

6. Conclusion
As a system surface computing has advantages like the four features, NUI which eased the learning of UI, Seamless- no wires or USB ports and disadvantages like its huge system not easily portable, currently very costly $5,000 to $10,000. Security issues should be considered, validating the system is high but the system has more applications in the business field so the all the disadvantages doesnt matters it more. As the business point of view, each professional home computer and the IT companies discussion rooms should be replaced by a multi-touch surface technology, so it will provide a great deal of multiuser with multiple application, best idea sharing table for IT professionals with reduced paper work. There can be an IT solution system for developing different applications for the customer requirements with BPO or online help system to manage the simple applications like guests magazines, Online gaming system, and the home appliance control system, so that there can be a best reliable and the trust worthy system validation. The information would not be stored locally, it can be on the cloud serve so that every client can access. Surface computing can be in any environments school, businesses & home where it is more useful. However, technology is huge it can change the way machine operates in the future. As for the companies, I would recommend looking at some options to bring the cost down so that it can be used widely in the world.

Dept of CSE, NCET, Bangalore

2010-11

Page 15

Surface Computing

BIBILOGRAPHY
[1] White paper assignment for Dr. Marshall G. Joness EDUC 651 class at Winthrop University, Rock Hill, SC. [2] Baudel, T. and Beaudouin-Lafon, M. (1993) Charade Remote control of objects using free-hand gestures. Communications of the ACM 36 (7), 28-35. [3] Forlines, C., Wigdor, D., Shen, C., Balakrishnan, R.: Direct-Touch vs. Mouse Input for Tabletop Displays. Proc. CHI '07, pp. 647--656. ACM Press, New York (2007) [4] http://www.microsoft.com/surface [5]http://www.libraryjournal.com/slj/printissuecurrentissue/858904/surface_computing_ coming_to_your.html.csp [6]Tagged%20objects%20and%20tag%20visualizations%20/%20Surface%20/%20Articl es_files/hugon.js [7] http://www.youtube.com/watch?v=-xRywyX8Xlo [8] http://www.hongkiat.com/blog/microsoft-surface- computing-technology/ [9]http://en.wikipedia.org/wiki/Graphical_user_interface [10]http://www.allfreeessays.com/search_results.php?query=surface+computing [11] http://www.seminarprojects.com/Thread-surface-computing?pid=36588 [12]http://www.microsoft.com/surface/casestudiesmicrosoftretailstores.aspx [13] http://www.engadget.com/2008/10/23/kids-on-with-the-smart-table

Dept of CSE, NCET, Bangalore

2010-11

Page 16