Multimedia Information Systems

Prof. Pallapa Venkataram

PET Unit, Electrical Communication Engineering Dept.
Indian Institute of Science, Bangalore, India

Prof. Pallapa Venkataram, IISc

Lecture-01

Outline of the talk

Definitions
Multimedia
Multimedia Applications
Multimedia Information System

Characteristics of a Multimedia System

Components of a Multimedia System
Issues/Challenges
Multimedia data

Prof. Pallapa Venkataram, IISc

Lecture-01

Definitions
Multimedia means that computer information can be
represented through audio, video, and animation in
addition to traditional media (i.e., text, graphics/drawings,
images).
A Multimedia Application is an application which uses a
collection of multiple media sources e.g. text, graphics,
images, sound/audio, animation and/or video.
Examples of Multimedia Applications include:
World Wide Web,
Multimedia Authoring, e.g. Adobe/Macromedia Director,
Video-on-demand,
Interactive TV,
Computer Games,
Digital video editing and production systems,
Multimedia Database systems.

Prof. Pallapa Venkataram, IISc

Lecture-01

Multimedia Examples

Figure: Multimedia Examples

Prof. Pallapa Venkataram, IISc

Lecture-01

Multimedia Applications

Figure: Multimedia Applications

Prof. Pallapa Venkataram, IISc

Lecture-01

Definitions

A Multimedia Information System is a computer

hardware/software system capable of processing
multimedia data and applications, i.e.,
Acquiring and Storing,
Indexing and Searching,
Manipulating (editing and quality enhancement),
Distributing,
Protecting.

Prof. Pallapa Venkataram, IISc

Lecture-01

Characteristics of a Multimedia System

A Multimedia system has four basic characteristics :

Multimedia systems must be computer controlled.
Multimedia systems are integrated.
The information they handle must be represented digitally.
The infterface to the final presentation of media is usually
interactive.

Prof. Pallapa Venkataram, IISc

Lecture-01

Components of a Multimedia System

Capture devices Video Camera, Video Recorder, Audio

Microphone, Keyboards, mice, graphics tablets, 3D input
devices, tactile sensors, VR devices. Digitising Hardware
Storage Devices Hard disks, CD-ROMs, DVD-ROM, etc.
Communication Networks Local Networks, Intranets,
Internet, Multimedia or other special high speed networks.
Computer Systems Multimedia Desktop machines,
Workstations, MPEG/VIDEO/DSP Hardware.
Display Devices CD-quality speakers, HDTV,SVGA,
Hi-Res monitors, Colour printers etc.

Prof. Pallapa Venkataram, IISc

Lecture-01

Issues/Challenges

How to represent and store temporal information.

How to strictly maintain the temporal relationships on pl
back/retrieval
What process are involved in the above.
Data has to represented digitally AnalogDigital
Conversion, Sampling etc.
Large Data Requirements bandwidth, storage,

Prof. Pallapa Venkataram, IISc

Lecture-01

Meeting Required QoS Parameters

Figure: Bandwidth Required vs. Latency Sensitivity for Different

Applications
Prof. Pallapa Venkataram, IISc

Lecture-01

Issues/Challenges

Compression & Delivery

Large Data Size: Digital video 720x480x24x30 = 249Mbps,
112 GB per hour
Variation in capabilities in receivers: cell phone, PDA, PC,
TV, HDTV
Variation in network: mobile, Wi-Fi, Internet, Satellite

Review image, video, audio, and graphics compression,

Survey scalable compression, joint source-channel coding,
multiple description, distributed compression

Prof. Pallapa Venkataram, IISc

Lecture-01

Data Compression

Figure: General Data Compression Scheme

Prof. Pallapa Venkataram, IISc

Lecture-01

Issues/Challenges

Segmentation
Break down complex audio-visual objects into atomic
units for retrieval

Basic image segmentation Video segmentation shot and

story level
Special events detection : Face and Object Tracking,
Video Structure Modeling
Feature Extraction
Represent an audio-visual object as a (multiple) vector with
a (multiple) distance function d(...) for measuring similarity

Various similarity functions : L-1, KL Divergence,

Earth-mover

Prof. Pallapa Venkataram, IISc

Lecture-01

Multimedia Segmentation

Figure: An Example of Region Segmentation

Prof. Pallapa Venkataram, IISc

Lecture-01

Issues/Challenges

Similarity Search & Indexing

Given a query vector q, find v in a database D of feature
vectors such that d(q,v) is the smallest (most similar).

Feature selection, dimension reduction, clustering-based

data structure (trees), and randomized search
Content Protection
Forgery detection
Authentication
Tracking
Secure distribution (scrambling)

Digital Watermarking for copyright, user ID, content ID,

authentication, data hiding.

Prof. Pallapa Venkataram, IISc

Lecture-01

Multimedia data

Multimedia data :
Digital audio, graphics, images and video, etc.,
Underlying concepts and representations of sound, pictures
and video,

Data compression JPEG/GIF, MPEG video and MPEG

Audio.
Transmission and Integration of media.
Multimedia applications : e.g. content based retrieval.

Prof. Pallapa Venkataram, IISc

Lecture-01

An Example: Image Compression

Figure: Format of Image Compression

Prof. Pallapa Venkataram, IISc

Lecture-01

Multimedia data

Multimedia data acquisition and formats: Audio, Graphics,

Images and Video
Audio/Video fundamentals including analog and digital
representations, human perception, and audio/video
equipment, applications.
Digital Audio signal processing, Image/Video Processing.

Prof. Pallapa Venkataram, IISc

Lecture-01

Text and Static Data

Source : keyboard, speech input, optical character
recognition, data stored on disk.
Stored and input character by character :
Storage of text is 1 byte per char / more bytes for Unicode.
For other forms of data (e.g. Spreadsheet files). May store
format as text (with formatting) others may use binary
encoding

Format : Raw text or formatted text e.g HTML, Rich Text

Format (RTF), Word or a program language source (C,
Pascal, etc..
Not temporal BUT may have natural implied sequence
e.g. HTML format sequence, Sequence of C program
statements.

Prof. Pallapa Venkataram, IISc

Lecture-01

Graphics

Format : constructed by the composition of primitive

objects such as lines, polygons, circles, curves and arcs.
Input : Graphics are usually generated by a graphics editor
program (e.g. Illustrator) or automatically by a program
(e.g. Postscript).
Graphics are usually editable or revisable (unlike Images).
Graphics input devices : keyboard (for text and cursor
control), mouse, trackball or graphics tablet.
graphics standards : OpenGL, PHIGS, GKS
Do not take up a very high storage overhead.

Prof. Pallapa Venkataram, IISc

Lecture-01

Images
Still pictures which (uncompressed) are represented as a
bitmap (a grid of pixels).
Input : digitally scanned photographs/pictures or direct
from a digital camera.
Input : May also be generated by programs similar to
graphic or animation programs.
Stored at 1 bit per pixel (Black and White), 8 Bits per pixel
(Grey Scale, Colour Map) or 24 Bits per pixel (True Colour)
Size : a 512x512 Grey scale image takes up 1/4 MB, a
512x512 24 bit image takes 3/4 MB with no compression.
This overhead soon increases with image size modern
high digital camera 10+ Megapixels 29MB
uncompressed
Compression is commonly applied.
Prof. Pallapa Venkataram, IISc

Lecture-01

Audio

Audio signals are continuous analog signals.

Input : microphones and then digitised and stored.
CD Quality Audio requires 16-bit sampling at 44.1 KHz
Even higher audiophile rates (e.g. 24-bit, 96 KHz)
1 Minute of Mono CD quality (uncompressed) audio requir
5 MB.
1 Minute of Stereo CD quality (uncompressed) audio requir
10 MB.
Usually compressed (E.g. MP3, AAC, Flac, Ogg Vorbis).

Prof. Pallapa Venkataram, IISc

Lecture-01

Video
Input : Analog Video is usually captured by a video camera
and then digitised.
There are a variety of video (analog and digital) formats
Raw video can be regarded as being a series of single
images. There are typically 25, 30 or 50 frames per
second.
E.g. A 512 x 512 size monochrome video images take 25 x
0.25 = 6.25MB for a second to store uncompressed.
Typical PAL digital video (720 x 576 pixels per colour
frame) 1.24 x 25 = 31MB for a second to store
uncompressed.
High Definition video on Blue-ray (up to 1920x1080 2
Megapixels per frame) 6.2x25 = 155MB for a second to
store uncompressed
Digital video clearly needs to be compressed for most
times.
Prof. Pallapa Venkataram, IISc

Lecture-01

Conclusions

Definition
Characteristics of a Multimedia System
Components of a Multimedia System
Issues/Challenges
Multimedia data

Prof. Pallapa Venkataram, IISc

Lecture-01

Thank you

Prof. Pallapa Venkataram, IISc

Lecture-01