Service Robotics: Past-Present-Future

• From Industrial Robots to Service Robots

• Current Applications and Markets
• Opportunities and Challenges

Martin Hägele
Haegele@ipa.fhg.de, +49 711 970 1203
Fraunhofer-Institut
Produktionstechnik und Automatisierung (IPA)
Stuttgart

April 16th, 2004

© Fraunhofer IPA Slide 1

Apr. 15, 04
MECHATRONIC
 Fraunhofer Society

Foundation in 1949
Resources Headquarter in Munich
12.000 Employees Non-profit research organisation
1000 M€ Annual turn-over
58 Research institutes
43 Locations in Germany
Branch-offices in USA (14),
Asia (9), Europe (8)

Research Fields
• Microelectronics
• Production
• Information and communication technologies
• Materials and components
• Life sciences
• Surface technology and photonics
Februar 2003
• Defence research and technology
© Fraunhofer IPA Slide 2
Apr. 15, 04
MECHATRONIC
 Fraunhofer IPA, Stuttgart
Fraunhofer Institut
Produktionstechnik und Automatisierung
Directors
Prof. Dr.-Ing. Dr. h.c. mult. Rolf Dieter Schraft (Speaker)
Resources
Prof. Dr.-Ing. Prof. E. h. Dr. h.c. Engelbert Westkämper
250 scientists, engineers Dr.-Ing. Dipl.-Wirtsch.-Ing. Wilfried Sihn (Deputy)
Turnover 2003: 32 M€
Industrial funding 57%
Offices: 5.100 m²
Laboratory area: 5.600 m²

Business Areas
• Corporate Management
• Enterprise Logistics
• Automation Systems
• Manufacturing Automation
• Manufacturing Technologies
Februar 2004
© Fraunhofer IPA Slide 3
Apr. 15, 04
MECHATRONIC
 Competences, Services in “Robot Automation” at IPA

Robot Systems

Processes Industrial Robots ServiceRobots

Joining processes Assembly

Market Studies - mechanical Material Flow
Personal Care

- hybrid - Palletizing Cleaning

Potential Analysis Transportation
- micro - Depalletizing
Systems Engineering 3D-shaping and - Warehousing Hazardous
forming - Sorting Environments
Specifications Cleaning Inspection/Repair
3D-Machining and
3D-Forming Rehabilitation
Hard- und Software Welding Entertainment

Realisations and
Implementations

Computer Controls Sensors Kinematics Interfaces

© Fraunhofer IPA Slide 4
Apr. 15, 04
MECHATRONIC
 Fraunhofer IPA, Center of Institutes

© Fraunhofer IPA Slide 5

Apr. 15, 04
MECHATRONIC
 50 Years in Robotics

Installation at Ford, USA, ca. 1964

Patent Claim 1954

© Fraunhofer IPA Slide 6

Apr. 15, 04
MECHATRONIC
 Robotics has Taken Many Forms

Inspection of reactor outer cores Liquid hydrogen refueling

Robot-framing-station Courier-robot in hospitals ASIMO (Newsweek 2/02)

Photos: KUKA Schweissanlagen, Ballard, Matsushita, Newsweek, IPA
© Fraunhofer IPA Slide 7
Apr. 15, 04
MECHATRONIC
 Industrial Robots World Wide: Installation and Shipments
Yearly installations of industrial robots 2002 Operational stock of industrial robots 2002
Number
of IR Number
Total Installationen Total Stock
60.000 2002: 68.566 of IR 2002: 769.889
2006: 91.100 (forecast) 500.000 2006: 875.300 (forecast)
50.000
40.000 400.000

30.000 300.000

20.000 200.000

10.000 100.000

0 0
2000 2002 2006 2000 2002 2006
Other Countries USA European Union Japan

IR per 10.000
Germany Japan • Assembly, welding, painting: 70 % of IR-stock
workers
• Growth potential: material flow automation
General 100 280 • Car industries: still 60 % of IR installations
Automotive 650 1273 • Applications in SME: less than 20 % of installations
industries
[Source: World Robotics 2003, UN/ECE, IFR]
© Fraunhofer IPA Slide 8
Apr. 15, 04
MECHATRONIC
 Robotics in New Processes and Applications

Drivers
• Decreasing cost of robots
• Growth industries
• Manual working conditions
• Quality assurance
• New processes

Current research objectives Robots in ware-houses:

• Rapid workcell deployment Palletising
Sausage packaging
• Programming, instruction IMT Nagler
• Integration into existing
environments
Postal automation
• Technical optimisation Siemens Dematic
(speed, flexibility, useability)
Brick-Laying

Baggage
handling

© Fraunhofer IPA Slide 9

Apr. 15, 04
MECHATRONIC
 Migration of Technologies into Industrial Automation
Entertainment Electronics
Flat Screens CD

PC Industrial Automation es
W - A g i
ind rchit n olo
ow ect ech Telecommunication
Computers s, ure - T
CE D
CC

Teleservice
n ted Teleoperation
O r ie
-
ct ues DECT, GSM
j e
Ob hniq Industrial CA
N
c sen -Bus
Te Automation
s
swi or
tch
es

Software Car-Electronics
© Fraunhofer IPA Slide 10
Apr. 15, 04
MECHATRONIC
 Why Robot Assistants?
Unit Costs
1000

Robot
100 automation
Automated transfer line
Manual
10 Manufacturing

Manual workplace Robot Automation

0,1 Units/Year
100 1.000 10.000 100.000 1.000.000

Graphs (left): Hasegawa, Boothroyd

Handbook of Industrial Robots, Wiley, 1984
© Fraunhofer IPA Slide 11
Apr. 15, 04
MECHATRONIC
 Why Robot Assistants?
Unit Costs
Push:
1000
140 Index
1990 = 100
120 Labour

gy
100

lo
Compensation
100

sh no
Cost

Pu ch
80

Te
Robot prices, reduction
not quality adjusted 10
60
40
Robot prices, quality adjusted

n
tio
20 1

ll ica
Flexibility

Pu ppl
0 increase
1990 1992 1994 1996 1998 2000 2002

A
0,1 Units/Year
Pull:
• Instant reaction to markets: Uncertain 100 1.000 10.000 100.000 1.000.000
production volumes, product life times
• Customized products at series prizes; „MC“ Robot helpers Rapidly
• Cost-effective productions close to markets at the manual deployable
• Simple and rapid deployment automation workplace robot work cells
• Managing the age-quake

© Fraunhofer IPA Slide 12

Apr. 15, 04
MECHATRONIC
 tea m @ work : Robot Without Fences
Reference collision
Characteristics targets measure

• Cooperation of worker
and robots
• Overlapping work spaces
• Variable degree of
automation
• Handling, assembly tasks

Challenges
• Intuitive programming
and instruction
• Safe physical interaction
• Overall work system
productivity and cost- Cameras, ceiling light
Conveyer
effectiveness SCARA-Robot
Cases
© Fraunhofer IPA Slide 13
Apr. 15, 04
MECHATRONIC
 Migration of Technologies out of Industrial Automation
Household Health Care

Cleaning Systems

Leisure

Industrial
Automation
Construction Machines Facility Management
© Fraunhofer IPA Slide 14
Apr. 15, 04
MECHATRONIC
 First Service Robots are Developed at the End of 80ies
Repair (Yaskawa Electric Corp., J) Sewer Maintenance (Hochtief, D) Vacuum cleaner (Panasonic , J) AMR (CASA, ES)

Floor grinding FLAT-KN (Shimizu Corp., J) Manus-Arm (Exact Dynamics, NL) FlorBot (GE Plastics, USA) Auto Sweepy (Toshiba, J)
© Fraunhofer IPA Slide 15
Apr. 15, 04
MECHATRONIC
 Definition “Service Robot”

„A robot which operates semi- or fully

autonomously to perform services
useful to the well-being of humans
and equipment, excluding manufacturing operations.“

© Fraunhofer IPA Slide 16

Apr. 15, 04
MECHATRONIC
 Industrial Robots in Service and Entertainment

RoboShop, Reis-Robotics Robo-Coaster, KUKA, LEGO, Billund

Automatic Drink Shop; Reis-Robotics Champaign-Bar Robo-Disc Jockeys (ZKM)

© Fraunhofer IPA Slide 17
Apr. 15, 04
MECHATRONIC
 Examples of ServiceRobots at IPA (1)

HAKO ACRO, 1996 Info-Mobil, 1996 IMP, 1995 Sewer Robot, 1998

Climbing robot, 1999 Tennis ball collector, 1997 Kärcher BR 700, 1996
© Fraunhofer IPA Slide 18
Apr. 15, 04
MECHATRONIC
 Examples of ServiceRobots at IPA (2)

Refuelling by robot, IPA, 1996 Skywash, PMW, 1998 Fire fighting robot, Iveco Magirus Surgical robot; URS, 2001

Museum robots, MPK Berlin, 2001 Care-O-bot I, 1998 Mona and Lisa of OPEL Zentrum, Berlin, 2003
© Fraunhofer IPA Slide 19
Apr. 15, 04
MECHATRONIC
 Examples of ServiceRobots at IPA (3)

Roof Cleaning, Lehrter Bahnhof, 2003 ARSE, 1998 DecontJet, Kärcher, 2003

Kärcher BR 700, 1996 Cleaning robot; Kärcher, 2000 RACCOON; P&G, 2002 Quirl, 2003
© Fraunhofer IPA Slide 20
Apr. 15, 04
MECHATRONIC
 Examples of ServiceRobots (4)

Rehabilitation-Equipment, Hitachi Prothetics, Otto-Bock MAid, FAW Ulm CMU Nursebot HelpMate, PYXIS Corp

CoWorker, iRobot TMSUK/SANYO: Quadruped LEGO Mindstorms K-Team, Khepera II R100, NEC
© Fraunhofer IPA Slide 21
Apr. 15, 04
MECHATRONIC
 Service Robots for Professional and Private Use
Service robots for professional use Service robots for private use
Number
of units Number of units
in thousands
10.000 1.600
Stock at end of 2002 1.400
8.000 Stock at end of 2002
projected installations 1.200 projected installations
in 2003-2006
6.000 1.000 in 2003-2006

800
4.000
600

2.000 400
200
0 0
Domestic Entertainment

er
ng

y
er

t p ling
n

ry
l
ica

ry
an form
rit
Un litio

to

th
at

st
ni

cu
ed

el

ra
rw

lo
re
ea

fu
Se

t
M

bo
fo
m

al
de

la
Cl

Re
De

La
e,

d
nc

bo
lla

re
ro
ei

tu
ile
rv

ul
Su

ob

ric
M

Ag

[Source: UNECE and IFR, 2003]

© Fraunhofer IPA Slide 22

Apr. 15, 04
MECHATRONIC
 An Industry is Being Created
Professional Applications Consumer-Products
• Application of industrial robots in non- • Extension of existing product lines by high-
manufacturing fields end
• Extension of existing machinery by automated automated products (e.g. robot vacuum
functions cleaners)
• Development of new robots from scratch for • Development of new products without
existing or new services. predecessors
• Installations by 2002: • Installations by 2002:
18.600 units, 2,625 Mio. US$ in value 606,840 units, 960 Mio. US$ in value
• Forecast 2003-2006: • Forecast 2003-2006:
29.780 new installations (3,280 Mio$ value) 2,163,000 new installations (2,750 Mio$ value)
Cyberknife, Accuray Brokk 330 MOSRO1, Robowatch Electrolux Trilobite Robox, Bluebotix Roboscience RS-01

© Fraunhofer IPA Slide 23

Apr. 15, 04
MECHATRONIC
 Indoor- and Outdoor Navigation

Scanner Gyros GPS Display US-Sensors Steering drive

PC µ Controler Control

CAN-Bus

CANtrol
TESS 1998-2000,
Kärcher 2001
Steering Throttle Incremental Motors Cleaning Interfaces
sensor weel sensor aggregate

© Fraunhofer IPA Slide 24

Apr. 15, 04
MECHATRONIC
 Software Framework for Mobile Robots

Lines of Anwendung
Code Plattform
45000 Robotics
40000 RealFrame

35000
30000
25000
20000
15000
10000
5000
0
1A 1B 1C 1 2

Robot 1A 1B 1C 1 2
Museum Guide Museum Guide Museum Guide Care-O-bot Cleaning Robot
Reuse of software 73,4 78,3 75,6 75,7 77,4
components in %
© Fraunhofer IPA Slide 25
Apr. 15, 04
MECHATRONIC
 Reuse of Software: Design Patterns and Repositories
Localization Robotics-Toolbox
Sensor-
control Design patterns

Position
control Realtime-Framework

Operating System

OSACA: Open System Architecture for

Controls within Automation systems
http://www.osaca.org/

http://www.orocos.org/

• Real-time motion kernel: K.U.Leuven

• CORBA-inspired communication primitives
for robotics: KTH Stockholm and FAW Ulm.
• Task execution sequencing: LAAS Toulouse
© Fraunhofer IPA Slide 26
Apr. 15, 04
MECHATRONIC
 Functionalities of Care-o-Bot

• Household tasks

• Fetch and carry-tasks

• Mobility support

• Communication and
social integration

• Monitoring and
safety

• Home Management

© Fraunhofer IPA Slide 27

Apr. 15, 04
MECHATRONIC
 R&D in Robot Sytems: The MORPHA Project (1999-2003)

Scenarios Cross-Functions
Home- and Manufacturing- - Channels for man-
Care-Assistant Assistant machine communication
- Scene-analysis and
interpretation
- Advising and adaptability
- Motion planning and
Spin-Off's
co-ordination
- Interactive task
planning
- Fail-safety, maintenance,
diagnosis

Demonstrations Procedures, Methods, Components

MORPHA was funded by the German BMBF

Pictures by Ruhr-UNI Bocum, DaimlerChrysler, IPA
© Fraunhofer IPA Slide 28
Apr. 15, 04
MECHATRONIC
 Hardware

electrical gripper

manipulator

wireless emergency gyroscope

button
wireless LAN
laser
scanner
platform control
robot control
battery position
encoder

© Fraunhofer IPA Slide 29

Apr. 15, 04
MECHATRONIC
 Initial Scenario: Assembly of Hydraulic Pumps

Storage of pump shafts

Robot-assisted workbench
Workbench assembly
assembly with

Brueninghaus Hydromatic,
Preassembly of pump shafts Bosch Rexroth, IPA, 2001
© Fraunhofer IPA Slide 30
Apr. 15, 04
MECHATRONIC
 Software Architecture

Man-Machine-Interface Robot System

Symbolic Domain
Goal list
Voice entry planner description

Scene editor
Action list
Command entry

Voice output Data channel Environmental Execution Plan

(CORBA) model module library
2D-Presentation

3D-Presentation Environmental Trajectory

Skills
data planner
Camera view

State output
Sensors Actuators

© Fraunhofer IPA Slide 31

Apr. 15, 04
MECHATRONIC
 User Interface

GUI-Configuration

3D-View

Command inputs

Symbolic map

© Fraunhofer IPA Slide 32

Apr. 15, 04
MECHATRONIC
 First Experiences with

+ -
• Operation time 12 hrs
Mechanical • Compact design • Only small payloads possible
Design • Safe mobility
Navigation • Collision free navigation • Limited mobility in industrial environment
• Fast path planning (doorsteps, cables, uneven floor)
• Unreliable computer vision (grasping etc.)
• Problems if deviation from map - real world
Task Planner • Robust and fast
• Difficult data acquisition in real situations
• Limited reliability of speech
Safety • Computer vision is • Safety sensors are only suitable for well-
promising for future defined control volumes (stationary safety)
safety systems • Controlled lighting for robust image
processing

© Fraunhofer IPA Slide 33

Apr. 15, 04
MECHATRONIC
 Limits and Constraints of
Possible Applications for Robot Assistants

• Industrial Engineering

Re-Engineering
• Tests

• Demonstrations

• Feedback & Discussions

Concentration on effective “Task Domains”

• Compact task
• Clear task allocation between worker and robot
• Clear interaction patterns
• Effectiveness in industrial environments reachable within years
© Fraunhofer IPA Slide 34
Apr. 15, 04
MECHATRONIC
 Definition of Constraint Task Domains

“Drill
“Weld Job”
Mate”

98°

“Mobile “Co-
Scanner” Processor”

© Fraunhofer IPA Slide 35

Apr. 15, 04
MECHATRONIC
 Scenario: Weld Mate

Video
Goals:
• Intuitive use
• Quality increase: constant welding velocity
• Instruction instead of programming

rob@work

FT-Sensor

Tool Coupler
Approach:
• PDA-User-Interface Welding Torch
• Implementation of
specific welding skills Worker hand
• Use of tactile information

© Fraunhofer IPA Slide 36

Apr. 15, 04
MECHATRONIC
 Example: Manual Velocity Vector Overide
1 Saw tooth pattern
5 cm

Ideal welding line Assisted welding line

V
E

2 Circle

• Cartesian stiffness control

• Support of constant welding feed
- Quality
- Documentation (customers)

© Fraunhofer IPA Slide 37

Apr. 15, 04
MECHATRONIC
 www.care-o-bot.de Robotic Home Assistants

Household Tasks
- Fetch and carry objects
- Execute everyday jobs
- Control home infrastructure
Mobility Aid
Communication and Social
Integration
- Media and time
management
- Communication with
medical and public facilities
- Supervision of vital
signs and emergency call

InHouse-Video (Juice)
InHouse-Video (Flowers)
Hannover Trade Fair, 2002
© Fraunhofer IPA Slide 38
Apr. 15, 04
MECHATRONIC
 Features (1)

Safe Manipulation in Human Environments

Problem/Motivation
• Collision-free manipulation
• Instant adaptation to altering positions
of objects and changing environments
Scientific Approach
• 3-D-scanning of environment
• “Rapidly-Exploring Random Trees” for
path planning
Novelty of Approach
• Using UB-trees for environment-model
to increase planning performance
• Global planning and local optimization
Results
• Collision-free 3-D paths within seconds

© Fraunhofer IPA Slide 39

Apr. 15, 04
MECHATRONIC
 Features (2)
Reactive Navigation of an Intelligent Robotic Walking Aid

Problem/Motivation
• Flexible path planning and modification
while being followed by a user
Scientific Approach
• Model of the robot followed by a user
• Fast Path Planning Algorithm for
non-holonomic vehicles
• Dynamic Path Modification “Elastic Bands”
Novelty of Approach
• Combining Elastic-Bands and “Dubin’s Car”
• Reacting on external forces applied to
walking aid handles
Results
• Time for path planning: 0.1 Sec.
• Smooth obstacle avoidance

© Fraunhofer IPA Slide 40

Apr. 15, 04
MECHATRONIC
 Features (3)
Interactive Robot Control for Robot Assistants

Problem/Motivation User
w
• Robustness in carrying out tasks Ne t a
da
in altering/un-known environments Symbolic Planner Domain World-knowledge

Feedback
• High-level commands

Goal
(„C“) (SQL-database)

Scientific Approach Action list

• Hybrid three-layer control Problem-
description
Facts

architecture
Execution Module “GO“
Novelty of Approach („Python“)
• Use of Scripting Language
Execute
• SQL-queries to reduce domain Action list Sensordata
State
Results
Robot Control:
• Execution Framework for con- Behaviors („C++“)
current tasks and error handling
• Task description language
Actuators Sensors

© Fraunhofer IPA Slide 41

Apr. 15, 04
MECHATRONIC
 Beispiel von Humanoiden (Japan)
Humanoid Robot Hadaly-2,
Waseda Universität, Japan
SDR-3X (Sony Corp.)

Human Symbiotic Robot WENDY

(Waseda ENgineering Designed sYmbiont) Der Honda-ASIMO: Anzeige in „Newsweek“ vom 5. Feb. 2001

© Fraunhofer IPA Slide 42

Apr. 15, 04
MECHATRONIC
 Honda-Humanoid: Anwendungsszenarien

Physical
Interaction

Services in public Servant

environments
Weiterentwicklung Assistant
• Gewichtsreduktion Public (Workplace) Privat
• Kosten use use
• Sicherheit
• „Social Harmonization“ Entertainment
Pet
Anwendungs-“Roadmap“
1. Entertainment (kurzfristig)
2. Attendant (Services)
3. Work Assistant Emotional/social
4. „Robot as a partner“ interaction
(langfristig) T. Fukui: „The Honda Humanoid Robot“, Plenarvortrag Diskussion, ISR, Seoul, 18. April 2

© Fraunhofer IPA Slide 43

Apr. 15, 04
MECHATRONIC
 Conclusion

• For more than 20 years robots have been entering non-manufacturing applications.
• Some 200 product ideas and applications have been collected (by IPA)
• Service robots have established themselves as a new high potential product category
• Service robots still are specialists designed for a specific application.
• The breakthrough of service robots have been depending on smart components
(e.g. scanner for mobile navigation)
• There is a high potential in sharing software components
• Assistive systems emerge as a category for both manufacturing and service automation
• Extensive RTD in system dependability (capabilities and safety)
• Man-machine interaction as a major challenge in service robotics
• But also good product design and engineering

© Fraunhofer IPA Slide 44

Apr. 15, 04
MECHATRONIC
 Safety System

Safety-Sensors:
• Laser scanner
• Image processing -
surface matching
• Image processing -
differential image

Result:
• Promising approaches
• Safety conformability
still open

© Fraunhofer IPA Slide 45

Apr. 15, 04
MECHATRONIC