Research Proposal

RESEARCH TOPIC:
Designing of Highly Transparent and Stable Teleoperation System for Disaster Response Robot
INTRODUCTION:
Teleoperation system has been used in many applications ranging from remote surgery, tasks in
dangerous environments, education, and training, but disaster like Fukushima Daiichi nuclear
accident (Fig. 1) shows that current disaster-response robots are not sufficient to be used to
support the rescue workers. First two robots which entered Fukushima nuclear plant were
PackBots, robots manufactured by the American Company iRobot. PackBots were built to survive
in hazardous environments and have been used by the military to diffuse bombs. The robots had
cameras and special sensors to get the needed measurements such as radiation level. They were
able to enter the plant and retrieve initial measurements of the situation. These measurements
were used to assess the location and duration for operators to go inside the plant while staying
within the safety limits of radiation [1, 2]. To detect the leaks in the reactors water pool, a robot
made by Hitachi-GE Nuclear Energy Company was also used. Robots, such as the ones developed
by Raccon from Atox, are expected to be used for the decontamination of certain areas. To
ensure operators safety, plugging of leaks is performed by the robotic arm developed by Kurion
[1, 3, 4].
However, till now disaster-response robots are just able to be used for inspection and
decontamination but, to deal with disasters like Fukushima nuclear plant, more complicated
robots are required that should be designed to perform special task [1].

Fig. 1 Fukushima nuclear accident

To address aforementioned issues, a human-robot teleoperation system needs to be designed

that is able to perform complex tasks in rough terrain and serious disastrous conditions.
The aim of my research during PhD is to develop a human-robot teleoperation system where a
human is telepresent with his whole body in an environment affected by a disaster, like
Fukushima. Following are the main goals of my PhD,
1. I will design highly transparent bilateral teleoperation system based on one of the
following architecture:
Four channel architecture(close loop master-slave system)
Model mediated architecture (open loop master-slave system)
Hybrid architecture (combination of the above two approaches)
2. In teleoperation system there is tradeoff between transparency and stability. Due to
delay in communication channel there is issue of stability. I am intending to use
passivity based state-of-the-art controller, Time Domain Passivity Approach (TDPA) to
tackle the stability problem.
3. To enhance the performance of teleoperation system, only kinesthetic haptic master
interface is not enough. To give the operator a real sense of the environment, I will
develop a 3-DOF fingertip cutaneous display.

METHODOLOGY:
I have divided three years of my research plan into the following four main phases,
Phase # 1, Development of 3-DOF cutaneous fingertip display (Six months):
Cutaneous display will play an important role to enhance the transparency of teleoperation
system. Cutaneous display is the device which gives the sense of touch as well as helps the
operator to recognize the shape and orientation of the remote or virtual object. Nowadays,
cutaneous displays are gaining significant attention due to their applications in different areas
like medical, virtual reality and teleoperation tasks. Several tactile haptic devices have been
designed to be portable or wearable. There are three main approaches used to generate
cutaneous feedback in wearable devices: 1) system generating vibrations 2) pin-arrays deforming
the skin to simulate given shape and 3) mechanism applying three-dimensional vector forces at
one or more contact points. Based on three different approaches mentioned for generating
cutaneous force feedback, several researches have been done. In [5-8], authors presented
different cutaneous displays based on the first approach. Although these cutaneous haptic
devices can be considered wearable, their force feedback is limited to vibrations, thus limiting
their possibility of simulating richer force patterns.

In [9-10], authors developed the cutaneous displays based on the second approach. Their
implementation achieved a compact design, but it still required to develop an external drive unit
for the actuation system, thus compromising portability.
The third approach to wearable tactile display consists of applying three-dimensional force
vectors. In [11-14], authors proposed different cutaneous displays based on the third approach.
Since in existing cutaneous devices, actuators are mounted on the fingertip which make the
overall system heavier. On the contrary, in order to perceive the real feeling of virtual or
remote environment, the overall system should be much lighter. The second issue in the
existing device is that the belt, responsible to exert the force on fingertip, is always in contact
with the fingertip, therefore, operator feels difficulty to distinguish when it is interacting with
virtual or remote environment. Third issue in the existing device is that it is driven by a gear
motors which causes the weight of the device to be increased. In [15], author tried to solve
mentioned issues and developed an effective 3-DoF wearable cutaneous display in which motors
were placed on the forearm and two cables for each actuated finger were necessary to transmit
the motor torque.
Proposed Idea:
In order to perceive the realistic cutaneous feeling, the cutaneous display itself should be as light
as possible and it should not be in contact with the human body when there is no virtual or
remote environment. I have proposed a new design based on the third approach for generating
cutaneous force feedback. All the actuators are attached to the wrist and use Bowden cables in
order to transfer motor torques to actuate the thimble attached to the fingertip. On fingertip
small assembly made by 3D printer is used, which has mass of only 5.6g which is really light and
operator does not feel it at all. On this assembly a spring is attached so that it detaches from the
fingertip when there is no virtual or remote environment. Fig.2 shows the prototype of the
proposed design. Gear in the actuator increases its weight, therefore, in order to reduce the mass
of the actuators, twisting wire actuators [16] will be used, which are light weight and give enough
power.

Fig.2 Proposed design for 3-DOF fingertip cutaneous display

After developing cutaneous display, rigorous testing will be done. Mechanical design and
experimental results will be published in one of the good conferences and journals in order to
benefit the robotics community.
Phase # 2, Designing of highly transparent bilateral teleoperation system (18months):
A bilateral teleoperation system allows the operator to manipulate the remote environment
through master-slave devices and also enable the operator to feel the environment force. In
bilateral teleoperation, master-slave are kinesthetically coupled with each other. This setting
allows a direct exchange of velocity and force signal between the master-slave devices, which
makes the teleoperation system more intuitive. However, there is always a tradeoff between
transparency and stability. The ideal bilateral teleoperation system acts as a massless, infinitely
stiff connection between the master and slave [17-18]. The ideal system should provide exact
mapping between environment impedance and impedance felt by the operator, while remain
stable at the same time. Realizing this ideal behavior is impossible, especially in case of
communication delay, therefore, different researchers have proposed different control
architectures to achieve transparency and stability both at the same time.
1. Four Channel Architecture
There are mainly three types of close loop control architectures used for bilateral
teleoperation system, namely: 1) Position-Position Architecture (P-P), 2) Position-Force
Architecture (P-F), and 3) Four Channel Architecture. Here, close loop means master and
slave directly send signals to each other. In all the three architectures there is always a
tradeoff between transparency and stability, P-P architecture is more stable but poor
transparency, P-F architecture is more transparent than P-P architecture at the cost of
stability, similarly among all the three architectures four channel (see Fig.3) architecture
is highly transparent but shows more instability. In this research project I am planning to
use four channel architecture to get higher transparency, whereas stability issue can be
solved by the passivity controller (in the third phase of the project).

Fig.3 Block diagram of Four Channel Architecture

2. Model Mediated Architecture

Due to large communication delay, the quality of feedback in bilateral teleoperation
degrades in case of close loop bilateral teleoperation system. Even, if it is properly
stabilized by means of some passivity based controller, delayed force feedback distracts
the operators perception about the environment and make it very difficult to manipulate
the environment.
To tackle this issue researchers have proposed a method called Model Mediated
Telemanipulation, in which rather than directly sending the slave data to the master,
model mediated teleoperation, abstract the data to form a simple model of the
environment and the model is transmitted to the master where it is haptically and
graphically rendered for immediate user feedback without any lag [19]. Model mediated
teleoperation (see Fig.4) enhances the performance of the overall teleoperation system
even in case of large delay, but in case of complex environment, accurate modelling of
the environment becomes a challenging task.

Fig.4 Model Mediated Telemanipulation Scheme (courtesy of Telerobotics Lab,

Stanford University, USA)

3. Hybrid Architecture
As discussed above both the architectures (Four channel and Model mediated) have
their advantages and disadvantages, therefore in my PhD research, I will develop a
hybrid control scheme which can take the advantages of both of the architectures and
make us able to design a highly transparent and stable teleoperation system.
Finally, based on the designed architecture, experimental results will be published in one of
good conferences and journals.

Phase # 3, Implementation of passivity based state-of-the-art controller to guarantee the

stability of the overall teleoperation system (Six months):
Third phase of the research will be done by the end of 2.5th year of the project. In teleoperation
system, there is always a compromise between transparency and stability, especially when
master and slave are closed in a feedback loop, this is so called bilateral teleoperation (see Fig.
6). The main goal of bilateral control architecture is to ensure the stability of the system while
maintaining the high transparency, so that the system does not vibrate or diverge, under any
operating conditions or any environment. The main source of instability in teleoperation system
is the delay in the communication channel which causes energy leak and ultimately makes the
system unstable (see Fig. 5). Therefore, a different bilateral control architecture has been
developed to achieve the desired performance of the teleoperation system [20, 21]. In [22-27],
several methods have been proposed based on linear circuit theory, linear robust control theory
and nonlinear control laws to guarantee the stability of teleoperation system.
Other approach to design a stable bilateral controller is to utilize the passivity criteria to
guarantee the stability of teleoperation system, which does not require the exact dynamics of
teleoperation system. In [28-32], authors proposed passivity based controller to guarantee the
stability of the overall teleoperation system.
Time Domain Passivity Approach [33] is the promising approach among all passivity based
approaches to guarantee the stability of the overall teleoperation system. To achieve the stability
of the teleoperation system, I am intending to use Time Domain Passivity Approach in this
project.

Fig. 5, Network representation of bilateral teleoperation

Phase # 4, Rigorous tests to validate the effectiveness of the designed human-robot system
and submission of final report (Six months):

Fourth phase of the research will be done by the end of 3rd year of the project. During this period
rigorous tests will be conducted to validate the performance of the developed overall humanrobot teleoperation system. Conference and journal papers will be published in one of the
reputed conferences and journals in the area of robotics like IROS, ICRA, RSS, World Haptics and
IEEE transactions on robotics etc. Finally, final report will be submitted for the PhD defense.
REFERENCES:
[1] E. Strickland, "Fukushima's Next 40 Years," IEEE Spectrum 51, No. 3, 46 (2014) [Republished as
"Dismantling Fukushima: The World's Toughest Demolition Project," 28 Feb 14.
[2] K. Mori, "Robotics in Nuclear Disaster Response," PH241, Stanford University, Winter 2013.
[3] M. Rodriguez, "Colorado Robot to Help Fix Japan Nuclear Disaster," Denver 9 News, 13 Apr 14.
[4] Kevin Mori, http://large.stanford.edu/courses/2013/ph241/mori1/
[5] R. Traylor and H. Tan, Development of a Wearable Haptic Display for Situation Awareness in AlteredGravity Environment: Some Initial Findings, Proc. 10th Symp. Haptic Interfaces for Virtual Environment
and Teleoperator Systems, pp. 159-164, 2002
[6] J. Lieberman and C. Breazeal, TIKL: Development of a Wearable Vibrotactile Feedback Suit for
Improved Human Motor Learning, IEEE Trans. Robotics, vol. 23, no. 5, pp. 919- 926, Oct. 2007.
[7] H. Kim, C. Seo, J. Lee, J. Ryu, S. Yu, and S. Lee, Vibrotactile Display for Driving Safety Information,
Proc. IEEE Intelligent Transportation Systems Conf., pp. 573-577, 2006.
[8] S. Scheggi, F. Chinello, and D. Prattichizzo, Vibrotactile Haptic Feedback for Human-Robot Interaction
in Leader-Follower Tasks, Proc. ACM Fifth Intl Conf. PErvasive Technologies Related to Assistive
Environments (PETRA 12), 2012.
[9] G. Yang, K. Kyung, M. Srinivasan, and D. Kwon, Quantitative Tactile Display Device with Pin-Array Type
Tactile Feedback and Thermal Feedback, Proc. IEEE Intl Conf. Robotics and Automation, pp. 3917-3922,
2006.
[10] I. Sarakoglou, N. Garcia-Hernandez, N. Tsagarakis, and D.Caldwell, A High Performance Tactile
Feedback Display and Its Integration in Teleoperation, IEEE Trans. Haptics, vol. 5, no. 3, pp. 252-263, JulySept. 2012.
[11] K. Minamizawa, S. Fukamachi, H. Kajimoto, N. Kawakami, and S.Tachi, Gravity Grabber: Wearable
Haptic Display to Present Virtual Mass Sensation, Proc. ACM SIGGRAPH, article 8, 2007.
[12] B. Gleeson, S. Horschel, and W.R. Provancher, Design of a Fingertip-Mounted Tactile Display with
Tangential Skin Displacement Feedback, IEEE Trans. Haptics, vol. 3, no. 4, pp. 297-301, Oct.-Dec. 2010.
[13] K. Kuchenbecker, D. Ferguson, M. Kutzer, M. Moses, and A.Okamura, The Touch Thimble: Providing
Fingertip Contact Feedback during Point-Force Haptic Interaction, Proc. Symp. Haptic Interfaces for
Virtual Environment and Teleoperator Systems, pp. 239-246, 2008.
[14] Prattichizzo, D.; Chinello, F.; Pacchierotti, C.; Malvezzi, M., "Towards Wearability in Fingertip Haptics:
A 3-DoF Wearable Device for Cutaneous Force Feedback," Haptics, IEEE Transactions on, vol.6, no.4,
pp.506, 516, Oct.-Dec. 2013

[15] M. Solazzi, A. Frisoli, and M. Bergamasco, Design of a Cutaneous Fingertip Display for Improving
Haptic Exploration of Virtual Objects, Proc. IEEE Intl Symp. Robots and Human Interactive Comm., pp. 16, 2010.
[16] Popov, D.; Gaponov, I.; Jee-Hwan Ryu, "A study on twisted string actuation systems: Mathematical
model and its experimental evaluation," Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ
International Conference on , vol., no., pp.1245,1250, 7-12 Oct. 2012
[17] G. R. Raju, G. Verghese, and T. Sheridan, Design issues in 2-port network models of bilateral remote
manipulation, in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 3,
Scottsdale, AZ, May 1988, pp. 13161321.
[18] Willaert, B.; Reynaerts, D.; Van Brussel, H.; Vander Poorten, E.B., "Bilateral Teleoperation:
Quantifying the Requirements for and Restrictions of Ideal Transparency," Control Systems Technology,
IEEE Transactions on, vol.22, no.1, pp.387, 395, Jan. 2014
[19] Mitra, P.; Gentry, D.; Niemeyer, G., "User Perception and Preference in Model Mediated
Telemanipulation," EuroHaptics Conference, 2007 and Symposium on Haptic Interfaces for Virtual
Environment and Teleoperator Systems. World Haptics 2007. Second Joint, vol., no., pp.268, 273, 22-24
March 2007
[20] S. Salcudean, K. Hashtrudi-Zaad, S. Tafazoli, S. DiMaio, and C. Reboulet. Bilateral matched impedance
teleoperation with application to excavator control. Control Systems, IEEE, 1 (6):2937, Dec 1999.
[21] Y. Yokokohji and T. Yoshikawa. Bilateral control of master-slave manipulators for ideal kinesthetic
coupling-formulation and experiment. Robotics and Automation, IEEE Transactions on, 10(5):605620,
Oct 1994.
[22] H. Arai, K. Tanie, and N. Shiroma. Nonholonomic control of a three dof planar underactuated
manipulator. Robotics and Automation, IEEE Transactions on, 14(5):681695, Oct 1998.
[23] H. Yabuno, M. Miura, and N. Aoshima. Bifurcation in an inverted pendulum with tilted high-frequency
excitation: analytical and experimental investigations on the symmetry-breaking of the bifurcation.
Journal of Sound and Vibration, 273(3):493 513, 2004.
[24] J. Yan and S. Salcudean. Teleoperation controller design using hinfin;-optimization with application
to motion-scaling. Control Systems Technology, IEEE Transactions on, 4(3):244258, May 1996.
[25] K.-H. Yu, Y. Shito, and H. Inooka. Position control of an underactuated manipulator using joint friction.
International Journal of Non-Linear Mechanics, 33(4):607 614, 1998.
[26] J. Ryu and D. Kwon. A novel adaptive bilateral control scheme using dual closed-loop dynamic
characteristics of master/slave manipulators. In Intelligent Robots and Systems, 2000. (IROS 2000).
Proceedings. 2000 IEEE/RSJ International Conference on, volume 2, pages 1173 1178 vol.2, 2000.
[27] W.-H. Zhu and S. Salcudean. Stability guaranteed teleoperation: an adaptive motion/force control
approach. Automatic Control, IEEE Transactions on, 45(11):19511969, Nov 2000.
[28] G. Niemeyer and J.-J. Slotine. Stable adaptive teleoperation. Oceanic Engineering, IEEE Journal of,
16(1):152162, Jan 1991.

[29] C. Seo, J.-P. Kim, J.-P. Kim, J. H. Yoon, and J. Ryu. Stable bilateral teleoperation using the energybounding algorithm: Basic idea and feasibility tests. In Advanced Intelligent Mechatronics, 2008. AIM
2008. IEEE/ASME International Conference on, pages 335340, July 2008.
[30] Y. Yokokohji, T. Imaida, and T. Yoshikawa. Bilateral control with energy balance monitoring under
time-varying communication delay. In Robotics and Automation, 2000. Proceedings. ICRA 00. IEEE
International Conference on, volume 3, pages 26842689 vol.3, 2000.
[31] M. Franken, S. Stramigioli, S. Misra, C. Secchi, and A. Macchelli. Bilateral telemanipulation with time
delays: A two-layer approach combining passivity and transparency. Robotics, IEEE Transactions on,
27(4):741756, Aug 2011.
[32] J.-H. Ryu. Bilateral control with time domain passivity approach under time-varying communication
delay. In Robot and Human interactive Communication, 2007. RO-MAN 2007. The 16th IEEE International
Symposium on, pages 986991, Aug 2007.