Sunteți pe pagina 1din 4

Psychomotor Learning in Martial Arts: an Opportunity for

User Modeling, Adaptation and Personalization


Olga C. Santos
aDeNu Research Group, UNED
Calle Juan del Rosal, 16. Madrid 28040
Spain
ocsantos@dia.uned.es
ABSTRACT1 sensorial support to correct those movements that significantly
differ from the instructor’s [2]. Literature shows that the
Psychomotor learning is crucial for many kind of tasks that monitoring is usually done either with optical motion capture
involve the acquisition of motor skills, such as practicing martial systems [3] and/or wearable inertial sensors [4]. In order to
arts. In the past two decades, diverse technological solutions compare the instructor’s and learner’s performance, a modeling
have been developed to support the learning of the process is required. Although 3D skeletal models are sometimes
corresponding motor skills. However, the UMAP (User provided by the motion capture system, usually ad-hoc
Modeling, Adaptation and Personalization) community has not algorithms are implemented, which take into account the
taken part in that research efforts and thus, resulting systems do specific user attributes obtained during the initial calibration
not adapt and personalize the system response to their users’ process [5]. Typically, the system response is visual, although
needs. This paper discusses the main features of existing systems auditory and haptic feedback can also be provided [6].
to learn martial arts and identifies research opportunities that There exist many tasks that involve the acquisition of motor
are to be included in the future agenda for UMAP research. skills, such as dancing, playing a musical instrument or
practicing martial arts. This paper focuses on the later. In
KEYWORDS particular, Section 2 analyses existing systems to learn martial
Personalization; Adaptation; User Modeling; Psychomotor arts, Section 3 analyzes the movement modeling and feedback
learning; Motor skills; Martial arts delivery approaches used in them, Section 4 discusses the user
modeling, adaptation and personalization (UMAP) issues
addressed in them, and finally Section 5 identifies opportunities
ACM Reference format:
for the UMAP research in order to personalize psychomotor
O. C. Santos. 2017. Psychomotor Learning in Martial Arts: an learning of martial arts.
Opportunity for User Modeling, Adaptation and Personalization.
In UMAP'17 Adjunct proceedings, Bratislava, Slovakia, July 2017
(UMAP’2017), 4 pages. 2 TECHNOLOGICAL SUPPORT FOR
DOI: 10.1145/3099023.3099107 MARTIAL ARTS LEARNING
Martial arts learning systems aim to improve the technical
1 INTRODUCTION execution of predefined movements by monitoring relevant
motion features that can serve to identify errors with respect to
Psychomotor learning is usually performed by imitation (i.e.,
the performance of an expert [7] and avoid injuries [8].
repeating an observed pattern) and involves memorizing
Technological solutions to learn and practice martial arts have
different kinds of information and behaviors [1]. Personalizing
already been proposed in the late 90’s. To my knowledge, the
psychomotor learning implies monitoring the physical
first approach reported in the literature is the Sensei system
interaction of the user to build the corresponding user model,
developed in 1997 [9]. It recognized some gestures of Tai Chi
comparing the movement performed by the user against the
martial art tracked with 3D stereo cameras and identified the
movement as it would be carried out by an instructor to
moment in the sequence where the learner’s gesture most
diagnose motion errors and, if needed, deliver the personalized
differed from the instructor’s.
Recognizing martial arts movements is more complex than
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or
other human activities such as jumping, sitting down or walking
distributed for profit or commercial advantage and that copies bear this notice and [10]. Nonetheless, not all martial arts have the same level of
the full citation on the first page. Copyrights for components of this work owned complexity. For instance, as commented in [11], movements in
by others than the author(s) must be honored. Abstracting with credit is permitted.
Tai Chi are i) done individually by each person replicating the
To copy otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee. Request permissions from posture of the instructor, ii) their nature is slow, and iii) learner
Permissions@acm.org. and instructor always face the same direction. These features
UMAP'17 Adjunct, July 09-12, 2017, Bratislava, Slovakia (and not so much its benefits on physical well-being [12, 13])
© 2017 Copyright is held by the owner/author(s). Publication rights licensed to
ACM. ACM ISBN 978-1-4503-5067-9/17/07 $15.00
seem to be the main reason for the popularity of systems for
http://dx.doi.org/10.1145/3099023.3099107 learning Tai Chi in the literature [9, 11, 14, 15, 16, 17]. Tai Chi
also differs from most martial arts in the lack of fights.
Learning techniques for fighting adds complexity because a gyro sensor to align the camera with the learner’s posture
movements are to be performed faster and depend on the when calibration is not done [15] or with a Kinect RGBD sensor
behavior of the partner. However, attacks can be practiced to reduce the cost of the camera [21], ii) a Kinect sensor without
individually by the learner striking to an imaginary learner markers [8, 22, 28], iii) video cameras and image processing [9,
placed in front (although this tends to make learners move 18], and iv) a drone camera that can capture the images of the
carelessly [18]). In that case, practicing attacks differs from user from any angle [14]. In turn, wearable sensors can be placed
performing Tai Chi movements in the speed (explosiveness of on parts of the user body (e.g., wrists, arms, legs, knees, neck,
the execution). Furthermore, when the goal of the techniques to backbone origin) to collect inertial information from the
learn is defensive (such as in the case of Aikido), the practice movement without any additional infrastructure but requiring
becomes more complex as a real opponent is needed to start an additional processing to compensate body-mounted sensors
attack so the learner can respond with the defensive technique. inaccuracies [17, 25]. Both approaches can be combined, as in
Defensive aspects deal with i) moving away from the line of [26] that uses camera and LED markers to follow the body
action of the unwanted incoming force that is attempting to position and an accelerometer on the wrist to measure tilt,
produce harm and ii) placing the attacker in a safe and movement and vibration, and as in [24] that uses a camera,
incapacitated situation that is no longer a threat [19]. microphone and accelerometer to measure wrist acceleration.
Additionally, weapons can be used in the attack and/or to The feedback provided to the learner is mainly visual, either
defend, thus, their movement also need to be modeled [20]. in i) large displays on the wall [16, 24, 26], ii) embedded in a
With this in mind, it is not surprising that most systems computer game [18, 22, 27] or iii) enhanced with 3D see-through
developed for learning martial arts that involve fights only head mounted displays that immerse the learner in a virtual
provide support for practicing specific postures and gestures by reality environment that always keeps the instructor in the line
mimicking given instructors’ movements (as it is done in Tai of sight [11, 14, 15]. Regarding the learner perspective, there are
Chi). This is the case of several martial arts systems for Karate two alternative views, i.e. first-person looking directly at the
[21, 22, 23, 24], Kung Fu [18, 25], Taekwondo [26] and Muay instructor [11, 14] and third-person watching both the learner
Thai [8], although in one of the Karate systems (i.e., [24]) it was and the instructor, as if looking at a mirror [14, 18]. In some
mentioned the possibility a future extension to consider fights. cases, it is possible to superimpose the learner’s body on the
Nonetheless, there are few works that address the dual instructor’s body [11, 14]. Diverse specific information of the
monitoring required for defensive martial arts, such as capturing performance can be provided as feedback, such as the center of
the movements of both attacker and defender in Muay Thai [27] gravity and the breathing status [15], consumed energy [17],
and in Seni Silat Cekak [28], or detecting the instability of a explosiveness [25], energy line with intensity proportional to the
posture when pulled by the opponent in Shorinji Kempo [29]. In error produced [16, 21, 28], statistical information of the
addition, devices that help judges scoring body kicks, such as the movements’ performance [16, 21, 28], acceleration data [26],
SensorHogu system for Taekwondo matches [30], can be used to progress indicator of correct position [26], score that shows the
evaluate the learning performance in terms of this scoring. accuracy of the learner’s position with the instructor’s [8] and
In any case, the only technological solution found for display of the wrong positions [8, 9]. Other sensorial channels
learning techniques to be used in fights is the interactive used are auditory (music to control the rhythm of training and
training tool Aikido3D [31]. In this tool, virtual characters sound variations proportional to wrist acceleration [24]) and
(created by recording high degree practitioners with motion tactile (vibration proportional to the amount of error [23]).
capture techniques) show the execution (from different angles) Multimodal feedback as in [16] can complement the visual
of all the techniques up to black belt. This solution is more information displayed on the wall (instructor showing the next
advanced than widely used YouTube videos as learners can movement and learner performing the current movement) with
control the position of the camera when watching the virtual sound changes that point to the location of the error and
characters performing the technique. However, Aikido3D does vibrotactile feedback whose intensity is proportional to error.
not collect nor process user data, thus, no movement modeling is The 16 systems analyzed aim to facilitate learning some
performed nor feedback delivered to the users. aspects of martial arts mainly relative to improving the
performance of predefined postures and gestures. To achieve
this, some of them have been developed as games for
3 MOVEMENT MODELING AND FEEDBACK edutainment [8, 11, 18, 22, 27]. In addition, it is also possible to
DELIVERY IN MARTIAL ARTS SYSTEMS interact in real-time with a virtualized instructor through
The previous section has identified 16 systems [8, 9, 11, 14, 15, Internet [8]. Moreover, the systems can serve also to produce
16, 17, 18, 21, 22, 23, 24, 25, 26, 27, 28] that model the movement enriched video materials with movements of real experts
carried out by the learner when practicing martial arts and obtained with motion capture technology [15, 26]. However, and
provide some feedback to support its learning. despite their existence for over 20 years, the technological
Movement modeling is performed in diverse ways. Some support provided is far from facilitating learners an adapted and
systems track the full body visually with: i) optical motion personalized support along their lifelong learning process of the
capture technology that uses infrared cameras and reflective corresponding motor skills to master the performance of the
markers on the body [11, 16, 23, 27] which can be combined with martial arts techniques.
4 UMAP ISSUES ADDRESSED IN MARTIAL be adjusted in terms of the performance achieved [18]. In
ARTS SYSTEMS addition, if the system provides scores of the performance, these
can be used to reward those users that follow complex preceding
As discussed above, no martial arts system has been found in the
patterns and penalize those users who focus on practicing only a
literature that provides adapted and personalized support for
subset of techniques because they are more effective and fast
learning the required motor skills to perform the corresponding
[18]. In this way, personalized gamification might be considered
techniques. Some of those works have suggested the viability for
when rewarding.
personalized training [26] as well as benefits of the personalized
Thus, there are many opportunities for UMAP community to
support in adapting the output to the learner’s skilled level [22,
research on personalizing and adapting the psychomotor
24, 25], so learners can practice at their own pace [24]. In this
learning in martial arts. Some ideas for them are compiled next.
sense [22] discusses that matching the skill level can increase the
likelihood of experience flow, but if adaptation is not properly
introduced, it can cause the opposite effect [32]. 5 OPPORTUNITIES FOR UMAP RESEARCH
Specific user features to be involved in the modelling are
Martial arts systems have been developed for the last two
considered in the calibration process when using optical motion
decades focused mainly on helping users mimic the postures and
capture to ensure that the image coordinates regarding
gestures of the instructor and providing non-personalized visual
kinematic data are correctly scaled to size [33]. In this way, it
feedback of the learner’s execution. In turn, as discussed
can cope with users’ anatomical differences when building the
elsewhere [37], the current edition of the UMAP conference
model of the body actions [34]. For the calibration, typically the
already presents some works focused on modeling the physical
height of the learner [8, 21, 22] or the length of the arms are used
activity of users, but considering other domains. Thus, this
[16], although a more detailed calibration of all joints can also be
seems a good time for the UMAP community to step in the
performed [11]. In addition, differences between beginners and
development of martial arts systems that fully support the whole
experts have also been mentioned in several works: 1) novice
learning needs and adapt in a multisensorial way not only to
learners struggle to memorize the sequence of movements, while
individual physical practice but also to physical practice in pairs
experienced learners focus on refining them [11], 2) beginners
(as well as in groups) performing both specific movements in
find difficult to pay attention to motions in various parts of the
sequence and interacting in free-style fights.
body and to check correctness of small motions, as well as they
Current martial arts systems already collect some user
need to be led to get success experiences, while experienced
information, mainly kinetic and kinematic [7], to model the
learners need to be encouraged to create new practice by
movements carried out. Although this information can be
themselves [24], 3) advanced users want the system to be more
valuable for personalization purposes, it may probably not
realistic [18], 4) experts consume less rotational energy and
suffice and should be extended with other types of information.
produces inertial signals smoother and more periodical,
In particular, as discussed in [18], physical performance in terms
movements are faster, feet make less pauses and choppy
of physiological parameters can be taken into account and can
movement [17], 5) expert’s inertial signals are clearly and more
serve to adjust the level of the tasks to be carried out by the
regular and can also serve to compute intra-user differences
learner, since these parameters can impact on techniques
between her left and right side to identify the better trained body
performance [38]. In addition, martial arts are not only about the
half [25]. Moreover, there also seems to be differences in
body, but also about the mind. Thus, the affective state of the
learning styles (improving gestures vs. postures) and gender
learner should also be computed with appropriate techniques
(males use power vs. females focus on technique) [26].
[39]. Moreover, the practice of martial arts requires memorizing
In turn, some open issues regarding the delivery of the
complicated forms of motion that are to be performed with
appropriate feedback have also being identified in the works
highly accurate positions and gestures, sometimes in a specific
analyzed. Depending on the learners’ features and their
sequence, thus some intellectual abilities are also of relevance to
performance, it might be better to provide feedback for praise or
solve the dynamic situations that arise in a fight, and thus
feedback for calling for attention [24]. The amount of feedback
psychomotor skills cannot be acquired just by repetition of
needs also to be controlled [24], not only because of the human
motor patterns without consider the interrelation between
limitation to sensing different feedback outputs (e.g., the number
cognitive processes and motor actions [40]. Thus, as discussed in
of different sounds that can be distinguished is low [24],
[37], a user model that extends cognitive and affective
vibratory movements that are too close cannot be distinguished
information with psychomotor aspects needs to be defined,
[23]) but because reducing the feedback can improve the motor
which should include, among others, information about the body
skill learning [35]. User habituation to the feedback signals needs
built, the physical condition, psychomotor goals and motor
also to be taken into account (as discussed in [23] for vibrotactile
skills. Furthermore, the appropriate personalized sensorial
feedback, nerve sensors cease sending signals to the brain when
support needs to be investigated and properly designed with
exposed to a constant stimulation [36]). Multimodality support
appropriate elicitation methodologies, such as TORMES [41]. In
has also been explored, so when several feedback modalities are
this way, users are supposed to be able to learn defensive martial
combined, the relevance of each of them should also be taken
arts such as Aikido and achieve the reported benefits of its
into account [23]. The speed of the movements to mimic can also
practice [42].
To conclude, UMAP researchers have to be aware that there [18] P. Hämäläinen, T. Ilmonen, J. Höysniemi, M. Lindholm, A. Nykänen. 2005.
Martial arts in artificial reality. In Proceedings of the SIGCHI Conference on
exist other technological challenges that are out of the scope of Human Factors in Computing Systems (CHI '05). ACM, NY, USA, 781-790.
the expertise of this community, so it would be wise to carry this [19] F. Goodman, A. Popovic, P. Brady. 2010. The Practical Step-by-step Guide To
Martial arts: T’ai Chi & Aikido. London: Hermes House, 512 pages.
research out in an interdisciplinary manner. Some of these [20] D. James, W. Uroda, T. Gibson. 2005. Dynamics of swing: A study of classical
challenges are: 1) improving movement modeling and in japanese swordsmanship using accelerometers. In A. S. Ujihashi S. (Ed.).
particular, the modeling of movement in fights, 2) improving Presented at the Asia-Pacific Congress on Sports Technology, Tokyo (2005).
[21] A. Timmi. 2012. Biomechanical analysis of Karate techniques based on the
interaction design to give more realism to the virtual learning evaluation of the body kinetic energy from 3D mocap data. 1st IMACSSS
environment (when available), 3) designing movement modeling (International Martial Arts and Combat Sports Scientific Society). Genova, 8-
10 June 2012,
algorithms that are sensitive to relevant motion characteristics [22] C. Chye, M. Sakamoto, T. Nakajima. 2014. An Exergame for Encouraging
but insensitive to the sensor inaccuracies, especially when low Martial Arts. Human-Computer Interaction, Part III, HCII 2014, LNCS 8512,
cost wearables are used, and 4) building virtual reality 221–232.
[23] A. Bloomfield, N. Badler. 2008. Virtual Training via vibrotactile arrays.
environments where realistic force feedback is provided. Teleoper. Virtual Environ, 17, 103–120.
[24] M. Takahata, K. Shiraki, Y. Sakane, Y. Takebayashi. 2004. Sound feedback for
powerful karate training. In Proceedings of the 2004 conference on New
REFERENCES interfaces for musical expression (NIME '04), Michael J. Lyons (Ed.). National
[1] R.W. Byrne, A.E. Russon. 1998. Learning by imitation: a hierarchical University of Singapore, Singapore, Singapore, 13-18.
approach. Behav Brain Sci. 21(5), 667-721. [25] E. A. Heinz, K. S. Kunze, M. Gruber, D. Bannach and P. Lukowicz. 2006. Using
[2] O.C. Santos. 2016. Training the Body: The Potential of AIED to support Wearable Sensors for Real-Time Recognition Tasks in Games of Martial Arts
Personalized Motor Skills Learning. International Journal of Artificial - An Initial Experiment. 2006 IEEE Symposium on Computational
Intelligence in Education, 26 (2), 730-755. Intelligence and Games, Reno, NV, 2006, 98-102.
[3] J.K. Aggarwal, L. Xia. 2014. Human activity recognition from 3D data: A [26] D.Y. Kwon, M. Gross. 2005. Combining Body Sensors and Visual Sensors for
review, Pattern Recognition Letters, 48 (15),70-80. Motion Training. In Proceedings of Advances in Computer Entertainment
[4] A.I. Cuesta-Vargas, A. Galán-Mercant, J.M. Williams. 2010. The use of inertial Technology, Valencia, Spain, 15–17 June 2005, 94–101.
sensors system for human motion analysis. Physical Therapy Reviews, 15 (6), [27] S. Phunsa, N. Potisarn and S. Tirakoat, 2009. Edutainment -- Thai Art of Self-
462-473. Defense and Boxing by Motion Capture Technique. 2009 International
[5] J. Van der Linden, E. Schoonderwaldt, J. Bird, R. Johnson. 2011. MusicJacket— Conference on Computer Modeling and Simulation, Macau, 152-155.
Combining Motion Capture and Vibrotactile Feedback to Teach Violin [28] G. Mustapha, M.F.A. Razak, M.S.M. Hamzah, N.H.M. Yahya, J. Mahmud. 2016.
Bowing. IEEE Trans. Instrum. Meas. 60, 104–113. The Development of a Low Cost Motion Analysis System: Cekak Visual 3D
[6] R. Sigrist, G. Rauter, R. Riener, P. Wolf. 2013. Augmented visual, auditory, V1.0. International Journal of GEOMATE, 11 (24), 2248-2252.
haptic, and multi-modal feedback in motor learning: A review. Psychonomic [29] Y. Shinagawa, J. Nakajima, T. L. Kunii, K. Hara. 1997., Capturing and
Bulletin & Review, 20, 21-53. analyzing stability of human body motions using video cameras. Computer
[7] E. Polak, J. Kulasa, A.V. Brito, M.A. Castro, O. Fernandes. 2016. Motion Animation '97, Geneva, 1997, 48-57.
analysis systems as optimization training tools in combat sports and martial [30] E. H. Chi. 2005. Introducing wearable force sensors in martial arts. IEEE
arts. Revista de Artes Marciales Asiáticas, León, 10 (2), 105-123. Pervasive Computing, 4 (3), 47-53.
[8] P. Thiparpakul, W. Limprasert. 2017. New Design System to Learning Martial [31] Aikido3D: http://www.aikido3d.com/.
Art Via Kinect 2.0. In Proceedings of the 5th International Conference on [32] K. Kiili. 2005. Digital game-based learning: Towards an experiential gaming
Information and Education Technology (ICIET '17). ACM, New York, NY, model. The Internet and Higher Education 8(1), 13–24.
USA, 89-93. [33] A. Canessa, M. Chessa, A. Gibaldi, S. P. Sabatini, F. Solari. 2014. Calibrated
[9] D.A. Becker. 1997. Sensei: A Real-Time Recognition, Feedback and Training depth and color cameras for accurate 3D interaction in a stereoscopic
System for T’ai Chi Gestures. M.I.T. Media Lab Perceptual Computing Group augmented reality environment. Vis. Commun. Image Represent, 25 (1), 227–
Technical Report No 426. Master of Science in Media Technology. 237.
[10] X. Sun, C.-W. Chen, B.S. Manjunath, 2002. Probabilistic motion parameter [34] L. Emering, R. Boulic and D. Thalmann. 1998. Interacting with virtual
models for human activity recognition. In Proceedings of the 16th humans through body actions. IEEE Computer Graphics and Applications, 18
International Conference on Pattern Recognition, 443-446. (1), 8-11.
[11] P.T. Chua, R. Crivella, B. Daly. 2003. Training for physical tasks in virtual [35] C.J. Winstein, R.A. Schmidt, 1990. Reduced frequency of knowledge of results
environments: Tai Chi. Proceedings of IEEE Virtual Reality, 87-94. enhances motor skill learning. Journal of Experimental Psychology: Learning,
[12] S. Wolf, R. Sattin, M. Kutner. 2003. Intense t'ai chi exercise training and fall Memory, and Cognition, 16(4), 677-691.
occurrences in older, transitionally frail adults: a randomized, controlled trial. [36] G.C. Burdea. 1996. Force and Touch Feedback for Virtual Reality. New York:
J Am Geriatr Soc., 1:188-189. Wiley.
[13] C. Wang, R. Bannuru, J. Ramel, B. Kupelnick, T. Scott, C.H. Schmid. 2010. Tai [37] O.C. Santos, M. Eddy. 2017. Modeling Psychomotor Activity: Current
Chi on psychological well-being: systematic review and meta-analysis. BMC Approaches and Open Issues. In UMAP 2017 Adjunct proceedings (in press).
Complementary and Alternative Medicine. 10:23. [38] M. Hausen, P.P. Soares, M.P., Araújo, F. Porto, E. Franchini, C.A. Bridge, J.
[14] P.-H. Han, Y.-S. Chen, Y. Zhong, H.-L. Wang, Y.-P. Hung. 2017. My Tai-Chi Gurgel. 2017. Physiological responses and external validity of a new setting
coaches: an augmented-learning tool for practicing Tai-Chi Chuan. In for taekwondo combat simulation. PLoS ONE 12(2).
Proceedings of the 8th Augmented Human International Conference (AH [39] O.C. Santos. 2016. Emotions and personality in e-learning systems: an
'17). ACM, New York, NY, USA, Article 25, 4 pages. affective computing perspective. In Emotions and Personality in Personalized
[15] T. Iwaanaguchi, M. Shinya, S. Nakajima, M. Shiraishi. 2015. Cyber Tai Chi - Services (ed. Tkalčič, DeCarolis, de Gemmis, Odić, Košir). Springer, 278-279.
CG-based Video Materials for Tai Chi Chuan Self-Study. 2015 International [40] J. Harasymowicz, R.M. Kalina. 2005. Training of psychomotor adaptation – a
Conference on Cyberworlds (CW), Visby, 365-368. key factor in teaching self-defence. Archives of Budo, 1: 19-26.
[16] O. Portillo-Rodriguez, O. Sandoval-Gonzalez, E. Ruffaldi, R. Leonardi, C. [41] O.C. Santos, J.G. Boticario. 2015. Practical guidelines for designing and
Avizzano, M. Bergamasco. 2008. Real-time gesture recognition, evaluation evaluating educationally oriented recommendations. Comput. Educ. 2015, 81,
and feed-forward correction of a multimodal Tai-Chi platform. International 354–374.
Workshop on Haptic and Audio Interaction Design. HAID 2008: Haptic and [42] O.C. Santos. 2015. Education still needs Artificial Intelligence to support
Audio Interaction Design. Lect. Notes Comput. Sci. 2008, 5270, 30–39. Motor skill Learning. A Case Study with Aikido. In: Les Contes du Marriage:
[17] K. Kunze, M. Barry, E.A. Heinz, P. Lukowicz, D. Majoe, J. Gutknecht. 2006. Should AI stay married to ED? A workshop examining the current and future
Towards Recognizing Tai Chi - An Initial Experiment Using Wearable identity of the AIED field. CEUR Workshop Proceedings, AIED 2015
Sensors. 3rd International Forum on Applied Wearable Computing 2006, workshops, 1432 (4), 72-81.
Bremen, Germany, 2006, 1-6.

S-ar putea să vă placă și