ENAC

COLE NATIONALE DE LAVIATION CIVILE

MASTRE SPCIALIS
AIR-GROUND COLLABORATIVE SYSTEMS ENGINEERING

MMOIRE DE FIN DTUDES

DRONE MISSION PLANNING APPLICATION

Abdelkader KOURDI
Rockwell Collins France, Blagnac
10 September 2015

Note to the reader

This thesis is the personal work of an ENAC student written during his/her end-of study project in a
company or research establishment.

Acknowledgement
I especially dedicate this work to my wife, mother and children also I would like to thank
the different people who were involved in the internship for the professional and the human
context in which it took place. I would like to thank my tutor Eric Thomas, for following my work
despite his tight schedule and for implicating me in the current projects besides my internship.
I especially want to thank Christian le Roux; Head of the Masters at ENAC, all jury
members and ENAC staff.

Rsum
Aujourdhui les aronefs tl-pilots appels galement RPAS (Remotely Piloted
Aircraft System), ou plus familirement drones , sont contrls uniquement par un
utilisateur au sol. Cependant ces aronefs automatiss sont amens voluer dans des
missions plus complexes, rendant le contrle humain de plus en plus difficile.
Afin de pallier aux nouvelles problmatiques lies au fait que le pilote nest plus bord
dune part, et que lintrt est plus dans la ralisation de la mission que dans le contrle du vol
lui-mme dautre part, il est primordial de repenser lapproche actuelle de cockpit dport
et de fournir une solution qui rponde au mieux ces nouvelles exigences.
La rponse apporte par le dveloppement que jai effectu durant mon stage, consiste
en une (interface humain-machine) IHM tactile, volutive et mobile, permettant la planification
pralable des missions au sol. Cette IHM permettra au tl-pilote de dfinir lenvironnement
dans lequel laronef devra voluer (y compris les zones que le drone ne devra pas traverser
ainsi que les obstacles mobiles quil pourrait rencontrer), ainsi que le plan de vol dfini par des
points de passage.
Dans un souci doptimiser la trajectoire parcourue par laronef, une autre partie
concernant le dveloppement dun planificateur de trajectoire a t ralise par une autre
quipe. LIHM sera amene communiquer avec ce planificateur qui elle soumettra son plan
de vol. En retour, le planificateur de trajectoire se chargera de proposer une trajectoire
optimale que pourra suivre laronef.

Abstract

Today RPAS (Remotely Piloted Aircraft System), also dubbed drones, are only piloted
and controlled by a user on the ground. However they are brought to evolve in more complex
missions, making the human control more difficult.
In order to mitigate issues related to the fact that the pilot is no longer on board the
aircraft on the one hand, and that the focus is more on the mission than in the control of the
flight on the other hand, it is essential to rethink the current approach of a remote cockpit and
supply a solution that answers at best these new requirements.
The answer brought by the development I have accomplished during my internship,
consists in a human-machine interface (HMI), tactile, scalable and mobile, allowing mission
planning on the ground. This HMI will enable the remote pilot to define the environment into
which the drone will have to fly (including zones not to cross and mobile obstacles the drone
can meet), as well as the flight plan defined by waypoints.

In a concern to optimize the trajectory traveled by the aircraft, a trajectory planner was
developed by another team. The HMI will have to communicate with this planner by first
submitting the flight plan. The planner will then propose an optimal trajectory the drone will be
able to follow.

Keywords: RPAS, mission planning, mobile application, tactile, Geo location

BLANK PAGE

Table of Contents
Note to the reader .................................................................................................................................... 2
List of tables .............................................................................................................................................. 9
Table of figures: ...................................................................................................................................... 10
List of annexes ........................................................................................................................................ 11
1

Introduction...................................................................................................................................... 13

Company presentation .................................................................................................................. 14

2.1

Rockwell Collins group .......................................................................................................... 14

2.1.1 Main activities sectors ....................................................................................................... 14

2.1.2

Rockwell Collins organization....................................................................................... 15

2.1.3 Key Figures ......................................................................................................................... 16

2.1.4 Activities ............................................................................................................................... 16

2.1.5

Rockwell Collins France ................................................................................................ 18

2.1.6

Research & Technology Innovation (R&T I) ......................................................... 18

Project environment: ...................................................................................................................... 19

3.1

RPAS Regulation in France .................................................................................................. 19

Project Organization ...................................................................................................................... 22

4.1

Project management .............................................................................................................. 22

4.1.1 Gantt diagram ..................................................................................................................... 22

4.2

Project specification ............................................................................................................... 22

4.2.1 Stakeholders: ...................................................................................................................... 24

4.2.1.1 Stakeholders identifications: .................................................................................... 24
4.2.1.2 Stakeholders analysis: .............................................................................................. 24
4.2.2 Capture original requirements: ......................................................................................... 27
4.2.3 Operational concept (CONOPS):..................................................................................... 29
4.2.4 Define requirements........................................................................................................... 34
4.3

Project realization ................................................................................................................... 36

4.3.1 Environment choice ........................................................................................................... 36

4.3.1.1 Development technology........................................................................................... 36
7

4.3.1.2 Operating system ....................................................................................................... 38

4.3.1.3 Geo location providers: ............................................................................................. 40
4.4

Development phase ............................................................................................................... 41

4.4.1 HMI Design .......................................................................................................................... 41

4.4.1.1 Target technology: ..................................................................................................... 41
4.4.1.2 User Profile .................................................................................................................. 42
4.4.1.3 Style .............................................................................................................................. 42
5

HMI Functionalities presentation ................................................................................................. 45

5.1

The Map ................................................................................................................................... 45

5.2

Action Bar ................................................................................................................................ 45

5.2.1 Mission Designer ................................................................................................................ 45

5.2.2 Flight Plan Features ........................................................................................................... 49
5.2.3 Flight Plan Simulation ........................................................................................................ 49
5.2.4 Dynamic Obstacles Designer ........................................................................................... 49
5.2.5 Planner ................................................................................................................................. 49
5.2.6 Settings ................................................................................................................................ 51
6

Project Tests & Integration ........................................................................................................... 52

6.1

Tests phases ........................................................................................................................... 52

6.2

Integration Phase ................................................................................................................... 54

Conclusion and future perspectives: ........................................................................................... 56

REFERENCES ............................................................................................................................... 58

ANNEXES ....................................................................................................................................... 59

List of tables

Table 1: Projects stakeholder ..................................................................................................................................24

Table 2 : Communication plan for requirement ........................................................................................................27
Table 3: Native vs Hybrid..........................................................................................................................................38

Table of figures:

Figure 1 : Rockwell Collins sectors ...........................................................................................................................14

Figure 2: Rockwell Collins locations .........................................................................................................................15
Figure 3: Rockwell Collins Organization ...................................................................................................................16
Figure 4: CineCopter II (Category E) ........................................................................................................................19
Figure 5: Quad Phantom 2 (Category D) ..................................................................................................................19
Figure 6: CineCopter II, Certified for scenarios: S-1, S-2, S-3 .................................................................................21
Figure 7: Quad Phantom 2, Certified for scenarios: S-1 and S-3 .............................................................................21
Figure 8: Requirement definition process .................................................................................................................23
Figure 9: Power-influence matrix ..............................................................................................................................25
Figure 10 : Stakeholder Analysis ..............................................................................................................................25
Figure 11: Flight plan creation ..................................................................................................................................30
Figure 12: Delete/ modifying flight plan ....................................................................................................................31
Figure 13: Save/Load flight plan ...............................................................................................................................32
Figure 14: Flight plan 3D display ..............................................................................................................................32
Figure 15: Trajectory planer .....................................................................................................................................33
Figure 16: Prototyping process .................................................................................................................................34
Figure 17: Test HMI functionalities ...........................................................................................................................35
Figure 18 : HMI presentation ....................................................................................................................................41
Figure 19: Rule of 3 clicks, applied to Flight plan design function............................................................................43
Figure 20: HMI organization .....................................................................................................................................43
Figure 21: Action bar ................................................................................................................................................45
Figure 22 : Flight plan designer ................................................................................................................................46
Figure 23: Example of trajectory consisting of 6 waypoints .....................................................................................47
Figure 24 : Geo-Fence..............................................................................................................................................47
Figure 25 : No Go Zones ..........................................................................................................................................48
Figure 26 : Request an Optimized Trajectory ...........................................................................................................50
Figure 27 : Optimized Trajectory in blue ...................................................................................................................50
Figure 28: Testing File dialog render in all android screen size, using the Emulator ...............................................53
Figure 29: Interface HMI and RPAS planer ..............................................................................................................55

10

List of annexes
Annex 1: Mock-up .....................................................................................................................................................60
Annex 2: Gantt ..........................................................................................................................................................70
Annex 3: Interviews questions ..................................................................................................................................74

11

BLANK PAGE

12

1 Introduction

RPAS (Remotly piloted Aircraft systems) are actually operating with full human control on
ground. With the complexity of the environment in which these systems are going to evolve in
a future, it is necessary to think for new ways to insure automated if not autonomous mission.
To handle this complexity, new systems have to be designed to guarantee mission
capabilities of the RPAS. The idea was to develop a system that allows a ground pilot or user
to plan RPAS mission in advance before launching. The role of the pilot will be limited to
indicating environment in which the RPAS is going to evolve, static and dynamic obstacles to
avoid and the objectives to be reached by the drone.
What I did during my internship was to propose a system characterized by an HMI
(human-machine interface) that can handle mission planning requirements in an easy way
around tactile and mobile technology, this with the main objectives to be easily transportable
and operational in any location. Also the HMI will be able to communicate with a service
center, called RPAS planer, that provides an optimal trajectory to be followed by the drone.
Starting with HMI and RPAS planer, we think in future to develop an UTM (unmanned
traffic manager) to manage drone traffic especially for low-altitude airspace, by providing
services such as dynamic geofencing, congestion management, terrain avoidance, route,
separation management and clearance.

13

2 Company presentation

2.1 Rockwell Collins group

2.1.1

Main activities sectors

Rockwell Collins has been recognized as a leader in the design, production, and
support of communication and electronics aviation for customers worldwide. Its activities are
split between civil and government sectors.
Commercial systems: as key customers
o Airliners
o Aircraft manufacturers
Government Systems: as key customers
o Military
o Governments

Rockwell Collins sectors

Commercial
Systems
Government
45%
Systems
55%

Commercial Systems
Governement Systems

Figure 1 : Rockwell Collins sectors

14

2.1.2 Rockwell Collins organization

The head quarter of Rockwell is located in USA, in Cedar Rapids Iowa. The company employs
approximately 19000 employees worldwide in 80 locations, 58 support centers and 64 sales
representatives offices.

Figure 2: Rockwell Collins locations

Rockwell Collins is organized in 4 main services over the world, as depicted in Figure 3
Commercial systems
Government systems
International & Service solutions
Information Management Services

15

Rockwell Collins

Commercial
systems

International &
Service solutions

Government
Systems

Air Transport
Systems

AMERICAS

Surface
Solutions

Business & Regional

Systems

EuMEA

Airborne Solutions

Flight Information
Solution/Cabin & Electro
Mechanical Systems

Asia Pacific

Communication &
Navigation Products

Service Solutions

Simulation & Training

Solutions (STS)

Information
Management
Services

Figure 3: Rockwell Collins Organization

2.1.3 Key Figures

Annual sales: $4.6 billion (2014)
Employees: approximately 20,000 worldwide

2.1.4 Activities
The main activities of Rockwell Collins are communications, navigation,
display/surveillance, embedded avionics, interactive systems and COTS (Commercial Off-TheShelf) systems, system awareness, cockpit applications, information management and
services as well as distribution of Collins solutions.

16

Communication:

Development of communication tools like:

VHF and HF data radio
Satellite communication

Navigation

Global satellite navigation systems

Situation awareness systems
Flight control systems

Display & surveillance

Display technology for commercial and

military aircraft, small and large display

5x5 Displays (PFD, ND)

8x10 Large Displays (PFD, ND,
EICAS)
8x6 Touchscreen Displays (3rd
party development)
Radars
Personal locator systems

17

2.1.5 Rockwell Collins France

Rockwell Collins France is a subsidiary company of Rockwell-Collins. It was created in 1959;
and has been present in Toulouse Blagnac since 1978. It annual sales in (2013), was 222
million euros.
Rockwell Collins France employs 750 salaries with:
55 % Engineers
28 % Technicians
17 % Administrative staff

RCF Employees
17%
28%

Engineers

55%

Technicians
Administratives

It is represented by 4 sites in France

The Headquarter in Toulouse-Blagnac
The sales offices in Thiais
2 Technical sites: Creil and Paris CDG

2.1.6 Research & Technology Innovation (R&T I)

My internship was taking place in R&T-I department its mission is to Identify, develop
and mature technologies that provide growth for Rockwell Collins, 20% of the research and
innovation in Europe are made in Rockwell Collins France.

18

3 Project environment:
My internship took place in EuMEA Reseach & Technology Innovation department,
which main target is to develop and produce innovative avionic products that insure the
competitor rank of Rockwell Collins. In my case, I was in charge of the design and the
development of an innovative HMI for drone flight planning, to insure drone automated mission.
To understand drone activity, it is essential to have a notion about the regulation and
rules applied in France. In a first chapter I will make a brief presentation about RPAS
regulation. In a second chapter I will present the project specifications process followed to
analyze customer requirements, after I will go through the development, test and integration
phases of the HMI including a presentation of the application. Finally I will discuss the future
evolution of the HMI and potential market that could make use of this product.

3.1 RPAS Regulation in France

An RPA (remotely piloted aircraft) or drone is an aircraft that is able to achieve a
mission with full or partial control from a remote pilot. As this activity is subject to regulation
from the country within which the RPA is getting to fly, in this section I will make a brief
presentation about drones according to the French decree of April, 11 2012, that regulates
RPAS activities.
In France, RPAS regulation depends on categories and scenarios of use. RPA are
classified in 7 categories from A to G (Figure 4 and Figure 5 illustrate two kinds of drones
categories E and D) this classification is made by DGAC according to the takeoff mass of the
drone.
Concerning scenarios of use, DGAC defines 4 scenarios as the base to use RPA safely.

Figure 4: CineCopter II (Category E)

Figure 5: Quad Phantom 2 (Category D)

19

Categories:

Category A
Model aircraft, Motorized or non-motorized with a maximum takeoff mass less
than 25 kg, or, for aircraft with an inert gas, total mass (structural mass and charge
carried) less than 25 kg, consisting of a single type of propulsion and subject to the
following limitations:
Engine: total capacity not exceeding 250 cm3;
Electric motor: less than or equal to 15 kW total power;
Turboprop: less than or equal to 15 kW total power;
Reactor: total thrust not exceeding 30 kN, with a thrust / weight ratio without fuel
not exceeding 1.3
Hot air: total mass of gas in on board cylinders not exceeding 5 kg
Any tethered or captive aircraft: captive means UAV grounded in a mobile that
cannot be lifted or moved by any action.

Category B
Any model aircraft more than 25 kg at takeoff and, which does not meet the
requirements propulsion of Category A, needs a flight certification. Pilots have to
demonstrate their ability to maneuver such vehicle.

Category C
Unmanned captive aircraft that are not model aircraft, with a maximum takeoff
mass of 150 kg

Category D
Unmanned aircraft that are not model aircraft, motorized or not, non-captive, with
a maximum mass of takeoff less than 2 kg.

Category E
Unmanned aircraft that are not model aircraft, which are not Class C or D,
motorized or not, maximum takeoff mass less than 25 kg or unmanned aircraft with inert
gas total mass (structural mass and charge carried) less than 25 kg.

Category F
Unmanned aircraft that are not model aircraft, maximum takeoff mass less than
150 kg and does not meet the requirements of the class C or D or E.

Category G
Unmanned aircraft that are not model aircraft, which does not meet the
requirement of the class C to F

20

Scenarios:
DGAC defines 4 scenarios of use, S-1, S-2, S-3, and S-4; any RPA could evolve in one
or more scenario according to its usage, (Figure 6 and Figure 7 show an example of drone
scenario of use),

Scenario S-1: any operation in direct sight of the remote pilot, taking place
outside the area populated, to a maximum horizontal distance of 100 m from the
remote pilot.

Scenario S-2: operation taking place off-sight, non-populated area, in a volume

of maximum horizontal dimension of radius of one kilometer and less than 50 m
above ground level (AGL) and artificial barriers, without any person on the
ground in vicinity.

Scenario S-3: operation taking place in urban areas or near people or animals,
sight and a maximum horizontal distance of 100 m from the remote pilot.

Scenario S-4: Special surveys, photographs, observations and monitoring air

taking place off direct view, non-populated area and do not meet the criteria of
the S-2 scenario. In addition, is required for the execution of this scenario:
possession of PPL1, airplane or helicopter and 100 hours of flying on these
aircraft with, in addition, 20 hours of flight with the unmanned aircraft.

Figure 6: CineCopter II, Certified for scenarios: S-1, S-2,

S-3

Figure 7: Quad Phantom 2, Certified for scenarios: S-1

and S-3

Private Pilot Licence

21

4 Project Organization
4.1 Project management
4.1.1

Gantt diagram2

The main task was to groundwork the project for developing the application and to build
an effective schedule. As you can see in the annex, the project was broken in many structures
with the purpose to delineate the steps required for building and delivering the application. The
Gantt helped me to clarify project ambiguities (integration steps) and raise the assumptions
earlier, especially for requirements captures and technologies choice. When developing the
Gantt, I imposed a level of details when defining the project structure to make tasks more
comprehensible and easy for completing.
Finally I identified the prioritized tasks and organized my effort according to this work
structure; with the Gantt model I set the time frames and task accomplishment.

4.2 Project specification

The main objective of this section is to present my approach to determine and capture
customer needs. I followed the process described in figure 11, starting by a stakeholder
analysis. After I focused my requirement capture around key user identified in the first step.
After, I made an operational concept analysis to define the main functions. As result for
this analysis I outlined the project requirements and made mock-ups for final requirement
validation with key users.

Annexes Gantt
22

Identify and analyse Stackeholders

Capture requirements

Operational concepts (CONOPS)

Prototyping

Requirement
validation
Figure 8: Requirement definition process

23

4.2.1 Stakeholders:
Before starting with the specification aspects of the project, I had to identify the
interested parties attracted by the project. By this process I tried to analyze qualitative
information to determine actors interests that should be taken into account when developing
and implementing the HMI.
In my analysis I focused on some stakeholders characteristics that can impact the
projects realizations:
1- Who are interested in the project?
2- Who are externals actors interested by the project?
3- Who are against the projects or harmfully affected by the project?

4.2.1.1Stakeholders identifications:
Stakeholders
Rockwell Collins

Definition
The sponsor of the project, who have a
business and economic interest in the project.

Regulator (DGAC)

Put rules for drone usage in area space,

external constraints that affects the projects,
especially if regulation changes during the
project. For the moment I refer to the decree
of April, 12, 2012.

Users

Persons, who are going to use the HMI for

flight planning
Population against RPAS and whose interests
are not particularly represented by the DGAC
regulations.

People

Table 1: Projects stakeholder

4.2.1.2 Stakeholders analysis:

Below is an Importance-influence matrix that summarizes, weight of each stakeholder,
based on their influence on the project schedules and implementations phases, also priority
given by me to them to satisfy their interests through the project.

24

HIGH
REGULATOR(DGAC)

ROCKWELL COLLINS

IMPORTANCE

USERS

A.

B.

C.

D.

LOW

PEOPLE

INFLUENCE
Figure 9: Power-influence matrix

HIGH

IMPORTANCE level

According to the above matrix, I associate similar stakeholder in a group, I made my analysis
in order to define the approach and the cooperative model to handle the relation with all the
stakeholders.

A: High importance
Low influence

B: High importance
High influence

Group A: Require special initiatives

and attentions to protect and decrees
their interests,

Group B: The one that can make, the difference;

we have to maintain and develop an excellent
relationship with them

C: Low importance
Low influence

D: Low importance
High influence
Group D: not involved directly in the project but,
may be source of conflict and risk.
Need to be carefully observed and monitored

INFLUENCE level
Figure 10 : Stakeholder Analysis

25

Group B (Rockwell Collins and users): Is the key player, the main group interested in the
project, I have to focus requirement capture on it and involve it in all project phases and
decision making.
Group A (DGAC): It has a high importance on the project; its requirements need to be
carefully implemented on the HMI design but with low influence, if DGAC decides to stop all
drone flight or change the regulation with a considerable impact on drone activity, we could
also continue our development, as the application can be used outside of France
Group D (People): not involved in the project but with big risk to struggle the project, they
have to be carefully observed and monitored.

26

4.2.2 Capture original requirements:

In this part of my internship I had to involve the key players (Group B) in all the requirement
identification phases, with the objective to clarify the needs, and eliminate disagreements due
to misunderstanding of the requirements. For that and before requirement capturing I had to
build a communication relation with the stakeholder that guaranty:

Clarity: in information exchange between myself and the stockholders (group B).
For that I built a questionnaire for attaining and extracting information needed to
clarify requirements and stakeholder needs, and also to share information with
stakeholders concerning my actions in order to make the application that reach their
requirements

Monitoring Changes: Stakeholders requirements are likely to change over time;

the aptitude to maintain a continuous communication with the stakeholder helps in
adapting these changes. By maintaining periodic meetings with the stakeholders I
was able to anticipate any requirement changes and provide a quick response to
their suggestions.

Table 2, illustrates the communication plan followed during requirement gathering

Target audience
RC Manager
RC Manager

Nature of communication
Project presentation
Project Requirements

Communication method
Meeting
Meeting
Email, Phone

RC Manager

Requirement analysis

Meeting
Email, Phone

RC Manager, Users
RC Manager, Users
RC Manager

Mockup
Prototype
Requirement validation

Meeting
Meeting
Meeting

Frequency
One time
Weekly
When
needed
Weekly
When
needed
Sometime
Sometime
Weekly

Table 2 : Communication plan for requirement

Once the communication plan of requirement put in place, I centered my attention to

requirement identification. My work was to clarify the customer needs and understand what the
customer was exactly looking for with this new application.
For that purpose, I organized one to two meetings and workshops by week with the project
manager, and I also prepared a questionnaire3 in order to identify:
Application context: The HMI shall be tactile and mobile; it shall allow to define a flight
plan on touchscreen and have the capabilities to manage dynamic obstacles.
3

Annex: Interview questions

27

The application shall be able to communicate with another application named trajectory
planner, which principal role is to define the optimum trajectory for the drone. Also the
HMI should present the flight plan in 3D using Google Earth and could be used indoor
as well as outdoor.
User profile: They have some knowledge of the context, with a complete understanding
of HMI functionalities and knowledge about the government regulation where the drone
is going to evolve.
Number of user: One user by application, each user is equipped with an android
device, to use the HMI
HMI Technology & Environment: The key players didnt have a clear idea about the
technology to use and environment to execute the HMI, I had to choose one, and
demonstrate the motivations of this choice.
Input/output of the application:
Input: came from the geo-provider that will be used for all geo location functionalities of
the application, also the application shall be able to load saved flight plan. The
application shall be able to read data coming from the RPAS planner.
Output: The HMI shall have the capabilities to save file plan, the format chosen to store
data is KML (an extension of xml used for geo location features)
The HMI shall submit the flight plan to the trajectory planner
Input/output of the project: In this case, I had to produce some documents, SRS
(software requirements specifications), Integration procedures, test scenarios, test
report, application user guide and demonstration video at the end of the project.
Application Integration: I had defined 2 integrations process and interfaces
A. The first interface: between the HMI and Google Earth (see application
context).
B. The second interface: between the HMI and the trajectory planner, the
customer wants to develop a web service to share data between the HMI and
the planner.

All outputs are in KML format; we will see later in this report a definition of this format
and its utility.
Application evolutions: should be developed to facilitate the futures evolution and
integration with other applications, EX. We plan to use it with an UTM (unmanned traffic
management) in the future, I will discuss that in future chapter.
28

4.2.3 Operational concept (CONOPS):

In this part of my internship I focused my study on the operational concept, with the
purpose to describe operational needs expected from the user and illustrate the conceptual
view of the system. I used use cases to describe the user systems behavior and to capture
the functional requirement, in another way representing the user point of view against the HMI.

Flight plan definition:

The flight plan is the main functionality made by the HMI. This latter proposes an
intuitive way to create and define the different components of the flight plan that are:

Waypoint: Virtual point that represents a 3D position, defined by its latitude, longitude
and altitude
Geo Fence: Virtual perimeter of a geographic area, where drone can evolve
No Go Zone: Restricted area, with a fixed altitude from the ground, the drone cant cross
this area, only fly over it if its altitude is greater than the no go zone altitude
Obstacles: Other drones or aircrafts the drone can meets in vicinity of its trajectory

HMI propose many functionalities:

- Flight plan creation
- Flight plan modification / deletion
- Flight plan save/load
- Flight plan display (Google Earth)
- Request trajectory from the planner

29

Flight plan creation:

Animate Flight plan

Add waypoints

Drone planner

Add Geo Fence

Add No go zones

Simulate obstacle

Set category

User

Set obstacle
trajectory

Define altitude

Figure 11: Flight plan creation

30

Delete/modifying Flight plan

Mod/Del trajectory

Del. waypoints

Del. altitude

Del. Geo Fence

Del. No go zones

Del. Simulate
obstacle

User
Del. obstacle
trajectory
Del. category
Clear plan

Figure 12: Delete/ modifying flight plan

31

Save/Load Flight plan

Create Flight
plan

Save Fight plan

In
cl
ud
e
Load Flight plan

User

Figure 13: Save/Load flight plan

Flight plan 3D display

Load Flight plan

Display Google
Earth

User

Create Flight
plan

Figure 14: Flight plan 3D display

32

Trajectory planer:

Load Flight plan

Request
Trajectory

User

Create Flight
plan

Figure 15: Trajectory planer

33

4.2.4 Define requirements

The CONOPS gives a good understanding of the required functions; it helps me to have
a clear statement of the goal and objectives of the application,
a good understanding of the HMI interfaces with external applications,
a good understanding of the user expectation.
In order to validate the final design and attest the stakeholders requirements, I made
some prototypes to better clarify the final needs and validate them. The purpose of the
prototypes is represented by the Figure 16,

Validate the
HMI
organization

Test basics
HMI
functionalities
in its
environement

Validate the
HMI design

Attest the
HMI
objectives

Figure 16: Prototyping process

I made a prototype in 2 steps:

1. The first was a mockup4, where I presented a general views about the HMI, using
graphical tools, I represent all the function, interaction and HMI behavior learned on
the CONOPS and requirement capturing session, it helps me to have a vision of the
visual aspect intended by the user, with this mockup I:
Validate the HMI objectives (validate the Input/output and interfaces)
Validate the functions (concept of use)
Validate the HMI design (buttons and icons positions)

Annexe Mock-ups
34

2. Once the first mockup validated, I had also to demonstrate HMI functionalities by
developing a very basic software prototype as depicted in Figure 17, to expose main
HMI functionalities for creating the flight plan and test the capabilities of the Geo
provider and the operating system chosen to run the HMI (see next chapters for
more details).

Figure 17: Test HMI functionalities

Once the final design approved by the customer and requirements validated, I wrote
the SRS that attest the requirements acceptation from the customer.
In the next chapter I will discuss the project realization, and application development
to build the final HMI.

35

4.3 Project realization

4.3.1 Environment choice
The main need was to design and develop a tactile HMI planner. Tactile means that the
application should be used on a device with a touch screen capabilities, tablet or smartphone.
Also the HMI shall be able to handle all geo location aspects, for modeling the flight
plan. Many questions had to be answered before starting the development:

Which technology to develop the HMI?

Which operating system to run the HMI?

Which geo-location provider that allows to build and display HMI map?

In this section I will study these questions in details, and explain the raisons of my
choice.

4.3.1.1

Development technology

The motivation was to look for a multiplatform technology, by multiplatform I mean a

technology able to be operated and deployed both on Android and iOS operating systems.
After a first research, and reading some books5 and documentation, I had a dilemma between
2 potentials solutions Hybrid development and Native development.

a) Hybrid solution
Consists of a new way of developing multi-platform applications that are able to run on
iOS and android.
Applications are developed using mobile web platform around basic web development
technologies HTML 5, CSS and JavaScript.
Hybrid applications are able with limited right to access device capabilities
accelerometer, contacts and more.
5

See References
36

Advantages
Fast development
Multiplatform applications
Drawbacks

Knowledge of the two worlds web and native

Only online mode
Very basic user interface
Device sensor access limited, need additional libraries
Full mobile capacity limited
No embedded development
No local storage (JavaScript is not able to write on physical disk)

b) Native solution
Native development tools of the OS (language, environment) are used to design and
build the application; application can be used only on particular platform or device.
Advantages

Get good performance when using native Software development kit (SDK)
Rich user interface
Local storage access
Entire Device Sensor accessibility
Full use of the device capabilities
Embedded development

Drawbacks
Dedicated teams for different Platforms (Android, iOS)
Higher Development Cost (for iOS)
Not easy to adapt application to/from another platform

c) Native vs. Hybrid

To compare the solution I chose some criteria that are necessary to develop the
application, the performance of the UI (user interface) was a very important criterion as we are
developing an HMI, and looking for a nice interface to propose a good user experience.
For the local storage, one of the requirements was to propose an application that is able
to save and load files; this was a very weakness for the hybrid approach, that cant propose a
solution to save local file on device.
37

For the full capabilities access of the device, this criterion was chosen as important for
needs of application evolution. If the customer needs in future to implement embedded
program, additionally to the HMI, he will able to do that. Concerning the development cost, I
will detail this point in the next section

App criterias

Native

Hybrid

Rich UI Experience

High

Low

Development Cost

Higher

High

Application Storage

High

Low

Device capabilities access

High

Low

Table 3: Native vs Hybrid

d) Solution chosen:
According to the previous analysis, the Native approach was the appropriate solution to
develop the application. Certainly it has the disadvantages that the HMI cant run on
multiplatform, but the needs to propose a riche user interface and a full access to devices
components including local storage was more important for the customer.

4.3.1.2

Operating system

There are currently 2 major systems, handle touchscreen technology dominating the
market of smartphone and tablet, iOS and Android.
In this section I will make a comparison between these two OS based on development
environment and licenses criteria. The reason for choosing these criteria was that I looked for
an environment that proposes a robust and complete development kit, including emulator for
testing and with an affordable and scalable license.

a) Development environment:

iOS: We need a mac computer in order to develop iOS applications; the

integrated development environment (IDE) is Xcode with the iOS SDK and

38

iOS simulator for testing application. It is mandatory to subscribe for an apple

developer account before downloading and using Xcode.
Xcode proposes a friendly development environment with much functionality
to enhance user interface design, and its simulator is very fast to use. The
great drawback is that this simulator is running using Mac resources and not
iPhone emulation. If we need to test applications on a real iPhone device, we
have to pay an additional license.

Android: application can be developed on Windows, Linux and Mac Os.

Android studio is the development environment with (IDE) with integrated
Android SDK.
Android studio proposes an intuitive environment for developments based on
Java language. The only drawback is its emulator, very slow and not really
efficient for fast development.

b) Licenses

iOS: is available with a restrictive license, and limited possibilities. It will not
be possible to use external libraries to enhance development and reduce time
developing, for example if an app is developed with external APIs and then
proposed to the iOS SDK, the application will be rejected while uploading for
approval by app store.

Android: Android is an Open source solution; it can be fully customized to

develop new Android versions. Also many external libraries and packages are
proposed that can enhance Android API, and reduce development time.

c) Conclusion:
After brief comparison between the IOS and Android, I decided to choose Android for its
low cost development, its intuitive IDE, except emulator, for that I tested the application directly
on an Android device. The rich Android API that allowed me implementing external libraries
helped me a lot in rapid development and application design. Finally, its Open source
approach opens doors to make many improvements on Android and makes application
evolution easier.

39

4.3.1.3

Geo location providers:

For the HMI development, I preferred using Google Maps, as Im familiar and worked
before with this technology. Also Google Map is natively integrated to Android, which is a good
point if we plan to deploy the HMI to other Android devices. No installation, no configuration
and no additional tests are needed to validate Google Maps on these devices.
On the other hand, Google Maps provides a rich and robust API for development. We
can also add external libraries to boost development and reduce time and cost.
Conclusion
To develop the tactile HMI I opted for a native approach for its robustness, chose
Android as Os for its rich API and Google Maps as geo location provider for its accuracy and
easiest deployment since natively integrated to Android.

40

4.4 Development phase

4.4.1 HMI Design
The HMI is composed of two parts:
1. A map where 2D image of the actual position is displayed. It is meant for the design
and the display of the flight plan.
2. A menu bar, called action bar, which handles all user actions of
creation/modification/save and load of the flight plan. It also manages all other
activities related to the organization of the application.
The HMI was designed taking into account 3 principles; Target technology (seen
before), User profile and Style

Figure 18 : HMI presentation

4.4.1.1

Target technology:

As seen before in the previous section, technology chosen to develop the HMI
answered the main technical requirements, it shall be tactile, mobile, easy maintainable and
inexpensive. The choice was Android as development platform.
41

In addition Android, is able to propose rich libraries and APIs to enhance development
of intuitive and friendly user interface. It integrates Google Maps API, which helped me a lot to
develop all geo-location aspects of the application. Also, its maintainability is very low as it's
open source platform with Java as main development language (that benefits from a large
community of developers).

4.4.1.2

User Profile

The HMI is designed for only experienced users, meaning that they have good area
knowledge, a nice experience with Android devices and with an expected level of training on
the HMI.

4.4.1.3

Style

To define the HMI style and improve user experience, I followed some rules, in order to
make a coherent design. For that purpose I had to:
-

Define the architecture of the HMI

Organize the contents & define navigation principle
Organize the interface
Manage object dynamism
Create dialog box

a. Define the architecture

To reduce user workload, the HMI content was organized in a way to easily find the
information needed by the user. All HMI functionalities are categorized by function family. Each
function has only one dedicated button, this to enhance recognition of functionalities, by
associating function to button.
For example, all flight plan-designing functions are grouped in a same sub menu, with
specified family name to be easily recognized by the user.
b. Organize the contents
Contents were organized in a menu, and sorted in order to reach the information with
three clicks maximum. Users must indeed have access to any information on the application
within three clicks from the Main menu. This in order to reduce user frustration, to not be
distracted by something else or forget the information he/she is looking for. Figure 19 below
describe the application of this concept to flight plan designer function

42

Also I tried to put at most 7 items by menu and submenu, as the HMI runs on Android
devices and screen size is very limited. Such rule insures coherence in the menu display and
avoids that the menu will be too loaded.

Main Menu
First clic

Flight Plan
Designer function
Second clic

Geo Fence
Third clic

No go Zone

Waypoints

Third clic

Third clic

Figure 19: Rule of 3 clicks, applied to Flight plan design function

c. Organize the interface

We have only one page, with one map and one menu bar on the top screen (see
Figure 20), as we have a limited screen size, I tried to preserve the visibility of the map
and not load the interface with many icons. All functionalities are grouped in the menus and
can be easily accessed to.

Menu

Map

Figure 20: HMI organization

43

d. Manage object dynamism

To preserve display quality of the different objects on the map (markers, icons,
buttons), icons are created following Android recommendations, as we have different
Android devices with different screen sizes. Each icon is generated using Android asset
studio6, an online platform that helps in generating icons, according to Android
recommendations. Each Information quality will be preserved on any Android device screen.
Each function and button has only one dedicated button, this to enhance recognition of
functionalities by the user, by associating a function to a button.
e. Create Dialog box
Dialog boxes were built with respect to the following criteria:
Use of a coherent vocabulary:
Each message is in direct speech, affirmative and shortens as possible to
be quickly understandable by the user

Displayed at the same place in the screen

For more comfort of use, all messages are displayed at the same place on
screen
Present only needed information
The message is built to correspond with the actual task; all messages will
refer to the actual user work.

Preserve actual task

Display message will not have any influence on the actual task, task will
never been interrupted and user can go on, on his activity as before message
alert.

https://romannurik.github.io/AndroidAssetStudio/iconsgeneric.html#source.space.trim=1&source.space.pad=0&size=24&padding=8&color=33b5e5%2C100&name=ic_e
xample
44

5 HMI Functionalities presentation

5.1 The Map
The Map is the display part of the HMI; it handles and manages all map objects. We
chose to use Google Maps as Geolocation provider for its data accuracy, accessibility and
reliability. Also, it proposes a large set of API to help in developing and designing maps.
Moreover, Google Maps is natively integrated to Android.
The map can be displayed in 4 formats: normal mode, hybrid mode, satellite mode and
terrain mode.

5.2 Action Bar

It regroups all actions, to build the Flight plan; its organized in 6 mains categories, Figure
21,

Figure 21: Action bar

Action bar categories:

1. Flight Plan Features
2. Mission designer
a. Flight Plan Designer
b. Flight Plan Manager
c. Flight Plan 3D
3. Flight Plan Simulation
4. Dynamic Obstacles Designer
5. Planner
6. HMI Settings

5.2.1 Mission Designer

In this menu we find all actions, and functionalities to create, and operate the flight plan.
Its organized in 3 submenus.

45

Figure 22 : Flight plan designer

1.

Flight plan designer

In this menu the user is able to design the flight plan by adding the 3 main objects that
compose a flight plan: waypoints to define the trajectory to be followed by the drone, Geo
Fence that defines the area in which the drone is going to evolve and the no go zone or
restricted area that are to be avoided by the drone during flight.
Waypoint:
WGS84 Coordinates of each step of the flight. Its defined by its:
o Latitude
o Longitude
o Altitude
A number identifies each waypoint and all waypoints are connected by a magenta
straight line, defining the drone trajectory as depicted in Figure 23

46

Figure 23: Example of trajectory consisting of 6 waypoints

Geo-Fence:
Its a virtual perimeter of a geographic area, within which the drone can evolve.
Its represented by a green polygon with a fixed height from the ground as depicted
Figure 24.

Figure 24 : Geo-Fence

47

No Go Zone:
A restricted zone, with a fixed altitude from the ground, the drone cant cross unless its
altitude is greater than the no go zone altitude. Its represented by Figure 25

Figure 25 : No Go Zones

2.

Flight Plan Manager

Composed of 2 items Save and Open
Save:
Saves a flight plan (including geofence, nogo zone and waypoints) locally in a KML
format. Whenever the user wants to save the file, a predefined
name
with
the
current date and time is proposed. The user can also choose its own file name.
Open:
The user can open a saved flight plan (including geofence, nogo zone and waypoints),
and then modify it by adding, removing or changing items to the flight plan.

3.

Flight Plan 3D
HMI can show map in 3D display using Google Earth installed on the Android device.

48

The advantage of using KML file is the ability to operate directly in Google Earth without
any code modification. In addition we can add customize tags to enhance and add new
features to the flight plan.

5.2.2 Flight Plan Features

In this menu, the user can set the altitude of each map components. He/she is also able
to delete map components

5.2.3 Flight Plan Simulation

Used to simulate the drone flight through its trajectory, it can help to check the flight plan
and make some changes before using it.

5.2.4 Dynamic Obstacles Designer

Obstacles are objects the drone can meet during its flight, like aircraft, or other drones.
They are modeled as sphere object known by their diameter. On the HMI, obstacles are
classified in 3 categories according to their diameter.
Before adding an obstacle, the user has to define the obstacle category and set the
obstacle trajectory. He can also simulate or animate the obstacle movement.

5.2.5 Planner
The Trajectory planner has the role to provide the user with an optimized flight plan.
Once the user designs its flight plan he/she can request from the planner an optimized flight
plan by submitting the flight plan on clicking the Planner icon see Figure 26 and Figure 27.

49

Figure 26 : Request an Optimized Trajectory

Figure 27 : Optimized Trajectory in blue

50

5.2.6 Settings
This menu handle all HMI setting and flight plan parameters, its organized in 3 classes:
1. Regulatory Framework
When user can set the max altitude that can drone reach, and the drone category, (see
regulation presentation in the beginning of this report)
2. Map Preferences
In this part, user can set color, width of flight plan features
3. Obstacle Preferences
In this part, user can set color, width of dynamic obstacle

51

6 Project Tests & Integration

6.1 Tests phases
During the project, I tested the functions at the same time as their development finalized.
The objective was to involve the user in all project phases. This helped me to anticipate
deviation from the origin requirement and real needs from the customer.
To organize the test activity, and before writing any code, I defined tests scenarios that
proved the code I would implement would work as expected by the requirement. I separated
each test function and tested it at different level:
Unit test level: I checked all components separately of the rest of the program, but taking
into account their interactions with other components, to insure that it met the functional
specifications and worked properly in all circumstances.
Acceptance tests level: Specified by the user, to check whether all the requirements
described in the SRS (software requirements specification) are met. During this phase, the
behavior of each function had to be checked according to its specifications, taking into account
reference test scenarios.
Pre-integration tests level: The idea was to anticipate any issue due to missing
integration procedures and to make sure that full integration would take place in good
conditions. Pre-integration consisted in:

First: Checking the environment in which the application was going to run,
testing the UI display in different android device using the android Emulator
as depicted in Figure 28 I also proceed to physical tests on Wiko phone, HTC
phone, and Samsung tablet with different screen sizes to make sure the HMI
behavior was the same for these devices.

52

Figure 28: Testing File dialog render in all android screen size, using the Emulator

Second: Checking that the interface between the HMI and the Trajectory
Planer were functional and answered customer requirements.

Once all these steps completed and bug corrected, I proceeded with the integration phases
as detailed in the next section

53

6.2 Integration Phase

The integration phase concerned the establishment of the communication and data
exchanges between the HMI and the Trajectory Planner.
Trajectory Planner has the role to calculate the optimal trajectory for the drone between
different waypoints and patterns, avoiding obstacles defined by the user in the flight plan.
To interface the two parts, it was decided to develop the integration around client-server
technologies, with the idea that the planer plays the role of trajectory service provider, and the
HMI the role of client requesting services form the planner.
The User submits the flight plan to the Trajectory planner in KML format, same as xml
since it is easier to extract information by parsing its tags.
Data were sent through network, using sockets but some problems appeared with Android
when implementing this solution. It was systematically crashing at each user request. This was
due to the main Android user interface thread that could not manage network calls and display
results in the main UI thread. To bypass this problem, Google recommended using
synchronized calls that allow performing background operations and publishing results on the
UI thread without having to manipulate threads.
After calculating the new trajectory, the planner sends a response to the user, containing
the new trajectory coordinates also in KML format. Once received, the HMI parses the KML file
to display result on the main map view.
The Figure 29 summarizes the interfaces between the HMI and the Trajectory planner.

54

Create or load a flight plan

Request to the RPAS planer

Trajectory

Flight plan
KML format

Parse the
Trajectory KML
format

Send data (flight plan) in kml

Trajectory

Network

Send data (flight plan) in kml

Trajectory

RPAS planer

Figure 29: Interface HMI and RPAS planer

55

7 Conclusion and future perspectives:

One of the main problem faced when developing the HMI, was the learning of Android. It
was very different from what I had seen with java, especially when developing the flight plan
features. It was not possible to modify any of them once created, the only way was to delete
and recreate the features. If I had had more time, I would have tried to develop and implement
a local library to handle this issue. But due to lack of time on the one hand and constraints of
other scheduled activities on the other hand, the only choice was to bypass this step of
development since a non-blocking problem.
Also another problem was the setup of the Android studio, as Google migrates his
development environment from Eclipse to Android Studio. I spent time before being familiar
with the new functionalities, especially the Gradle, an advanced build toolkit that manages
dependencies and allows you to define custom build logic as defined by7.There is no way to
build your app and define dependencies without using Gradle on Android Studio. This was a
big disadvantage. At the beginning of my development I noticed that Gradle took much time to
build the application. After some research, I found a solution8 that helped me to solve this
issue.
We didnt have the opportunity to make a real test, and check the RPAS behavior in real or
simulated situation. This was due to the fact that this was a prototype and proof of concept for
future development. The HMI is able to build a flight plan mission in an easy way, thanks to
Google Maps and Android API that permitted a real progression in the development of these
functionalities.
During this internship I was able to apply what I had learned at ENAC and proposed many
solutions to different problems, both functional and technical, and enhanced my experience in
the systems engineering area. Nevertheless, a lot remains before the real exploitation of this
HMI.
I preconize to include new functionalities such camera filming or recording as the user or
pilot works with a remote cockpit. It would be very useful to have a vision of the environment
in which the RPA is evolving especially for search and rescue missions. With this HMI we
could develop a real unmanned traffic management, to manage RPAS traffic in lower altitude.
Also we could develop a new market for RPAS flight plan provider services, by providing
optimal trajectory to users.
It would also be interesting to implement a support service for user, to handle errors and
provide a change management support for those who have habit to use other system than this
HMI.
7
8

http://stackoverflow.com/questions/23815831/android-gradle-what-is-it
https://www.timroes.de/2013/09/12/speed-up-gradle/

56

Finally, an important point is to implement a knowledge base, to collect user experiences

feedback. We could then learn much more on how the users operate their system, and define
new enhancements for future versions of the HMI.

57

8 REFERENCES
Chapter in a book
Nizamettin Gok, Nitin Khanna, Chapter 3 Android Fundamentals & Chapter 6 HTML Architecture for Hybrid
Applications, Building Hybrid Android Apps with Java and JavaScript. Country of publishing: O'Reilly Media,
July 2013, pages 156

Websites:
Android Developers, April to August 2015, https://developer.android.com/training/index.html
Design, Android Developers, April to August 2015, https://developer.android.com/design/index.html

Electronic Course
Video2Brain.fr, Devenez un Dveloppeur Android Vol 1 et Vol 2, publishing date: August, 19 2009

Articles
Melanie Tory, Torsten Moller, Human factors in Visualization Research, Country of publishing: IEEE Transactions
on visualization and computer graphics, Vol 10 No. 1, January/February 2004

58

9 ANNEXES

59

Annex 1: Mock-up

60

Flight Plan Designer

61

62

63

Flight Plan Manager

64

65

Flight Plan 3D

66

Flight Plan animation

Dynamics Obstacles Designer

67

68

69

Annex 2: Gantt

70

71

72

73

Annex 3: Interviews questions

How:
Question: How do you plan to use this HMI?
Answer: The application will be used to define a flight plan for the drone, and be
able to define dynamic obstacles. The application shall be tactile and mobile.
Question: How the user would access and use to the HMI?
Answer: Using a tactile device, the user shall be able to design his flight plan on
touchscreens
Question: How many users?
Answer: Some users are able to use the HMI; we prefer to have one user by device

Where:
Question: Where the HMI will be used?
Answer: Anywhere, in indoor or outdoor

Who:
Question: Who will use the HMI?
Answer:
Drone pilot
Tester engineer, user with some experience of the context
Question: Who will be interested to learn about the HMI?
Answer:
Innovation services
Engineers

Which:
Question: Which application will deliver the inputs for the HMI?
Answer:
Geo location provider, to define trajectory coordinates
RPAS Planner: for which the HMI shall submit its flight plan and receive the
optimal trajectory from the RPAS as KML file.
HMI itself, shall be able to read and load saved flight plan
74

Question: Which application will receive the output of the HMI?

Answer:
The RPAS planner application, for which the HMI shall submit the flight plan,
its role is to define the optimal trajectory for the drone and resend the output
to the HMI.
Google Earth for 3D visualization
HMI itself, shall be able to save file locally

What:
Question: What is the future evolution of the HMI?
Answer: We plan to develop an UTM (unmanned traffic management) for drone
Question: What kind of license do you plan to use for the HMI open source or
commercial?
Answer: we prefer Open source license

75