Hanoi Data Treatment Oct17 PDF

Validation and data treatment
Véronique Ghersi|Head Forecast and Diagnosis Team|16-10-2017

Sommaire
1. Data validation
2. Data analysis
3. Practical case on Hanoi data
Validation and data treatment | 15-10-2017 2

1 Data validation

Data validation objective
Input data:
Campaign
&
Monitoring Network
Data validation
Validation and data treatment | 15-10-2017

Data validation objective
Guarantee that :
► Technical validation : Data are technically compliant and

reliable
► Environmental validation : Data are consistent with the knowledge

in terms of photochemical behaviour of
atmospheric polluants
and representative of the behaviour and
the area around the station, defined by
the station typology

Risk of unvalidated data
► To communicate values higher than reality
► Loss of credibility
► Problems for modeling calibration
► Not to be able to evaluate trends properly and so the

efficiency of reduction measures

Technical validation
► Offset, deviation
► Wrong data after maintenance
► Instability (flow problems for example)
► Negative data…
► Both automatic corrections and human validation

Environmental validation
► Knowledge :
Monitoring network (types of stations, methods…)
Physico-chemical behaviour of atmospheric pollutants
Emission sources
Statistics records
National standards
► Tools :
comparison with levels on other monitoring stations
comparison with other pollutants on a same station
consistency with the theoretical knowledge about the physico-
chemical behavior of atmospheric pollutants
consistency with the knowledge about emissions
(ex : high levels of CO without NO on a traffic site ; NO / NO2
ratio)
Near road traffic

Correlations between pollutants from a same source
Correlation between NO and NO2
NO
NO2

Correlation between CO and NO

Correlations between pollutants
Correlation entre NO et NO 2 in summer
NO2 levels because of ozone

NO peaks car in the afternoon :
NO+O3=NO2+O2
NO2

Consistency between monitoring sites
NO traffic - background
Near traffic
Background

NO traffic - background
Near traffic
Background

In background situation

Impact SO2 - NO
Industrial impact :
Correlation NO–SO2

Consistency between pollutants
Anti-correlation between O3 and NO2
NO2 O3

O3 background: Urban / Rural
Rural station
Urban station

Environmental validation
can identify a technical problem

Inconsistency with emission data
Example : anticorrelations NO/NO2 on traffic sites

Inconsistency with emission data
Example : NO2 without NO on traffic sites

Inconsistency with atmospheric photochemistry
NOx impacts without ozone reaction
Not possible !

Ozone « blast »
Not possible !

PM10/PM2.5 correlation
PM2.5 ~60 to 70 % of PM10
Not possible !

Consistency between pollutants
PM10 and PM2.5

200
180
Trung Yên PM10_12
160 Trung Yên PM25_12
Concentration (µg/m3)
140
120
PM10 < PM2.5 not possible
100
80
60
40
20
0
1
4
7
10
13
16
19
22
25
28
31
34
37
40
43
46
49
52
55
58
61
64
67
70
73
76
79
82
85
88
91
94
97
100
103
106
109
112
115
118
121
124
127
130
133
136
139
142
Source : DONRE

How to identify a technical problem ? Pratical case
Site du Parc Omnisport

Suzanne Lenglen
About 900 m
Station d’Issy-les-Moulineaux
First tests show no technical problems

How to identify a technical problems ? Pratical case
Ex campain Paris 15 SL
NO2 levels very low compared to the other stations
80
C on ce ntra tio n en N O 2 e n µg /m 3
60
40
20
Paris Stade Lenglen Issy-les-Moulineaux

45
Paris Stade lenglen
40
Issy-les-Moulineaux
35
30
25
NO2] en µg/m3
20
15
10
0
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Heures

µg/m3
Nord
70
60
50
40
30
20
10
Ouest 0 Est
Sud
Issy-les-Moulineaux
Paris Stade Lenglen
Impact all the day, in every wind direction not a local source

After changing the analyser :

Mantes-la-Jolie
Gonesse
Lognes
100
Evry
Lognes
Villemomble
Tremblay-en-France
80 Argenteuil
C o n c e n tra tio n e n N O 2 e n µ g /m 3
Versailles
Saint-Denis
Champigny
Gennevilliers
60
Neuilly-sur-Seine
Aubervilliers
Bagnolet
Paris 13ème
40 Bobigny
Paris 12ème
Paris Centre
Issy-les-Moulineaux
Paris Stade Lenglen
20
Vitry-sur-Seine
La Défense
Ivry-sur-Seine
Paris 18ème
Paris 7ème
Paris Stade Lenglen Issy-les-Moulineaux
0 5 10 15 20 25 30 35 40 45 50
[NO2] en µg/m3

Data can be technically valid
but not representative

Representativeness
► « Environmental » validation : guarantee that data are

representative of the behaviour and of the area around the
monitoring station, defined by the station classification
« Non representative » = pollution data technically valid but

which is not consistent with the expected behaviour
considering the sampling point classification and which is not
representative of the spatial scale of the monitoring station
≠ « non correlated » data, for which the chemical existence

of the data is questionable

Spatial scale : urban background stations
► Located in urban areas
► Pollution level not dominated by a single source type (e.g.

traffic)
moderate levels
Average exposure of general population
(or vegetation and natural ecosystems)
► Representativeness:
A district in very densely urbanized
areas to several km2

Spatial scale : rural background stations
► Located in rural areas
► Far from any pollution source

minimum levels
Average exposure of general population or vegetation and
natural ecosystems)
► Representativeness:
several dozens km2

Spatial scale : traffic stations
► Located in close proximity to a single major road (less than 10 m

from the road).
► Maximum levels to which the population is exposed
(high levels but shorter exposure for the general population)
► Pollutants : from road traffic

NOx, particles, CO, HAP et benzene
Representativeness:
100 meters
“Micro-scale” environment

How do we judge that data are not representative ?
► The station has an atypical behaviour (very high levels,

instability…)
Ex : works or temporary sources, as diesel generators in the
immediate vicinity of the sampling point. Ex : Tremblay -
Summer 2008

Example of non-representative data
Demolition works in the immediate vicinity of the sampling

point - Vitry (2003)

But we keep large scales events
► Eiffel Tower fireworks (Opéra, 14.07.12)

But we keep large scales events
► big fire in the « Sentier » area 2008, April 9th
226 rue Saint-Denis
> 700 m as the crow flies
PA01H

How to identify a local source ?
► Very unstable profile = often very close source

► Correlation with other pollutants
if PM10/PM2.5 correlated with NOx = combustion source
if not, and PM10 >> PM2.5, often resuspension
► Pollution rose

How to identify a local source ? Pratical case n°1
► Example of the validation campaign of Paris 15 Stade

Lenglenn
Campaign
Paris Stade
Lenglenn
Mobile site: PM10
About 900 m
Reference site
Issy-les-Moulineaux

Distribution of hourly average concentrations

100
Stade Lenglen>
Reference
80
Concentration en PM10 en µg/m3
60
40
20
Paris Stade Lenglen Issy-les-Moulineaux Paris Centre Vitry-sur-Seine Gennevilliers

35
Paris Stade lenglen
30 Issy-les-Moulineaux
25
20
[NO2] en µg/m3
15
Hourly average concentrations regularly higher than the
10
reference site
5
Not a temporary event
0
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Heures

Is this a technical problem or a local influence?

> 4 m/s 2 - 4 m/s 0 - 2 m/s
Nord
12%
10%
8%
hourly average PM10 6%

4%
concentrations Ouest
2%
0% Est
Impact Sud
(PM10 Stade Lenglen – PM10 reference)

with wind direction
5
-1
-2

Building under construction
Local influence confirmed in PM10
-1
-2

End of impact
confirmed at the end
of the construction

How to identify a source ? Pratical case n°2
Few SO2 sources in IdF, slow formation (1 to 2 days)

Good tracer of long range transport

Impact of an Icelandic volcano

49
???
But Sulfate doesn’t explain everything…

EBC 2/3 from local

combustion sources

Which criteria to invalidate the data ?
► the number of stations --> area concerned

► the distance from the event
► the intensity of the phenomenon
(max of the day, baseline statistics)
► the duration of the phenomenon
► the recurrence of the phenomenon

In any case, ensure validation consistency
Between monitoring stations

In any case, ensure validation consistency
Between pollutants (NOx/O3 for ex.)

2 Data Analysis

Role of meteorology
0-1 m/s 2-3 m/s 3-4 m/s >4 m/s
70
Concentrations en µg/m3
60
50
40
30
20
10
0
NO2 NO PM10
► pollution levels decrease when conditions are more

favorable to dispersion (better mixing of the air )
► When the wind is null or very low : atmospheric stability and
higher pollution levels.

Role of meteorology
► PM10 impact more important during dry meteorological

conditions :
Resuspension by traffic and industrial activities
Humidité relative
<50% 50-60% 60-70% 70-80% 80-90% >90%
100%
80%
Impact (%)
60%
40%
20%
0%
Gymnase d'Epluches Rue de la Plage Ecole Effel Rue d'Epluches
-20%
-40%
Impact = difference between reference background site and campaigns sites

Role of meteorology
► Wind direction :
An area is “downwind” of a source when it is located after
the emission source according to the wind direction
Levels are influenced by the emission source
Sampling point Wind direction

« downwind » of the
emission source
Emissions source
Sampling point

Role of meteorology
► Wind direction :
An area is “upwind” of a source when it is located before
the emission source according to the wind direction
Levels are not influenced by the emission source
Sampling point « upwind »

of the source
Emissions source
Sampling point
Wind direction

Background pollution

Influence of Paris emissions
Neuilly-
sur-Seine
Bagnolet
Paris

Influence of Paris emissions
South-West winds
East winds
Bagnolet « upwind » of Paris Bagnolet « downwind » of

< Neuilly « downwind » Paris > Neuilly « upwind »

Daily profile – Influence of the distance to the agglomeration
► Two maxima at rush hours

P OISSY Cergy-P o nto ise Gennevilliers
One peak when
35
meteorological conditions
30 are less favorable to
pollutants dispersion
Concentrations en µg/m3
25
20 One wider peak at the end

15
of the afternoon
10 PM10 levels at Poissy range
between Gennevilliers
5
(close to Paris) and Cergy-
0
1h 3h 5h 7h 9h 11h 13h 15h 17h 19h 21h 23h
Pontoise (limits of the
H e ure Léga le agglomeration)

Trend analysis

Réduction des émissions de polluants primaires
monitoring of sulfur dioxide (SO2) over 45 years...

… and levels divided by 15
400
Dioxyde de soufre
364
Méthode de mesure : depuis 1994

Acidité forte jusqu'en 1993 sur un échantillon
350
324
évolutif de stations
321
urbaines
290
280
300
275
273
272
270
250
222
221
212
201
198
198
3
192
µg/m
200
173
172
171
161
148
146
143
150
122
109
90
100
82
76
76
69
66
50
47
46
43
41
50
32
28
25
25
22
21
19
16
14
13
13
12
10
10
8
6
5
0
56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 00 02 04 06 08

Trends – Example : NO2/NOx near traffic
Decrease for NOx but steadiness for NO2
500 MOVING MEAN CONCENTRATION OVER 3 YEARS

IN THE PARIS AGGLOMERATION
407 395 (CONSTANT SAMPLE)
400 378
353
333 315
311 297
300 279 268 262 260
3
µg/m
200 100
85 85 84 84 83 84 83 84 83 84 84 85
100
80
0
60
µg/m3
1996- 1997- 1998- 1999- 2000- 2001- 2002- 2003- 2004- 2005- 2006- 2007-
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
40
20
0
1996- 1997- 1998- 1999- 2000- 2001- 2002- 2003- 2004- 2005- 2006- 2007-
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Evolution of the NO2/NOx ratio on traffic stations

Evolution of the NOx impact : different trends according to the stations

Impact of measures on traffic
120
100 Outside Paris

+5%
Concentration en µg/m3
80
60
Inside Paris
- 5%
40
Quai des Célestins

Importance to have reliable data !
20
Moyenne Paris intra-muros sans Célestins
Moyenne Grandes Voies de circulation
0
01-09 / 12-09
07-09 / 06-10
01-10 / 12-10
07-10 / 06-11
01-11 / 12-11
07-11 / 06-12
01-12 / 12-12
07-12 / 06-13
01-13 / 12-13
07-13 / 06-14
01-14 / 12-14
07-14 / 06-15
01-15 / 12-15
07-15 / 06-16
01-16 / 12-16
07-16 / 06-17
Année glissante

Impact of traffic

Impact of traffic
N.104

Impact of traffic
50 par vent de " Nord" (au vent N104)

par vent de " Sud" (sous le vent N104)
40
30
20
10
0
N104 à 60m Evry M ontgeron M elun
(Etiolles)
Down-wind of the N104 road : increase of NO levels

On the contrary, on the reference stations : decrease of
concentrations

Impact of traffic
Ring-road
The mobile
laboratory is
located at 140m
west of the A4
highway, and
200m South of the
Paris ring-road
A4 Highway

Rose of pollution impact
The rose pollution of the impact

Nord
40 points at :
35
30
25 ► Influence of the A4 highway
20
15
10
5
NO2 levels 30 µg/m3 higher
Ouest
0
-5 Est
than the background level
at Ivry-sur-Seine with South-
West winds
Sud
► Influence of the ring-road
impact of 10-25 µg/m3 with
North-West wind

Sensors data treatment

Metrological tests on microsensors and ministations
35 brand tested
► Test from few weeks to few months

► On background and traffic
situation
► Pollutants measured: NO2, NO, O3,
CO et PM10/2.5

NO2 with sensor (MN_030)
Very good temporal correlation but results less accurate than

reference methods
Offset +15

NO2 with sensor (MN_030-15µg/m3)

Relative deviation MS/Ref. en NO2
Average relative deviation (without correction)

90 86
80
71
70
60 57
Pourcentage (%)
50
40
32
30
20
20
14
10
10 5 5
3 3 4
0
S1 S2 S3 S4
Tube/auto Cairpol/Tube
Sensor 1/Tube GreenBee/Tube
Sensor 2/Tube
Possibility of correction on the weekly scale thanks to the tube

measurement.
Example of correction week by week with tubes
Comparison reference NO2 without tube correction
Comparison reference NO2 with tube correction

NO2 sensors
► Very good temporal correlation but results less accurate

than reference methods
► Necessity to make a data treatment to ajust the data :

correction with another measurement method
or statistical correction if enough sampling points
► Useful to test several sensors simultaneously with reference

method

PM10 measurements

3 Practical case on Hanoi data

Example of technical validation
Strong instability
Data source : DONRE

Example of technical validation
Very wide range of

concentration for SO2
and CO. Accuracy ?
Data source : DONRE

Mean levels from 13/02 to 16/05
CO Average - Daily data from 13/02 to 16/05

7000
6000
5000
4000
3000
2000
1000
0
Trung Tân Mai Tây Mỗ Hoàn Kim Liên Mỹ Đình Phạm Thành Nhổn Hàng
Yên CO 19 CO 11 Kiếm CO CO 17 CONO 142 Average
Văn - Daily
Công data from
CO_8 13/02
Đậu COto 16/05
CO_12 10016 Đồng CO 18 15
90 CO 3
Concentration µg/m3
80
70
60
Traffic sites
50
40
Background sites 30
20
10
0
Trung Yên Tân Mai Tây Mỗ Hoàn Kim Liên Mỹ Đình Phạm Văn Thành Nhổn Hàng Đậu
NO2_12 NO2 19 NO2 11 Kiếm NO2 NO2 17 NO2 14 Đồng NO2 Công NO2 NO2_8 NO2 15
16 3 18

Mean levels from 13/02 to 16/05
Traffic sites
Background sites
PM10 average
140
Concentration µg/m3
120
100
80
60
40
20
0
Trung Yên Tân Mai Tây Mỗ Hoàn Kiếm Kim Liên Mỹ Đình Phạm Văn Thành Công Nhổn Hàng Đậu
PM10_12 PM10 19 PM10 11 PM10 16 PM10 17 PM10 14 Đồng PM10 PM10 18 PM10_8 PM10 15
3

Daily profile13/02 to 16/05
CO daily profile
10000 Trung Yên CO_12
Profile with 2 distinct peaks, typical of road traffic sites
9000 Tân Mai CO 19
8000 Tây Mỗ CO 11
7000 Hoàn Kiếm CO 16
6000 Kim Liên CO 17
5000 Mỹ Đình CO 14
4000 Phạm Văn Đồng CO 3
3000 Thành Công CO 18
2000
Nhổn CO_8
1000
Hàng Đậu CO 15
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
HOURS

Profile with 2 distinct peaks, typical of road traffic sites
NO2 daily profile

140 Moyenne de Trung Yên NO2_12
Moyenne de Tân Mai NO2 19

120
Moyenne de Tây Mỗ NO2 11

100
Concentrations NO2
Moyenne de Hoàn Kiếm NO2 16
80 Moyenne de Kim Liên NO2 17
Moyenne de Mỹ Đình NO2 14

60
Moyenne de Phạm Văn Đồng NO2 3

40
Moyenne de Thành Công NO2 18
20
Moyenne de Nhổn NO2_8
0 Moyenne de Hàng Đậu NO2 15
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hours

PM10 daily profile

200
Trung Yên PM10_12

180 Phạm Văn Đồng PM10 3
Nhổn PM10_8
160 Hàng Đậu PM10 15
140
120
Concentrations µg/m3
100
80
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hours

Trafic site Hàng Đậu (site n°15)
► High levels of NO2 and CO, consistent with a traffic type of station
► Daily profile with 2 distinct peaks, typical of traffic sites
► PM10 mean level seems a bit low
CO daily profile
10000 Trung Yên CO_12
9000 Tân Mai CO 19
8000 Tây Mỗ CO 11
7000 Hoàn Kiếm CO 16
6000 Kim Liên CO 17
5000 Mỹ Đình CO 14
4000 Phạm Văn Đồng CO 3
3000 Thành Công CO 18
2000
Nhổn CO_8
1000
Hàng Đậu CO 15
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
HOURS

Traffic site Phạm Văn Đồng (site n°3)
► NO2 and SO2 a little higher than background levels

The monitoring site is implemented a little bit far
from the road ?
► Very high PM10 levels during the night influence of the

works ?
PM10 daily profile
200
Trung Yên PM10_12

180 Phạm Văn Đồng PM10 3
Nhổn PM10_8
160 Hàng Đậu PM10 15
140
120
Concentrations µg/m3
100
80
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hours

Traffic site Nhổn (site n°8)
► CO profil shows an influence of the traffic, but not very

intense (distance from the road ?)
► NO2 too low and NO/ NO2 ratio not consistent
► Comparison with the background site
► PM10 quite high influence of the metro work ?

Resuspension ?

► NO2 profile and NO/NO2 ratio not consistent with traffic
sites
Background site Trung Yen (site n°12)

10
15
20
25
30
35
10
15
20
25
30
35
5
0
1 1
4 4
7 7
10 10
13 13
16 16
19 19
22 22
25 25
28 28
31 31
34 34
37 37
40 40
43 43
46 46
49 49
52 52
55 55
58 58
61 61
64 64
67 67
70 70
73 73
76 76
79 79
82 82
85 85
88 88
91 91
94 94
97 97
100 100
103 103
106 106
109 109
112 112
115 115
118
118
121
121
124
Trung Yên NO_12
Trung Yên NO2_12
124
127
127
130
130
133
133
Nhổn NO_8
Nhổn NO2_8
136
136
139
139
142
142
145
145
93
► PM10/PM2.5 ratio ? PM10 often > PM2.5
200
180
Trung Yên PM10_12
160 Trung Yên PM25_12
140
120
100
80
60
40
20
100
103
106
109
112
115
118
121
124
127
130
133
136
139
142
1
4
7
10
13
16
19
22
25
28
31
34
37
40
43
46
49
52
55
58
61
64
67
70
73
76
79
82
85
88
91
94
97

► Instability of CO levels
3500 35
Trung Yên CO_12
Trung Yên NO_12
3000 30
2500 25
2000 20
1500 15
1000 10
500 5
0 0
1
4
7
10
13
16
19
22
25
28
31
34
37
40
43
46
49
52
55
58
61
64
67
70
73
76
79
82
85
88
91
94
97
100
103
106
109
112
115
118
121
124
127
130
133
136
139
142
145
L’Observatoire au service de la Santé
et de l’Action
airparif.fr
Contact : veronique.ghersi@airparif. fr | 01 44 59 47 64

Hanoi Data Treatment Oct17 PDF

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Hanoi Data Treatment Oct17 PDF

Încărcat de

Drepturi de autor:

Formate disponibile

Validation and data treatment

Véronique Ghersi|Head Forecast and Diagnosis Team|16-10-2017

Validation and data treatment | 15-10-2017 2

Validation and data treatment | 15-10-2017 3

Validation and data treatment | 15-10-2017

► Technical validation : Data are technically compliant and

► Environmental validation : Data are consistent with the knowledge

Validation and data treatment | 15-10-2017 5

► To communicate values higher than reality

► Problems for modeling calibration

► Not to be able to evaluate trends properly and so the

Validation and data treatment | 15-10-2017 6

► Wrong data after maintenance

► Instability (flow problems for example)

► Both automatic corrections and human validation

Validation and data treatment | 15-10-2017

Validation and data treatment | 15-10-2017 9

Correlation between NO and NO2

Validation and data treatment | 15-10-2017 10

Correlation between CO and NO

Validation and data treatment | 15-10-2017 11

Correlation entre NO et NO 2 in summer

NO2 levels because of ozone

Validation and data treatment | 15-10-2017 12

Validation and data treatment | 15-10-2017 13

Validation and data treatment | 15-10-2017 14

Validation and data treatment | 15-10-2017 15

Validation and data treatment | 15-10-2017 16

Anti-correlation between O3 and NO2

Validation and data treatment | 15-10-2017 17

O3 background: Urban / Rural

Validation and data treatment | 15-10-2017 18

Validation and data treatment | 15-10-2017 19

Example : anticorrelations NO/NO2 on traffic sites

Validation and data treatment | 15-10-2017 20

Example : NO2 without NO on traffic sites

Validation and data treatment | 15-10-2017 21

NOx impacts without ozone reaction

Validation and data treatment | 15-10-2017 2222

Validation and data treatment | 15-10-2017 23

Validation and data treatment | 15-10-2017 24

PM10 and PM2.5

Validation and data treatment | 15-10-2017 25

Site du Parc Omnisport

First tests show no technical problems

Validation and data treatment | 15-10-2017 2626

Paris Stade Lenglen Issy-les-Moulineaux

Validation and data treatment | 15-10-2017 2727

Validation and data treatment | 15-10-2017 28

Validation and data treatment | 15-10-2017 29

After changing the analyser :

Validation and data treatment | 15-10-2017 30

Validation and data treatment | 15-10-2017 31

► « Environmental » validation : guarantee that data are

« Non representative » = pollution data technically valid but

≠ « non correlated » data, for which the chemical existence

Validation and data treatment | 15-10-2017

► Located in urban areas

► Pollution level not dominated by a single source type (e.g.

Validation and data treatment | 15-10-2017 33

► Located in rural areas

► Far from any pollution source

Validation and data treatment | 15-10-2017 34