ch6 Bayesian Learning.

DE
D M
N
D M
N
D M
N
DE
D`
Dd
DE
Dd
mon[p
"!

$#&%(')'*,+- /.02143657*8*:9;<7=)>?<A@0>B<C"!
GF*IH;B2J)1K2L
O P > M Q RF&STJ)+8(*8*
O P U V1XW(Y1Z*
5[WB5[L %(L *X3/1X5[SB+85[J)W U 2W('+8 S(1K5[W?365\S U
GF*]J)S(+85[L U 3 U *X*85_^21
a5[b) E cF)* U a1KW(21
. aL S U )9 Q a1KW(5[W(' J"b21I+8/.0+ (a+K
GF*,5eaW fT U 5[/gW(2+ihjJ1Zk&*
.0ST3/+Ka+85[JW M ".05[L 5\la+85[JW U ')J)1K5[+8(L
qsr$t$uwvyxr/z{qs|s}yr$z-~x,u{r$uy7cV[$7[[[4yV|ut7r$qsqVtox{A-|sqsqmA[[

P X1 J b&57<* SB1Z)3/+85732 U U V1XW(5\W(' U ')J)1K5[+8(L *:9

D` a5[b) E cF)* U a1KW(5[W('
DE GF*,5eaW fT U 5[/gW(2+ihjJ1Zk U a1KW(5[WB'
D JL f(5[WB S(1K5[J)1 k WBJ"h U <' ! S(1K5[J)1 SB1XJ)f af(5 U 5\+85G*#"
h 5\+8 J)f?*821Kb) (a+
D %$ %(5[1K* S(1K5[J)1 S(1KJ)f?Vf(5 U 5[+-5*
P X1 J b&57<* % *,/g % U 36JW?362S(+-%? U g 1ZaL 2hjJ)14k
DP 1KJ"b&57 * &')J U *,+KVW?(a14(' g J)1 2b U %?V+85[W(' J)+8(Y1
U a1KW(5[W(' U ')J)1K5[+8BL *
DNO (<5[+85\J)W? U 5[W?*85[')R+ 5[W0+-J ) 3232aL *C*]1ZVl2J)1
mon,+

! "

D ! "
D ! "
D ! "
D ! "
mon[
! " ! "
! "
S(1K5[J)1 S(1KJ)f?Vf(5 U 5[+

F JVg RF&STJ)+8(G*,57*
SB1X5[J1 S(1KJ)f?af(5 U 5[+iF JVg +814a5[W(5\W(' Ba+K
S(1KJ)f?afB5 U 5[+
F JVg
'5[b)2W
S(1KJ)f?afB5 U 5[+
F JV
g ')5[b2W

! "

! " ! "
! "
2WB21Z U[U F hVW0+]+-( L J *8+]S(1KJ)f?af U 0F&STJ)+8B*,5e* )' 5[b)2W

8+ ( +814a5[WB5[W(' (a+K

0F&STJ+8(*857* "! 9
"! a1K' L ". ! "
#$%
! " ! "
a1K' L ".
! "
#$%
a1K' L ". ! " ! "
#$%
& g *X*8%(L !' " !"( "+8(2W 32aW g %(1K+8(21 *85[L S U _5 g F>
aW? 3 (J&J *, +-( )
* + ,-
/.),0
213445 ! M Q "
0F&STJ+8(*857*

mon;:
6 a1K'L . ! ' "

#879$%

J&G* ?S a+85\2W0+ Gb) 32aW 3621 J)1IW(J+

O S?a+85\2W0+ + )ka* U Vf +8*8+ aW? +8( 1K*,% U +
36J)L *]f?3k STJ *,5\+85[b))H]( +8G*,+ 1K2+8%B1XW?*I
36J)1K1X3/+ STJ *,5[+-5[b) 1K*,% U + 5[W J)W U F J g +8(
32)*8* 5[W hI(573Z +8( <57*8)*8 57* )36+-%? U[U F S(1K*,YW0+2>
aW? 36J1X1K36+IW(2' a+85[b) 1K*,% U + 5[W J)W U F J g
+8( 32)*8* 5[W h B573 +8( <5e*,*, 5e*NW(J+ SB1X*82WR+2
>
JVg +8( 2WR+85[1K STJ)S(% U a+85[J)W
%(1K+8(Y1XL J1X)
?Gb) +8(57* 32aW 3621
! "
! "
! $ "
mon[
! "
!"! # "
!"! $ "

45
) 3,; 9 (S 1KJ)f?Vf(5 U 5[+
F ! "jJVg
36J)W %(W?36+85\J)W JVg
+ hjJ Yb)2W0+ * O a W? E 9
D
!
"
! " ! " ! " ! "
,; 9 S(1KJ)f?afB5 U 5[+

F JVg <57*%BW?36+85[JW JVg +
hjJ
2b)2WR+K* O a W E 9
D
!
"
! " ! "
!
"
31 ,

,0
9 5_$ g Yb)2W0+ * !

"# $
a1X L %(+8%? U\U F :. 3 U %?*85[b) hI5[+8 % '' & ! ' " )= >(+8B2W
$
! " ''( & ! ) ' " ! ' "
m+*-,

= )J 1I3 0 F&STJ)+-(*857* 5[W

STJ *,+-21X5\J)1 BS 1XJ)f af(5 U 5\+iF
! "

>(3Y U 63 % U a+8 +8(
! " ! "

! "
@R ) %B+8S(%(+I+8( RF&STJ)+8(*857* "! h \5 +8 8+ ( (5\')(*8+

STJ *,+-21X5\J)1 SB1XJ)f af(5 U 5\+iF
"! a1X'L ". !
"
# $%
m+*ym

+

J)W?*857<21 J)%(1 %?*8%? U 36JW?362S(+ U a1KW(5[W(' +K)*Kk

D 5[W?*8+KaW?36 *,S?36 >(0F&STJ+8(*857* *,S?36 > +814a5[W(5\W('
/. aL S U *
D 36J)W?*857<21 +8( U Ga1XWB5[W(' U ')J)1K5[+8(L ! J)%(+8S(%B+K*
L J *8+I*8ST365_^T3 0F&STJ)+8B*,5e* g 1XJ)L +8( b)214*85[J)W *8S?)36
% "
?V+ hjJ)% U E GF)*]1K% U SB1XJR<% 36 )*]+8( M O P

0F&STJ+8(*857*
J&G* J)%(+8SB%(+ M O P 0F&STJ)+-(*857*
m+*[n


+

O *K*,%(L ^(.0 *,Y+ J g 5[W?*8+KaW 36*

"
O *K*,%(L 5e*N+8B *82 + J g 3 U )*K*,5 ^T32a+85\J)W?*
! "

" ! "
(J&J *8 ! "49
m+*-*

+

O *K*,%(L ^(.0 *,Y+ J g 5[W?*8+KaW 36*

"
O *K*,%(L 5e*N+8B *82+ J g 3 U )*K*,5 ^T32a+85\J)W?*
! "

" ! "
(J&J *8 ! "
D ! " = 5_g 36J)W?*857*8+82W0+ h 5\+8
D ! " J)+8(Y1XhI57*8
(J&J *8 ! "j+8J fT
) 57*,+-1X5[fB%(+85[J)W

g J)1 U[U 5[W
!

"

D
%
H](2W>

5_g 57* 36J)W?*857*8+82W0+ h 5\+8

! "

J)+8B21XhI57*8

m+*

P(h )

P(h|D1)
hypotheses
(a)
m+*[p

P(h|D1, D2)
hypotheses
( b)
hypotheses
( c)

Inductive system
Training examples D
Candidate
Elimination
Algorithm
Hypothesis space H
Output hypotheses
Equivalent Bayesian inference system

Training examples D
Output hypotheses
Hypothesis space H
Brute force
MAP learner
P(h) uniform
P(D|h) = 0 if inconsistent,
= 1 if consistent
Prior assumptions
made explicit
m+*,+

y
f
hML
e
J)W?*857<21 aW0F 1XG U b U %( +Ka1K')2+Ng %(W?3/+85[J)W

H14a5[W(5[WB' /. aL S U * ' ' Z>(hI(21K ' 57* W(J)57*8F
+814a5[W(5\W(' b U %B
D ' ! ' " '
D ' 57* 1ZVW?<J)L b a1K57af U ! W(J)57*8 "<14GhIW
5[W?<2STYW?<2WR+ U F g J)1 3 ' )3236J1Z<5\W(' +8J *8J)L
a%?*K*,5eaW <57*8+81X5\f(%(+85[JW hI5[+8 L aW
H](2W +8( L .05[L %(L U 57kV U 57(J&J 0F&STJ)+-(*857* 6 57*
+8( J)WB +-?a+ L 5\W(5[L 5[lY* +8( *8%(L JVg * $ %?a1K 21K1XJ)14*29

6 a1K' L 5[W ' ( & ! ' ! ! ' " "
#$%
x
m+*[

6 a1K')L
#$%
a1K')L
#$%
a1K')L
#$%
M ".05[L [5 lY W?a+8%B1Z U U J)'

m+*;:
6 a1X'L
#$%
a1X'L
#$%
a1X'L
#$%
a1X'L
#$%
.
! ' "

=
77
. ' &

@
JVg +-(57* 5[W *,+8G) 7
. ' &
". '' ( & U W

! "
". '' ( &

". ''( &
5[W ' ( & ! '
= '
@ @
= ' ! ! ' "
! ' "
! ' " "
! '
!
! ' " "

J)W?*857<21 S(1XG<5736+85\W(' *8%(1Kb&5[b U S(1KJ)f?af(5 U 5[+iF g 1XJ)L

S?a+-5[2W0+ (a+K
H14a5[W(5[WB' /. aL S U * ' ' Z>(hI(21K ' 57* = J)1
VW0+I+8J +814a5[W W(Y%(1Z U WB2+ihjJ)14k +8J J)%(+8S(%B+

,0
')5\b)2W ' ! WBJ)+ J)1 = "
& W 8+ B57* Y3 )*, 32VW 8* (J h

6 a 1X'L ". ' ( & ' U W ! ' " ! = ! ' " U W !=
#$%
Y5[')0+ %(S (a+8 1X% U g )J 1 *,5['L J)75 %BW(5[+29
(
( (
hI(21K

( ( ! ' ! ! ' " " ' (
'' &
m+*[
! ' " "

) 3232aL *$*]14al2J)19 S(1X:g 21 +-( *,(J1X+8G*,+I0F&STJ+8(*857*
Q 9<SB1X/g 21 8+ B R F&STJ)+8(G*,57* +8?a+IL 5[WB5[L 5[l2G*

6 a 1K')L 5\W
! "
! "
#$%

hI(21K
! "5e*N+8B *X3/1X5[SB+85[J)W U 2W('+8 JVg %BW?<21

2W?3/JR<5[W('
M
<3657*85[J)W +81K2*2> +814a5[WB5[W(' (V+K

d . aL S U 9
U afT U *
! "57* fB5[+K* +8J <*K361X5\fT +81K2
D
! "5e* f(5\+K*N+-J <*K361K5[fT ')5[b)YW
D

J)+8
! "
5_g /. VL S U * 3 U )*K*,5 ^
`
ST21-g 3/+ U F f0F ` 2G JW U F <*K361K5[fT /. 3/2S(+85[JW?*

D 2W?36 6 +-1Z) * J
+81K2 *85[l2 g J)1 +814a5[W(5\W('

21X1KJ)14*
m,

"! a1K'L . ! " ! "

#$%
a1K'L . U J)' ! " U J)' ! "

#$%
a1K'L 5\W ! U J)' ! " ! U J)' ! " != "

#$%
& WR+821K*8+85[W(' g )36+Ig 1XJL 5[W g J)1KL a+85[JW +-(2J)1KFT9
H;B JS(+85[L U ! *,(J1X+8G*,+ :.0ST36+8 36JR<5\W('
U 2W(')+- "36JR< g J)1 aW 2b)YW0+IhI5[+8 S(1KJ)f?Vf(5 U 5[+
F 57*
! U J)'
f(5[+ *:
#&J 5[WR+821KS(1XY+ ! = "Z9
g %BW?<21IJ)S(+-5[L U 3/JR<
D ! U J)' ! "57* U 2W('+8 JV
g ')5\b)2W %BW?<21
D ! U J)' ! "j57* U 2W(')+- JV
J)S(+85[L U 36JR<
S(1K/g Y1 +8( 0F&STJ+8(*857* +8?a+ L 5[W(5[L 5\l2*
! "
m m
"
!

M J ,* + P 1XJf?af U U )*K*85_^T32a+-5[J W VJ g ` 2h & W?*8+KaW 36*

&# J gV1 hj *wb) *8J)%(')R+I+8( L J *8+]S(1KJ)f?Vf U 1 13
2
')5[b2W +8( (V+K ! 5)7> "! "
5[b2W W(2h 5\W?*,+ aW?36 > hI?a+ 57* 5[+K* L J *8+]SB1XJ)f af U
),;

D "! ! "j5e*NW(J+ +-( L J *8+]S(1KJ)f?af U 3 U )*K*,5 ^T32a+857J)W
J)W?*857<219
D H;B1X2 STJ *K*85[f U 0F&STJ)+-(*8*:9
! "

! "
! "

D 5[b)2W W(Yh 5[W?*8+KaW 36 >
!
" !"
! !"
!

D ?a+ *C* L J *,+S(1XJf?af U 3 U )*X*85_^T3Ya+857J)W J g

m n

"!
#%$'&)(
/
*0#%(

,-
!. ( ' " ! ' "
$ # 7 $%
#+*
124365
"
!
"
!

"
!
!"! "
!"! "

!"! "
! "

! "

! "

7
7
8:9 3;$'3=<>?$@3
# 7 $%
(
# 7 $%
' D'HCJI
A B
CED ' F
D ' CJI
B
A
ML
CED GF
!
B
#%N O
#P$'&"(
Z *
,G-RQ
#+*
S:T Q=U
.V+CJDWXF
B
DWHCJI
[]\M^M_a`cbd\=eE[]f]gc\Meihkjlbm_E\Mnl_pocjqjlrtsPu4vMwqxzyG{}|{pu4~pyxy4l4cPf_J^q\M[][P^bEif][][kZ44

#?3

>61 8 (
#P2=2q# 3 $
%
# N # 3 3=* 1 3 *
N !63 <
- #)#. #%24&6>?$ :
8 9 ( 5
7 ,/
> >0m3 >?N 3

D CEI F
1 ,2
1$@>! O3"$# 3 8

9
1 > 8:9 3"m3(,
#%N+
(
$'3"&%2 8(' #)% 8
9
1 > 8:9 3"
# 8
' #(=>?$ O N&
$#%N O0>?(
8 >
B
8:93
m3
8 >
2q# <
5 %$@1$ N&
<# 8 576
8%8=9<8 N O>>
< 9 >>
88:9; 3 8
3
8 (=> 9 O N ;
BDCFE EHGEFI WKJLJLMONQP

A
5 %1 1 >0m3
%(
?
N. 8 #%N.=3
N3(4
1 9 > 9
8:9 3
O 9<8 4
>?N
,@
9 3 N 5
BRCFEEHGES3TVUXW MZY[]\ W_^ TV` N
1 A
> 9a9 3 8(' %N <> 9 >
8 >
>6N. 3 1 8
1 9 > 9
O 89 #3% 8 >?N
> !?3 9
'
8:9 3;N
fe
bdc
8 Ng
9
1 > 8:9 3
< 9 >>
5 ' 4
48:9
%N <> 9 >
1 9 >i# 8 # 2 48
bdj 8
3lk 1 3 8 3O
>?1 8 >

3 99 > 9
N >
> 9 m3
8:9*8 N
8 4
3
8 ?3"
8 2

4>?N ;
i9
4
Ng# 9F' >?N

9 ;N
>
:9
>P<

Z

9 ' N % 9]8 N 4
>0 39<8 8 8:9 N N ;
>
> 9 e ' N 8:9
>
:9 > O)l,
%*
9 #3%
O
>6N
> O
9]8
b
b
g
> 9
8:9;
i9.8
>?N O >?N 8
9<8: N N ;
O
9 #

8 ! 8: 8
#
N. 8 N*
N O
N O N
;i ! N
8:9
8 8 >6N
5 %. H<Z% 8 8 >?N.

b
b
Z
8:; N>0
/ 8 a <Z N ;
k
O>+ %3>
;N

&%3>
8:9;

N. 8 N*

g
39
8: !
# 8 #
#$ %&
'
#$ %&
'
4

Z
8
;i !

<Z%N. >?N
O
"a 9 # O
'
3
#g
LK
89 #3%
' 4 9

<,
8
"!
8:9; > 8 k
# .V -
/
011
!
(*)RQ+
,
2

1
- # V ! # V !
8:9; > 8 k
,
,!
2

/

0

1

1

/

.

*( )RQ+
8:9; > 8 k
2
,
1
- # V ! # V !
( ) Q+
,
,
! 8 %
8 &%3>

N
2
4
011
- # V ! '
7 8:9*;< = 8>?<@ ABC8:@D@9*EF<GIH #J
5W
'
2
W6- # V !
8:9; >
( ) Q +
8 k
# V ! 5W
,
2
W6- # V !

3
8: !

8
%,
( #'& *! )

,
8
43
8:9
C

8:9;
! 8 %
>
8
8 9a #3%

2
+ - '# &
! ) >
8
(4
#J

8
8,
'
8:9; >

( ).- +
"! C$#
#'&
#'& !
! 8 %
8
!
8 k

+ 8

+ - '# & !
( #'& ! 5 (
T0/
2- 1 ,
,
8 9a #3%
2
+ - '# & !
576989:<;>=?62@A57B7C>69@EDGF4=H:A69I4:KJ>FLF4MON,PRQ9SLTVUXWZY[WKPR\KU]TÛR_4`Racb>d,Be:f8hgL69575i`]d,8kjl=AmonqpEB7575G`ZorRrR3

/

=8:;08: ' ,8
# " ! C
! '
,

>
%
# J
S
'
'
8:9a; >
()- +
#
8 k
! <? $&%
G:G
# & ! 5
R

'
8

' $)( ' +* $,-% ' # E G (
2
+ - # & !
# -:
. ! # G:G ! - :. ! ' )$ ( ' - :. ! # EHG (
,
,
,
,
,
/ ! # ! # GHG ! ! ' )$ ( ' ! # EHG (
,
,
,
,
,
/.:!
1798
Z
#J
S
'
- .:! ' 1 02043

- ! ' 1 0 165

8
,/
!
.8 R
, , , #3% =
99
8:9a; >
()- +
174r
B

>

8: !

8 a%3>

" >
R
3;
8
9

8:9a; >
()- +
9
8 k
8,3 ' 5

N C
8
9
9

2
+ - # & !
,
9

8
2
+ - # & ! '
8

5
R
9 e =% 9 )9a 3;
( # & ! 5 (
+ ,
,
8 k

b

2
/
1
- # & ! '
,

3
#'& - ! #
8 ,
4
# & !
9

11D
- # & !
8 .8 +
% 39 8 8

1 ,4

8=
8:9;
3 8
@
# '

'

b
b
Z

&
%3>
8 !
# & ! 5

+
,
, , ,
2
+ - # & ! '
8 8
R R
9<8: 3 );
')8
>

2
+ - '# & !)
k 8 >

C
8

+
! 8 %

8
89 #3%
2
+ - # & ! '

R
9<8: 3 3;

9

4
2
+ - '# & !
4
'
39

,(

# 9
%3>
R
&
4 9
b
%

! 8 %

# 9

>
;
! 9 % 8
k 8 >
k 8 >
C
8
=; !

39
4
9
2
+ - '# & !
9
, ,
# '
%)>
#'&
# 9

q

8H9 8:9
8H9 8
*8:;

b
b
8H9

9
! "

" #
# " " lk
$# %
3
Z cZ

q

- # $,( #
5
$#
"
" # "
"
" " !
"
,
,

%#
,
ZFE!G
$
#
)(+*!,-/.0 21
%'&
+453 , 2
+ ' 5
- #'& ! " "7
'# &
# -# & ! '
87 " :9 0
+ ' 5 6
, $ 5 6;
< < " # = < >
?9 0
+ '
%# # - !
,
%# # -Y !

-
Y

"
$#
"

!
3

"
-# & ! '
:9A@ '
5

- #'& !
"
- #'& !)BDC$)B0

9 "!2 # B
!#" !%$
'& ")(+*
* " $-, ( $."0/ $1,
$2"3!4!657(
9 "!2 #
! "
< " < "7 "
# 98 ! 9A . )
<:9 9 "! #
;7 !
!65
$ $-, ( #< 5 ( * = 1> &
* = 5 6)? > &
@ (+"BA
A $ $ (#C
I<
ED < " 7 " # >%F HG
J K ML6N FPO # <MQ 9SR TVU WN Q < "
87 7 "7
" # > F
.0ZY\[]
1

%
>
F
O
X
=
X_^`ba @c X [ X
J
J d Ve-FfO HG %/=& I < # G " I
<gG G HG ! " <MQ K MLhN F
J i O < G # " <MQ
< " I HG d je1F <#kG HG
I #
< " I # =
J Q < " 7
< " I 5 6 HG )L 98mlnU 9AWo
p i 6 O G # " <MQ
< " I 5 6 < # " HG
d je1F
p = 5 6)? >qF O rht Xxw r#0suY aztky|v { a~}4X
%
H
ZFE
___z3MH4HHHM_M#Amz__ZHH
ZFE
% 6 S
' MLq
T gN e i N O !
< " I < !< G HG ML "7
<gG HG <:9 G Q <#kGI HG ML 98mlnU 9AWo
# " G > ;
87 "
> " = 1>%F
= 9 ? %> F
] w
c)0u[ * 0 r [


G " G
) +& G < $%$ Q+*
, 7 7 G /, $ & <!
G I < "!# G %$ Q'& <# 7 & <(!

G - , I < "!# G .$ / <&I G <
# Q'& <#

<(# 10 & 7 $

# $ 0 Q <& $ !
<# 10 < $ 0 # $32 , GI < , $ 0 # $
& 0 ! < $
<# 40 $ * $ 0 5 # 0/ 0 7 &I , & & 0 # < <&"* ! $
<(# 10 $ * $ 0 # 0 7 &I , & & 0 $ <& 0 5 $ 5 !
<# 106, G I < , $ 0
& 0 $ <& 0 7 < 49 *
0 "7 $ #
$ < 0 & <gG 0 7 & $D G
9 0 & <gG 0 # $
9 0/ < $ 0 # I $"
9 /0 < $ 0 # $
9 /0 < $ 0 k
! G $
$ 0$
$ 0 "&%*
$ '0 ! & <gG $
$ 0# I
7 (8 9 * 8 /$ :<;(=?> "@ $%$ BA DC F E G /G!& "*
ZFE!E
G

1
"!$#"%&('*)+-, '*")'*/.#'0&1#3241)-5$&6)7'8"+-,+"1 '&

9:+;!<. ="="=>?"?"? 'A@"@"@B'&1# CED"!$" F!&G
H #"IJ"&K L&( MN") ?&"+(OE) INKP"P&)K 1 6Q!R+)K CE!$%NSN-!
F&( T U%+ V"W X"V(T"W(W"V Y-Z"M
[ !$Q? -!<.".& !$ & \]NSP])
^ I"# & ._!:)K
$! I,]N-!$#) 1N1& `;!R+ %Q&") )S#+%+N")& N") U&=
aNKNKb ' c& S.& NKPQ? !$#&1 ) 1&`&)"NK,&
1&`&)&S. ^ I"# &OE) -&"+;? .#S ._!R+& '
Y"+&" )Kb&+ 1 +"1 )SQ! CQ!$#P 6N")S & 6&+&
._!R+& "S#+&G
' [< `Kd^ & %+&""? .#S ""!$6&1
& \]NSP]) ! :+1& 6? "#P& 1&`&)"NK,&
:NIN-:NK"? #Q e;!R`"`&? ' \&"? c:+#1&? N") !$Q? &
INKP"P&)K 1N")%"%Q!:NS.& N` ?!$# Q!$#P & 6) ?
P!"!R1 ! I&P]NS 6NK' f#d^ I&)Kd^ &OE) !$Q?
.&1N-!:+& P!&1&+B' U I&"&+ SQ!:N"& 6Q!$#QR1 I&
M!<.) H 1)K:+;!<.3^ Q!$#P Q! +;!$#P ? `#Q !R`
N") !$6" ^ I"# I&#)& )-!<.& "#P]) NS M!R+;!$! 1&"N1&1
hgji
Z
3

20News
100
90
80
70
60
Bayes
TFIDF
PRTFIDF
50
40
30
20
10
0
1000
10000
-"!& "* $ 0 100

8

/

8

8
&
$ C $
, C @ Q E & C 8 $ C
hgH
Z
___z3MH4HHHM_M#z__ZHH

]
C 8 & 8 $ C 5 8 ! $ 8 :
7 8 9 * 8 $ $ $ !# C E E Q E C
8 8 8 8 C E E & 8 $ C & C 8
9 ! C C $ C & C. 5 @ 8 , C E ! C $ E # 8
$ $ !# C E $ 0 0F0
9 * 8 $ 98 @ 8 Q 8 C , E & $ 8 $ G& 5 8
8 8 8 8 # E
E Q
@@ E , $ E # 5 & E & E , @ 8 8
' 8 8 8 8 $ # E & 5 @ 8 $
C & DC%
E @
$!
E C E @
& 5 @8 $
5 E ! C
, C E5 $ 8 & 8
@ $ E @@ 8 9 * 8 $ 7 8 C $
hg
Z
___z3MH4HHHM_M#z__ZHH

B

$ ! #" E$
% '&)( 8 * & $ C 8 ,+ E5.-D5 & @)& C * /&10 C +2& 5,3 C & E
4 E ( 8 + & 4 &506& 8 8 8 C E$ C 8 ( - @ 38 E$ %
3 8 E$7* 8 C - C &50:9.& $
4 &)( 8 C 8 ( - @
<;>=.?@BACD@FEG IH J =K?ML % J ACN@* J EG J H J
Q E +2R S E Q - S C @)T'9.UVR U+2& C R
H
=K?OLP*
L % W@ * J H PL *
XZY - Q @)R :\[ ,

] ^._a`cbDd 1& 0 S E /& C & E .- @@1T & /RR /R C E$
e fg _ 9 4 &)(R h gji ].kl_ g _ i
H [ ],^K_a`cbDd L e f g _ @mh gji ].lk _ g _ i J H [ ],^._n`cbDd Loh jg i ].kp_ g _ i
q - &)(R r - T'R0 3 B0 R0 Sts vu& /RauwBs x 3 B0 wB& $ T
H @ % LP* J H L % @t* #H % LP*
J H PL * #H % LP*
hg)y
Z
z|{~}~<{Woz||<{~o{~P<1)~1P)P))<}1{~z|z}!m|z|zly)y)
EG

Storm

BusTourGroup
S,B
Lightning
Campfire
S,B S,B S,B
0.4
0.1
0.8
0.2
0.6
0.9
0.2
0.8
Campfire
Thunder
ForestFire
q RNwMUVs'+ +2R,+2R0R w20 - 0 R w6s $ Sts

& R R /R StR - 020BRN+ w&)s 0
/& wB&)s .-

s R &50 - 020RN+2wBR wBs R WS s /&)w&)s .- T
/RR /R w s $ &)w20 s /R02StR - 2w 0N9 4 &)('R &)w20
Q R & - wBR ,+2R /RSWR0 0Bs'+0:u
&)+2RStwBR - StT S &1S 4 + -
X - S
&
&Q
!
#"%$
z|{~}~<{Woz||<{~o{~P<1)~1P)P))<}1{~z|z}!m|z|zly)y)

Storm

BusTourGroup
S,B
Lightning
Campfire
S,B S,B S,B
0.4
0.1
0.8
0.2
0.6
0.9
0.2
0.8
Campfire
Thunder
ForestFire
R + R0BR w20 x s& w + s
- ,& & wOT
&10w+ &% 3 wB&)s
s ('RN+ -%
( - +2& - R 0
R'u 4 u9 H
k d @ ^N[ ^Kd d ^ @N@ dbk g d b
- 9
& 4 R,RN+
$
A F@"#N@2A$ J % H
A ?#L H f dbD_ak)
% ? *
H
!
?'&(
U,+ RN+ R H f dbD_ak)
% ? /R#,s'wBR0 & Q Q R/& - wR
,+ R-/RStR020Bs'+F0s $ % ? & 4 + - (+
0s,9 x#s'& w /&10BwB+ & 3 wB&)s &10 $ 3 % T R/.0,R T 4 + - (+a9
)3 06*w +,R H
A ? L H f d bD_ak1
% ? 2
#"<
z|{~}~<{Woz||<{~o{~P<1)~1P)P))<}1{~z|z}!m43|z|zly)y)

s U S
s,R 's +
( - )3 R0

- s ,R & $ RN+ w*+,R

,+2s - ,& &)wB&1R0 s $ ( -)3 R0 s $
Q s+ R ,RNwOUVs+ ( - +2& - R 0:9 4 & ('R s .0BNR +2('R
s $ s'2w +,RN+0
T R0 ,
RNw Sts w - & .0 -% & $ s'+ Q - wB&)s
r -
w2,
+ &10 & $ NR + R#.StR
s T s,R ( - +2& $
& $ R + &)w
R U&)w2+ 3
4 R,RN+ - S - 0R'9 ,+ s

,+ - Stw&1StR'9>S -

RNR/R $ s'+
( -)3 R'9.R - 0BT wBs
,s U
R Q &10 q

+ - +
0 3 SNStRNR- & Q - T S - 0R 0
XY - SWw6& $ NR +2R.SWR Q R w2+,s ,0 UVs'+

UVR
,RNwOUVs+ 0w + 3 SWw 3 +2
R 0
% $ s+ 0s Q R
s Bw R - + s Q RNw2+,s ,
0 B 0B& Q 3 1- wBR 2w +,R ,
RNwMUVs'+

+ - /s Q T Bw s S - S 3 1- Bw R - ,+ s Y & Q - wR 0Bs )3 Bw &)s.0
#"

RN(RN+ - ( - +2& - w 0 s $ w*+,&10
R - + ,& 4 w - 0
R > 's + \
q RNwOUVs'+
0BwB+ 3 Stw 3 2+ R Q & 4 + w
+ - & ,& 4 R Y - Q R 0 Q & 4 + w ,+2s (c& /R ( -)3 R0 s $ '
,RNwOUVs+ ( - +2& - R 0:9s'+ x 3 0w

$ 0BwB+ 3 Stw 3 + R ,s U, - s .0BRN+2('R -% ( - +2& - R 0
+ R & w ~0VR - 0BT - 07wB+ - &,& 4 - q - & ('R r - T'R0VS 1 - 0 0B&'.KRN+
#"

c3 s 0R 0BwB+ 3 Stw 3 +2R ,s U n9 ( - +2& -
s .
0BRN+2( -
R
R0
- + w& -% T
R'u 4 u19,s .0BRN+2('R

9 3 ,
w s'w !#"$%
&" ('*) ,+ u1u1u
-0/ -%2143 ws w
.
9( - @w ?
A 1CB w:D B:1

1 -%#- wEF
7sHC7 3 6C7 wBs
N
PO Q I

3 s

#"SR
3514- - % 6 8 7:9 3;1 % 7Nw=<Vs 3
7 41 3 %7 wE<Vs
w 1 ,7 ?(9? - %6
%N7 w=<Vs 3 I
< - w2+
+ -(
>7#
B s - w - s 1

6 351 - 7 w 1 ? B 7# :w G
1 J-0/ 0- L M7 ?
*w + 1 w
s B:1% F / K
3

7 >7# 7 %7 7 3 F - 2+%7 B - - 1

3 1 -%0- =F 1 7 A 3 H 143@- 1 7 - 2+%7 %7=< 3
N
! "# Q N $&%#')(+*
2+%7 0- ?, - . A!H 1 %9 7M?
7/ 6/ D - A0 1 $ !23 %4' D2+%7# -56 / - 6+ 7
7 * %
89 - 8 - % % - :
7 3 A; 3@/ 6 3;1 - 7< 1 ? B 7 F 3 77 1 7- F
= / 9( 1 7 1 > 9? - %6 3;14- - 86 1 1 O
NJI
" 9 -56 QLK
? A@ B DFCEHG

M /N2 +%7D 3 7O 3/ 10-0L 7 2 +87 P 1 ?? 9 3 7
RQ >S =
UT V > V =
W#"YX
Z\[^]^_a`cbd[HeZ\f\gc[êPhjikb>_[^lk_nmcioikpNq#rYs^touwvyx{z|xnrY}nv~uvYkY<#f_]o[^Z\Z#]b43Pf\Z\ZjWYY

=/
M/
1 6
1 B 9
1 ?? 9 /
1 B 9

<,+87 3 7
1 B 9
B:

3@- 2+ / B:1

1 7 3
1
- %6 I
1 7 87:<
O < -
1 7- 3

A O
1 ?>
7 9 ? 7/ 7 7 1 -7 F
- 0- - 7,? !A 9(O

? 7 3 H 7 H 41 3@- 1
M7 ?:D
> , / 1J-0/ 0- L 7 N

O Q I
B 9 >7M?
!2+
? 7 3 CH 7 1
1
-0- - 7 ? A 9(
? 7 3 CH 7- H 143 -21
7M?
+%7# ?, 3 9 B 9 3 7 9( OK< o/ /k/

1 3;B + , 1 (4? 9
?> ;3 1CB
6 3@- 2+ / ? 9? 7 6 3 7:7>F ? 7 4
7>6C7M? 1 O >7M?
-2B
B - HC7 3 7M? 7 143;B + ,
W
Z\[^]^_a`cbd[HeZ\f\gc[êPhjikb>_[^lk_nmcioikpNq#rYs^touwvyx{z|xnrY}nv~uvYkY<#f_]o[^Z\Z#]b43Pf\Z\ZjWYY

/
K<

/
- 7 3@- 3 K< 7>6 7 < -
? 7 3 HC7 1 1
1 B A 3- 3 K< 7>6C7 <%7 B 33 7 B G - ?,
7 3 %7 ? 1/ 7 B / 7 J- =F
B - HC7 3 7M? 7 143;B 143 7 1
J 7 A 3 /
! 7 1 , 3 7 1 ! H 1 9%7 H 143@- 1
7,?
" 143;14/ 7 7 3-0L 7 - ?, 3-
%9 - ? - ?, 7 1 A 1
7M?
J 7 , # 3 >? 3 >7 3- ?> 7 1 A$ 3 %! ? - - 1
? F>? 7 / ?
& 3 7 7(' 7 B - HC7 - A 7 3 7 B 7 / 7 O )%?
/o/k/
WY
Z\[^]^_a`cbd[HeZ\f\gc[êPhjikb>_[^lk_nmcioikpNq#rYs^touwvyx{z|xnrY}nv~uvYkY<#f_]o[^Z\Z#]b+*Pf\Z\ZjWYY

%7 P 9 ? 7

1 1 - ? F 1 3 -21 F
? 7 3 H 1
7
? 9 7 3 H - ? 7 B 9?> 7 3- 6 143 6C7 H 1 9%7
9
? 7 3 H 1
7
9 7 3 H - ? 7 7 13 %- 6 $?, / 7 - ?, 1 B 7 1 , 3@-
%9 7M?
9
? 7 3 H 1
7
/ 7 9? 7M?
3514- (1 FC7,? - 1 7 0- 7 A 7E< 3 ?
? 9 7 3 H - ? 7 B 9?> 7 3- 6
7 13 %- 6 - >7 & 143 KH & )>7 ?
W

p(x)
A

B

= / O

Y
M /

WY

;
! ,

Y

7 9 )9 :

& ) 7 9 9 :

" 7 9 9 :
=


& Y

!

7 9
" Y

7

9 : /

Q

'

'

Q
& O !
7 9 :

Y
7 9 :

! /!

7 9 :/

#
Q
"

?
Q "

WcW

,

W

|
;

!

Z\[^]^_a`cbd[HeZ\f\gc[êPhjikb>_[^lk_nmcioikpNq#rYs^touwvyx{z|xnrY}nv~uvYkY#f_]o[^Z\Z#]b+*Pf\Z\ZjWYY

A

9 9 "

"
"

"

! #"
& $

&
')(*
%
& &

&

+-,/./021435,768+-9-:4,/6<;>=?3@08,/A?0CB4=D=?EGFHI/JDKMLONQPNCHRCLSKTLU?VWYX9Z0[.D,/+-+\V.)]^3+*`_ba<9-+-+>Vc'`dd(

#
%

! "
&

(

')(10
! #"%$'&

%

% % % &
! "

*)+ -,* ( ! #"%$'&

&

.%/

%
+-,/./021435,768+-9-:4,/6<;>=?3@08,/A?0CB4=D=?EGFHI/JDKMLONQPNCHRCLSKTLU?VWYX9Z0[.D,/+-+\V.)]^3+*`_ba<9-+-+>Vc'`dd(

ch6 Bayesian Learning.

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

ch6 Bayesian Learning.

Încărcat de

Drepturi de autor:

Formate disponibile



    "!

P X1 J b&57<* SB1Z)3/+85732 U U V1XW(5\W('  U ')J)1K5[+8(L *:9

S(1K5[J)1 S(1KJ)f?Vf(5 U 5[+

   

 2WB21Z U[U F hVW0+]+-( L J *8+]S(1KJ)f?af U  0F&STJ)+8B*,5e* )' 5[b)2W

6 a1K' L  . !  ' "

J&G* ?S a+85\2W0+  Gb) 32aW 3621 J)1IW(J+

    

  ,; 9 S(1KJ)f?afB5 U 5[+

31        ,    

    

= )J 1I3  0 F&STJ)+-(*857* 5[W

>(3Y U 63 % U a+8 +8(

!  " ! "

@R ) %B+8S(%(+I+8( RF&STJ)+8(*857*  "! h \5 +8 8+ ( (5\')(*8+

    

J)W?*857<21 J)%(1 %?*8%? U 36JW?362S(+ U a1KW(5[W(' +K)*Kk

 ?V+ hjJ)% U  E GF)*]1K% U  SB1XJR<% 36 )*]+8( M O P

    

O *K*,%(L  ^(.0 *,Y+ J g 5[W?*8+KaW 36* 

    

O *K*,%(L  ^(.0 *,Y+ J g 5[W?*8+KaW 36* 

  

  

     

Equivalent Bayesian inference system

 J)W?*857<21 aW0F 1XG U b  U %( +Ka1K')2+Ng %(W?3/+85[J)W

!  '  "

". '' ( &  U W

!  "

". '' ( &

5[W ' ( &  !  '

! ' "  

! ' " "

! ' " "

  

 J)W?*857<21 S(1XG<5736+85\W(' *8%(1Kb&5[b  U S(1KJ)f?af(5 U 5[+iF g 1XJ)L

& W 8+ B57* Y3 )*, 32VW 8* (J h

! ' " "

   

) 3232aL *$*]14al2J)19 S(1X:g 21 +-( *,(J1X+8G*,+I0F&STJ+8(*857*

Q 9<SB1X/g 21 8+ B R F&STJ)+8(G*,57* +8?a+IL 5[WB5[L 5[l2G*

<3657*85[J)W +81K2*2>  +814a5[WB5[W(' (V+K

ST21-g 3/+ U F f0F  ` 2G JW U F <*K361K5[fT /. 3/2S(+85[JW?*

   

"! a1K' L  . !  " ! "

M J ,* + P 1XJf?af U   U )*K*85_^T32a+-5[J W VJ g ` 2h & W?*8+KaW 36*

   "!

!. (  ' " ! '   "

!   "

> >0m3 >?N 3

1$@> !  O3"$# 3 8

$'3"&%2 8(' #)% 8

' #(=>?$ O  N&

BDCFE EHG EFI WKJLJLMONQP

BRCFE EHG ES3TVUXW MZY[]\ W_^ TV` N

 > 9a9 3 8(' %N  <> 9 >

O   89 #3% 8 >?N

%N  <> 9 >

3lk 1 3 8 3O

>?1 8 >

Ng# 9F' >?N

9  ' N % 9]8  N  4

"!

P X1 J b&57<* SB1Z)3/+85732 U U V1XW(5\W(' U ')J)1K5[+8(L *:9

S(1K5[J)1 S(1KJ)f?Vf(5 U 5[+

2WB21Z U[U F hVW0+]+-( L J 8+]S(1KJ)f?af U 0F&STJ)+8B,5e* )' 5[b)2W

6 a1K'L . ! ' "

J&G* ?S a+85\2W0+ Gb) 32aW 3621 J)1IW(J+

,; 9 S(1KJ)f?afB5 U 5[+

31 ,

= )J 1I3 0 F&STJ)+-(857 5[W

>(3Y U 63 % U a+8 +8(

! " ! "

@R ) %B+8S(%(+I+8( RF&STJ)+8(857 "! h \5 +8 8+ ( (5\')(*8+

J)W?857<21 J)%(1 %?8%? U 36JW?362S(+ U a1KW(5[W(' +K)*Kk

?V+ hjJ)% U E GF)]1K% U SB1XJR<% 36 )]+8( M O P

O K,%(L ^(.0 ,Y+ J g 5[W?8+KaW 36*

O K,%(L ^(.0 ,Y+ J g 5[W?8+KaW 36*

J)W?*857<21 aW0F 1XG U b U %( +Ka1K')2+Ng %(W?3/+85[J)W

! ' "

". '' ( & U W

! "

". '' ( &

5[W ' ( & ! '

! ' "

! ' " "

J)W?857<21 S(1XG<5736+85\W(' 8%(1Kb&5[b U S(1KJ)f?af(5 U 5[+iF g 1XJ)L

& W 8+ B57* Y3 ), 32VW 8 (J h

) 3232aL $]14al2J)19 S(1X:g 21 +-( ,(J1X+8G,+I0F&STJ+8(857

Q 9<SB1X/g 21 8+ B R F&STJ)+8(G,57 +8?a+IL 5[WB5[L 5[l2G*

<365785[J)W +81K22> +814a5[WB5[W(' (V+K

ST21-g 3/+ U F f0F ` 2G JW U F <K361K5[fT /. 3/2S(+85[JW?

"! a1K'L . ! " ! "

M J ,* + P 1XJf?af U U )K85_^T32a+-5[J W VJ g ` 2h & W?8+KaW 36

"!

!. ( ' " ! ' "

! "

> >0m3 >?N 3

1$@>! O3"$# 3 8

$'3"&%2 8(' #)% 8

' #(=>?$ O N&

BDCFE EHGEFI WKJLJLMONQP

BRCFEEHGES3TVUXW MZY[]\ W_^ TV` N

> 9a9 3 8(' %N <> 9 >

O 89 #3% 8 >?N

%N <> 9 >

3lk 1 3 8 3O

>?1 8 >

9 ' N % 9]8 N 4

>0 39<8 8 8:9 N N ;

> 9 e ' N 8:9

:9 > O)l,

>?N O >?N 8

N. 8 N*

8 8 >6N

5 %. H<Z% 8 8 >?N.

N. 8 N*

<Z%N. >?N

7 8:9;< = 8>?<@ ABC8:@D@9EF<GIH #J

8 9a #3%

8 9a #3%

! <? $&%

' $)( ' +* $,-% ' # E G (

- .:! ' 1 02043

" >

q

! "

q

___z3MH4HHHM_M#Amz__ZHH

___z3MH4HHHM_M#Amz__ZHH

G I < "!# G %$ Q'& <# 7 & <(!

<(# 10 & 7 $

7 (8 9 * 8 /$ :<;(=?> "@ $%$ BA DC F E G /G!& "*

___z3MH4HHHM_M#Amz__ZHH

"!$#"%&(')+-, '")'*/.#'0&1#3241)-5$&6)7'8"+-,+"1 '&

___z3MH4HHHM_M#Amz__ZHH

3

___z3MH4HHHM_M#z__ZHH

___z3MH4HHHM_M#z__ZHH

B

XZY - Q @)R :\[ ,

z|{~}~<{Woz||<{~o{~P<1)~1P)P))<}1{~z|z}!m|z|zly)y)