Documente Academic
Documente Profesional
Documente Cultură
Neuro-Fuzzy Systems
Building a fuzzy system requires
z prior knowledge (fuzzy rules, fuzzy sets)
113
N
SF
EURO
UZZY
Learning Fuzzy Sets: Problems in Control
After an error was computed, any fuzzy set learning procedures can be used
114
N
SF EURO
UZZY
115
N
SF
EURO
UZZY
Example: Prognosis of the Daily Proportional Changes of the DAX at
the Frankfurter Stock Exchange (Siemens)
116
N
SF
EURO
UZZY
Fuzzy Rules in Finance
Trend Rule
IF DAX = decreasing AND US-$ = decreasing
THEN DAX prediction = decrease
WITH high certainty
Turning Point Rule
IF DAX = decreasing AND US-$ = increasing
THEN DAX prediction = increase
WITH low certainty
Delay Rule
IF DAX = stable AND US-$ = decreasing
THEN DAX prediction = decrease
WITH very high certainty
In general
IF x1 is P1 AND x2 is P2
THEN y=K
WITH weight k
117
N
SF
EURO
UZZY
Classical Probabilistic Expert Opinion Pooling Method
UZZY
E experts, R quantiles for N quantities
o each expert has to asses R·N values
stat. Accuracy:
R si
C 1 F R2 >2 N I s, p @,
I s, p ¦ si ln
i 0 p
information score:
1 Nª R 1 pr 1 º
I ¦ «lnvi, R 1 vi,o ¦ pr 1 ln »
N i 1¬ r 1 vi ,r vi ,r 1 ¼
ce I e idD ce
weight for expert e: we
E
¦eE 1 ce I e id e ce
e
outputt= ¦ we output t
e 1
T
roi = ¦ y t sign output tDM
t 1
119
N
SF
EURO
UZZY
Formal Analysis
Sources of information
R1 rule set given by expert 1
R2 rule set given by expert 2
D data set (time series)
Operator schema
fuse (R1, R2)fuse two rule sets
induce(D) induce a rule set from D
revise(R, D) revise a rule set R by D
120
N
SF
EURO
UZZY
Formal Analysis
Strategies:
z fuse(fuse (R1, R2), induce(D))
z revise(fuse(R1, R2), D) m
z fuse(revise(R1, D), revise(R2, D))
121
N
SF
EURO
UZZY
From Rules to Neural Networks
1. Evaluation of membership degrees
122
N
SF
EURO
UZZY
Neuro-Fuzzy Architecture
123
N
SF
EURO
UZZY
The Semantics-Preserving Learning Algorithm
124
N
SF
EURO
UZZY
Return-on-Investment Curves of the Different Models
125
N
SF
EURO
UZZY
Neuro-Fuzzy Systems in Data Analysis
Neuro-Fuzzy System:
zSystem of linguistic rules (fuzzy rules).
zNot rules in a logical sense, but function
approximation.
zFuzzy rule = vague prototype / sample.
Neuro-Fuzzy-System:
zAdding a learning algorithm inspired by neural
networks.
zFeature: local adaptation of parameters.
126
N
SF
EURO
UZZY
A Neuro-Fuzzy System
is a fuzzy system trained by heuristic learning techniques derived from neural
networks
can be viewed as a 3-layer neural network with fuzzy weights and special
activation functions
127
N
SF
EURO
UZZY
Learning Fuzzy Rules
Cluster-oriented approaches
=> find clusters in data, each cluster is a rule
Hyperbox-oriented approaches
=> find clusters in the form of hyperboxes
Structure-oriented approaches
=> used predefined fuzzy sets to structure the
data space, pick rules from grid cells
128
N
SF
EURO
UZZY
Hyperbox-Oriented Rule Learning
y
Search for hyperboxes
in the data space
Create fuzzy rules by
projecting the
hyperboxes
Fuzzy rules and fuzzy
sets are created at the
same time
x
Usually very fast
129
N
SF
EURO
UZZY
Hyperbox-Oriented Rule Learning
y y y y
x x x x
130
N
SF EURO
UZZY
Structure-Oriented Rule Learning
large y
by a fuzzy grid
Detect all grid cells that
contain data (approach by
Wang/Mendel 1992)
small
UZZY
Structure-Oriented Rule Learning
Simple: Rule base available after two cycles through the training data
z 1. Cycle: discover all antecedents
z 2. Cycle: determine best consequents
132
N
SF
EURO
UZZY
Learning Fuzzy Sets
Gradient descent procedures
only applicable, if differentiation is possible, e.g. for Sugeno-type fuzzy
systems.
133
N
SF
EURO
UZZY
Learning Fuzzy Sets: Constraints
Mandatory constraints:
z Fuzzy sets must stay normal and convex
z Fuzzy sets must not exchange their relative positions (they must
not „pass“ each other)
z Fuzzy sets must always overlap
Optional constraints
z Fuzzy sets must stay symmetric
z Degrees of membership must add up to 1.0
The learning algorithm must enforce these constraints.
134
N
SF
EURO
UZZY
Example: Medical Diagnosis
135
N
SF
EURO
UZZY
Example: WBC Data Set
699 cases (16 cases have missing values).
136
N
SF
EURO
UZZY
Applying NEFCLASS-J
Tool for developing Neuro-Fuzzy Classifiers
Written in JAVA
137
N
SF
EURO
UZZY
NEFCLASS: Neuro-Fuzzy Classifier
Unweighted connections
Fuzzy rules
138
N
SF
EURO
UZZY
NEFCLASS: Features
139
N
SF
EURO
UZZY
Representation of Fuzzy Rules
Example: 2 Rules
140
N
SF
EURO
UZZY
1. Training Step: Initialisation
Specify initial fuzzy partitions for all input variables
y
large
c1 c2
medium
small
x
x y
small medium large
141
N
SF EURO
UZZY
2. Training Step: Rule Base
Algorithm: Variations:
for (all patterns p) do Fuzzy rule bases can
find antecedent A, also be created by
such that A( p) is maximal; using prior
if (A L) then add A to L; knowledge, fuzzy
end; cluster analysis, fuzzy
decision trees, genetic
for (all antecedents A L) do
algorithms, ...
find best consequent C for A;
create rule base candidate R = (A,C);
Determine the performance of R;
Add R to B;
end;
Select a rule base from B;
142
N
SF EURO
UZZY
Selection of a Rule Base
Performance of a Rule :
• Order rules by
performance.
N
1
Pr
N
¦ 1
c
Rr x p , with • Either select
p 1 the best r rules or
the best r/m rules per
class.
0 if class(x p ) con( Rr ), • r is either given or is
° determined automatically
c ® such that all patterns are
°1 otherwise. covered.
¯
143
N
SF EURO
UZZY
Rule Base Induction
NEFCLASS uses a modified Wang-Mendel procedure
y
large
c1 c2
medium
R1 R2 R3
small
x
x y
small medium large
144
N
SF EURO
UZZY
Computing the Error Signal
Ej sgn(d ) 1 J (d ) , with d tj oj
c1 c2 § ad ·
2
¨¨ ¸¸
and J : o >0, 1@, J (d ) e © d max ¹
Rule Error:
x y Er W r 1 W r H Econ( R ) , with 0 H 1
r
145
N
SF
EURO
UZZY
3. Training Step: Fuzzy Sets
x a ½
°b a if x [a, b) °
Example: ° °
triangular °c x °
membership P a ,b,c : o [0,1], P a ,b,c ( x) ® if x [b, c] ¾
function. °c b °
° °
°0 otherwise °
¯ ¿
V P ( x) if E 0
Parameter f ®
updates for an ¯V 1 P ( x) otherwise
antecedent 'b f E c a sgn( x b)
fuzzy set. 'a f E b a 'b
'c f E c b 'b
146
N
SF EURO
UZZY
Training of Fuzzy Sets
y
large
initial fuzzy set
P(x)
medium
reduce enlarge
0.85
0.55
small
0.30 x
147
N
SF EURO
UZZY
Training of Fuzzy Sets
Algorithm:
Variations:
repeat
for (all patterns) do • Adaptive learning rate
accumulate parameter updates; • Online-/Batch
accumulate error; Learning
end;
modify parameters; • optimistic learning
until (no change in error); (n step look ahead)
148
N
SF
EURO
UZZY
Constraints for Training Fuzzy Sets
149
N
SF
EURO
UZZY
4. Training Step: Pruning
Goal: remove variables, rules and fuzzy sets, in order to
improve interpretability and generalisation.
150
N
SF
EURO
UZZY
Pruning
Algorithm: Pruning Methods:
151
N
SF
EURO
UZZY
WBC Learning Result: Fuzzy Rules
R1: if uniformity of cell size is small and bare nuclei is fuzzy0 then benign
R2: if uniformity of cell size is large then malignant
152
N
SF
EURO
UZZY
WBC Learning Result: Classification Performance
Predicted Class
malign benign not sum
classified
malign 228 (32.62%) 13 (1.86%) 0 (0%) 241 (34.99%)
benign 15 (2.15%) 443 (63.38%) 0 (0%) 458 (65.01%)
sum 243 (34.76%) 456 (65.24%) 0 (0%) 699 (100.00%)
153
N
SF
EURO
UZZY
WBC Learning Result: Fuzzy Sets
uniformity of cell size
sm lg
1.0
0.5
0.0
1.0 2.8 4.6 6.4 8.2 10.0
bare nuclei
1.0
0.5
0.0
1.0 2.8 4.6 6.4 8.2 10.0
154
N
SF
EURO
UZZY
NEFCLASS-J
155
N
SF
EURO
UZZY
Resources
Detlef Nauck, Frank Klawonn & Rudolf Kruse:
Beta-Version of NEFCLASS-J:
http://www.neuro-fuzzy.de/nefclass/nefclassj
156
N
SF
EURO
UZZY
Download NEFCLASS-J
157
N
SF
EURO
UZZY
Conclusions
Neuro-Fuzzy-Systems can be useful for knowledge discovery.
Not an automatic model creator, the user must work with the tool.
158
N
SF
EURO
UZZY