356

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 2, MARCH 2000

The Equivalence Between Fuzzy Logic Systems and Feedforward Neural Networks
Hong-Xing Li and C. L. Philip Chen
AbstractThis paper demonstrates that fuzzy logic systems and feedforward neural networks are equivalent in essence. First, we introduce the concept of interpolation representations of fuzzy logic systems and several important conclusions. We then define mathematical model for rectangular wave neural networks and nonlinear neural networks. With this definition, we prove that nonlinear neural networks can be represented by rectangular wave neural networks. Based on this result, we prove the equivalence between fuzzy logic systems and feedforward neural networks. This result provides us a very useful guideline when we perform theoretical research and applications on fuzzy logic systems, neural networks, or neuro-fuzzy systems. Index TermsEquivalence on fuzzy logic systems, fuzzy logic systems, feedforward neural networks, interpolation representation, rectangular wave neural networks.

I. INTRODUCTION ANY researches focused on combining neural networks and fuzzy logic systems, such as neuro-fuzzy systems, or fuzzy neural networks [1][3], [10][16]. They have shown valuable results undoubtedly. However, this paper takes a different approach to demonstrate the equivalent relationship between fuzzy logic systems and neural networks. Based on this result, we wish to provide more significant theoretical result on combining both systems. To provide this, we first introduce representations of fuzzy logic systems and then we elaborate the idea of interpolation representation of fuzzy logic systems and provide new results under some weaker conditions. We first prove rectangular wave activation function neural networks can represent nonlinear neural networks. Based on this result, we further prove that fuzzy logic systems and neural networks are equivalent essentially under some restriction. First of all, we introduce interpolation representation of fuzzy logic systems. We prove that the antecedents of inference of a fuzzy logic system are the base functions of interpolation and the consequents of inference only relate to their peak values but not to the shape of the membership functions. Second, we define mathematical model of rectangular wave neural networks, where their activation functions are rectanManuscript received February 8, 1999; revised July 29, 1999 and November 22, 1999. This work was supported by the National Natural Science Foundation of China, U.S. AFOSR Grant F49620-94-0277, and NSF-EIA-9601670. H.-X. Li is with the Department of Mathematics, Beijing Normal University, Beijing 100875, China (e-mail: lhxqx@bnu.edu.cn). He is currently on leave with Wright State University, Dayton, OH 45435 USA (e-mail: hxli@cs.wright.edu). C. L. P. Chen is with the Department of Computer Science and Engineering, Wright State University, Dayton, OH 45435 USA (e-mail: pchen@cs.wright.edu). Publisher Item Identifier S 1045-9227(00)01743-4.

gular waveforms. Then we prove that a nonlinear neural network can be represented by a rectangular wave neural network. By means of this result, we prove the equivalence between fuzzy logic systems and feedforward neural networks. This conclusion provides an important theoretical tool or basis for fuzzy logic systems or neural networks, especially when combining both of them. Similar work can be found from [12] and [16]. Jang [12] treated the system using radial basis function and Sugeno model. The equivalence is found by finding the mapping between Sugeno model and radial basis function without some rigorous proof. Buckley et al. [16] focused on the equivalence for fuzzy expert system. In this paper, we study the interpolation and equivalence under weak condition, i.e., Kronecker's property. II. INTERPOLATION REPRESENTATIONS SYSTEMS
OF

FUZZY LOGIC

This section reviews and elaborates the interpolation mechanism or representations of fuzzy logic systems. The interpolation mechanism or representation of fuzzy logic systems is discussed in detail in [1]. Here, we review, briefly, some main issues under weaker restrictions as follows. A. Some Necessary Concepts and Notations Let us quickly recall fuzzy logic systems, taking a two inand be puts and one output system as an example. Let the universes of input variables and , respectively, and be the universe of output variable . Denote and , where and , and and is the family of all fuzzy sets on and , respecand as linguistic variables, so that a tively. We regard group of fuzzy inference rules is formed as follows: If where is and is then is (1)

, and and are called base variables. According to the Mamdanian algorithm, the inference relath inference rule is a fuzzy relation from tion of the to , where

10459227/00$10.00 2000 IEEE

LI AND CHEN: THE EQUIVALENCE BETWEEN FUZZY LOGIC SYSTEMS AND FEEDFORWARD NEURAL NETWORKS

357

Since inference rules should be joined by or (corresponding to set-theoretical operator "), the whole inference , i.e., relation is (2) and , the conclusion of inferGiven can be determined as ence by means of composition of relations inference (CRI) method, where is the composition of fuzzy relation, and where (3) To change crisp quantity inputs to fuzzy sets, for example, and as crisp inputs, we use fuzzification given as follows:

Moreover, we call a base element or base function of . Then we also call a group of base elements of . In this paper, ( is the set of real numbers), we always when . assume that B. Interpolation Representation of the Mamdanian Algorithm with One Input and One Output and be the universe of input and output variable, Let and be a respectively, and family of fuzzy sets on and , respectively. Usually we can suppose that and are real number intervals, i.e., and , holding and , where and are the peak and respectively. When and are general points of measurable sets, the following result is also true. In addition, and as integrable functions. we always regard Lemma 1: Under the above condition, there exits a group of such that the Mamdanian base functions, algorithm with one input and one output is approximately an for its base unary piecewise interpolation function taking functions (7) holds Kronecker's property. When the fuzzy logic where is just a fuzzy partition of . Moreover system is normal, is an equidistant partition, degenerates when into , i.e., (8) Proof: According to Mamdanian algorithm, the inference rules If is then is and form (9) so the

By the above expressions and using (2), we can get the inference result

(4) In order for the fuzzy set, , to be used for a practical operating quantity, the fuzzy set should be turned into a crisp number. The commonly used defuzzification method is called the centroid method (5) This is just the basic description of a fuzzy logic system. In fact, it is a fuzzy logic controller. In some occasion, a fuzzy logic system and a fuzzy controller are used alternatively. For the sake of convenience, we introduce several concepts. be a family of normal Given a universe , let where is fuzzy sets on , i.e., called the peak point of . Usually has Kronecker's property

and the inference relation of the th rule is , i.e., whole inference relation is

(10) For a given input response value , being similar to (4), we have . Also similar to (5), we can obtain a crisp

So in this paper, we always assume that this property holds. In many cases, is a fuzzy partition of , which means that it satisfies the condition Let and (6) Clearly, (6) implies that has Kronecker's property. When the input linguistic variables of a fuzzy logic system are fuzzy partition, the fuzzy logic system is called a normal fuzzy logic system. necker's property, of definite integral, we have . Because

(11) and and have Kro, based on definition

358

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 2, MARCH 2000

Denote . Then

and

Lemma 2: Under the above conditions, there exists a group of base functions such that the Mamdanian algorithm with two inputs and one output is approxifor mately a binary piecewise interpolation function taking its base functions (12) (13) Proof: Let . For given inputs and , we have

which means that is approximately represented as an unary piecewise interpolation function taking for its base functions. and let If we write , then we get (7). holds Kronecker's property. In fact, Now we prove that , clearly . When , we have when

Let In addition, when the fuzzy logic system is normal, we can is just a fuzzy partition of . Actually, prove that and

We have

(14) Thus Kronecker's condition holds for . is an equidisAt last, if the system is normal and . Then we have tant partition, then If we write (13) D. Interpolation Representations of , then we get -Centroid Algorithm

, i.e., . So which is just (8). Note 1. From we know that is . For is only related to a kind of weighted form of and is identical to . Besides, the antecedents of the inferences, , are just the base functions of interpolation and the peak points, , (of the consequents of inferences, ) appear in the interpolation equation. Under the Kronecker's property, the shape of membership functions of the consequents has nothing to do with the interpolation. This provides us with a very important guideline for designing a fuzzy controller. C. Interpolation Representation of the Mamdanian Algorithm with Two Inputs and One Output and Reviewing the notations in Section II-A, let and real number intervals: is the peak point of holding where . And suppose that and are integrable functions. be ,

Therefore

-centroid algorithm proposed in [5] is thought by The -centroid alsome people to be more convenient than the gorithm (i.e., the Mamdanian algorithm). -cenLemma 3: Under the condition in Lemma 1, the troid algorithm with one input and one output has the same conclusion as in Lemma 1. is replaced by we have Proof: In (10), after (15) . From For a given input , where (11), it is easy to obtain that and , which is just (12). The other results in the lemma are clear. The lemma means that result of the -centroid algorithm is the same as the result of the Mamdanian algorithm in the case with one input and one output. Lemma 4: Under the conditions in Lemma 2, there exists a such that group of base functions -centroid algorithm with two inputs and one output is the

LI AND CHEN: THE EQUIVALENCE BETWEEN FUZZY LOGIC SYSTEMS AND FEEDFORWARD NEURAL NETWORKS

359

approximately a binary piecewise interpolation function taking for its base functions (16) Furthermore, when the system is normal is an equidistant partition, we have and

output is approximately a binary piecewise interpolation function taking for its base functions (22) where and (23)

(17) Proof: In (2), after is replaced by

The proof is omitted for it is similar to the proof of lemma 4. F. Interpolation Representation of the Simple Inference Algorithm The simple inference algorithm is proposed in [7] and [8] which is thought by some people to be a kind of quick and simple algorithm with respect to fuzzy inference. In fact, it has been most widely used in neuro-fuzzy systems involving learning mechanisms. In the algorithm, the fuzzy sets representing inference consequents are replaced by numbers. For example, inference rules with two inputs and one output are as follows: If is and is then is (24) is calcu-

(18) For given inputs and , we get (19) Similar to the proof of Lemma 2, we have

Then for a given input lated in the following steps: Step 1)

, the response value

or Step 2) (20) (25) where and and we get (16). When the system is normal and equidistant partition, it is easy to see that (17) is also true. E. Interpolation Representations of Weighted Algorithm . Let is an . So Lemma 6: The simple inference algorithm with two inputs and one output is exactly (not approximately) a binary piecewise interpolation. When (26) -Centroid where

-centroid algorithm is proposed in [6], The weighted which means the inference rules are joined by weighed or. Taking (18) for an example, we should have When and the system is normal

(27)

(28) (21) , and usually . where Lemma 5: Under the conditions in Lemma 2, there exists a such that group of base functions -centroid algorithm with two inputs and one the weighted The proof is omitted for it is not difficult. G. Interpolation Representation of Function Inference Algorithm The function inference algorithm is proposed in [9] which is the generalization of the simple inference algorithm. We con-

360

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 2, MARCH 2000

sider the case with two inputs and one output for an example, where the inference rules are as follows: If Here is and is then is (29)

bounds, the factor only plays a part of normalization in the whole (34) but is not essential. So we can delete them from (34). Thus (33) becomes the following: (35) Note 3. The fuzzy inference rules should be If is and is then is

are real functions and usually they are linear: , where coefficients and should be determined by some methods. , where For a given input is a general operator, for example (or ). Then similar to (25), we have (30) Lemma 7: There exists a group of base functions such that the function inference algorithm is exactly (not approximately) a binary piecewise interpolation function with variable nodal points (the term of numerical analysis; i.e., the nodal points, , are no longer constant but functions of variables and ) as follows: (31) where . When and the system is normal

Write . Clearly, . Thus, the fuzzy inference rules can also be represented as follows: If is then is (36)

, we can form Generally speaking, for any a group of inference rules as given in (36). The interpolation representation of the fuzzy logic system based on the group of inference rules can be given by the following expression: (37) When , (37) turns into the following: (38)

(32) Note 4. We reconsider the fuzzy inference rules The proof is omitted since it is straightforward. H. A General Inference Algorithm of Fuzzy Logic Systems From above results we can give a general algorithm of fuzzy logic system by means of a -norm (33) where (40) (or ). Thus, Clearly the interpolation representation of the fuzzy logic system can be given by If is and is then is

They can be rewritten as follows: If is then is (39)

is actually a fuzzy relation It is easy to understand that can be denoted by , where between and . . Usually we take

(34) and means a -norm. Note 2. Actually, the factors is not very important, because it is and ; in fact, in the factor in essence formed by means of

(41) Conversely, for any group of fuzzy relations , we can form a group of fuzzy inference rules If is then is (42)

is the main part which is only a kind of weighted average of the set . From the viewpoint of numerical

LI AND CHEN: THE EQUIVALENCE BETWEEN FUZZY LOGIC SYSTEMS AND FEEDFORWARD NEURAL NETWORKS

361

which represents a fuzzy logic system. Naturally, the interpolation representation of the system can be given by the following expression:

This provides us a general guideline to construct models or algorithms for fuzzy logic systems.
Fig. 1. MP model of artificial neurons.

III. RECTANGULAR WAVE NEURAL NETWORKS NONLINEAR NEURAL NETWORKS

AND

As two of basic tools for proving the equivalence of fuzzy logic systems and feedforward neural networks, we define and discuss rectangular wave neural networks and nonlinear neural networks in this section. First, let us recall the basic structure of a neuron. At present, the models of neurons most in use are the units with multiinput and single output, for example, MP model (i.e., McCullochPitts model, see Fig. 1). It is well known that the relation between the inputs and outputs of the neuron is expressed as follows: (43) , then , If we take which means that the threshold value has been absorbed into . We reuse the notation instead of and (43) is changed as the following: (44) where in is regarded as a -ary function

Definition 2: A neuron is called a rectangular wave neuron if its activation function is just a characteristic function of a certain real number interval (including infinite intervals; see Fig. 2). A neural network is called a rectangular wave neural network if every neuron of the neural network is a rectangular wave neuron. Usually for the sake of convenience, we do not distinguish between a neuron and its activation function. So in Fig. 1, we use the notation of the activation function of a neuron to stand for the neuron. It is worthwhile to mention that rectangular wave (pulse signal) neural networks can be easily implemented by hardware. Lemma 8: A nonlinear neural network can be, approximately, represented by a rectangular wave neural network. Proof: For a given nonlinear neural network, without loss of generality, we take the neural network as in Fig. 3, where and are the activation functions of these neuron with nonlinear activation function. Also without loss of generality, we can suppose that these activation functions are all nonlinear and continuous functions. According to the relation between the input and output of the neural network, we have (45) have the universe (domain) ( is the Let , we create a partition of , set of real numbers). For any , where , and denoted by as form a group of zero-order base functions, shown in Fig. 4 otherwise such that This means that where . So (46) .

Let . Clearly, is just the mathematical representation of the artificial neuron. Pay attention to the simple fact that is the composition of and , where plays a role in the synthesizing of -dimensional Euclidean space and plays a is just a kind of multifactorole in activating signals. In fact, rial function (see [10]); so usually we call a space synthesizer or a synthetic function of space. And is called a signal activator or an activation function. Now we should understand that a neuron can be interpreted as a function of several variables (including unary functions) and from mathematical viewpoint a neural network is regarded as a certain combinatorial form of some functions of several variables. Definition 1: A neuron is called a nonlinear neuron if its activation function is a nonlinear function. A neural network is called a nonlinear neural network, if there at least exists one nonlinear neuron. The following definition shows an important concept in this paper.

(47) and for any , in a similar For any way, we can obtain zero-order interpolation functions (48)

362

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 2, MARCH 2000

Fig. 2. The activation function of a rectangular wave neuron.

such that is universe of is as in Fig. 5. Let

, i.e., , where the and the base function

Fig. 3. A nonlinear neural network.

(49) which is just the relation between the input and output of the simple neural network shown in Fig. 6. We have shown . We now prove that can approximate that within an arbitrary accuracy. In fact, we consider the following expression:

Fig. 4. Rectangle wave activation functions as the basic functions.

Fig. 5. Zero-order activation function g

(v ).

, since is a continuous function, , when that, . Because of Let . Thus arbitrary, we can make

such . being So arbitrary, we can take . This means . Also because of . Hence being

LI AND CHEN: THE EQUIVALENCE BETWEEN FUZZY LOGIC SYSTEMS AND FEEDFORWARD NEURAL NETWORKS

363

Fig. 6. A rectangular wave neural network, where I (x) =

111

= I (x) = I (x) = I (x) = id(x).

, i.e., can approximate within an arbitrary accuracy. IV. THE EQUIVALENCE BETWEEN THE TWO SYSTEMS Theorem: A fuzzy logic system is approximately equivalent to a feedforward neural network. Proof: Necessity: Arbitrarily given a fuzzy logic system, for the convenience of the proof, we consider the case with two-input and one-output. Without loss of generality, based on the conclusions in Section II, the system can be regarded as an interpolation function So . Clearly, the neural network is a nonlinear neural network. According to Lemma 8, it can be changed into a rectangular wave neural network. This finishes the proof of the necessity. Sufficiency: Arbitrarily given a feedforward neural networks, without loss of generality, we consider the case with two inputs and one output and three layers as an example shown in Fig. 8. We consider the output of the network (50) From (50) we should get a fuzzy logic system such that is approximately the output of the fuzzy logic system. Using

Now we create a feedforward neural network shown in Fig. 7, and are the neurons (regarded as functions of sevwhere eral variables), holding , i.e., considered as hyperbolic functions, and is given as ; that is

Thus, the output of the network is as follows:

364

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 2, MARCH 2000

Fig. 7. The three-layer feedforward neural network with two inputs and one output.

where otherwise Thus

Fig. 8.

An arbitrarily given neural network.

(50), because can be regarded an unary function (such that there exist at most finite discontinuous points) and for any , there exists a group of base functions such that

(51)

LI AND CHEN: THE EQUIVALENCE BETWEEN FUZZY LOGIC SYSTEMS AND FEEDFORWARD NEURAL NETWORKS

365

where set

is the characteristic function of the , (clearly, , for any ), and

(52) If we form a group of fuzzy inference rules is then is (53) If for a fuzzy logic system, then (51) is just the interpolation representation of the system according to the statement of Section II-H. This finishes the proof of the sufficiency. V. CONCLUSION REMARKS We briefly summarize our main conclusions in this paper as follows. 1) Fuzzy logic systems are equivalent to feedforward neural networks, which means that, an arbitrarily given fuzzy logic system can be approximately represented by a feedforward neural network. Conversely, an arbitrarily given feedforward neural network can be approximately represented by a fuzzy logic system. It is worth noting that here approximately used by us means the approximation can reach an arbitrarily given accuracy. 2) We define mathematical model of rectangular wave neural networks and nonlinear neural networks. Then we prove that nonlinear neural networks can be represented by rectangular wave neural networks, which can be implemented easily by hardware. 3) From Lemma 1 to Lemma 7, we discover a very important and interesting conclusion: under the condition of Kronecker's property (see Section II-A), the membership functions defined on output variable universes do not take effect in fuzzy logic systems but only their peak values do. So, in many real applications, as long as the Kronecker's property is satisfied, we do not need to care for the shapes of these membership functions but only need to consider whether their peak values are suitable. REFERENCES
[1] H.-X. Li, The mathematical essence of fuzzy controls and fine fuzzy controllers, in Advances in Machine Intelligence and Soft-Computing, P. P. Wang, Ed. Durham, NC: Bookwright, 1997, vol. IV, pp. 5574. [2] W. A. Farag, V. H. Quintana, and G. Lambert-Torres, A genetic-based neuro-fuzzy approach for modeling and control of dynamical systems, IEEE Trans. Neural Networks, vol. 9, pp. 756767, 1998. [3] C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems: A Neural-Fuzzy Synergism to Intelligent Systems. Englewood Cliffs, NJ: Prentice-Hall, 1996. [4] M. Mizumoto, The improvement of fuzzy control algorithm, part 4: (+; )-centroid algorithm, in Proc. Fuzzy Syst. Theory, vol. 9, 1990, p. 9. (in Japanese). [5] P. Z. Wang and H.-X. Li, Fuzzy Information Processing and Fuzzy Computers. New York: Science, 1997. [6] T. Terano, K. Asai, and M. Sugeno, Fuzzy Systems Theory and Its Applications. New York: Academic, 1992.

[7] M. Sugeno, Fuzzy Control (in Japanese). Tokyo, Japan: Japan Ind. Press, 1988. [8] T. Takagi and M. Sugeno, Fuzzy identification of systems and its applications to modeling and control, IEEE Trans. Syst., Man, Cybern., vol. SMC-15, pp. 1116, 1985. [9] H.-X. Li, Multifactorial functions in fuzzy sets theory, Fuzzy Sets Syst., vol. 35, pp. 6984, 1990. [10] B. Kosko, Neural Networks and Fuzzy Systems. Englewood Cliffs, NJ: Prentice-Hall, 1992. [11] M. Brown and C. Harris, Neurofuzzy Adaptive Modeling and Control. Englewood Cliffs, NJ: Prentice-Hall, 1994. [12] J.-S. R. Jang and C.-T. Sun, Functional equivalence between radial basis function networks and fuzzy inference systems, IEEE Trans. Neural Networks, vol. 4, pp. 156159, 1993. [13] D. Nauck, F. Klawonn, and R. Kruse, Neuro-Fuzzy Systems. New York: Wiley, 1997. [14] L. H. Tsoukalas and R. E. Uhrig, Fuzzy and Neural Approaches in Engineering. New York: Wiley, 1997. [15] J.-S. R. Jang, C.-T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing. Englewood Cliffs, NJ: Prentice-Hall, 1997. [16] J. J. Buckley, Y. Hayashi, and E. Czogala, On the equivalence of neural nets and fuzzy expert systems, Fuzzy Sets Syst., vol. 53, pp. 129134, 1993.

Hong-Xing Li received the B.S. degree from Nankai University, China, in 1978 and the Ph.D. degree in mathematics from Beijing Normal University, China, in 1993. He has been a Professor at Beijing Normal University since 1994. He was a visiting Professor at Department of Computer Science and Engineering, Wright State University, Dayton, OH, during 1998 to 1999. His research interests are fuzzy control theory, adaptive fuzzy control, neural networks, and mathematical theory of knowledge representation. Dr. Li is on the editorial board of Journal of Fuzzy Systems and Mathematics, and Journal of Systems Engineering.

C. L. Philip Chen (S88M88SM94) received the B.S.E.E. degree from NTIT, Taiwan, in 1979, the M.S. degree from the University of Michigan, Ann Arbor, in 1985, and the Ph.D. degree from Purdue University, West Lafayette, IN, in Dec. 1988. In 1988 to 1989, he was a Visiting Assistant Professor at the School of Engineering and Technology, Purdue University, Indianapolis, IN. Since September 1989, he has been at the Computer Science and Engineering Department, Wright State University, Dayton, OH, where he is currently an Associate Professor. He is also a Visiting Research Scientist, at the Materials Directorate, Wright Laboratory, Wright-Patterson Air Force Base. He has been a Senior Research Fellow sponsored by National Research Council, National Academy of Sciences. He also worked as a Research Faculty Fellow at NASA Glenn Research Center during summer of 1998 and 1999. He was on sabbatical leave to Purdue University and Case Western Reserve University for Fall 1996 to Fall 1997. His current research interests and projects include neural networks, fuzzy-neural systems, neuro-fuzzy systems, intelligent systems, robotics, and CAD/CAM. Dr. Chen was a Conference Cochairman of the International Conference on Artificial Neural Networks in Engineering in 1995 and 1996, a Tutorial Chairman for the International Conference on Neural Networks in 1994, the Conference Cochairman of the Adaptive Distributed Parallel Computing in 1996, a Program Committee of OAI Neural Networks Symposium in 1995, the IEEE International Conference on Robotics and Automation in 1996, and the Intational Conference on Intelligent Robotics and Systems (IROS) in 1998 and 1999. He is a member of Eta Kappa Nu. He is the founding faculty advisor of the IEEE Computer Society Student Chapter at Wright State University and a recipient of the 1997 College Research Excellent Faculty Award.