Proceedings of the 29th Annual Hawaii International

Conference on System Sciences -

1996

Applications of Neural Network in Manufacturing

Ramesh Rajagopalan,Ph.D. * and Purnima Rajagopalan+ * Industrial and SystemsEngineering, Mercer University, Macon, GA 3 1207 rajr@egr.mercer.peachnet.edu + M IS Manager, Galaxy Services Group, Macon, GA 3 1210 Abstract
s Neural network is a model of brains cognitive process. Neural network originated as a model of how the brain works. Neural network research has its beginnings in psychology. Today neural network methods are being used to solve numerous problems associated with manufacturing operations. A review of neural network applications to problems in production and operations management is presented. Applications reviewed in this paper include character, image andpattern recognition, managerial decision making, manufacturing cell design, tool condition monitoring, real-time robot scheduling and statistical process control. Methods and structures of neural network are explained. called neurons. Neurons get data from the surrounding neurons, perform some computations, pass the results to other neurons. Connections between the neurons have weight associated with them. In neural network, the knowledge is stored in the network interconnection s weights in an implicit manner, learning takes place within the system and plays the most important role in the construction of an neural network system. The neural network system learns by determining the interconnection weights from a set of given data [40]. Learning in neural network can be supervised, unsupervised or based on a combined unsupervised-supervised training [20]. In supervised learning, a set of data, called a training data set, is used to help the network in arriving at the appropriate weights [40]. A teacher teaches the network and gives results of the output corresponding to the input. The inputs as well as side information indicating the correct outputs are presented to the network[20]. The network is also programmed to know the procedure to be applied to adjust the weights and thus the network has the means to determine whether or not its output was correct and the means to apply the learning law to adjust its weights in response to the resulting errors[20]. Weights are generally modified on the basis of the errors between desired and actual outputs in an iterative fashion and one of the widely used training algorithms is the Delta Rule [20]. The neural network learns the desired outputs by adjusting its internal connection weights to minimize the discrepancy between the actual outputs of the system and the desired outputs [32]. In unsupervised learning a neural network operates in self-organization mode. In this self-organization mode, a competition mechanism is used to select processing elements due to which their weights are modified. The first part of the competition involves competition between the simple-elements that he within same sample. Winners from these competitions compete with other processing elements in their layer that have won similar competitions in order to be single processing element on their layer

I. Introduction to neural network

Over the past several years, various methods based on such areas as operations research, statistics, computer simulation, control theory have been developed and applied to solve a wide spectrum of problems in manufacturing. Today manufacturing environment is s characterized by complexity, inter-disciplinary manufacturing functions and ever growing demand for new tools and techniques to solve difficult problems. Neural network offers a new and intelligent alternative to investigate and analyze challenging issues related to manufacturing. In this section, an introduction to neural network and two commonly used neural network methods will be provided. Neural network is used to capture the general relationship between variables of a system that are difficult to analytically relate. Neural network has been described as brain metaphor of information processing or as a biologically inspired statistical too1[35]. It has the capability to learn or to be trained about a particular task, its computational capabilities and the ability to formulate abstractions and generalizations. Neural network has an organization similar to that of a human brain and it is a network made up of processing elements

1060-3425/96 $5.00 0 1996 IEEE

447

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 1996 IEEE

Proceedings of the 29th Annual Hawaii International which has the weight changed during training. In selforganized training, the network is given many input characters and no information is given to the network as to what each example corresponds to. There are many layers and each layer corresponds to one character more than the other layers. The importance of unsupervised learning is that the system does not need to know the correct answer in order to solve a problem. The system learns a pattern from repeated exposure to it and is able to recall the learned pattern when it solves a categorization or pattern matching problem [40]. Neural network can also employ a hybrid approach in which learning is based on combined unsupervisedsupervised learning. The hybrid approach first uses unsupervised learning to form clusters and the labels are then assigned to the clusters identified and a supervised training follows[20].

Conference on System Sciences - 1996

III. Neural network applications

Neural network offers a method for incorporating and processing qualitative knowledge and have the additional advantage of formalizing machine learning in an explicit manner, it aspires to imitate human intelligence in its totality [40]. Neural network has been applied to a wide variety of problems, ranging from traveling salesman optimization to vision problems. Neural network is best at solving classification problems and it also has the added advantage of performing successfully where other methods often fail - recognizing and matching complicated, vague, or incomplete patterns [40]. Neural network is used to convert text to speech, for natural language processing, for example, for deriving language rules, recognition of characters and handwriting, image processing and pattern recognition. This line of research has immediate use in banking, credit card processing, and other financial services where reading and recognizing handwriting on documents is crucial [40]. Due to rapid development of neural network methods and tools, neural network has generated tremendous amount of interest solving manufacturing related problems. Neural network is used to address issues relating to manufacturing process planning and process control, manufacturing system design, operational decision making, and resource scheduling. The Table 1 provides a summary of various neural network applications in the general and manufacturing categories. The remainder of this section is organized into two subsections. Section III. 1 describes general applications of neural network in pattern and character recognition. Applications of neural network to manufacturing problems are presented in section III.2.

II. Neural network methods

Two methods are commonly used in neural network applications - neocognitron and back propagation. Neocognitron is a hierarchical network made up of many layers and it organization is like that of the visual cortex. s Neocognitron is a method of pattern recognition. It has the capability to recognize shapes and sizes of characters even if it involves noise and distortion. Back propagation allows the training ofmulti-layer networks, it is a powerful and practical tool for solving problems that would be quite difficult using conventional computer science techniques and these problems range horn image processing to speech recognition to character recognition forecasting to optimization[5]. Back propagation is the best-known supervised learning method for Neural Network with three or more layers[40]. In the back-propagation procedure, the network fast uses the input data set to produce its own output, and then compares this with the desired output, if there is no difference, no learning takes place else the weights are changed to reduce the error term [lo]. One of the most important steps in the development of a neural network is the development of the training data set and the training data set must be constructed carefully to include examples of many different operational scenarios. Approximately equal numbers of examples of each In the scenario should be included in the data set. following section applications of neural network to a wide variety of manufacturing planning, design and control problems are presented.

III.1 General applications

A character recognition problem using neocognitron to translate alphabets in one language to alphabets in another language is the described by Sankamarayanan [ 3 11. The author used the method of neocognitron to translate English letters to Greek letters. The method is quite versatile and may find widespread application as a general pattern-recognition paradigm. A software system using PASCAL was developed [31]. The software system implements a neocognitron based self-learning neural network. The input is a character with low-resolution. Neocognitron is not affected by scale change, shift in position, noise and distortion. It shares the weight during training. This weight sharing method is used so that whenever the weight of an element in a layer is modified,

448

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 1996 IEEE

Proceedings of the 29th Annual Hawaii International

Conference on System Sciences -

1996

Table 1. Summary of neural network applications.

through a visual system and recognized correctly by the multilayered hierarchical network using self-organization. To recognize handwritten digits using a machine and using pattern recognition is difficult. This is of some interest to recognize ZIP codes in addresses written by hand in mail. Neural network and its computational properties have attracted the interest of researchers in the area of machine perception by presenting an exciting, complementary alternative to symbolic processing paradigms[29]. The neural network gives results quickly using unsupervised training. There are many different neural network models. The method of neocognitron is used to recognize handwritten alphanumeric characters. This neural network can acquire the ability to recognize patterns by learning, and can be trained to recognize any set of patterns, it also has a large power of generalization, presentation of only a few typical examples of deformed patterns is enough for the learning [13]. The neural network not only makes use of classic notions of dynamic learning systems, but also has at its core, the idea that a system may use relatively static sets of previously learned information analogous to what cognitive psychologists might refer to as schemas or strategies[25]. In particular applications, such as optical character recognition, it is possible to train a network by presenting it with the finite universe of prototypical sets of pixels that may lead to a correct solution most of the time[25]. When the training is completed the neural network is found to be stationary. Rarely when the neural network is static, it fails to recognize the input figure and in that case the dynamic neural network system is used. These two networks are combined in an image recognition unit that has the capability to exceed the expected Neural performance of human beings at this task[25]. network is used in the image recognition and pattern recognition methods. The neural network model should be synthesized in order to endow it with pattern recognition capability like that of a human being. A cell in a deeper layer generally has a tendency to respond selectively to a more complicated feature of the stimulus patterns and, at the same time, has a larger receptive field and is less sensitive to shifts in position of the stimulus patterns, SO each cell of the deepest layer of the network responds selectively to a specific stimulus pattern and is not affected by the distortion in shape or the shift in position of the pattem[l4].

Fukushirna et a1.[13j

all other processing elements in the layer immediately adopt new weights. The input to the network is in the form of zeroes and ones (binary). Tests are conducted for many iterations. The goal of the network is to identify accurately character present in the input image. At each layer of network input image is analyzed within the surrounding area that becomes bigger and bigger till the entire input character is classified. Several examples of translation of English letters such as a and b to corresponding Greek letters such as ~1and p are given in [31]. Neural network using neocognitron can recognize handwritten numerals of various styles of penmanship correctly, even if they are considerably distorted in shape[31]. Although the author shows results for the recognition of alphabet, the neocognitron can be trained to recognize other set of patterns such as Arabic numerals, geometrical shapes, or others. A pattern can be seen

III.2 Manufacturing applications

Moon and Chi [27] have utilized the generalization

449

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 1996 IEEE

Proceedings of the 29th Annual Hawaii International capability of neural network models to solve the part family formation problem. The approach presented by the authors, combines the useful capacities of the neural network technique with the flexibility of the similarity coefficient method [27]. Manufacturing information, such as the sequence of operations, lot size, multiple process plans were given special consideration in their approach to solve generalized part family formation problem. The authors also point out that the method is flexible and can be efficiently integrated with other manufacturing functions. The authors conclude that neural network can address the Group Technology (GT) family formation problem efficiently. Kaparthi and Suresh [20] present a neural network clustering method for the part-machine grouping problem. This method is based on a neural network algorithm to support procedures like production flow analysis. A neural network clustering algorithm using similarity coefficients, is used to solve the part-machine grouping problem. The neural network method is based on unsupervised learning. In the part-machine matrix, each row (part) is considered as a vector in a higher dimensional space and every dimension corresponds to a machine type, and the number of dimensions is given by the total number of machine types required for all parts [20]. The authors have shown that the neural network is capable of handling large data sets. Lee et al., [24] present a method for part family formation, machine cell identification, bottleneck machine detection and the natural cluster generation is done using a self-organizing neural network. The authors argue that the generalization ability of the neural network makes it possible to assign the new parts to the existing machine cells without repeating the entire computational process. The authors show that neural networks can learn from a given set of patterns and are able to generalize this knowledge to other similar problems. This property makes them useful in small and medium-size batchmanufacturing systems where training data are limited and new parts are continuously encountered. The authors point out that their method based on neural networks is not significantly influenced by the size of the machine-part matrix and hence it is appropriate for solving large-scale industrial problems. Moon [26] presents a neuro-computing model to Similarity identify part family/machine groups. coefficients based on, for example McAuley method, is s The similarity coefficients are used as determined. connection weight values for neurons within the machine and part layer. A neuron from either the part layer of the machine layer is then chosen arbitrarily. The neural

Conference on System Sciences -

1996

network is started by providing an output signal to the chosen neuron. A threshold value, which determines the number of clusters as well as the degree of similarity within a cluster is then decided. Neurons with activation values greater than the threshold value are grouped together. Thus part families and machine groups are identified. The method then chooses a neuron arbitrarily which has not been assigned to any family or group. These steps are iteratively carried out until all the neurons are clustered. Wu describes two applications of neural networks to solve manufacturing problems - to form group technology based manufacturing cells and to monitor cutting tool condition. To form a manufacturing cell, a neural network was made to operate on the classical machine/components matrix Patterns representing respectively the machine characteristics and the component characteristics are used as input to the network. This approach was tested on two such matrices, one consisting of 10 jobs and 15 machines, and the other 14 machines and 24 parts. The author concluded that the results produced by the neural network approach compare well with the classical clustering techniques [ 3 71. The monitoring of tool condition during metal cutting operations is a classical production topic for which a substantial amount of research work has been carried out, the problem is to monitor accurately the condition of the cutting tool on-line so that a worn-out tool can be detected quickly and subsequently replaced [37]. The methods to detect tool failure are based on the level of cutting force, vibration or acoustic emission, and these are measured by sensors that are attached to the tool. But the main difficulty is how to detect the signal pattern produced by a worn tool. Some of the signal components are selected from a number of raw input data which are more sensitive to tool ware, but less sensitive to process noise. A set of training inputs are chosen which consists of signal patterns representative of both fresh tool cutting, and worn tool cutting and these are used to train a single hidden layer back-propagation neural network with a 8-3-l structure, i.e. 8 input processing units, 3 hidden processing units and 1 output processing unit, processing units are the building bricks of neural network [37]. Target outputs of 0.01 and 0.99 aTe set for fresh and worn cutting patterns. After training, each layer of the neural network acts as a signal filter so that the neural network suppresses noise and increases the signal/noise ratio step by step as the input

450

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 1996 IEEE

Proceedings of the 29th Annual Hawaii International patterns propagate through the network, the trained neural network produced a 94% success rate for worn pattern detection [37]. Neural network monitors successfully the tool condition in metal-cutting operations. Yih et al.,[39] uses a hybrid method that combines human intelligence, optimization and neural network to solve a real-time robot scheduling problem in a circuit board production line. The hybrid method is divided into three phases. ln phase I, a simulator collects data from human schedulers. Phase II constructs Semi-Markov decision models which are then used to fmd solutions to the scheduling problem. Solutions derived from phase II are then fed as training data for the neural network. In the neural network model, there exists three layers: an input layer, hidden layer, and output layer. The nodes between the input layer and hidden layer are fully connected, as are those between the hidden layer and the output layer. The input layer contains nine nodes, each correspond to one attribute in the state definition and the output layer has six nodes that represent six possible decisions in the system. For the hidden layer twenty nodes are selected after systematically experimenting with various hidden nodes based on the performance of the resulting neural network. The neural network was trained for twenty hours on a 486 IBM-compatible personal computer. The duration of training was determined by observing the level of errors of the neural network and the decrement in error rate. Training the neural network was concluded when the error levels were low and decrement in error rate was approaching zero. The resulting network model was then evaluated in the real-time robot scheduling problem [39]. Cook and Shannon [lo] present a methodology to predict the occurrence of out-of-control process conditions in a composite board manufacturing facility. This method is developed using neural network theory. The neural network, using back-propagation method, was successfully trained to represent the process parameters. The trained neural network was able to successfully predict the state of control of the specific manufacturing process parameters with 70% accuracy. The learning rule used in this research was the generalized delta rule which is an error-correcting rule that has been used in various applications including converting printed text to speech, controlling robot arms, and selecting good loan The back-propagation learning applications [lo]. algorithm uses a gradient- or steepest-descent heuristic that helps a network to organize itself in ways that improve its Continuous manufacturing performance over time. processes with measurable process parameters are promising application areas for neural network modeling

Conference on System Sciences -

1996

The relative importance of various manufacturing decision making criteria and the overall performance of a manufacturing system is impossible to analytically estabhsh[ll]. Chryssolouris et al., [l l] explore the use of neural networks for identifying the relative importance of decision criteria. Simulation and a neural network are used to establish adequate weights of the criteria for the decision-making process at the work center level. A procedure for determining operational policies for manufacturing systems has been presented. The authors use simulation results to train a neural network which then prescribes an operational policy suitable for achieving a set of goal performance measures [ 111. The proposed neural network procedure determines suitable criteria weights for an entire sequence of multiple-criteria decisions, The authors conclude that the method is better suited to complex applications involving chains of decisions, such as job shop scheduling, while the conventional methods are likely better suited to isolated, single decisions [ 111. Yao et a1.,[38] uses a distributed neural network of coupled oscillators to solve an industrial pattern recognition problem. The problem addressed is machine recognition of industrial screws, bolts, etc. in simulated real time in accordance with tolerated deviations from manufacturing specifications. Inputs are preprocessed and represented as 1 x 64 binary vectors. The supervised neural network uses the backpropagation method to accomplish the pattern recognition task. The authors state that the neural network method performs better than a standard Bayesian statistical method.

IV. Summary
The methods, structures and applications of neural networks are presented in this paper. Ever since the beginnings of research in artificial intelligence, neural network has shown tremendous potential in solving complex problems. Neural network methods based on supervised and unsupervised learning are used in numerous manufacturing applications. Solution methods based on neural network are elegant and efficient in solving a wide range of problems in manufacturing that are critical for companies survival in today competitive s environment. Applications of neural network presented in this paper are only a small sample of a large number of potential applications.

BIBLIOGRAPHY
VI.
Abutaleb, A.S., A Neural Network for estimation of forces acting on Radar Targets, Neural Networks, vo1.4, 1991, pp.667-678.

[lOI.

451

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 1996 IEEE

Proceedings of the 29th Annual Hawaii International

Conference on System Sciences -

1996

PI.

Atiya, F. A., An Unsupervised Learning Technique for Artificial Neural Networks, Neural Networks, vo1.3, 1990, pp. 707-711. Baran, N., The Outlook for Pen Computing, BYTE, September 1992, The Worldwide Computing Authority. Bebis, G.N., and Papadourakis, G.M., Object Recognition using invariant object boundary representations and Neural Network Models, Pattern Recognition, Vo1.25, No.1, 1992, pp.25-44. Blum, A., Neural Networks in C++ - An ObiectOriented Framework for Building Connectionist Systems, John Wiley and Sons, New York, NY, 1992, pp.l-10. Carpenter, G.A., and Grossberg, S., The ART of adaptive pattern recognition by a self-organizing neural network, Computer, 21 (3), 1988, pp.77-88. Caudill, M., Back Propagation Networks, Naturallv Intelligent Svstems, Chapter 14, 1990. Caudill, M., Neural Networks, Primer Part II, AI Expert, February, 1988, pp. 55-61. Caudill, M., and Butler, C., A Neural Network Model for Selective Attention, Proceedings of the IEEE First International Conference on Neural Networks. Vol. 2, 1987, pp.ll-18. Cook D.F., and Shannon, R.E., A predictive neural network modelling system for manufacturing process parameters, International Journal of Production Research, 1992, Vol. 30, No.7, pp.1537-1550. Chryssolouris, G., Lee, M., Domroese, M., The Use of Neural Networks in determining Operational Policies for Manufacturing Systems, Journal of ManufactutingSystems, Volume lOiNo.2, pp.166175. Eberts, R., Using Neural Nets for Assistance in Interacting with Computerized Systems, Proceedings ofthe I992 NSF Design and Manufacturing Systems, pp. 687-690. Fukushima, K., and Wake, N., Handwritten Alphanumeric Character Recognition by the IEEE Transactions on Neural Neocognitron, Networks. Vol. 2,1991, pp. 355-365. Fukushima, K., Miyake, S., and Ito, T., Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition, IEEE. Trans. Systems,Man, and Cybernetics, Vol. SMC-13, No. 5, 1983, pp. 826-834.

D51.

Fukushima, K., and Miyake, S., Neocognitron: A New Algorithm for Pattern Recognition Tolerant of Deformations and Shifts in Position, Pattern Recognition 15(6), 1982, pp. 455-469. Fukushima, K., Cognitron: A Self-organizing Multilayered Neural Network, Biological Cybernetics 20, 1975, pp. 121-136. Jacobs, R.A., Increased Rates of Convergence through Learning Rate Adaption, Neural Networks, ~01.1, 1988, pp.295-307. Johnson, K., Daniell, C., and Burman, J., Feature Extraction in the Neocognitron, IEEE International Conference on Neural Networkq Vol. 2, 1988, pp. 117-126. Kaparthi, S., and Sure&, N.C., A neural network system for shape-based classification and coding of rotational parts, International Journal ofProduction Research, 29 (9), 1991,pp.1771-1784. Kaparthi, S., and Suresh, N.C., Machine-component Cell Formation in Group Technology: A Neural Network Approach, International Journal of Production Research 1992, Vol. 30, No. 6, pp. 13531367. Khotanzad, A. and Lu, J.H., Distortion Invariant Character Recognition by a Multi-layer Perceptron and Back-Propagation Learning, IEEE International Conference on Neural Networks 1988 - Volume I, pp.625632. Lippmann, R.P., Pattern classification using neural networks, IEEE Communications Magan ne, November, 1989, pp.47-64. Lippmann, R.P., An introduction to computing with nemal nets,IEEE A S S P Magazine, April, 1987, pp.422. Lee, H., Malave, CO., and Ramachandran, S., A selforganizing neural network approach for the design of cellular manufacturing systems, Journal ofIntelligent Manufactuting (1992)3,325-332. Mehr, D., and Richfield, S., Neural Net Application to Optical Character Recognition, IEEE Zntemafional Conference on Neural Networks, Vol. 4, 1987, pp, 771-776. Moon, Y.B., Establishment of a neurocomputing model for part family/machine group identification, JoumalofIntelligentManufacturing, (1992)3, pp.173182.

[33.

V61.

M.

P71.

r51.

U81.

161.

D91.

[71.

PO].

PI. PI.

Pll.

uw

P21.

[ill.

~231.

WI.

~241.

P31.

~251.

[141.

[261.

452

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 1996 IEEE

Proceedings

of the 29th Annual Hawaii

International [401.

Conference

on System Sciences -

1996

v71.

Moon, Y.B. and Chi, SC., Generalized Part Family Formation Using Neural Network Techniques, Journal ofManufacturing Systems, Volume 1 l/No. 3, pp.149-159. Pao, Y., Adautive Pattern Recoenition and Neural Networks, Addison-Wesley, New York 1989. Pawlicki, T. F., Lee, D-F., Hull, J.J., and Srihari, S.N., Neural Network Models and their Application to Handwritten Digit Recognition, IEEE International Conference on Neural Networks, Vol. 2., 1988, pp. 6310. Rumelhart, D.E., Hinton, G.E., Williams, R.J., Learning Internal Representations by Error Propagation, Parallel Distributed Processing, Chapter 8, vol. 1, 1986. Sankamarayanan, P., Application of the Neocognitron to the Character Recognition Problem, unpublished Master Thesis, Central Michigan s University, December 1992. Srinivasa, N., Kalle, P., Ziegert, J.C., and Smith, S., Neural Network Prediction of Machine Tool Error Maps, Proceedings of the I992 National Science Foundation (NSF) Design and Manufactun ng Systems, pp. 1099-1103. Vogl, T.P., et al., Accelerating the Convergence of the Back-Propagation Method, Biological Cybernetics, vol. 59,198X, pp.257-263. Wasserman, &Ural Computing, pp. 167-188. Wilson, R., and Sharda, R., Neural Networks, OR/MS TODAY, A Joint Publication of ORSA and TIMS, August 1992, pp. 36-42. Winograd, T., and Flores, Comnuters and Coenition. F., m

Zahedi, F., An Introduction to Neural Networks and a Comparison with Artificial Intelligence and Expert Systems,Znterfnces 21: 2 March -April 1991, pp.2538.

P81. P91.

[301.

[311.

~321.

[331.

[341. [351.

[361.

[371.

Wu, B., An introduction to neural networks and their applications in manufacturing, Journal of Intelligent Manufacturing (1992)3, pp.391-403. Yao, Y., Freeman, W.J., Burke, B., and Yang, Q., Pattern Recognition by a Distributed Neural Network: An Industrial Application, Neural Networks, Vol. 4, 1991, pp.103-121. Yih, Y., Liang, T-P, and Moskowitz, H., Robot Scheduling in a Circuit Board Production Line: A Hybrid OR/ANN Approach, ZZE Transactions, Volume 25, Number 2, March 1993, pp.26-33.

1381.

L391.

453

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 1996 IEEE