Keynote Papers

Keynote Papers
Presented at the Opening Session Sensor Technology in Assembly Systems
M.Santochi ( I ) , G. Dini (2) Department of Production Engineering, University of Pisa, Italy
Abstract This paper describes the present state of the art of sensor technology in automated assembly and analyzes the trends of research performed in the last years. After having examined which are the main reasons for using sensors, the assembly operations requiring sensors are shown. Then the working principles of the most diffused sensors are described together with open problems and some examples of typical applications: force, tactile, optical and vision, mechanical probes and positional sensors. In addition the paper examines the main trends of present research with special reference to new types of sensors. to new application areas, to sensor fusion and artificial intelligence for their management. Finally typical industrial assembly applications are described where sensors have been successfully used.

Kevwords: Process Monitoring, Assembling, Sensor Technology.

1. Introduction

In the last years new types of assembly systems have been developed under the thrust of modern market needs and thanks to the available devices for automation and robotics. Two recent keynote papers (41,80) described the evolution of assembly systems under the thrust of globalization: from dedicated assembly machines typical of the years before 1980 to flexible and reconfigurable systems using robots and flexible part feeders. It was also pointed out that ideal assembly systems should have good productivity, flexibility and dexterity. The last two specifications are easily obtained in manual assembly plants, still dominant in various fields and countries, but are more difficult to obtain in highly automated systems. In particular dexterous assembly, i.e. human like assembly work made by robots, requires reliable sensory feedback to be efficiently used in industry. The importance of sensors is therefore connected both with the development of new concepts of assembly plants and with the improvement of the performances of existing automated assembly plants and with recently arisen assembly problems. Sensors in assembly have a greater importance than in monitoring of machine tools: simple sensors have always been widely used in assembly plants much more frequently than in even highly automated machining systems. Some reasons are that assembly operations are very different one from each other, are generally complex or request complex movements. In addition, even if assembly robots operate in a structured working environment, some variables like part tolerances, positioning errors, unknown positions of parts always influence the process. Furthermore quality assurance makes it necessary to include automated inspection

among usual assembly activities. A proper installation of sensors also allows the use of standard machines (robots, pneumatic manipulators. feeders, etc.) with a lower positioning accuracy, with consequent economical advantages. Another advantage of sensor is during the programming stage. In fact, the same sensors used to control the process allow an easier and faster programming of handling and insertion trajectories: a high-precision positioning of the end effector is not requested during selflearning, since the sensor itself provides the necessary information to correct the path in the operative phase. In addition, off-line programming is made easier and more efficient since sensors can compensate the scarce precision of a CAD representation. Since when the importance of sensors in assembly was stressed in a keynote paper in 1978 (149) the evolution of assembly systems has been remarkable: many indus!ries have started to use sensors in their assembly plants. robotics research has produced a lot of work in the sensorial aspect, new kind of sensors are available on the market. Therefore it has seemed important to examine and discuss the state of the art of sensor technology both in industry and in research in order to have an Overview of the main applications, the existing problems and the current trends.

2. Assemblv oDerations
Apart from auxiliary activities such as cleaning. deburring. etc., the main operations carried out in assembly plants in which sensors are required can be classified as follows: Parts mating: peg in hole, hole on peg, multiple peg

Annals of the ClRP Vol. 47/2/1998

503

Table I - Attitude of sensors for the application in assembly operations. Assembly operations Force sensors Tactile sensors Compliant devices Optical sensors Vision systems

Mechanical probes

Positional sensors

= very suitable

1-

= quite suitable

7 difficult to adopt 1 =
Literature on sensor technology in assembly is very wide since the even wider field of robotics is involved. This paper is based both on a critical analysis of the literature and on the processing of a questionnaire distributed among ClRP members, sensor producers and industries using assembly plants. Very simple sensors like mechanical, capacitive or inductive microswitches will not be considered in this paper even if largely used in assembly plants.

in hole are typical operations belonging to this group where sensors are necessary since simple or multiple insertions are affected by problems of tolerances or faults of parts and robots inaccuracy. 0 Parts joining: this activity is estimated to cover about 50% of assembly activities. Examples are: 0 screwing where torque monitoring and checking of the connection quality are necessary; 0 press fits where the applied force must be controlled; 0 glueing and sealing in which the amount and continuity of the distributed material must be checked. 0 riveting, very popular in aircraft industry 0 Parts handling: automated feeding and robot grasping often require good sensorial capabilities when the shape or the position of the parts are not completely known or can change during the operation itself 0 Parts recognition: recognition of position and orientation of randomly fed parts are typical examples where sensor are unavoidable. 0 Inspection for quality control: soldering of electronic components on Printed Circuit Boards (PCBs) is a typical example, where solder past volume and shape must be controlled. In addition sensors are very useful or unavoidable in more recent applications or fields: 0 automated soldering of SMDs (Surface Mount Devices), very popular in modern electronic industry especially in 3-MIDs (Molded Interconnected Devices); 0 assembly of deformable parts, very difficult to handle and control during assembly as in automotive industry: 0 automated disassembly of products for recycling purposes, where the conditions of the product are not known in advance; 0 microassembly, where scale effects can lead to unforeseen situations.

3. Sensors in ass8mbly
The sensors typically adopted to perform the previously described tasks can be classified in the following 3 wide categories: 0 stress sensors, able to measure stresses generated by the contact between two or more components (fingers and grasped object, peg and hole, etc.). They obviously operate in-process and can be mainly divided in: 0 force sensors; 0 tactile sensors; 0 sensorized compliant devices; 0 dimensional sensors, able to measure dimensional quantities (recognition of shape, position, orientation or simply presence of an object). With respect to the previous ones, they can also operate in pre-process measurements (e.g.: recognition of part orientation before grasping) or in post-process measurements (e.g.: part inspection at the end of the assembly process). This kind of sensors can be mainly divided in: 0 optical sensors; vision systems; mechanicalprobes; positional sensors. other sensors. Other typologies of sensors are used in specific experimental applications or in other inspection tasks related to the typical assembly

504

Keynote Papers

operations. Among them, the most important are: temperature sensors; pressure sensors; acoustic sensors; accelerometers. Table I summarizes the attitude of stress and dimensional sensors for the application in assembly operations. Fig.la reports a graphical distribution of the main categories of sensors used in assembly, as results from literature analysis. Fig.1b shows, for the same papers, the distribution of the assembly operations where force and vision systems are used. Sensors used in assembly Multisensor
7 Q/n

A further classification derives from the operative principle. In most cases it is based on the measurement of strains generated by external forces on a deformable structure. This measurement is often performed by electrical strain gages (due to their high reliability, easy use and low cost), but other systems are also adopted such as piezoelectric, inductive, capacitive or Eddy etc. current transducers, potentiometers (50),

Force

33%
(a) Involved assembly operations
70%

Easy installation High flexibility Multicomponent sensors

50%
40%

30%
20%
10%

Limits on payload Increasing of endeffector size

Multicomponent sensors No limits on payload and endeffector size Difficult installation flexibility

Local stresses can be measured

Motor current can be used for sensing

Small-size Only the joint torque or Integrated can be measured sensors Difficult installation

Fig.2 - Installation sites for force sensors. A very important aspect in the application of a force sensor in assembly is also represented by a proper selection of the installation site on the manipulator (Fig.2). The choice is based on different considerations concerning the force component to be measured, weight, sensor size, flexibility, etc. The more common uses are: recognition of overload situations. owing to programming errors or unforeseen situations. These cases are mainly detected by binary sensors mounted on the wrist and able to measure from one to three components of the force. These devices can be integrated with compliant systems in order to protect the structure from collisions; rn grasping, concerning the control of grasping force, fulfilled by simple load cells installed on each finger, or more simply presence detection of the part by measuring the load on the wrist; recognition of missing parts by force sensors mounted on the wrist. This process is feasible only if the presence of a part influences the force measured

0Y o

Fig.1

- Statistical data about sensors in assembly

(b)

3.1 Force sensors The idea of using a sensor able to generate a signal corresponding to the force exerted between two objects in contact, was one of the first solutions adopted to obtain a powerful and reliable feedback in automated assembly. These sensors can be divided in two main categories: binary and analog sensors. A second kind of classification can be made on the basis of the number of force components the sensor is able to measure: the choice can range from one-component sensors (usually known as load cells), to ForcelTorque sensors (FTT sensors) able to separately measure the 6 components of force and torque (101).

505

by the sensor: comparing the sensor output with a reference value, the missing part can be detected; screwing, pressing, riveting o insertion operations.

Fig.3 - Eddy current torque sensor used in screwing (source: Feldmann).

In screwing operations, the force sensor is used to measure the torque and it is usually integrated within the screwdriver body. A typical configuration is represented by the Eddy current sensor illustrated in Fig.3 (11,38). Two hollow cylinders with longitudinal slots are mounted coaxial at the ends of the main torsion shaft. A set of fixed coils, powered by a high frequency current, is positioned around the hollow cylinders. During the shaft rotation, the coils induce on the cylinder a current proportional to the faced surface. During the torsion of the main shaft, the alignment between the slots changes, generating an increase of the faced surface and, therefore, a variation of the Eddy current detectable measuring the subtraction of energy from coils. In pressing operations, since no rotating components are present, the measuring of the force is simpler. Hydraulic pressing units with integrated load cells are available on the market (105); the process is monitored by a control system in function of the pressing force and the position of the punch. In insertions, the force measurement is fundamental to successfully complete the operation. A very common application consists in an FTT sensor mounted on the robot .wrist (116,120,144,148,149,163): the in-process measurement of the 6 components allows the detection of anomalies and, consequently, the application of recovery procedures or new trials of insertion (active compliance).

consists of sensitive sites made of conductive elastomer, which changes its resistance according to the applied stress (piezo-resistance effect). By measuring the resistance of each site it is therefore possible to evaluate the pressure distribution. A second transducer principle currently available consists of sensitive sites made of PVDF (polyvinylidene fluoride) films used for transmitting and receiving ultra-sounds (60). Each matrix site is bonded on a rigid substrate and covered by a thin layer of rubber. During working, the sites transmit ultra-sound pulses which cross the rubber layer, hit the object surface and return to the sites. The time of pulse flight is therefore proportional to the contact pressure (since the rubber thickness changes accordingly to the contact pressure) and can be measured for each site. Tactile sensors are usually positioned on the finger-tip surfaces or on the workbench directly in contact to the part to be grasped or to be assembled (Fig.4).
Pressure exerted on sensors

(a 1

Tactile sensors

Figd - Applications of tactile sensors in part mating. Typical applications concern: recognition of parts and/or their positions. These operations can be made by using a tactile matrix directly positioned on the workbench: from the analysis of the pressure distribution exerted on the sensitive surface, the part and its position can be identified (92); verification of grasping operations (part presence, control of grasping force, etc.). The knowledge of pressure distribution between. finger and object generates more detailed information such as: detection of the actual grasp position; detection of part slipping phenomena, etc.; execution' of high precision insertions. Similarly to force sensors, tactile sensors can be used to actively compensate insertion misalignments. Corrections can be performed in function of the pressures between the finger and the grasped element (Fig.4a), or in function of the pressures generated between the whole product and the workbench (Fig.4b). The main problems related to the use of these sensors concern the following 2 aspects: the high damageability of the sensitive surface and the complexity of the algorithms to be implemented in order to transform the sensor output in reliable information. In particular the first aspect is

3.2Tactile sensors A tactile sensor can be defined as a device able to measure the contact pressures exerted between two objects. The difference from the force sensors is the capability to supply, not only the resultant force, but also its distribution on the contact surface, allowing the evaluation of more details on the interaction between two components. These sensors are made of a thin matrix of sensitive sites, able to measure perpendicular forces. The data coming from this device may be elaborated with pattern recognition techniques in order to identify particular contact conditions during handling or assembly operations. Several transducer principles have been experimented in this kind of sensors. One of the most diffused principle

506

Keynote Papers

crucial, since it leads to a strong limitation of the use of this sensor in industrial practice: the well known aggressiveness of industrial environments can rapidly wear the sensor surface, consequently changing the output signal (interchangeable pads are usually adopted to reduce this problem).

3.3 Sensorized compliant devices

It is well known that, in peg-in-hole operations, the robot structure should have a proper compliance in order to passively modify the position and the orientation of the gripper, according to the forces and torques generated during insertion, in order to avoid jamming phenomena and correctly complete the operation. This is the working principle of the passive compliance devices and, in particular, of the RCC devices (Remote Center Compliance) which represent one of the simplest and most utilized systems to accomplish tight tolerance pegin-hole insertions (71,101,102,103). These devices, by means of an appropriate structure, locate the center of compliance of the system close to the point of first contact between the peg and the hole. The standard configuration of an RCC is based on 3 elastomeric bearings, assuring the proper compliance, positioned between two plates used respectively for the connection to the wrist and the gripper. Other configurations, different from the previous one but based on the same principle, have been successfully experimented (4,119). An RCC, however, cannot be considered a sensor since it does not produce any signal related to a physical quantity to be used for an active control of the operation. Its sensorized variant, often known with the acronym IRCC (Instrumented RCC), also evaluates the deformation of the device and therefore the forces applied during assembly. An IRCC behaves like a force sensor, but in addition it results compliant in appropriate directions, allowing a faster insertion. Two approaches are normally used to sensorize a compliant device: the former consists in the integration of the sensor within the device structure (64), the latter consists in separately installing a passive device and a force sensor on the robot wrist (18,93,157). An example of the first approach is given in (33) where pneumatics is used both for sensing and for actuation. The device consists in three simple pneumatic sensors and three actuators arranged radially. A lateral load will cause the output pressure of the corresponding sensor to increase and hence the corresponding actuator will expand causing the immediate assembly to move in the direction of the applied force. A compliant system made by a rubber element mounted with an F/T sensor is used in (85) for the assembly of panel fastener using robots. An example of the second approach is described in (84): a passive device is mounted on the wrist in order to compensate lateral and angular deviations in screwing operations. If errors are too great, the passive system cannot correctly operate: this situation is detected by a force sensor which stops the insertion and activates a search procedure of the hole performed by an optical sensor. An interesting study on the maximum admissible

angle of deviation between the bolt and the hole, useful to design this kind of device, is presented in (151). A particular configuration is described in (145). A 5-axis compliant wrist has been used in close tolerance insertions (Fig.5). This device can be used according to 2 different strategies: in pure passive accommodations, setting the stiffness of each axis through the regulation of motor torques: in pure active accommodations, performing a closed-loop control of each axis in function of the signals generated by force sensors.

Fig5 - Five-axis compliant wrist for high precision insertions (source: Van Brussel). Generally speaking, the main problems related to these systems can be summarized as follows: 0 low flexibility: in most cases, the configuration of a passive device has to be set manually, according to the dimension of the part to be assembled; a correct use of these systems requires the presence of chamfers on the parts. In chamferless peg-in-hole conditions, supplementary sensors are therefore needed: the system compliance can meet with difficulties in horizontal or oblique insertions. 3.4 Optical sensors Optical sensors can be defined as devices that use light to obtain information from the environment. They are based on 2 components: a light emitter (lamp, LED, low-power laser, etc.) and a light receiver, such as a phototransistor, a photoresistor, a PSD (Position Sensitive Device), eventually arranged on mono- or bi-dimensional arrays. The light travel from the emitter to the receiver can be properly configured by lens, mirrors, optical fibers, etc. The working principle varies in function of the measurement to be done. In this regard, optical sensors can be mainly classified as follows (Fig.6): binary sensors (102). They sense the presence of an object through the interruption or the reflection of the light beam. If properly placed, they are able to provide information on part position and orientation; displacement sensors (109). They measure small displacements of an object with respect to a reference distance. The light produced by the emitter is diffused by the object surface and focused on a PSD. If the surface is shifted, the light spot intercepted by the PSD moves accordingly, generating a variation of the output signal (triangulation principle). This kind of sensor can be

507

IP

Optical sensors Typical uses in assembly

I Binary sensors I Part recognition

Displacement sensors Part inspection

Laser scanners Part recognition Part inspection

I I
Optical micrometers Part recogn., Inspection Part mating

Fig.6 - Classification of optical sensors used in assembly operations. also used in force measurement, by sensing the deflection of elastic structures; laser scanners (132). They mainly consist of a laser beam scanned on the working area and an array sensor which detects the reflected light. These devices are usually able to supply information about the intensity of the reflected light overall the scanned surface, in order to detect anomalies (comparing it with a reference map) or about the heights of the parts contained in the scanned surface (by means of triangulation principle), in order to detect defects, missing parts, etc. optical micrometers. These sensors are able to recognize the dimensions and the position of a part. The typical configuration consists of a lamp emitting a parallel light flux and a receiver made by a linear photosensor array. The width and the position of the portion of flux interrupted by an object can be easily detected. These devices usually reach sensitivities up to 10 pm; better values (up to 0.1 pm) are obtained by means of laser scan micrometers. In assembly plants, these sensors are more commonly used in: part recognition. Optical micrometers or binary sensors can detect some features of the part (shape, position or orientation). An interesting application is represented by the problem of orienting unstable objects in automated feeders (15113,137,162). In these situations, in fact, the standard devices (wiper blades, scallops, etc.) are not sufficient to correctly orient the pads; binary sensors are used to detect the position assumed by the part along the feeder and, accordingly, to actuate the more convenient procedure (part rejection, programmed orientation by actuators, etc.). Another application may consist in determining the position and the shape of a grasped object by arrays of binary sensors placed on the finger-tips; pre- and post-assembly inspections. Typical applications of displacement sensors concern: measurement of PCB deformations, measurement of pin deflections in electronic components, part presence detection, measurement of glue or seal thickness. The typical advantage is the absence of contact (i.e.: faster operation, sensor installed on a fixed device, etc.); on the contrary, their accuracy can be influenced by the surface reflectivity. Inspections can be also accomplished by optical micrometers, as in (150), where a laser sensor controls the correct geometry of bolt-type screws before insertion; error compensation in insertions. Several methods have been adopted to approach this problem. A first example consists of optical micrometers used to detect the position of the insertion hole in electronic assembly (62,97): the positioning error between a PCB and a manipulator is sensed through an expanded laser beam projecting the PCB holes on a PSD.
3.5 Vision systems A vision system represents a very powerful device to recognize a manufacturing environment (140) owing to the great amount of information which can be extracted from an image in relatively short times. However, its convenience has to be properly evaluated due to its higher costs and the difficulties of implementation with respect to other more conventional sensors. In assembly systems, this device has been widely applied (123), especially in recent years, in which a very impressive enhancement of accuracy and computing speed has been obtained: high resolution CCD cameras are currently used at reasonable costs. Since the task of a vision system is to acquire an image and extract from it dimensional information, its use is limited to operations like part recognition, part inspection and error compensation. Fig.7 reports a typical application in PCB inspection.
imoge memory A

imoge memory B

Fig.7 - Use of a vision system in PCB inspection (source: Toenshoff). It can be noticed that these applications substantially coincide with those ones analyzed for optical sensors. Particularly, in the field of PCB assembly, the huge number and the complexity of details to be measured in

508

Keynote Papers

short times have forced to a large diffusion of this sensor as a standard component in automated plants (27,135). Some typical examples of application are: part recognition. A common application in this field concerns the detection of the position and orientation of objects randomly placed in a magazine (32,140,152). A particular system is described in (70,121) and concerns the development of a Grey Scale Sensor able to recognize randomly oriented parts in industrial scenes, based on the scene models containing 3D information. Other examples deal with tracking and grasping of objects moving on conveyors (140), orientation on unstable parts on automated feeders (6,28,53), multi-purpose systems for inspection of workpiece dimensions using stereovision techniques (124). A meaningful example is given in (57), where a method to quantitatively figure out the 3D geometrical information of an object from a single picture of both the object and its mirror image is described; pre- and post-assembly inspections. A great amount of applications have been implemented in this field, especially in electronic assembly. A particular case is represented by the assembly of SMD components, where the use of a vision system is necessary to detect anomalies such as misplacement or loss of components before soldering, tombstoning of devices, etc. A critical problem is the configuration of the lighting system which can deeply simplify the image analysis: an example of commercial system is shown in Fig.8: a programmable lighting system, mounted around the objective of the camera, has been arranged with 4 sectors which can be independently activated in order to produce appropriate shadows around the components, verifying the anomalies such as missing parts, lead misalignments, etc. (104). Structured light inspection is often used to collect and assemble 3 0 images of solder paste deposits: color enhancement of the image makes it easy to spot problems caused by excess or insufficient solder paste (117);

assembly. An example of a special vision sensor integrated on the robot arm has been described in (35) to solve the problem of identifying the position of a capacitor lead in order to allow an easy assembly on an electronic board. With one camera two images are taken thanks to a rotating system installed on the robot arm itself. A stereoscopic triangulation routine is then run on the images. The same problem was previously solved in (8) using two cameras. The main problems in using a vision system in assembly operations regard: a) measurement accuracy, which depends from many parameters, such as camera resolution, optical lenses, lighting, extension of the measuring area, etc.; b) system calibration, particularly critical in assembly problems in which very high precision operations are requested: d) software development which can be expensive; e) high processing time for 3D images.
3.6 Mechanical probes It is well known that mechanical probes are able to locate a point of an object through the contact between a stylus and the object surface. As soon as the contact occurs, the stylus bends, the probe sends a signal to the controller and the current coordinates of the point are stored. In comparison with other dimensional sensors (optical sensors and vision systems), the use of this device requires a direct contact with the part to be sensed and therefore a relative motion between the probe and the surface. For this reason, in assembly applications, this sensor can be conveniently applied if machines with controlled axes are already present in the assembly area (e.g.: assembly robots). In these circumstances, the possible configurations are the following: probe installed on the end-effector of the robot and object to be measured on the workbench, or probe on the workbench and object grasped by the robot. Typical applications of this device mainly concern part inspection (115,154) or part recognition (56).

/ Toroidal mirror
/

4aector optical fiber Parabollc mirror

movement Part

n Q

Camera lens \

4ciizii&
Part

Fig.8 Programmable lighting system for inspection (source: Mitutoyo). error compensation in insertions. Also in this case, the most frequent applications concern electronic

3.7 Positional sensors This class of sensors generates a signal proportional to the position assumed by a mobile element. They can be classified in binary and analog sensors accordingly to the nature of the output signal or in linear and angular sensors accordingly to the movement they can measure. The positional sensors more commonly used in assembly operations are: LVDT sensors (Linear Variable Differential Transformers) (108). These linear analogic sensors consist of a cylindrical body, in which the primary and the secondary windings are contained, and a rod mobile along the body axis. The position of the rod is detected through the differential transformer principle; encoders. These well known sensors are usually preferred to other ones (resolvers, potentiometers, etc.) mainly for their high performancekost ratio. LVDT sensors are used in assembly when the control of the position assumed by a linearly moving device is requested; an example is the control the correct execution of pressing operations, where it is not sufficient to monitor the exerted force but also the punch position (105).

509

Another application of LVDTs concerns the measurement of deflections of elastic structures, commonly used to protect the grippers and the manipulators from collisions. Encoders can be used both to measure the position of linearly moving element (through appropriate kinematic chains), and to measure the angular position of a rotating element. This last situation occurs in screwing operations controlled by torque-angle techniques, which represent the most diffuse method to execute automated screwing (11,106,107). The screwing units, in this case, are equipped with a torque sensor (par.3.1) and an encoder for the measurement of the angle covered by the rotating shaft during the insertion of the screw. The operation is driven by the torque value: each kind of screwing has a torque threshold value, beyond that the angle measurement starts, and a torque value which corresponds to the tightening torque requested to correctly perform the operation. The constant monitoring of the torque-angle relation allows the verification of friction conditions on screw threads and the presence of geometrical errors or anomalies of the material.

reported in (156). It consists of two rings connected by six tractive force sensors through socket joints. Force sensor are used in a very unusual method for detecting the exact position of a part after it has been picked up by a robot (114). The object is picked up on a magnetized platform mounted on the robot wrist and the platform is on the top of a column machined from the same piece. A solenoid provides an initial pulse to start the natural vibration of the system (Fig.9). Strain gauges mounted on the column allow to measure the static deflection of the column loaded by the weight of the part and the natural frequency of vibration of the whole system. The output signals processed by a transputer give the coordinates and orientation of the part on the platform with sufficient approximation.

3D BT VR digitizer force

I ethernet
jc

4. New developments in research activities

TY
2.)

operator

In order to have a better overview of the recent trends of research activity, the huge amount of results available in the literature has been divided in the following 4 subsections 4.1, 4.2, 4.3 and 4.4, and the activity of the last ten years has been mainly considered.
4.1 Improvement of existing sensors Research activities on sensors described in the previous section have continued aiming to the improvement of their performances or to the extension of their applications.

real-time controller
Fig.10 Hirai).

- Apparatus for human motion measuring (source:

Platform

I
i

Fig.9 - Device used for part position sensing (source: Pham). Some original principles to develop new force sensors have been used as illustrated in the following. An intelligent screw insertion (133) by means of a mechatronic electric powered screw driver uses a low cost torque capacitive sensor. The sensor uses the torsion movement of a dielectric material between two capacitor plates to measure the twist angle. A new configuration of 6-component F/T sensor is

Two problems encountered in robotic sheet metal assembly (i.e.: car body panels) are the vibration of sheet metal parts during handling and the control of their position before welding. For the first problem, in (161) it is proposed to use small grippers equipped with strain gauge sensors to monitor part's vibration and feed it back to the robot control unit in order to modify the trajectory or the orientation of the part to reduce vibration. The second problem was solved by integrating data coming from the strain gauges and an F/T sensor providing feedback on the contact condition. Vibration amplitude was reduced up to 45% and angular error between the parts was reduced up to 50% in 1.7 s. There are particular assembly tasks where automation can be difficult to be introduced and at present they are made by workers. An idea to solve the automation problems is to measure human operations, to extract human skills and transfer them to the robot. An original approach (54) regards the insertion of deformable tubes into a rigid plug: human demonstration of the task is first measured using a position and a force sensor, then the measurements are analyzed and force sensations are extracted. Finally human motions are transplanted to a robot (Fig.10). In the last years some efforts have been directed in integrating tactile sensors in robotic grippers and in developing identification and localization algorithms. An example is a very unusual shape compliant device (44) which incorporates a matrix of actuating probes into the fingers. The probes produce an image by measuring distances between the object surface and the sensor

510

Keynote Papers

plane, making localization.

possible a

3D

identification and

In the field of IRCC devices, research has been oriented in designing new kinds of devices which can ensure versatility and good performances. In (159,160) a device is reported for peg-in-hole insertions which combines a passive compliance system and a displacement sensor and has the necessary flexibility to be set both in the position control and in the force control mode. The passive compliance is provided by a rubber structure, while the sensing mechanism consists of a sixjoint serial linkage with a transducer at each point. precise device

Impacts are registered by the accelerometers and the FTT sensor is used during the peg-in-hole insertions: a device for electrical contact is also used. A new interesting and versatile design is the so called "magic wrist" (111) for very accurate and delicate assembly operations, able to be used in different operating modes. The base of the RCC device is an exagonal rotor named "flotor" with six flat-wound coils levitated within fields supplied by six permanent magnets in a stator. The gripper is attached to the rotor. The rotor has 6 DOF with respect to the stator. The motions allowed are 2 5 mm in translation and 5 5" in rotation. Its position is measured by means of three light beams and three PSDs on the rotor. A pneumatical docking mechanism allows the rotor to be rigidly clamped in its center position. A different approach is used (5) in the case of assembly of big parts in automotive industry. As illustrated in Fig.11, two cooperating robots are used, each one having a coarse and a precise positioning device serially connected. The two robots are placed in parallel and the object is grasped by grippers mounted on compliant systems. An original example of active device for compensating robot inaccuracy (156) consists of a linear high accuracy driven unit that moves the element to be adjusted relative to the fingers using an adjustment pin.

Fig.11 - Precise positioning device for the assembly of big parts (source: Arai). Other working principles have been used as well as to simplify the IRCC devices. Such principle is used in (16) where a soft material is mounted between a rigid skeleton of the gripper and the hard part to be grasped. Positioning errors cause different pressure in the sensorized fingers which can be processed according to a particular strategy. A compliant system designed for press fit operations was integrated in the "Robohammer" press tool (153). The system consists of a spherical spring loaded segment contained in a housing. Positioning errors are compensated by the rotation of the system around the tool center point. The pressing force generated by an impact unit is transmitted to the part to be assembled through the spherical contact surface instead of elastic elements: for this reason the RCC device has large angular stiffness and compensate large errors. The Robohammer is sensorized through Eddy current and linear induction sensors which measure the travel per strike and the total stroke of the component to be assembled. New geometries of IRCC devices have been widely studied as well: an example is a 6 Degrees of Freedom (DOF) force sensor based on the mechanism of the Steward platform (95). The sensor consists of two platforms coupled by six spring loaded pistons whose length variations are measured by six LVDTs. Another method to monitor the assembly operations is to track the events of expected and unexpected impacts between the part grasped by the robot and the fixed part. This method (75) uses a gripper equipped with a compliant device, an F/T sensor and 3 accelerometers.

Fig.12 - Laser scanner used in part handling (source: Weck). Original passive adaptive compliant wrists even if not sensorized are described in (go), based on a micro SCARA mechanism with two eccentrically positioned rotating discs instead of two joined links, and in (138): in this last case the coupling has 6 DOF and works on the "lock and free" principle: the coupling is locked and behaves like a stiff member during robot motion and is free during constrained motion. It is based on a combination between an air bearing, a variable stiffness air spring and a center locking mechanism. Research in optical sensors has recently increased especially in the use of laser light for its well known

51 1

advantages. A first example is a laser sensor mounted in a central position above an assembly cell (86,88). It scans all the objects in the environment using a laser beam and evaluates the diffusely reflected light by comparison with preset values. The sensor evaluates the presence of parts, prevents collisions, decodes the assembly carrier units, counts the parts in magazines.

distance between the sensor center and the gripper axis by means of piezotraslators (Fig.13). The claimed accuracy is Ipm. Automatic component placement in PCB assembly is one of typical application areas of optical sensors: in (31) the classical video camera is replaced with linear arrays without optical lens (Fig.14). A silhouette of the pins is generated by point like illumination on the linear array. Using a central optical projection the dimension of the silhouette depends on the distances between the object light source and detection plane. A placement accuracy of 0.01 mm is obtained. Inexpensive light beams are used (146) for object localization in a recognition station of an assembly plant. The technique involves passing the polyhedral object through a set of crossed light beams and storing the positions when any beam is broken or reconnects.

Fig.13 - Optical sensor used in part mating (source: Schweigert). A similar system aimed to collision avoidance (156) consists of a dual axis scanning unit and a rotational axis around the vertical axis of the robot. A pulsed infrared laser sensor is used to scan the area where the gripper moves. A schematic view is given in Fig.12. A laser range finder (91) is used to guide a robot to assemble a rubber belt with fixed pulleys. This is an example of resolution of a very complex assembly task i.e. to manipulate deformable objects without any analytical model. A mechanical gripper including a laser based distance f measuring system is used (13) to control the position o an assembly robot. A 30 laser radar for object recognition which could have higher performance than stereo camera and shorter scanning time is being studied (143). An optical system for solder joint inspection on PCBs (125) is able to detect 3D shapes of specular objects with high reliability and high speed. The system uses a galvanometer scanning unit which steers the laser beam incident position, a mirror unit which gathers the reflected light and a sensing unit covered with photo diode arrays. A two dimensional piezoelectrically actuated optical scanner is described in (112). The sensor head consists of a laser diode with collimating lens, a photo detector and an amplifier circuit. The application is shape matching, performed by comparing the obtained scanning data to the preinput original data stored in memory. The shape matching problem has been made easier by an operative system based on genetic algorithms (46). PSD sensors have been used as well owing to'their interesting possibilities. The position of a hole center and its inclination (81) obtained from a PSD is used to achieve shaft inserting. When the hole is inclined a black ellipse image is focused on the PSD which detects hole inclination by its current difference. A PSD sensor is used (128) to measure the position errors in the peg-in-hole operation and to compensate the

f,

lcm

f,

Fig.14 - Optical sensor for PCB placement (source: Doemens). In the field of vision systems research efforts have been directed to improve the performances of 3D systems or to find new applications. In a 3 0 recognition system (1) a code lighting sequences is projected onto a pallet of workpieces. A range map is constructed using a sequence of binary images. The range data are processed and surfaces are matched to the data sets to extract surface models of the components. Since industrial applications require short processing time, high accuracy and robustness in recognizing complex objects, new hardware and algorithms have been developed like the VPSEE (Videorate Pipeline Subpixel Edge Extractor) (67) and successfully experimented in car

512

Keynote Papers

assembly. The system is based on a edge based binocular stereo vision able to obtain 3 0 geometrical primitives both from a depth map and from 2D primitives and then match a scene with previously prepared models. Algorithms have been implemented on specially designed processors in order to reduce computational costs and increase speed.
D'O'.S'or

height mor8ur.d ahift as aeon by triangulation by the came,a

I17 mm x 25

mm)

@I-

Fig.15 - Contour tracking sensor (source: Toenshoff). Triangulation is used in a method for contour tracking by robots (141). In the system, based on structured light, a wide stripe of light is projected onto the contour surface of the part by a miniature slide projector and the height along the edges of the stripe is measured using the triangulation method (Fig.15). From very simple computations made between consecutive frames the contour tracking is possible with very good accuracy. In (158) an adhesive dispensing head equipped with a CCD camera is used in PCB production. Vision feedback on the plan area of each dispensed blob is obtained: the plan area is adopted as a measure of the dispensed volume instead of more complex 3D systems. A good and reliable in process control of solder joints (83) is based on 3D sensor and laser triangulation. It uses a 2D gray level image and a 3 0 range image. Each pixel carries information on its brightness and its height. The images are used in the inspection system to recognize different types of faults. Contemporary use of laser light and CCD is another sensing method (126,127). The sensor is based on a laser projecting a double stripe onto the object and a video camera. Optical narrow bandpass filters are used on the camera lens to shield it from the ambient light. In addition an automatic planning and simulation system is proposed to determine the optical location of sensors in case of multisensor systems. An important point is that the reference model of the geometry to be checked or the shape to be recognized is a CAD model. The integration with CAD is investigated also in (155) for the recognition of complex 3D objects, very frequent in supervising assembly operations, in case of very small batches where the teach-in step should be avoided for time reasons. The reference data for the vision system are automatically computed from the CAD model: procedures have been developed to generate the artificial reference image and to bring it in coincidence with the real image taken by the camera. A research in the same direction of exploiting CAD potentiality has been made in (6566) to detect errors in assembled products. The

algorithm is trained on synthetic images generated using the CAD models of the different components of the assembly. In addition the method allows to simulate and optimize the position of the light source and camera, using the component material properties and contact information from the same CAD model. A very special use of stereo vision (94) is made to insert a flexible wire into a hole: forces acting on the elastically deflected wire are indirectly measured by tracking its shape observed by a stereo vision system (Fig.16). A recent original use of vision systems has been made for quality control in electronic production (43). This system, named "coherence radar", is based on Michelson white light interferometer and it is able to perform 3D inspections of components (dimensional anomalies in the range of 30 pm can be easily detected). Vision systems have been used also for guiding the assemblyldisassembly robots i.e. as visual guidance. Examples are given in (74,139) in the field of disassembly where it is necessary to sense the presence and conditions of a component and to guide the robot arm towards the selected component to be dismantled. Good results have been obtained in moving the manipulator to a partly occluded battery of old cars.

..
Insert into hole i

Contacton hole

Fig.16 - Different situations of insertion of flexible wires controlled by a stereo-vision system (source: Nakagaki). 4.2 New types of sensors New sensors working on different principles have been studied and tested like temperature measurement and acoustic methods. Temperature sensing in soldering operation is very important to achieve high quality of assembly of components on PCBs but it is difficult to perform. In an experimental study aimed to obtain a process model for single point soldering (96) a radiation pyrometer is used even if the emission ratio changes during the phase from solid to liquid and vice versa. The sensor measures the temperature over an area and not in a point. IR sensors are used for non contact temperature measurement in a monitoring and control system for reflow soldering of components on a PCB (26). Since the PCB moves at a constant speed through the oven a two dimensional thermal image may be created from the compilation of successive lines of temperature

513

measurements scanned at 90" to the PCB movement. The movement of the measuring spot is achieved by a rotating mirror reflecting the infra-red energy onto a fixed receptor.

200 kHz of the ultrasonic excitation the range of distance measurements is significantly increased. The system can perform measurements of distance, angles of plane surfaces and object identication among a limited number of possible shapes. An ultrasonic sensor based on a PVDF foil is used in a mechanical assembly process (14) to generate a signal which is very characteristic of the specific faults.

comd

I I

clbtnu

IAumblyoKJwaaI

CLbW

4.3 Sensors in advanced areas

New sensors or new versions of classical sensors have been used in particularly new and advanced research areas. One of these areas is microassembly. The assembly of very small components for the development of complex micro-electromechanicaldevices (MEMs) has very peculiar features like tolerance problems, force scaling problems, interference factors, influence of usually disregarded forces as electrostatic and surface tension forces which require very special sensors (118). Since the effect of those forces can be roughly estimated, the development of micromanipulation techniques must necessarily use sensors A small number of examples can be found in literature since this item is quite recent.

Fig.17 - Noise patterns detected in different clip assembly situations (source: Milberg). Another new promising technique is based on acoustic sensors. The philosophy of such systems is based on the fact that experienced workers on the assembly lines can easily understand the result of some assembly operations by their senses. In some joining processes the noise emitted contains information on the correctness or failure of the process. The systematic analysis of sounds allows an on line reliable quality assessment.

Fig.19 - Active compensation obtained by a sensor integrated system (source: Hoehn). A miniature optical positional sensor (17) consists -of an optomechanical measuring head, an optoelectronic box and a six fiber optic cable (Fig.18). The physical principle is light intensity modulation. The position of the shadows resulting from the illumination of a pin is determined by two wedge shaped differential detectors. The claimed accuracy is k0.003 mm. The sensor uses commercial optoelectronic component and has been tested in automatic microassembling operations of watch driving wheels. A precision positioning system used to assemble SMD components on three dimensional boards (55) uses both a CCD sensor to monitor the assembly tolerances and a triaxial accelerometer to monitor the vibrations of the handling device (Fig.19). In micro end effectors adhesive forces must be reduced. A new micro end effector whose grip surface is covered with micropiramids fabricated by micromachining techniques has been experimented (3). The micro end effector is equipped with an integrated piezoresistive force

Fig.18 - Optical measuring system used in microassembly (source: Braunschweiler). A wide research work has been performed in (87,89). An example is shown in Fig.17 and regards the assembly of an axially symmetrical plastic clip used in automotive industry for body panels. The sound signal is evaluated through a Fast Fourier Transformer (FFT) and the quality characteristics chosen are in the 700-1800 Hz range; incorrect assembly situations give different frequency spectra. Ultrasonics is used (76) in a low cost device consisting of four 40 kHz transducers mounted around one 200 kHz transducer. By changing the frequency between 40 and

514

Keynote Papers

sensor fabricated with the same techniques. The sensor has good linearity, negligible hysteresis and high accuracy.

4.4

Sensor integration, sensor fusion, Artificial Intelligence techniques. Apart from studying new types of sensors or improving the performances of already existing ones, research has been addressed to optimize their use through the concepts of integration, fusion and artificial intelligence.

Systems using sensor integration can be classified in two groups, in function of the methodology of interaction among sensors: parallel interaction. This is the typical situation occurring in screwing, with the contemporary detection of the torque and the angle, or in pressing, with the measurement of the force and the punch displacement. Parallel interaction is also used to monitor the same variable with different sensors (sensor fusion), obtaining other advantages such as: complementary information like in human reality, reduction of uncertainty and increasing of intelligence and flexibility, learning capabilities, etc.; sequential interaction. In this case different sensors are used to monitor the different steps of a process. Peg-in-hole cases are typical examples which could request a sequential interaction of sensors. In a first step, when the contact of the peg and the hole does not occur owing to high positioning errors, a vision or optical sensor is used; in a second step, after the first correction, a force sensor is used to control the final insertion (10,18,45,62). The following examples are available: combination of force, torque, proximity and laser sensors on the wrist of a robot (123), combination of force and vision to estimate the contact position of a grasped object (59) combination of accelerometers, forceltorque, electrical contact (12), combination of four ultrasonic transducers with two force sensors (20). Using different sensors the problem is to cope with their outputs in an integrated monitoring system: for this reason new promising Artificial Intelligence techniques are needed like Artificial Neural Networks (ANNs), fuuy sets theoty and so on. ANNs have been used in different ways in sensor technology: to analyze complicate outputs from one sensor only, to analyze signals coming from more than one sensor, to take decisions on robot movement based on signal from one single sensor (typical example is pegin-hole insertion). In an experimental cell (24,36) vision systems are used to check the position of objects, a force sensor detects the success or failure of a grasping, a tactile sensor on the gripper indicates parts orientation. A neural network based on a 2 dimensional Kohonen map is used to process the continuous sensory input and to give a diagnosis of the source of errors. Then a replanning of the assembly operations is made possible based on this feedback.

An example of a vision-force system is reported in (73); the artificial vision is used to obtain a first correction of the robotic arm, when the peg is completely outside the hole; the force sensor is used to give the final correction, in order to successfully complete the assembly operation. A well known problem in electronics production is quality control. In (42) ANNs are used in collecting and analyzing data coming from sensors for the monitoring of the following operations: solder paste application, component placement, reflow soldering. Implementing integrated test and quality assurance in production lines in electrotechnical industries is described in (83). Innovative data processing technologies based on ANNs are used both for acoustic signal interpretation as well as for image processing. The acoustic test is performed between assembly and packaging using a special vibration sensor, as reported in Fig.20. Image processing is used for PCB inspection: solder joint images are directly processed by a fully connected feedfoiward network, consisting in three layers, 2 x 900 input neurons, a hidden layer of 10 neurons and 4 output neurons corresponding to the 4 quality classes. Good results are obtained for the recognition of defects like "blow out" and "excess of soldef'
vibration sensor gripper motor

decouplin test stan8

Fig.20 - Acoustic motor processing system (source: Mengel). A similar method has been used in (125) with a good accuracy of classification of 94.2% of solder joints. An example of sensor fusion in the peg in hole case in a partially unstructured environment is reported in (136). In a first case the fusion is among optical sensors and two cameras: the cameras provide the approximate locations of the hole to the vision sensors to minimize the searching time. In a second case another fusion is discussed between vision and tactile sensors, to be used when obstacles are near the hole and cannot be detected by vision sensors only. A lot of research work has been performed in the field of using ANNs for developing strategies for assembly operations (like peg in hole) using only one sensor in order to solve the problems connected to the unreliable off-line description of the world around the assembly robot. ANNs have been widely used to evaluate the correct relation between the measured physical quantities and the corrections to the gripper position (7,22,25,30,47,78, 110,134). Fig.21 schematically shows an example of working procedure of such a system (29) divided in 2 phases: a preliminary phase, concerning the collection of

515

data for training and validation of the neural network, and an operative phase of peg insertion into the hole, using an FTT sensor. Combining task level knowledge and sensor based information is the aim of the perception based learning system proposed in (21,23). In this system signals coming from a six-component force sensor are fed in the network which gives a map of the contact states.

& 0 z
SJ

c
no

with a force controlled active wrist of a Cartesian robot where a hybrid position force algorithm is implemented in the control system. A hybrid sensor control system has been developed (39,40) where a specific function is assigned to a designated computer component. Fast and slow sensory control have been implemented on a plug in board making use of transputers. Special software tools allow an optimal distribution of the processes to the single processors. Involved assembly tasks require a 6 DOF robot to obtain the positioning of each component together with suitable sensor technology like laser scanners or fast vision systems. In another recent example (37), the concept of Open Architecture Control is used to have an intelligent assembly with possibilities of trajectory planning, collision detection, sensor integration. Sensors are in this case laser, optical ones and stereo vision. Research Positional Other
_n,

3s

CORRECTIONS ND FORCEfTORQU

Multisensor

7%

Force
9401

Vision 28% Industry Other Positional 8% Fig.21 - Active compensation by using a force sensor and a neural network system (source: Dini). Other researchers focus their interest on the strategy for peg-in-hole insertions (48,4932). Fuzzy logic is here used to search the best position based on signals coming from usual force sensors. In SMD assembly systems (19) it is described the use of ANNs connected to a vision system to measure the relative positioning error between pins and footprint. In the first experiments the learning phase took 10 minutes and estimating the misalignment needs about 1.7ms, with a throughput of about 21,000 components per hour. The system works in presence of noisy and low contrast images. Even if many types of sensors are available on the market, their integration with the control units of robots are not completely achieved; in other words, a control unit is not adequately prepared to accept and manage sensor signals (e.g.: a programmed path cannot easily modified according to sensor inputs). Various attempts to solve this problem have been made. An intelligent control of an assembly robot (58) uses vision and force sensors, able to modify the planned trajectory in a screwing task. Another example is in (2) Multisensor

Optical 14% Tactile 2%

Force

, ,f 3%

IRCC

Vision 57% Fig.22 - Distribution of sensors in experimental research and industrial applications.

5. Industrial applications

A sensor system must fulfil several requirements (often in contrast to each others) in order to be conveniently applied for the control and monitoring of assembly operations in an industrial plant: high reliability, accuracy, robustness, low maintenance, easy installation, easy access for maintenance, easy use and processing of the output signal, small dimensions and weight, noise immunity, low cost. A complete (or partial) satisfaction of these requirements is achieved in assembly with classes of sensors which can be defined as ((mature)) or, in other words, sensors based on well-developed technologies. Very simple

516

Keynote Papers

devices like mechanical switches or proximity sensors, or more complex systems (anyhow well experimented) such as load cells, LVDTs, torque-angle systems can be considered standard elements in assembly plants.

presence, overload situations, etc.). In this context, another interesting comparison can be made on the kind of the assembly operation involved in the sensor application (Fig.23). It can be easily seen that part inspection and part recognition represent the most frequent subjects in technical papers describing industrial applications, while in the works carried out in research centers the most efforts are focused on problems, perhaps more stimulating from an academic point of view, but less felt by industry. In the peg-in-hole problem, for example, the current tendency of industrial applications is oriented to the use of less complicated techniques with respect to very refined algorithms for the control of 6component force sensors or arrays of tactile sensors; the reason can be found in the obvious industrial trend to prefer simple and more reliable techniques even if this fact can involve the renouncement of more efficient and flexible solutions. The next paragraphs report, for some industrial sectors, some meaningful examples of sensor applications in assembly operations.

Fig.23 - Distribution of sensorized assembly operations in experimental research and industrial applications. Sensors based on ((modern)) technologies or sensors which request proper algorithms to process the outputs (F/T sensors, tactile sensors, vision systems, etc.) are gradually spreading in the industrial practice, although their application is limited to ((very difficult problems)) which can be approached only by using very specific sensors together with refined control strategies. In this regard, Fig.22 compares the distribution of the use of such sensors in experimental researches and in industrial plants, relatively to the data contained in the technical papers collected by the authors and describing innovative solutions in the field of assembly. In particular, in the experimental researches, it can be noticed the prevalence of vision systems or F i l sensors, together with a not negligible presence of devices using optical sensors, IRCC systems, etc. As far as the industrial applications are concerned, it is well evident a high predominance of vision systems, confirming the recent progress of this technology aimed to satisfy the aforesaid requirements. On the contrary, other sensors such as optical or F/T sensors, currently analyzed in research works for solving complicated problems in assembly, seem to meet some difficulties of applications in industry, obviously excepting very simple uses of these devices (verification of pdrt

5.1 Electronic industry The applications in this field mainly concern part inspection tasks accomplished by vision systems or optical sensors. Some examples are related to the following phases of the PCB assembly process: inspection of PCBs in the pre-solder stage of assembly (63). A CCD camera is placed over the board and acquires an image size of 12.5 x 12.5 mm. The camera is also moved at the speed of 230 mm/s along 2 perpendicular axes in order to cover the whole PCB area of 254 x 204 mm. The system is able to acquire about 300-500 images for each board; these images are analyzed, compared with an inspection database and any negative results are written to an error file in less than one minute; inspection of components before assembly. Vision systems are currently used to provide on-line quality control by inspecting the components prior to placement (98). If the component does not meet tolerance requirements, or presents defects such as incorrect lead length, lead damage, etc., it is rejected; inspection of solder paste deposit (9). The laser beam emitted by an optical displacement sensor is scanned perpendicularly to the direction of PCB movement along the assembly line. A controller simultaneously processes the data in order to obtain the area and the volume of the solder paste deposition, its average height and the presence of bridging. The system is able to measure with a resolution of 10 pm at a maximum scan speed of about 40 mmls; inspection of components after assembly. A sensor integration is described in (122). In this application, 500 different variants of mini-contactors are inspected after assembly by two cameras. Within a cycle time of one second, the two images are processed with different techniques (grey-level and slitlight techniques) and then integrated in order to detect anomalies such as missing parts, misalignments, presence of parts belonging to other variants, etc.

517

Another example in this field concerns the inspection and setting of head position in assembly of VCR devices (68). The system consists of a CCD camera for the acquisition of two images of the head (respectively with and without interferometric fringes), used to accurately calculate the head positioning errors. The calculated parameters are then transferred to an adjustment mechanism. The images are processed in 31,5 ms and the total cycle time is contained within 30 s.

second camera locates the bolts on the hub. The robol is therefore able to position the wheel on the bolts in accordance with the instructions of the vision system. assembly of windshields and doors (147). The main problem in this case is to exactly locate the car body before assembly with a sensor system. Vision systems are normally used for this purpose. Vision sensor .. .~
~

I
Manipulator-+
SYD ---+

Slit bea

ITIIJ-~amera~

m
!

Slit beam image

y4-l

Placement site

Fig.24 - Typical configuration of a vision system used for error compensation in PCB assembly (source: Mc Walter). Other interesting cases concern error compensations during placement of components on PCBs. Also in this situation, vision systems are largely used. A quite usual solution (82,99) consists in using two cameras as illustrated in Fig.24. A first camera is mounted on the manipulator to locate the position of the placement sites on the PCB; a second camera is placed on the workbench to detect the exact position of the components. The system described in (82) combines the two measurements, calculates the correction and updates the assembly program to obtain the positioning of components with an accuracy of about k0.2 mm and a placement time of about 4 s.
5.2 Automotive industry In this particular sector, a distinction between car body assembly and engine assembly has to be made, being the assembly tasks very different in terms of process execution, part .dimensions and requested accuracy. In car body assembly, sensors are mainly implemented for accomplishing part inspection or part recognition tasks, or in the assembly of components which do not request high precision of positioning. The sensors usually installed are the vision systems, as results from the following examples: measuring of body fit of completed vehicles on continuously moving assembly line conveyor (69). A camera is mounted on a robot and measures the clearance of adjacent body panels using light section principle, with an accuracy of i O . 1 mm in a cycle time of 0.7 s for one point (Fig.25). This system is preceded by the vehicle position and orientation measuring system, using another camera and an optical sensor, which provides the robot with the data it needs to make the adjustment, obtaining a final sensor positioning of k3 mm. assembly of wheels on the hubs (147). A first camera identifies and locates the wheel on a conveyor, while a

Coordinate data Fig.25 - Operating principle of body panel positioning sensor in automotive industry (source: Kondo).
r

2 7 L L
dFl/dl

TOP COMPRESSION RING

OIL RING

BOTTOM COMPRESSION RING

(a)

dFl /dl

(b) Fig.26 - Example of force sensor output in detecting missing part: a) presence of 3 piston rings; b) absence of the second ring (source: Little). Different problems are related to the assembly of engines. The involved automated operations, in fact, usually request higher accuracy and the inspection may concern the measurement of different parameters which may require different kinds of sensors. Some examples are: a screwing of bolts. An interesting application of tactile sensors is described in (100) for the measurement of

Keynote Papers

tightening torque in screwing. A piezoelectric thin film (0.02 mm) is applied to fastener heads in order to obtain an ultrasonic transducer. These modified fasteners are used with conventional assembly tools equipped with a spring-loaded probe which assures an electrical contact with the film and the measurement of the times-of-flight of ultra-sounds, proportional to the value of torque. identification of missing rings in assembly of pistons (77). An F/T sensor is here used to monitor the assembly of pistons into engine cylinders. The sensor can verify the presence of rings during assembly through the analysis of the axial force and its derivative, as illustrated in Fig.26. inspection of O-rings in injectors. A vision system is able to control the presence and the integrity of small O-rings mounted on the mobile rods of injectors (72). inspection of sealing operations. The inspection is normally performed after the seal material has been deposited on the part; a vision system is used to verify the continuity of the seam, the width and its position. inspection of rotating elements. The correct execution of the operations related to the assembly of rotating elements (camshafts, crankshafts, etc.) is checked by verifying the free rotation of these elements through acoustic sensors (comparing the sound emitted with a reference model) or torque sensors (verifying that the torque remains below a threshold value).
5.3 Other industries The use of sensors in assembly is becoming an usual application in other kinds of industries. The following examples regard meaningful situations in mechanical, electromechanical, packaging, shoes and Gther industries: 0 assembly of air-conditioners. A high-speed bin-picking technology from randomly piled-up parts of airconditioner is described in (51). The system comprises two cameras: the former placed over the parts is used to control the operation of picking up; the latter is used to analyze the posture assumed by the part between the fingers of the gripper in order to activate the final correction. assembly of bearings. Vision systems are used to verify the correct ball positions and the amount of grease. b assembly of electric motors. In the example described in (79), 450 different motor styles are produced with 8 significant differences in terms of assembly requirements. In this scenario, vision systems are used for positioning, orienting recognition and inspection of the parts; 0 transport of packaging materials. In order to automate the supply of packaging machines with packaging materials (foil bobbins), a transport system sensorized with a camera has been developed (142). The task of the vision system is to analyze the scene in the vicinity of a pallet, detect bobbins on it and locate the top bobbin that can be freely removed by a hexapod robot. Only one camera is here used, thus a quasi-3D algorithm that computes the third image dimension from the known diameter of the bobbin core has been

implemented. The appreciable advantages of this system is the low cost and the robustness under industrial environments; assembly of shoes. Vision systems are commonly used in automated process to identify the shape, position and orientation of shoes components before stitching operations. Automated stitchers equipped with this kind of sensors are currently available. A set of rollers transports the component under a light source and the system evaluates the features (e.g.: decorative pictures) which uniquely identify the part and its orientation; the appropriate stitch pattern is then retrieved from a database and corrected according to the previously detected features; woodworking industry. Examples can be found in assembly of rack furniture or kitchen cabinets where vision systems or laser scanners are used for the recognition of part orientation (131); lamp wiring industry. In this area, a critical problem is the automated assembly of conductors which are items difficult to manipulate and retain in position. In the example described in (130) different kinds of sensors (vision systems, force sensors, etc.) are used to obtain error compensations and measurement of insertion forces; clothing industry. Sensors are used in an automated sewing cell (129) to give the information of the actual orientation of workpieces (e.g.: trouser leg) before sewing.

6. Conclusions

In this paper the present state of the art of sensor technology in automated assembly system has been analyzed and the trends of research have been outlined upon the basis of the wide collected literature and the opinions of many ClRP members. In the last years the importance of sensors in this 'aspect of production has been increasing together with automation and robotics. Sensors have been used both in the typical assembly processes to improve reliability and to perform in line quality inspection and in new assembly areas or products like microassembly and assembly of large deformable parts. In the past, the use of high-technology sensors in assembly applications was restricted to research centers or development institutions. Nowadays, the recent results in sensorics, such as a higher data processing speed, a lower cost and a better efficiency, have enhanced the trend to adopt such systems even in industrial fields. However, it is well known that the transfer of such technologies to an industrial environment characterized by very tough conditions meets with many difficulties and is therefore slow. As a result mpst of industries have installed very simple sensors which are well known , experimented and reliable, but have opened their interest also to more advanced sensors particularly to vision systems, often connected to laser light: the reasons are that they are general purpose systems, they are flexible since they are

519

programmable, they replace the best human sensorial capability. In other words vision systems are going to become a standard component of advanced assembly lines. Other sensors even if simpler and cheaper continue to find some difficulties to be largely accepted in industry. Reasons might be that they are too much specific, they need an initial adjustment phase, they have not yet reached a sufficient reliability. Some suggestions can be made as a consequence of this analysis with regard to future research efforts: 0 research efforts should be addressed to classical sensors like force, optical, IRCC in order to improve their reliability and versatility; 0 acoustic sensors seem very promising and their use could be extended in many areas of assembly; 0 future robot control unit should be designed in order to easily interface any type of sensor and process their signals: 0 vision systems, even if widely diffused, should be simplified in the programming phase today still expensive
Acknowledgements: the authors wish to thank all who have contributed to this paper with very helpful documents and suggestions and in particular Prof. T. Arai, Prof. H. Bley, Prof. T.R. Crossley, Prof. K. Feldmann, Prof. N. Martensson, Prof. T. Pfeifer, Dr. N. Roth, Prof. G. Seliger, Prof. H.K. Toenshoff, Prof. M. Weck, Prof. E. Westkaemper, Prof. H. Weule, Prof. H.P. Wiendahl, Prof. D. Williams. A special thank to Dr. M. Vasta for his valuable work in collecting contributions. The following sensor producers and users are also acknowledged for their cooperation: Asatec srl, AT1 Industrial Automation, Chrysler Corporation, Comau S.p.A., Denso Corporation, F. Schunk GmbH, Sommer Automatic, SKF, Toshiba Corporation.

7.

8.

9.

10. 11. 12.

13.

14.

15.

16.

Biblioaraphy

17. 1. Alshawish, H.M.M., Allen, C.R., 1994, 3D object recognition using coded light projection for robot assembly applications, 21st IEEE industrial Electronics Conf.. vol. 2, Los Alamitos: 1240-1247 Amirat, Y., Pontnau, J., Babaci, S.,Francois, C., 1995, Flexible assembly cell integrating a parallel manipulator for accurate automatic assembly tasks, Int. IEEEIIAS Conf. on lnd. Autom. and Control: Emerging Technologies, Piscataway: 308-313 Arai, F., Fukuda, T., Iwata, H., Itoigawa, K., 1996, Integrated micro endeffector for dexterous micromanipulation, 7th Int. Symp. on Micro Machine and Human Science, IEEE, Piscataway: 149-156 Arai, T.. Kinoshita, N., 1981, The part mating forces that arise when using a worktable with compliance, Assembly Automation, August: 204-210 Arai, T., Osumi, H., Fukuoka, T., Moriyama, K., 1995, A Cooperative Assembly System Using Two Manipulators with Precise Positioning Devices, Annals of the CIRP, vol. 44I1: 23-26 Arnstroem, A., Groendahl, P., submitted by

18.

2.

19.

3.

20.

4.

5.

21.

22.

6.

Sohlenius, G., 1985, A System for Automatic Assembly of Different Products, Annals of the CIRP, VOI. : 17-20 3411 Asada, H., 1990 Teaching and Learning of Compliance using Neural Nets: Representation and Gneration of Non-linear Compliances, IEEE Int. Conf. on Robotics and Autom., V01.2: 1237-1244. Asano, T., Maeda, S., Murai, T., 1983, Vision System Aligns Leads for Automatic Component Insertion, Assembly Automation, February: 32-35 Asjes, R.J., Kooijman, K.S., Vermeulen, O.T., Duijve, R., 1996, Sensor and control electronics for an industrial 3D scanner, Proceedings of the Society of Photo-Optical Instrumentation Engineers, vol. 2599, SPIE, Bellingham, WA. USA: 171-176 Baird, S., Lurie, M., Precise Robotic Assembly Using Vision in the Hand, Robot Sensors: 85-94 Bauer, G.M., Screwing techniques and devices, Robert Bosch GmbH. Bergquist, C., Soderquist, B.A.T., 1994, On combining accelerometers, forceItorque sensors, and electrical sensing for detecting contact errors during assembly, IEEEIRSJIGI Int. Conf. on Intelligent Robot Systems, Piscataway, NJ, USA, vol. 3: 1736-1743 Bley, H., Avgoustinov, N., Seel,U., Featurebasierte Montage - Konzepte und Realizierungsbeispielefur den Datenaustausch in komplexen automatizierten Montagesystemen , VDI Berichte 1322: 161-178 Bley, H., Seel, U, Guenther, K.G., 1996, Solving technical problems in assembly systems design, Annals of the CIRP, Vo1.45/1. 11-15. Boothroyd, G., Dewhurst, P., 1982, Parts Presentation and Feeding for Robot Assembly, Annals of the CIRP, vol. 3111: 377-381 Borovac, B., Seslija. D., Stankovski. S., 1992, Soft sensored gripper in assembly process, IEEE Int. Conf. on Robotics and Automation, vol. 2, Los Alamitos: 1283-1288 Braunschweiler, A., Llosas, P., Bertholds, A., Baur, C., 1995, Optical position sensor for automatic microassembling operations, Optical Engineering, vol. 34, n. 9, September: 2660-2664 Brooks, T., Cunningham, R., 1982, Adaptive Robot and Sensory System for Attachment of Electrical Connectors to Solar Arrays, Applied Machine Vision Conference, SME, Dearborn. Michigan, USA Burel, G.. Bernard, F., Venema, W.J., 1995, Vision feedback for SMD placement using neural networks, IEEE Int. Conf. on Robotics and Automation, vol. 2, Piscataway: 1491-1496 Caicedo, E.F., Martin, J.M., Calderon, L., Freire. T., Cees, R., 1994, Sensorial system for parts identification and robotic assembly, IEEE International Symposium on Industrial Electronics, IEEE, Piscataway, NJ, USA: 316-319 Cervera, E., del Pobil, A.P., 1995, Geometric reasoning for fine motion planning, IEEE Int. Symp. on Assembly and Task Planning: 154-159 Cervera, E., del Pobil, A.P., Marta, E. Serna, M.A., 1995, A Sensor-Based Approach for Motion in

520

Keynote Papers

23.

24.

25.

26.

27. 28.

29.

30.

31

32. 33.

34.

35.

36.

37. 38.

39.

40.

Contact in Task Planning, Proc. of IEEE Int. Conf. on Robots and Systems: 468-473. Cervera, E., Del Pobil, A.P., Marta, E.,Serna, M.A., 1996, Perception-based learning for motion in contact in task planning, Jou. of lntell. and Robotic Sys.: Theory & Applications, vol. 1713, Nov.: 283-308 Chu, H., EIMaraghy, H.A., 1993, Integration of Task Planning and Motion Control in a Multi-Robot Assembly Workcell, Robotics & CIM, vol. 101235-255 Connolly, T.H., ffeiffer, F., 1993, Neural Network Hybrid Position1Force Control, Proc. of IEEElRSJ Int. Conf. on Intelligent Robots and Systems, Yokohama, Japan, July 26-30: 240-244. Conway. P.P., Whalley, D.C., Wilkinson, M., Williams, D.J.. 1997, Automated Adaptive Control of the Reflow Soldering of Electronic Assemblies Cowan, D., Davies, D., 1986, Automatic system copes with variable PCB production Cronshaw, A.J., Heginbotham, W.B., Pugh, A., A Practical Vision System for Use with Bowl Feeders, Robot Sensors Dini, G., 1997, Active compliance in high precision assembly by neural networks, Proc. Of 3d Aitem Conf., Salerno, Sept. 17-19, 259-266. Dini, G., 1998, Application of Neural Networks in Robotized Peg-in-Hole Insertions Using a PreProcess Learning Method, accepted for the publication on Int. Jou. for Manufacturing Science and Production, Freund Publishing. Doemens, G., Rummel, P., Stockmann, M., 1995, A Novel Approach to 3D Vision for Automatic Component Placement, International Conference on Recent Advances in Mechatronics, vol. 1: 220-225 Domm, M., 1989, A better view of PCB assembly, Assembly Automation, November: 187-191 Drazan, P.J., Hopkins, S.H.. submitted by Tilsey, R., 1984, Semi Autonomous Systems for Automatic Assembly, Annals of the CIRP, vol. 3311: 291-294 Dreisbach, B.. submitted by Guenther, K.G., 1985, Robot Aided Assembly in Massproduction with High Flexibility (Example: Assembly of Electrical Motors), Annals of the CIRP, vol. 34/1: 13-15 Driels, M.R., Collins, E.A., submitted by Boothroyd, D.G., 1985, Assembly of Non-Standard Electrical Components Using Stereoscopic Image Processing Techniques, Annals of the CIRP, vol. 3411: 1-4 El-Maraghy, H.A., Syed, A.. Chu, H., 1993, A 2-D Mapping Approach for Monitoring and Diagnosing the Execution of Robotic Assembly Tasks, Annals of the CIRP. vol. 4211: 9-12 EIMaraghy W., Preyn, T., 1997, An intelligent approach to intelligent assembly, 27 ISlR Conf. Feldmann, K., Steber, M., 1992, Screw Fastening in Flexible Automatic Assembly with ComputerIntegrated Process Control. Annals of the CIRP Feldmann, K., Schoenherr, U., Zeller, J., 1994, Fast Sensory Feedback for Demanding Robot Applications in New Technologies, 25th International Symposium on Industrial Robots, Hannover, Germany: 549-554 Feldmann, K., Schoenherr, U., Zeller, J., 1995, Fast

41.

42. 43.

44.

45.

46. 47.

48.

49.

50.

51.

52.

53.

54.

55.

56. 57.

Sensor Guided Robots for Flexible Manufacturing Experimental Results, 26th International Symposium on Industrial Robots, Singapore: 241-246 Feldmann, K., Rottbauer, H. Roth, N.. 1996, Relevance of assembly in global manufacturing, VO1.4512, 545-552. Feldmann, K., Goetz, K.. Quality assurance in electronics production Feldmann, K., Felbinger, C., Feuerstein. R., 1998, Analysis of different cause-effect mechanisms for quality assurance in electronics production, Production Engineering, Vol.V/I, 139-142. Friedman, G.M., Derby, S.J., Versatile 3-D tactile array gripper using a digitally encoded sensor, Flexible Assembly Systems - 1992 ASME, Design Engineering Division (Publication) DE, vol. 48, ASME, New York, NY. USA: 111-117 Fujita, N., 1988, Assembly of Blocks by Autonomous Assembly Robot with Intelligence (ARI), Annals of the CIRP, vol. 3711: 33-36 Grudzien, W., Sonntag, A., Seliger, G., 1997, Operating System for Laser Scanners, CAPE 97 Gullapalli, V.. Grupen, R.A., Barto, A.G., 1992, Learning Reactive Admittance Control, Proc. of IEEE Int. Conf. on Robotics and Automation, Nice, France, May: 1475-1480. Gurocak, H.B., de Sam Lazaro, A., 1995, Fuzzy logic and position sensing for precision assembly, Journal of Robotic Systems, vol. 12, n. 2, February: 135-146 Gurocak, H.B., 1996, Fuzzy rule base optimization of a compliant wrist sensor for robotic assembly, Journal of Robotic Systems, vol. 13/7, July: 475-487 Hanafusa, H.. Asada, H., A robot hand with elastic fingers and its application to assembly process, IFAC Symp. on manufacturing tech., Tokyo, 1977. Hanai,M., Matsumoto, K., lida, Y., Fujimoto, H., Kato, T., 1997, High-speed bin-picking system based on human performance, 29 CIRP int. Sem. on Manufacturing Systems, Osaka, Japan. Hara, K., Yokogawa, R.. 1992, Precision insertion by heuristic search with fuzzy pattern matching, JSME Int. Jou. Series 3: Vibration, Control Engineering, Engineering for Industry, vol. 35, n. 2, June: 201-206 Heginbotham, W.B., 1982, Flexible assembly module with vision-controlled robot, Assembly Automation, February: 36-39 Hirai, S.,Noguchi, H., Iwata, K., 1996, Humandemonstration based approach to the recognition of process state transitions in insertion of deformable tubes, 13th IEEE 1nt:Conference on Robotics and Automation, vo1.3, IEEE, Piscataway: 2006-2011 Hoehn, M., 1996, Mikropositionerinnichtung fur die (ed): Prazisionsmontage, In. Reinhart, G. Mikrotechnik fur die produktion, Munchen,pp.93-103, IWB. Seminarberichte22, Verlag Hollingum, J., 1989. Consortium aims for multiproduct assembly, Assembly Autom.. Aug. 137-141 Honda, T., Kanedo, S.. Takeyama, H., 1985, Pattern Recognition of Part andlor Workpiece for Automatic Setting in Production Processings, Annals of the CIRP, VOI. 29-32 3411:

521

58. Ishikawa, J., Kosuge, K., Furuta, K.. 1990, Intelligent control of assembling robot using vision sensor, IEEE Int. Conf. on Robotics and Automation, Los Alamitos: 1904-1909 59. Ishikawa, T., 1996, Estimation of contact position between a grasped object and the environment based on sensor fusion of vision and force, IEEE Int. Conference on Multisensor Fusion and Integration for Intelligent Systems, Piscataway: 116-123 60. Iversen, W.R., 1993, Tactile Sensing, 1990s Style, Assembly, February-March: 23-26 61. Iversen, W.R., 1994, Focus On Machine Vision, Assembly, April: 16-19 62. Janocha, H., Menzel, E., 1989, Assembly of PIHDevices with Robots, Annals of the CIRP, vol. 38/1: 29-32 63. Jolliffe, N., 1990. Clinching inspection of electronic circuit boards, Sensor review, April: 71-73 64. Kazerooni, H, 1988, Direct-drive active compliant end effector, IEEE Jou. of Robotics and Automation, June, Vo1.4 n0.3, 324-332. 65. Khawaja, K.W., Tretter, D., Maciejewski, A. A., Bouman. C. A., 1994, Automated assembly inspection using a multiscale algorithm trained on synthetic images, IEEE int. Conf. on Robotics and Automation, vo1.4, Los Alamitos: 3530-3536 66. Khawaja, K. W., Maciejewski, A. A., Tretter, D., Bouman, C. A., 1996, Camera and light placement for automated assembly inspection, IEEE International Conference on Robotics and Automation, vol. 4, Piscataway, NJ, USA: 3246-3252 67. Kimura, Y.,Naito, T., Nakano, M., Moribe, H., Kuno, T.. 1995, Stereo vision system for car assembly, IEEE International Conference on Robotics and Automation, vol. 2, Piscataway, NJ, USA: 1403-1409 68. Komatsu, T., Nagashima, S., Tsukada, H., 1989, An Automatic Adjustment System for VCR Magnetic Heads on Cylinder Units, Annals of the CIRP, vol. 3811 9-12 69. Kondo, M., Tachiki, S., Ishida, M.. Higuchi, K., 1995, Automatic measuring system for body fit on the automobile assembly line, IEEE International Conference on Robotics and Automation, vol. I. IEEE, Piscataway, NJ, USA: 538-543 70. Kutzer, E., 'Roth, N., Rummel, P., submitted by Guenther, K.G., 1986, A Model-Based Grey-Scale Sensor System for Robotics, Annals of the CIRP, VOI. 35/1: 17-20 71. Lane, J.D., 1980, Evaluation of a remote centre compliance device, Assembly Automation, November: 36-46 72. Lanzetta, M., 1998, The quality control of critical assembly components: visual inpection of O-rings, 2"d Int. Conf. On Planned Maintenance Reliability and Quality, Oxford April 2-3, 132-137. 73. Lanzetta, M., Dini, G., 1998, An Integrated VisionForce System for Peg-in-hole Assembly Operations, Proc. of CIRP Seminar ICME 98, July 1-3, Capri (Naples). 74. Lee, K., Bailey-Van Kuren, M. M., 1997, Supervisory control of an automated disassembly workcell based

75.

76.

77. 78.

79. 80.

81.

82.

83.

84.

85.

86.

87.

88.

89.

90.

91.

on blocking topology, IEEE Int. Conf. on Robotics and Automation, vol. 2, Piscataway: 1523-1528 Linderstam, C., Sijderquist, B. A. T., 1996, Monitoring the generic assembly operation for impact from gripping to finished insertion, IEEE International Conference on Robotics and Automation, vol. 4, Piscataway, NJ, USA: 3330-3335 Lindstedt, G., Olsson, G., 1993. Using ultrasonics for sensing in a robotic environment, IEEE International Conference on Robotics and Automation, vol. 3, Los Alamitos, CA, USA: 671 Little, R., 1992, Forceltorque sensing in robotic manufacturing, Sensors, Nov.. Vo1.9 no.12. Majors, M.D., Richards, P.J., 1995, A Neuralnetwork-based Flexible Assembly Controller, Proc. of 4th Conf. on Artificial Neural Networks, Cambridge: 268-273. Makhlin, A.G., 1981, Using vision in electric motor assembly, Assembly Automation, May: 144-148 Makino, H., Arai, T., 1994, New developments in assembly systems, Annals of the CIRP, Vo1.4312. 501-511. Masayuki. K., Koji, 0.. Nuio, T., Koji, I.. 1996, Hole position and inclination sensor with 2D PSD, and its application to automatic shaft inserting, IEEE Int. Conf. on Multisensor Fusion and Integration for Intelligent Systems, Piscataway, NJ, USA: 78-85 Mc Walter, K., Machine Vision Boosts PCB Assembly Processes, 1993, Assembly Automation, vo1.13, n. 2: 32-34 Mengel, P., Roth, N., Schwarz, P., 1992, Application of Advanced Data Processing Technology for Integrated Inspection in Electrotechnical Assembly, Annals of the CIRP, vol. 4111: 29-32 Milberg, J., Maier, C., submitted by Warnecke. H.J., 1985, Contribution to the Automation of Screwdriving with the Aid of Industrial Robots, Annals of the CIRP, VOI.3411: 49-52 Milberg, J.. Riese, K., 1988, Assembly of Panel Fasteners Using Robots, Annals of the CIRP, vol. 3711 17-20 Milberg. J., Schmidt, M., 1990, Flexible Assembly Systems - Opportunities and Challenge for Economic Production, Annals of the CIRP, vol. 39/1: 5-8 Milberg, J., Wisbacher, J., 1992, Acoustic Test Procedures - A Powerful Method for Quality Assurance and Process Monitoring in Assembly, Annals of the CIRP, vol. 41/1: 25-28 Milberg, J., Rockland, M., 1993, Flexible, Modular Systems for Assembly and Commissioning, 1: Production Engineering, vol. 1 1 139-142 Milberg, J., Reinhart, G., Lofler T., 1997, Audio monitoring of joining processes, Production Engineering, Vol.IV12. Milutinovic. D., submitted by Milacic, V., 1996, Micro Scara Robot as Universal*Adaptive Compliant Wrist, Annals of the CIRP, vol. 4511: 31-34 Miura, J., Ikeuchi, K., 1995, Assembly of flexible objects without analytical models, IEEE International Conference on tntelligent Robots and Systems, vol.

522

Keynote Papers

2, IEEE, Piscataway, NJ, USA: 77-82 92. Morris, K. A., 1985, Application of Tactile Sensors in Assembly, Lord Library of Technical Articles, LL1205 93. Morris, K.A., 1992, Sensor-Based Robotic Spool Valve Assembly, Robotics World, June 94. Nakagaki, H., Kitagaki, K., Ogasawara, T., Tsukune, H., 1996, Study of insertion task of a flexible wire into a hole by using visual tracking observed by stereo vision, IEEE Int. Conf. on Robotics and Automation, vol. 4, Piscataway, NJ. USA: 3209-3214 95. Nguyen, C.C., Antrazi, S.S., Zhou. Z., 1991, Analysis and implementation of a 6 DOF Stewart platformbased force sensor for passive compliant robotic assembly, IEEE SOUTHEASTCON, Piscataway, NJ, USA, VOI. 880-884 2: 96. Niemeier, J., Seliger, G., 1997, Process Model for Single Point Soldering, 29th CIRP International Seminar on Manufacturing Systems 97. N.N., 1983, GADFLY - the answer to electronic component assembly, Assembly Automation, February: 20-22 98. N.N.. 1990, Boost to SMD placement performance, Sensor review, April: 53 99. N.N., 1990, Robotic odd-form placement brings its rewards, Assembly Automation, November: 174-176 100. N.N., 1995, Ultrasonics Motor Clamp Load, Assembly, October: 23 101, N.N., 1997, Developing Products for Manufacturing Productivity, brochure of AT1 Industrial Automation, Garner NC. 102. N.N.. 1997, Automation Components, brochure of Sommer Automatic. Straubenhardt (D!. 103. N.N., 1997, Gripping Systems, brochure of Schunk GmbH, Lauffen (D). 104. N.N.. 1997, Quick Vision, 3 0 Optical Measuring Systems, brochure of Mitutoyo. 105. N.N., Pressing Technique,-brochureof Robert Bosch GmbH 106. N.N., Torque Angle Controller with Electric Drive, brochure of Rotor Tool, Cleveland 107. N.N., Macs Plus Assembly System, brochure of Atlas Copco. 108. N.N., Miniature LVDT, brochure of RDP Electronics Ltd. 109. N.N., Laser Displacement Sensors, brochure of Keyence Cow. 110. Nuttin, M. Van Brussel, H. Peirs, J., Soembagijo, AS., Sonck, S. 1995, Learning the Peg-into-hole Assembly Operation with a Connctionist Reinforcement Technique, Proc of 2nd Int. Workshop on Learning in Intelligent Manufacturing Systems, Budapest: 335-357. 111. Oh, S. R.. Hollis, R. L., Salcudean, S. E., 1993, Precision assembly with a magnetically levitated wrist, IEEE Int. Conference on Robotics and Automation, vol. 1, Los Alamitos, CA, USA: 127-133 112. Ohkura, K., Goto, H., Imanaka, K., 1995, Miniature scanning type of in-line shape-matching sensor, Proceedings of the Society of Photo-Optical Instrumentation Engineers, vol. 2595, SPIE,

Bellingham, WA. USA: 74-79 113. Page, C.J., Hassan. H.. The Orientation of Difficult Components for Automatic Assembly, Robot Sensors 114. Pham, D.T., submitted by Chisholm, A.W.J., 1992, An Intelligent Sensor for Robotics, Annals of the CIRP, VOI. 4111: 501-504 115. Pfeifer. T., Schmitt, R., 1993, Robot assisted operation of flexibly automated measuring and test equipment, 3'' Int. Symp. On Measurement and Control in Robotics, Sept. 21-24, Torino. 116. Qiao, H., Dalay, B:S:. Parkin. R.M., 1993, Robotic peg-hole insertion operations using a six-component force sensor, Proc. of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, vol. 207, n. 5: 289-306 117. Reid, K.J., 1995, Automated Inspection For Electronics Assembly, Assembly, January: 20-22 118. Reinhart, G.. Hoehn, M., submitted by Milberg, J., 1997, Growth into Miniaturization - Flexible Microassembly Automation, Annals of the CIRP, vol. 4611 7-10 119. Romiti, A.. Belforte. G., D'Alfio, N., 1982, Precise Assembly by Low Precision Machines, Notiziario Tecnico Amma, n. 10 120. Romiti, A,, Sorli, M., 1992, Force and moment measurement on a robotic assembly hand, Sensors and actuators, Physical, vol. 3211-3, April: 531-538 121. Roth, N., Guenther, K.G., Rummel, P., Beutel, W., 1989, Model Generation for Sensor-Guided Flexible Assembly Systems, Annals of the CIRP, vol. 3811: 5-8 122. Roth, N.. Mengel, P., 1992. Sensor integration getting the whole picture, Sensor review, vol. 12, n. 1 28-33 123. Roy, D., Deb, S.R., 1995, Stochastic model and performance measure of a multi-sensory system for robotized assembly operation, IEEE International Conference on Industrial Automation and Control, Piscataway, NJ. USA: 285-290 124. Rueb. K.D., Wong, A.K.C., 1988, Knowledge-Based Visual Part Identification and Location in a Robot Workcell, International Journal of Machining. Tools and Manufacturing, vol. 28. n. 3: 235-249 125. Ryu, Y. K., Cho. H. S.. 1996. Visual inspection scheme for use in optical solder joint inspection system, IEEE International Conference on Robotics and Automation, vol. 4, Piscataway, NJ, USA: 3259 126. Schuler, H., Hsieh, L.H., Seliger, G., 1994, A Flexible Laser Scanner System for On-Line Process Monitoring, International Symposium on Manufacturing Science and Technology 127. Schuler, H., Hsieh, L.H., Seliger, G., 1994, Computer-Aided Planning System for Multiple LaserScanner Applications, Int. Symp. on Laser Assisted Net shape Engineering, vol. 2: 523-532 128. Schweigert, U., 1992, Sensor-guided assembly, Sensor review, vol. 12, n. 4: 23-27 129. Schweizer. M.,Scholpp, C., Robot Technology in the Clothing Industry 130. Schweizer, M., Scholpp, C., Robot Technology in the

523

Wiring Industry 131. Schweizer, M., Hoerz, T., Robot Technology in the Woodworking Industry 132. Seliger, G., Hsien, L.H., 1991, Sensor aided programming and movement adaptation for robot guided deburring of castings, Annals of the CIRP, VO1.40/1, 487-490. 133. Seneviratne, L., 1996, On the use of mechatronics for intelligent screw insertions, IEEE Symposium on Emerging Technologies and Factory Automation, vol. 1, IEEE, Piscataway, NJ, USA: 193-198 134. Simons, J., Van Brussel, H. De Schutter, J. Verhaert, J., 1982, A Self-Learning Automaton with Variable Resolution for High Precision Assembly by Industrial Robots, IEEE Trans. on Automatic Control, AC(27), October. 135. Snyder, J., 1996, Seeing the Light, Assembly, August: 28-32 136. Son, C., Ahn, T., Sensor fusion techniques used for learninghdentifying features of robot's part assembly tasks, Proceedings of the Institution of Mechanical Engineers, Journal of Engineering Manufacture, vol. 210, n. 85: 429-435 137. Swift, K.G., Dewhurst, R.J., 1983. A flexible system handles engineering components, Assembly Automation, May: 86-89 138. Tian-Soon, S.. Ang, M.H.jr, Kah-Bin. L., 1997, A Compliant End-Effector Coupling for Vertical Assembly: Design and Evaluation, Robotics & CIM, vol. 13, n. 1: 21-30 139. Tonko, M., Schafer, K., Heimes, F.. Nagel, H. H., 1997, Towards visually servoed manipulation of car engine parts, IEEE Int. Conf. on Robotics and Automation, vol. 4, Piscataway: 3166-3171 140. Tonshoff, H.K., Janocha, H., Seidel, M., 1988, Image Processing in a Production Environment, Annals of the CIRP, vol. 37/2: 579-590 141. Tdnshoff, H.K., Seidel, M., Greulich, J.C., 1992, A Triangulation Sensor Measuring 5 Degrees of Freedom for Contour Tracking and Automatic Programming of Industrial Robots, Vision & Voice Magazine, vol. 6, n. 4: 279-282 142. Tonshoff. H.K., Soehner, C., Isensee. G., 1997, Vision-Guided Tripod Material Transport System for the Packaging Industry, Robotics & ComputerIntegrated Manufacturing, vol. 13, n. 1: 1-7 143. Tanshoff, H.K., private communication. 144. Tsusaka, Y.. Koide, M.. Tanaka, M., Nomura, H., Kuno, T., Nagamatsu, S., Akao, N., 1995, Development of a fast assembly robot arm with joint torque sensory feedback control, IEEE Int. Conf. on Robotics and Automation, Piscataway, NJ, USA, v01.3: 2230-2235 145. Van Brussel, H., Simons, J., submitted by Peters, J., 1979, Robot Assembly by Active Force Feedback Accommodation, Annals of the CIRP, vol. 28/1: 397401 146. Wallack, A., Canny, J.F.. Manocha, D., 1993, Object localization using crossbeam sensing, IEEE International Conference on Robotics and Automation, vol. 1, IEEE, Los Alamitos: 692-699

147. Wallster, O., 1995, OptiMaster robot guidance system, Sensor Review, vol. 15, n. 2: 23-26 148. Wapenhans, H., Seyfferth, W., Pfeiffer. F., 1993, Robotic force control for flexible assembly, IEEE International Conference on Robotics and Automation, Piscataway, NJ, USA, vol. 1: 225-231 149. Warnecke, H.J., 1978, Assembly systems now and in future Research and Development, Annals of the CIRP, VOI.27/21597-605 150. Warnecke, H.J., Pavel, G., Kuhn, G., 1981, Optical Testing of Bolt-Type Screws, Annals of the CIRP, VOI. : 461-466 3011 151. Warnecke, H.J., Abele, E., Walther, J., Fischer, G., 1985, Investigations of the Screw Driving Process with Sensor-Controlled Industrial Robots, Annals of the CIRP, vol. 34/1: 41-44 152. Warnecke, H.J., Domm, M.E., 1989, Assembly by Industrial Robots with CAD/CAM and Vision Control Periphery, Annals of the CIRP, vol. 38/1: 41-44 153. Warnecke, H.J., Schraft, R.D., Schweizer, M., Wurtz, G., 1990. Automating the interference fit process, Assembly Automation, November: 195-199 154. Weck M., Dern U., 1985. Integrated Manufacturing and Assembly, Annals of the CIRP, vol. 3411: 45-48 155. Weck, M., Etscheidt, K.. 1993, Avoiding Teach-In by Using CAD-Data for Model-Based Recognition of Complex 3D-Objects, Production Engineering, vol. 1/1: 167-170 156. Weck, M., private communications. 157. Wennstroem, L., Martensson, N., 1988, ForcelTorque Sensing for Monitoring and Programming of Assembly Operations, Annals of the CIRP. VOI.37/1: 21-24 158. West, A.A., Williams, D.J., Hinde, C.J., 1995, Application of intelligent control paradigms to real manufacturing processes, Proc. of the Institution of Mechanical Engineers, Part 1: Journal of Systems and Control Engineering, vol. 209: 293-308 159. Xu, Y., Paul, R.P., 1992, Robotic instrumented compliant wrist, Journal of Engineering for Industry, ASME Trans, vol. 114, n. 1. February: 120-123 160. Xu, Y., Paul, R.P., 1992, A robot compliant wrist system for automated assembly, IEEE International Conference on Robotics and Automation, Los Alamitos, CA, USA: 1750-1755 161. Yuen, K., Bone, G. M.. 1996, Robotic assembly of flexible sheet metal parts, 13th IEEE International Conference on Robotics and Automation, vol. 2, IEEE, Piscataway, NJ, USA: 1511-1516 162. Zenger, D., Dewhurst, P.. submitted by Boothroyd, G.. 1984, Automatic Handling of Parts for Robot Assembly, Annals of the CIRP. vol. 33/1: 279-281 163. Zupancic. J., 1990, A calibration technique in robotic assembly of hybrid circuits, 8th IEMT International Electronic Manufacturing Technology Conference, IEEE, Piscataway, NJ, USA: 137-146

524