The Ready Application of Neural Networks

October 28th, 2002 4:00 to 5:30 p.m.

Presented by: William E. Lutz WEL Associates www.welassociates.com

This paper is derived from a presentation given at the Chicago October 2002 URISA Urban Regional Information Systems Association conference.

Introduction
With the advent of microprocessors and, more importantly, the means to universally distribute information (via HTML, Perl, etc.) through the Internet, data now becomes something that can be reviewed, sorted and transported rapidly; thus the rise of neural networks. With greater processing and distributing speed, developers now view intelligence not so much as a matter of larger processors, but rather that of faster and more network distribution over wider areas. As of this writing, cost effective neural network software exists (particularly for mid- to larger sized agencies) automatically reviewing, sorting and analyzing street reportage (arrest / report records) or other relevant material employed in a wide variety of services. When queried, neural networks can automatically review and determine patterns, activities and delineate trends that were not easily recognized all in mere seconds. Neural networks represent a fundamental shift in the way organizations operate. Planners and other professionals must now come to recognize the potentiality and power that neural networks offer, embracing and mastering these developments while defining our roles in regards to these powerful tools.

What are Neural Networks?

Neural networks differ radically from regular search engines, which employ Boolean logic. Search engines are poor relatives to neural networks. For example, a user enters a keyword or term into a text field such as the word cat. The typical search engine then searches for documents containing the word cat. The search engine simply searches for the occurrence of the search term in a document, regardless of how the term is used or the context in which the user is interested in the term cat, rendering the effectiveness of the information delivered minimal. Keyword engines do little but seek words - which ultimately becomes very manually intensive, requiring users to continually manage and update keyword associations or topics such as cat = tiger = feline or cat is 90% feline, 10% furry. Keyword search methodologies rely heavily on user sophistication to enter queries in fairly complex and specific language and to continue doing so until the desired file is obtained. Thus, standard keyword searching does not qualify as neural networks, for neural networks go beyond by matching the concepts and learning, through user interface, what it is a user will generally seek. Neural networks learn to understand users interest or expertise by extracting key ideas from the information a user accesses on a regular basis. A detective assigned to burglaries, for example, would receive

automatic notifications of any new patterns developing within the last 24 hours from a trained neural network; similarly, that same detective who would conduct their inquiries would work with a neural network that knows what kind of information that particular detective seeks when any new queries are entered. Neural Networks are an evolutionary stage in computer development involving creating advanced, dynamic relationships between users and data. Once the domain of high-end users, neural networks are increasingly being offered to local and regional government entities at costeffective levels. Neural networks try to imitate human mental processes by creating connections between computer processors in a manner similar to brain neurons. How the neural networks are designed and the weight (by type or relevancy) of the connections determines the output. Neural networks are digital in nature and function upon pre-determined mathematical models (although there are on-efforts underway for biological computer networks using biological material as opposed to hard circuitry). Neural networks work best when drawing upon large and/or multiple databases within the context of fast telecommunications platforms. Neural networks are statistically modeled to establish relationships between inputs and the appropriate output, creating electronic mechanisms similar to human brain neurons. The resulting mathematical models are implemented in ready to install software packages to provide human-like learning, allowing analysis to take place.

Neural Networks vs. Artificial Intelligence

Neural networks differ from artificial intelligence; the two should not be confused with one another. Prior to the 1990s, the idea of artificial intelligences were primarily the domain of larger entities. Starting with corporate and governmental consortiums lead by such notable figures as retired Admiral Billy Ray Inman (formerly the Director of the National Security Agency) leading to the successful effort by International Business Machines (the creation of Big Blue), the search for advanced computerized processing meaningfully and truly reflecting human intelligence has been one of great difficulty and challenge and still largely remains the domain of science fiction.

Neural Networks: Can simulate choices based upon pre-determined values. Can suggest patterns and offer active linkages to other relevant data sources. Can learn users preferences based upon set pre-conditions and desired routines. Cannot pass the Turing Test An Artificial Intelligence: Cannot act as a fully cognitive entity. Cannot act outside of its programming framework. Cannot expand or change its inherent programming dynamics. Could (theoretically) pass the Turing Test

What is the Turing Test?

Dr. Alan Turing (1912 to 1954) was a British mathematician whose role in helping the Allies defeat the Nazis by creating a series of algorhythms cracking the Nazi Enigma coding machine was invaluable. During his tenure, Dr. Turing theorized upon the possibility of an intelligent machine and designed the Turing Test. The Turing Test involves a person and a computer engaging in casual conversation. If, by the nature of the conversation taking place, the person cannot tell whether or not it is a machine or a person responding back, arguably, that machine is intelligent. Neural networks do not fall into this category of passing the Turing Test.

There are presently a wide variety of functional neural networks. By definition, neural networks primarily serve as reliable pattern recognition engines and powerful subject classifiers, with the ability to generalize (within statistically accepted norms) in making decisions about imprecise input data. Neural networks are often classified by their method of learning, as some employ supervised training while others are referred to as unsupervised or self-organizing. Supervised training is akin to a student guided by an instructor while unsupervised algorithms cluster data into similar groups based on the measured attributes serving as algorithmic inputs in this case, the student learning totally on their own. Neural networks offer ideal solutions to a variety of classification problems such as speech, character (written and/or print) and signal recognition, as well as functional prediction and system modeling where the physical processes are not understood or are highly complex. Neural networks offer many advantages chief among them their resilience against distortions in the input data and their capability of learning. Neural networks represent a revolutionary method of programming - and deploying - computing processing. Neural networks are exceptionally good at performing pattern recognition and other tasks difficult to program using conventional techniques. Programs employing neural nets are capable of learning on their own and can adapt to changing conditions something which any busy network administrator can well relate to.

Historical Background on Neural Networks

Following McCullock and Pitts groundbreaking work, in 1949 the mathematician Claude Shannon determined the means by which information can be quantified by its value within the context of any given communication. Shannons published argument of the Entropy or Measurement of Uncertainty illustrates the less frequently a unit of communication occurs, the more information it conveys. Ideas communicated less frequently within the context of any message tend to be more indicative of the overall messages meaning. Shannons theory not only aided the telecommunications industry by expanding upon the notion of trunking communications, but also greatly contributed to neural networks functionality by allowing any neural network to readily determine the most important (or informative) concepts within a document or a whole series of documents. But Shannons work alone was not the groundbreaking work contributing to the development of neural networks.

Neural networks (in theory) existed since the 1950's, but it wasn't until the mid-1980's that algorithms became sophisticated enough for real neural network applications.
1943

McCulloch and Pitts groundbreaking work, A Logical Calculus of the Ideas Immanent in Nervous Activity lay the theoretical groundwork for neural network processing. 1950 - International Business Machine research efforts in advanced computing link academic institutions to private research; this model became the basis of future research and largely, the standard operational method by which additional advanced computing research is carried out. 1958 - The Perceptron model was developed, becoming the primary basis by which adoptive mathematical models mimicked the basic processing functionality of the human neuron. 1960 - ADALINE (ADAptive LInear Element) developed by Bernard Widrow and Marcian Hoff. Also known as the least-mean-squares rule, ADALINE minimises output error, enabling a neural network to learn and develop self-correctional ability as it functions. 1969 - Minsky of MIT and Papert question the entire concept of neural networks viability; development is slowed and the concept is thrown into ill-repute. At this time, computational hardware plateau, and thus it was not until the arrival of more advanced processing in the coming decades did neural network research resume in strength. 1985 With enhanced computing processing resulting in advanced microchips, ART (Adaptive Resonance Theory invented by Stephen Grossberg in 1976) neural networks attain sophisticated algorythms allowing for more stronger approximations to biological neuron learning models, thus paving the way for practical neural network applicability. 2003 - Neural networks are now regarded as cutting edge technology with viable commercial applications.

Neural Networks and the Human Mind

Neural networks mathematically emulate observed properties of biological nervous systems, principally drawing upon the analogies of adaptive learning. What makes neural networks unique is their information processing system structure, composed of a large number of highly interconnected processing elements (networks analogous to neurons) and are wired together (routers and connectors analogous to human brain synapses). Like the learning biological systems undertake by adjustments to the synaptic connections existing between neurons, so too do neural networks learn. And, like biological nervous systems, learning occurs by example through training, or exposure to input/output data whereby the training algorithm iteratively adjusts the connection weights (synapses). Like human minds, neural networks learn reinforced patterns through repeated behavior.

Specific aspects of a human neuron, identifying its functionalities, suggesting potential mathematical / mechanical replication.

Input is received

Dendrites (Routers)

Soma / Core (Processor)

Combine / Connects / Processes Input Connects with other processors

Axom (Output to User Interface)

Synapses (Routers)

Processes results into outputs

Neuron Processing Model: Biological and (Hardware) Function Comparisons

How Neural Networks Work

Based on the preponderance of one pattern over another in unstructured information, neural networks understand that there is X% of probability that an item in question regards a specific subject. The network then extracts a documents digital essence, encodes the unique signature of the key concepts, allowing a host of operations to be performed on that item. The highperformance pattern-matching algorithms providing the sophisticated contextual analysis and concept extraction automate the categorization and cross-referencing of information; this, in turn, improves the efficiency of information retrieval. Neural networks enable an automatic understanding of a page of text, web pages, e-mails, voice, documents and people while automating file / data operations. Artificial / electronic neurons are similar, but far more basic and inherently less sophisticated than biological neurons. In the diagram on the left, neural network inputs are represented by x(n), with each input multiplied by a connection weight (data value) represented by w(n). Inputs are processed, fed through a pre-determined transfer function to generate a result, and then sent out. Neural network types determine network design and how the results are processed. (Source: Saint Louis University School of Business and Administration)

In this chart of a Perceptron neural network model (on the left), we transfer the electronic neuron into a template, viewing an individual electronic neural in a two-dimensional design.

Note the structural similarities between the biological and mathematical schematics, as the electronic model seeks to mimic the biological in basic functionality.

Functional neural networks are multilayered three dimensionally, better approximating the human brain both in terms of function while adding more sophistication in terms of being able to selflearn through feedback and correction. Referring back to the Perceptron model, we dimensionally expand the model to add more dimensionality and hence, increase connectivity and potential computing power. Within neural networks, computing power is largely pre-determined by the amount of connectivity and speed between connections. In one recently celebrated event, 30,000 desktop PCs were linked together to track proteins within human bodies, breaking down the calculating time from 30 years to just two years (UPI, October 20th, 2002, Nature). Similar efforts take place with international volunteers allowing SETI (Search for Extraterrestrial Intelligence) to link users worldwide via their desktop, underscoring the power of distributive computing.

Common Neural Network Types and Typographies

Neural networks are generally divided into three (3) categories. Neural network software emulation packages using statistics based upon specific computing models and network typographies emulating human neurological activity are the primary topic of this discussion. Neuromorphic Systems are concerned with real-time implementations of neurally inspired systems, implemented directly in silicon, for sensory and motor tasks, while Biological Neural Networks is the science literally focused upon developing biological computers. The latter two are higher level and far less likely to be found in most agencies, while the first can be readily obtained through ready to purchase computer software.

Below, we view four (4) primary neural network typographies:

Type Perceptron Multi-LayerPerceptron Backpropagation Net Hopfield Net Kohonen

Used in: simple logical operations and pattern classifications complex logical operations and pattern classifications. speech analysis pattern association and optimization problems. optimization problems and simulations

Perceptron

Hopfield

Back Propagation

Kohonen

Neural network typographies possess inherent abilities ideally suited to meet specific functions with most applications of neural networks falling into the following five categories: Classification: Inputted values are reviewed to determine classification such as reviewing video data pictures. Pictures / images are reviewed for matching specific types and for data filing / classification. Data Association: Similar to Classification, Data Association recognizes data containing errors, not only identifying scanned characters but notifying when scanners are not working properly. Data Conceptualization: Inputs are analyzed so that relationships can be inferred, such as extracting the names of those most likely to buy a particular product or are involved in criminal enterprises. Data Filtering: Used for input signal enhancement i.e., noise removal from telephone signals or identify specific voice tones for classification and future retrieval; also used in voice recognition systems. Prediction: Inputted values are reviewed for predictive determination - such as choosing investments, weather predictions or identify people with specific health risks.

What Neural Networks Mean for the Planning / Governmental Professional

The focus of neural networks is not to supercede the role of the professional, but rather to enhance analysis. Within a context of a neural network, patterns can emerge and activities reviewed, but if only we knew what is happening from the wide range of data routinely entered on a daily basis. Many reports, incidents or other forms of data entry, storage and retrieval can be superfluous. What a neural network would do is to eliminate the clutter and sort through the mundane to seek out the extraordinary or to seek out a specific set of data based upon predetermined patterns. A neural network, for example, would be able to cull from the vast data a series of burglaries occurring over a specified time span involving the same modus operandi and specifically aimed toward victims sharing the same characteristics automatically.

Neural networks adaptive probabilistic concept modeling analyzes correlations between features found in items relevant to a query or profile, finding new concepts and item. Concepts important to sets of files can be determined, allowing new files to be accurately classified. Neural networks function within the context of fetches that is, software / hardware / communication interfaces accessing other databases. Other fetches include interfacing with large-scale databases (such as Oracle) or GIS routines or mass-media sources. Neural networks are designed to access a multitude of databases and files over wide areas; the greater access a neural network has, the more powerful that network becomes.

Practical Applications of Neural Networks

Neural network systems are designed to access hard copy/document imaged electronic files, emails, internet files and recorded calls (converted to data input) for review. Owing to time and limited resources, many data sources previously before not considered are now fair game. Neural networks software packages are now available after undergoing trials, acceptance and usage at several federal agencies. Total cost for such systems are based upon the amount of data involved, number of processors involved and the desired fetches or data processing routines sought, with costs usually ranging between approximately $275,000 to $400,000 for midsize to larger agencies (pricing naturally varies). Interfacing with powerful databases (such as Oracle) and electronically scanned imaged files (as well as GIS enabling automated mapping review), neural networks yield greater and faster - returns while dramatically transforming the role of the crime analyst. What neural networks offer at a practical level is the ability to conduct far more analysis at far greater efficiency across a larger scope. Some usages for neural networks involve: Quality Control Financial Forecasting Economic Forecasting Credit Rating Speech & Pattern Recognition Biomedical Instrumentation Process Modeling & Management Laboratory Research * Oil & Gas Exploration Health Care Cost Reduction Targeted Marketing Defense Contracting Bankruptcy Prediction Machine Diagnostics Securities Trading Some commercial neural network users are: MasterCard Visa American Express Thrifty Car Rental WalMart Gannett Media Some governmental agencies presently employing neural networks are: National Security Agency Federal Bureau of Investigations Defense Intelligence Agency Central Intelligence Agency

Some federal applications of neural networks involve: Military / Defense: War Games Anti-Terrorist Initiatives Long-term Economic forecasting Large-scale transportation railroad controls Utility / power grid system management Airline traffic oversight Political role playing / Scenarios Strategic analysis encompassing all of the above Neural networks are not solely for larger commercial and federal agencies. Some suggested state applications for neural networks can involve: Income tax / revenue forecasting Labor market / job pool determinations Highway / transportation network cost management Weather determinations (high snows raising the demand for more road salt, hurricanes impacting against shorelines, rain leading to flooding, etc.) And, some suggested applications of local / county government can entail: Population studies Aging localities Income level shifts Demographic / Ethnicity shifts Redevelopment / Development Targeting likely areas for redevelopment Calculating future assessment values on an individual block level Merging census data and other sources for extrapolation Calls for Services: Efficiency studies Fire / Emergency Medical Services Police Services Public Works / Road Construction Tax revenue distribution Neural networks are increasingly employed by law enforcement. In the Camden / Philadelphia HIDTA (High Intensity Drug Trafficking Agency), efforts are underway enhancing existing networks toward cross-referencing criminal records, case files and other relevant crime information into a region encompassing three states, over a dozen counties and a multitude of local jurisdictions. For law enforcement, neural networks are a boon, readily accessing:

NCIC / SCIC (National Crime Information Center and State Crime Information Center) enhanced
access Experian / Equifax / credit bureau review Regional (such as the Philadelphia / Camden HIDTA High Intensity Drug Trafficking) network enhancement Probation records (monitoring criminal recidivists) Federal, state and local court records Interpol / international data services Inter-state record records (such as MAGLOGEN) On-Line Internet databases

Similarly, at the municipal level, other records can be accessed: Zoning and Planning Tax Assessors Office Finance / Grants / Procurement Redevelopment (Brownfields and Tax Title Liens) Fire and Police Court County, State and Federal Services

Neural networks are potentially powerful means of managing and coordinating large-scale and multi-jurisdictional GIS / mapping databases, merging previously disparate databases into cohesive services allowing for large-scale automated data retrieval and/or GIS routines.

Cost savings also play an important role when considering neural networks. Neural networks offer a powerful means to initiate cost savings while enhancing existing network service routines. Neural networks do not require the introduction of large and expensive big iron mainframes; rather, they ideally function within existing distributive network environments. With selected hardware installations and upgrades, medium to largescale / regional / local governments can enjoy the power of neural networking.

Neural networks, in conjunction with Intranets, offer potent new means to distribute information universally across many platforms. Using neural networks accessed through Intranet platforms, users can access, review and obtain vast stores of information whether they are in the field or in their offices that otherwise would be time and/or resource consuming.

But just as neural connectivity is vital, so too does network speed play a large part in determining neural networking capability. Telecommunications advancements benefit neural networking development and applications, more readily allowing for wide-area systems encompassing both private and public informational sources and processing services. Neural networks could be accessed from work, home or from wireless services. The potential for applications and access is only limited to the budgetary constraints and defined by need. In some situations, shared cost servicing offers viable means of both ensuring distributive services and equal database access. Additionally, the power of voice recognition routines offer a means to both issue commands as well as access recorded messages across many mediums.Neural Networks are not artificial intelligence, but the foundations for more advanced computing. Despite such developments as IBMs Big Blue, we are still short of achieving Turings ideal of artificial intelligence. Nevertheless, neural networks represent an important step in computer evolution.

(Picture courtesy of MGM studios, from the film 2001: A Space Odyssey)

About the Author

William E. Lutz is an active member of URISA and also participates as a Publications Committeeman of the International Association of Crime Analysts (IACA) and the International Association of Computing Consultants (ICCA), and appeared at various conferences and locales - among them, Victoria Arson Squad, Melbourne, Australia; MapWorld 2000, San Antonio; U.S. Department of Housing and Urban Development (U.S. Office of Urban Empowerment Zones) and St. Josephs University in Philadelphia (criminal justice course on basic crime mapping techniques) as well as the June 2002 Delaware Valley Regional Conference on Anti-Terrorism. Mr. Lutz has been interviewed by multiple media outlets on specific issues, inclusive of The Economist of London (Urban revitalization); New Jersey Network News (Wireless / Internet crime mapping systems); Dateline NBC (The 911 / Eddie Polec beating incident of Philadelphia); The Philadelphia Inquirer (regional crime mapping systems) and Government Procurement magazine (drafting technology specifications), with over eighty published articles to his credit and has over fifteen years of governmental service.

Suggested Reading and Contacts

Avelino J. Gonzalez & Douglas D. Dankel, "The Engineering of Knowledge-based Systems", 1993 Prentice-Hall Inc. Fatemeh Zahedi, "Intelligent Systems for Business: Expert Systems with Neural networks, 1993 Wadsworth Inc. Haykin Simon, "Neural Networks", 1994 Macmillan College Publishing Company Inc. Autonomy, Inc.: www.autonomy.com International Business Machines (IBM): www.ibm.com SER Brainwave: www.sersolutions.com Visual Analytics: www.visualanalytics.com