The Topological Skeleton of Cellular Automaton Dynamics

Howard Gutowitz1 2 and Christophe Domain1 1 Ecole Superieure de Physique et Chimie Industrielles 10 rue Vauquelin 75005 Paris and 2 The Santa Fe Institute 1399 Hyde Park Road Santa Fe, NM 87501 April 10, 1996
;

We have developed statistical techniques to study the structure the state-transition graphs of cellular automata with periodic boundary conditions, in the limit of large system size. We organize our results around the concept of a topological skeleton. The topological skeleton is the set of physically relevant states. Covering this skeleton is a surface, typically thin and dense, which contains the bulk of the set of states. States in the skeleton have some long histories. States on the surface, by contrast, have only short histories; they are reached only near the beginning of cellular automaton evolution. We study in detail a sequence of rules which exhibit mostly skeletal to mostly surface structure.

Abstract

1 Introduction
Consider a cellular automaton operating on a periodic lattice of cells. If the number of cells in the lattice is s, and the number of cell states is k, then there are ks possible lattice con gurations. Each con guration maps to a new con guration under the action of the cellular automaton rule. Thus, we think of each con guration as a node in a graph, with an out-going arc leading to its successor con guration. As the number of possible con gurations is nite, any initial con guration must map eventually to a temporally periodic cycle. Under an irreversible rule some con gurations have more than one possible ancestor, so the evolution cannot be run deterministically backward in time. For these rules the state space (the set of con gurations on the lattice) is organized as a collection of cycles, with a forest of transient trees rooted on each cycle. The leaves of the trees are con gurations which have no preimages, the so-called Gardens of Eden 4]. In the opposite extreme, con gurations on cycles have preimages in nitely far back in time. Con gurations on the branches of the trees have an intermediate maximum-length history. In this paper we characterize cellular automata by the shape of their transient trees. We seek to describe these shapes in the limit of system size s tending to in nity.

Background. Study of the topology of the state-transition graphs of cellu-

lar automata in nite geometries has been largely limited to numerical work. However, some analytical results concerning the set of temporal cycles have been obtained in a sequence of papers by Jen 23, 24, 25] for some large classes of rules, and for various rules with special characteristics e.g. 29]. In addition, McIntosh 30] has collected a variety of mathematical techniques, which serve, for example, to count one-time-step preimages of small con gurations. Published numerical work on state-transition graphs mostly concentrates on lattices small enough that complete enumeration of all possible trees and cycles is possible. Typically, such enumeration is limited by practical considerations to lattices of size 20 or less. For example, the appendices to 34] contain catalogs of state-transition diagrams, counts on the number of cycles etc., for nearest-neighbor automata. These catalogs have been extended in 38]. Some authors 26, 22, 32] have attempted to extrapolate data on smallsystem cycles and transients to describe in nite-size behavior. Such extrap2

olation is di cult, since the structure of state-transition diagrams tends to depend sensitively on system size, especially when the system size is small. Some numerical investigation has been performed on cellular automata operating on systems larger than those which can be attacked by complete enumeration. For instance, Grassberger 18] studied transient times in rule 22. He found de nite non-statistical uctuations in transient times as a function of system size. Nonetheless, he was able to convincingly establish that transients grow exponentially with system size for this rule. Sinha and Jayaprakash 22] claim to have observed both exponential and power-law growth in transient times for a variety of rules, but their simulations are not su cient to quantitatively estimate the exponents involved. Additional work along these lines is reported in 34, 12, 14, 26, 27]. oncile two sets of results which may seem in contradiction. On one hand, it has been shown 41] that for most rules the probability that a randomly selected con guration is a Garden of Eden con guration ( i.e. has no preimages) rapidly approaches 1 as the size of the con guration increases. This suggests that state-transition graphs typically contain highly branched transient trees of low depth. Such a structure would have most of its mass in the leaves at the extremities of the graph. On the other hand, the length of the transient trees and cycles increases exponentially with system size for some typical rules 12, 18]. This suggests a structure with most of its mass in the interior, in the branches and roots. The topological skeleton hypothesis holds that, in the limit of large system size, the state-transition graph typically consists of an (exponentially long) skeleton of con gurations with long possible histories, covered by an (exponentially thin and dense) surface consisting of con gurations with vanishing histories. This distinction between skeleton and surface is de ned in terms of the length of histories. To explore this issue, we must have a method to measure the length of the possible histories of a con guration. To this end, we have developed a technique for statistical inverse iteration of irreversible cellular automata. This technique is an extension of that proposed by Wuensche 38], modi ed for e ciency, and to take account of the physical properties of cellular automata. 3

Topological Skeleton. The topological skeleton concept is meant to rec-

we summarize previously published information concerning the rules under study (section 2). Our results are presented in three parts, the rst (section 3) begins our study by means of some usual methods for presenting cellular automata and the corresponding state-transitions graphs. We examine transient lengths on nite and in nite systems. Then (section 4) we concentrate on measuring the extreme surface of these graphs. The third part (section 5) describes our inverse iteration method, and some of the results we have obtained with it. Finally, (section 6) we present some general conclusions and perspectives.

Organization of the paper. After a general introduction (section 1)

2 The Rules Studied

We have chosen to study in detail a collection of one-dimensional rules with two states per cell, and nearest-neighbor interactions. By convention, these rules are referred to by their rule numbers 34]. In the six rules chosen for study, 30, 22, 18, 54, 110, and 4, we have a roughly graded sequence from a rule whose state-transition graph is almost purely skeleton (rule 30) to a rule whose state-transition graph is almost purely surface (rule 4). Along this sequence, a number of behavioral properties co-vary with the structure of the state-transition graphs. Near the rule 30 end, rules exhibit generally chaotic behavior on in nite systems. As we approach rule 4, the rules increasingly exhibit properties associated with complex behavior, power-law scaling of various statistics, long-range spatial and temporal correlations, etc. It remains for future work to establish if these suggested relationships between state-transition graph structure and statistical-mechanical properties of in nite systems hold generally in the space of cellular automata. For the present we claim only a device for organizing our results. Let us summarize now what is known about these six rules.

Rule 30. Under rule 30 (001, 100, 010, and 011 ! 1, else 0) the standard measure, that is, the measure which assigns equal probability to all blocks of the same length, is invariant. This measure is stable in that apart from a measure-zero set in the set of measures, all initial measure tend to the standard measure under the action of the rule 28]. Numerical evidence 35] suggests that the random character at the level of the measure on the
4

invariant set has a direct correspondence in the randomness at the level of con gurations. That is, rule 30 generates con gurations with a high degree of temporal and spatial randomness. This randomness has been exploited for cryptographic purposes 37, 31, 13]. 1, else 0) resembles rule 30 in that it has a unique stable invariant measure, called the natural measure 8]. As in the case of rule 30, typical initial measures rapidly approach the natural measure under the action of this rule. However, while the natural measure of rule 30 is trivial, that of rule 22 is highly complex. Though correlations show an overall exponential decay, there are intricate ne-scale deviations, indicating subtle long-range e ects 9, 16]. Rule 22 is a representative chaotic or Wolfram class III rule 36]. As the radius or number of states of cellular automata increase the fraction of rules which are chaotic approaches 1. Hence the behavior of rule 22 should be representative of a large class of rules.

Rule 22. From numerical evidence 18, 43, 11], rule 22 (001,100, and 010 !

Rule 18. Rule 18 (001,100 ! 1, else 0) was one of the rst rules to be care-

fully analyzed as a statistical-mechanical system 16, 15, 28]. The space-time patterns generated by rule 18 are composed of "even" and "odd" domains within which rule 18 behaves like linear rule 90. Eloranta 10] proved that the boundaries between these domains ("defects") execute a random walk. Jen 24] showed that the defects in the patterns generated by rule 18 can be "repaired" so that rule 18 behaves exactly like linear rule 90. Nonetheless, the invariant measure of rule 18 is not uniform ( 16]), as would be the case for a linear rule. Jen's linearization procedure permits some features of the state-transition graph (e.g. temporal periods) to be calculated exactly. Crutch eld and Hanson have developed a technique for extraction of domains and interactions of domains from patterns generated by cellular automata. This has been applied to rule 18, as well as a variety of other rules 20], notably rule 54 21].

Rule 54. The behavior of rule 54 (001,100, 010, and 101 ! 1, else 0) is

transitional between very simple Wolfram class I and II behavior, and chaotic class III behavior. Like class I and II rules, random initial con gurations tend under rule 54 toward simple temporally periodic states. Unlike rules in 5

classes I and II, however, random initial con gurations exhibit long chaotic transients before reaching a periodic state 3]. For small times the behavior of rule 54 resembles that of a robustly class III rule. Rule 54 is a marginal or critical rule in many respects. While it appears to possess a natural measure, this measure is approached very slowly from typical initial conditions. The patterns it produces are described by large coherent regions, punctuated by slowly moving defects. This is re ected both in the correlation functions, and the Lyapunov exponents. A good deal of research treating cellular automata as physical systems focuses on critical behavior, motivating the study of a representative critical rule.

Rule 110. It is widely accepted that rule 110 (001,010,011,101,110 ! 1 else 0) is among the most complex nearest-neighbor rules with two states per cell, perhaps the only one which quali es a member of Wolfram's class IV. It supports a variety of gliders which propagate on a periodic background. These gliders have been cataloged in 36] and 17]. Li and Nordahl 27] have shown in a Monte-Carlo study that the transient times for rule 110 increase as a power law, a result which we con rm here. The ordering of rule 110 with respect to rule 54 is somewhat arbitrary. Both exhibit critical properties, and lie at a margin between ordered and chaotic behavior.
con gurations with only isolated 1's, and is achieved in one time step from any initial con guration. The statistical properties of this rule can thus be completely analytically determined 11].

Rule 4. Rule 4 (010 ! 1, else 0) is a simple rule. The limit set consists of

3 General Features of State-Transition Graphs

The standard presentation of the evolution of a one-dimensional cellular automaton is the space-time diagram ( gure 1). The space-time diagram shows a temporal sequence of con gurations, with cell states represented by colors, here black and white. The initial condition is a randomly generated conguration of length 200, with periodic boundary conditions imposed. Time proceeds downward, for 200 generations. Figure 1 shows that on such large 6

3.1 Space-Time Diagrams

con gurations our rules form a sequence from chaotic rules (30,22) which generate little immediately evident structure, through a chaotic rule with domains (rule 18), through complex rules (54,110) with defects separating large organized domains, to a simple rule (4) which generates trivial patterns. For small systems, the state-transition graphs of cellular automata can be drawn as a planar diagram ( gure 2). To produce these diagrams, a random initial condition is chosen and iterated until it entered a temporal cycle. Each con guration is represented as a point, and the temporal cycle is shown as a ring of points in the center of the diagram. In the next largest concentric ring are shown the preimages of the con gurations on the cycle, connected to their image by a line segment. The preimages of these are shown in the next concentric circle, and so on. It is di cult to draw conclusions concerning the ultimate shape of statetransition graphs from diagrams such as shown in gure 2. The structure of small systems tends to be dominated by features which result from the interaction of number-theoretic properties of the system size with properties of the rule table. Large systems, on the other hand, contain too many states to be conveniently displayed in a single image. Nonetheless, we can see in gure 2 that the structures generated by rule 30 are composed of long, bare trees, and the structures generated by rule 4 are composed almost exclusively of leaves. The other rules show an intermediate degree of branching, with a trend toward greater branching as the sequence passes from rule 30 to rule 4. There is a subtle relationship between statistical transients, the time it takes statistical measures on in nite systems to reach equilibrium, and topological transients, the time it takes for con gurations on nite, periodic systems to enter an exact temporal cycle. It is one of the aims of this report to clarify this relationship. Thus, we have performed experiments on the two types of transients. 1) Statistical transients. A random con guration of size 2 106 with periodic boundary conditions is iterated forward in time for 7

3.2 State-Transition Diagrams

3.3 Forward Iteration

30

22

18

54

110

Figure 1: Space-Time Diagrams This sequence is ordered according to properties of the state-transition graph, from rules whose graph is mainly composed of skeleton, to those whose graph is mainly composed of surface. Note that this corresponds to a progression from rules which generate chaotic space-time patterns to those whose spacetime pattern is simple, passing though intermediate patterns with defects moving on an organized background. 8

Figure 2: State-Transition Diagrams This shows the state-transition diagram for a chosen spatio-temporal cycle of spatial period 20, for each of the rules. These diagrams were produced using a program of A. Wuensche 40]. 9

2 104 iterations. The density (probability of a '1') is measured at each time step ( gures 3 and 4). 2) Topological transients. The average time for a randomly chosen con guration with periodic boundary conditions to reach an exact cycle is measured as a function of system size, for relatively small systems ( gure 5). 1000 initial con gurations were used each system size. The statistical transient data are plotted in two ways. In gure 3 the vertical scale is chosen to show deviations from the large-time density, D1 , in the range D1 0:02. Fluctuations around D1 are fairly random for the rules at the chaotic end of the spectrum (30,22,18), while the more complex rules (54, and 110) exhibit intermittent bursts in density. The simple rule (4) does not uctuate in density; the nal density is achieved exactly in one time step. In gure 4, these same data are replotted to examine more closely the convergence to equilibrium. Here, ln(jDt ? D1 j) is subjected to a moving average in time (window size = 50). The exact values for D1 are di cult to determine numerically, and are the subject of some controversy. The values used here are 0.5, 0.35095, 0.25, 0.5, and 0.4805, for rules 30, 22, 18, 54, and 110 respectively. Figure 4 shows that the more chaotic rules 30 and 22 converge rapidly to a stable limit value. Rules 18, 54, and 110 take a slower course to equilibrium. The corresponding curve for rule 4 is unde ned. Turning to topological transients, gure 5 shows that rules 30 and 22 exhibit an exponential scaling with respect to system size in the length of topological transients (cf. 18, 12]). Rules 18 and 110 exhibit a power-law scaling (cf. 27]) and the transients of rule 54 scale as a stretched exponential, intermediate between exponential and power law (cf. 12]). The transients for rule 4 are of length 1, independent of system size. The uctuations in topological transient time shown in gure 5 are largely non-statistical; the error bars are comparable with the line width. Comparing gures (3,4) with gure 5, we see a generally tight correspondence between statistical and topological transients. Chaotic rules tend to have short statistical transients and exponentially long topological transients. Complex rules tend to have long statistical transients and slowerthan-exponential topological transients. Simple rules have short transients, both statistical and topological.

10

0.02

0.01

-0.01

-0.02 0.02

0.01

-0.01

-0.02 0 0.5 1 1.5 2 0 0.5 1 1.5 2 0 0.5 1 1.5 2

Figure 3: Statistical Transients The evolution of the density (probability of a "1") in a con guration of size 2 106 . The vertical scale is adjusted so that the large-time density is approximately 0. Deviations from this limit density of up to 0:02 are shown.

11

-10

-5

-10

-5

0 0

0.5

1.5

2 0

0.5

1.5

Figure 4: Statistical Transients:deviations for limiting density As in gure 3 this gure shows the evolution of the density (probability of a "1") in a con guration of size 2 106. Here the log of the absolute value of the di erence between the density at time t and the large-time density is plotted on the ordinate. In this and succeeding gures the natural log is used, unless otherwise noted.

12

Figure 5: Topological Transients This gure shows the average topological transient time as a function of system size. The transient time is measured by randomly selecting sets of 1000 con gurations for each system size, and iterating them until they reach a temporal cycle. The error bars are comparable with the line width. The growth of topological transients follows an exponential scaling for rules 30 and 22, a stretched-exponential scaling for rule 54, and a power-law scaling for rule 18 and 110. 13

4 One-time-step Preimages
In this section we will examine statistical properties of one-time-step preimages. We will look at the dependence of the number of immediate preimages on system size and on depth in the state-transition graph. This information on temporally local structure of state-transition graphs is to be compared with the temporally global information from the last section. It is in this comparison that the skeleton idea emerges: state-transition graphs are highly branched locally, but of limited branching on larger time scales. A Garden of Eden is a state which has no preimages. The Gardens of Eden form the leaves of the state-transition graph, at the extreme surface. As mentioned above, Wuensche 41] has obtained evidence that in the limit of large system size almost all con gurations are Gardens of Eden, for typical rules. Here we con rm this result, using a sampling technique slightly di erent from his. For each of a range of system sizes, we select 1 104 con gurations at random and test for the possession of a preimage ( gure 6). This gure shows that the fraction of Garden of Eden states goes to 1 exponentially for all but rule 30. For these rules, then, all but a measure 0 set of states is at the surface of the state transition graph, in the limit of large system size. By contrast, for rule 30, the fraction of Garden of Eden states goes to 0 exponentially, and in the limit almost all con gurations are on a long transient or a cycle.

4.1 Fraction of Garden of Eden states

4.2 Time-series of preimage number

In gure 7 we follow the evolution of a random initial con guration (of size 48) under each rule, counting the number of preimages at each iteration. Except for rule 30, the number of preimages can vary wildly over time (note the logarithmic scale). In some gures (rules 18 and 54) a spatio-temporal cycle is reached, resulting in cyclic preimage numbers. Apart from this e ect, it appears that the uctuations are relatively independent of the depth of forward iteration. For rule 4 (not shown) the number of preimages is constant, after one time step. 14

Figure 6: Fraction of Garden of Eden States The fraction of Garden of Eden states (states which have no preimages) is determined by selecting 1 104 con gurations at random and testing for the possession of a preimage. This is repeated for a range of system sizes. An exponential convergence to 1 is shown for rules 22, 18, 54, 110, and 4. The corresponding exponents are -0.04, -0.13, -0.07, -0.05, and -0.19 respectively. Rule 30, on the other hand, has no Gardens of Eden in the limit, and this limit is approached with exponent -0.16. Solid line: data, dashed line: exponential t. Prinf : probability that a random con guration is a GOE in the limit of large system size (Prinf = 0 for rule 30, and Prinf = 1 for the other rules). Prs : corresponding probability for size s. 15

Figure 7: Time Series of Preimage Number A single con guration of size 48 is run forward in time under each rule, and the number of preimages counted at each time step. These preimage numbers are subject to large uctuations, but the uctuations appear to be relatively independent of the depth of forward iteration. (See, however, gure 8.) Here the log is taken base 10. 16

We saw in the previous subsection that the number of preimages of a con guration can be vastly di erent from the number of preimages of its successor. To see how the distribution of preimage number depends on depth into the state-transition graph, we selected for each rule 2000 Garden of Eden con gurations on systems of size 48, and iterated them forward in time, counting the number of one-time-step preimages at each step, up to a depth of 15 ( gure 8). We see that, for all rules, the distribution of the number of preimages converges with depth. For rule 30 (not shown) the convergence is immediate. The convergence is considerably faster for the more chaotic rule 22 than for the more complex rules 18, 54, and 110. This is consistent with a "thin surface" portrait of chaotic rules. It seems that, on average, con gurations near the surface have more preimages then those further in. Note that, given the logarithmic scale of gure 8, these di erences can be quite signi cant. These data support the idea that the bulk of con gurations lies very near the surface of the state-transition graph; the peak of the distribution moves to smaller values as the number of iterations increases. For all rules studied, the peak of the distribution shifts toward larger values as the size of the system increases (not shown).

4.3 Distribution of preimage number

5 Statistical Inverse Iteration

Under quite general conditions formally irreversible cellular automata can be run backward in time, in a statistical fashion. To x ideas, consider a one-dimensional automaton on two states per cell. Let r be the radius of the rule. Under such a rule, the preimage of a block of size n is of size q = n +2r. If n and r are su ciently small (q < 30 in practice), then the preimages of all blocks of size n can be tabulated. Our goal is to construct preimages for large con gurations based on the information tabulated for preimages of small blocks. We proceed, therefore, to break the large con guration into blocks of size n, and construct a preimage for the large con guration by matching together preimages for the small blocks. To build a preimage for a con guration of size s = nm, break it into blocks b0 : : : bm of size n (see gure 9). Preimage construction proceeds by systematic enforcement of a matching constraint on block preimages. The 17

Figure 8: Distribution of Preimage Number The (smoothed, window size = 10) distribution of preimage number on congurations of size 48, after 1-15 forward iterations, in steps of 2. Increasing line thickness indicates increasing depth of forward iteration. 2000 con gurations/rule.

18

|preimage| = n+2r |overlap| = 2r

|block| = n

Figure 9: Method for building preimages of large blocks This shows how preimages of n-blocks are matched together to form preimages of larger blocks.

19

matching constraint for preimages of two adjacent blocks bibi+1 requires that the 2r right-most cells of the preimages of bi must agree with 2r left-most cells of the preimage of bi+1. Assume that the preimage of bi has been previously chosen. There may be no preimage of bi+1 which satis es the matching constraint. In this case, it is not possible to continue building a preimage for the con guration. On the other hand, there may be several possible preimages of bi+1 which satisfy the matching constraint. In this case, a choice must be made among the possibilities. This choice may be made on the basis of local and/or statistical considerations. Statistical considerations will be discussed below (section 5.1). Among the local constraints is the look-ahead constraint: that the preimage of bi+1 may be continued (to the right). That is, one may select from the set of preimages of bi+1 only those which match at least one preimage of the block bi+2 to the right of bi+1. With appropriate data structures, satisfaction of this constraint is rapidly evaluated. Once a preimage for bi+1 has been selected, the process may be iterated with the pair bi+1bi+2, and so on (When n = 1 and all possible paths are followed recursively, this procedure reduces to that of 38]). Since we are constructing preimages for periodic con gurations, the preimage chosen for bm must match the preimage chosen for b0. There are many ways to attempt to satisfy this constraint. For instance, having arrived at bm, one may nd that the chosen preimage for bm does not match the previously chosen preimage for b0. In this case, it may be possible to select a di erent preimage for b0 which does match bm. If it is not possible to simultaneously satisfy the look-ahead constraint, then this will create a mismatch to the right of b0. In this case the procedure can be applied to the pair b0b1, and so on. Other local constraints and methods to insure their satisfaction could be considered. However, as our aim here is understand how local constraints interact with statistical constraints, we will assume henceforth a xed algorithm, as just sketched. The inversion algorithm described above exploits local constraints to limit the size of the set of con gurations which must be searched in order to nd a preimage. For any nite number of constraints, situations will be encountered in which a non-deterministic choice must be made in the construction process. 20

5.1 Statistical inversion

The weights given to the set of possibilities are crucial to the success of the inverse iteration process. To see this, we compare inverse iteration in which choices are made randomly with inverse iteration driven by choices which are in a well-de ned sense "natural" for the rule. That is, given a cellular automaton state produced after many iterations of the automaton, we select an antecedent state which is among the most "natural" or "physical" antecedents of that state. A physical antecedent will be one which 1) is a probable state under the invariant measure of the automaton, and 2) has itself a preimage. With a su ciently complete description of the invariant measure of a cellular automaton, the procedure can be iterated many times. It will permit us to examine state-transition graph structure on time scales intermediate between a single time step (section 4), and the length of topological transients (section 3.3). A probability measure on (in nite) con gurations is built from an assignment of probability to contiguous, nite blocks of cell states. We restrict attention to measures which are translationally invariant and consistent in the sense of Kolmogorov 7]. It can be empirically demonstrated that most rules possess a Ruelle-Eckmann natural measure 8]. A natural measure is a unique measure which is invariant under the evolution of the rule and stable to small perturbations. The natural measure represents the physical states of the system, those states which may be reached by the system in the limit of large time. Consider the case of an irreversible rule. A block of states may have many preimages under the rule, or it may have none. Typically, these preimages do not have the same probability under the natural measure of the rule. Assume we know this natural measure. If we take a block from a con guration which has been produced by many iterations of the rule, we may predict its preimage according to the natural measure. One anticipates that a preimage likely according to the natural measure is also likely to have itself a preimage. After all, the natural measure represents the set of con gurations with in nite histories, and the con guration under consideration has a long history by construction. Example inverse iteration patterns produced by this method are shown in gure 10. Here, as in the following, the natural measure is estimated by counting the occurrence of 8-blocks in con gurations of size 1200 iterated forward 200 times. 21

22 18 54 110 Figure 10: Example inverse iteration patterns First column: forward iteration pattern. Second column: backward iteration pattern, using the last line of the pattern from the rst column as initial state. Third column: pattern of di erences between the forward and inverse iterations. These backward iteration patterns were driven by the natural measure, from a con guration which had been forward iterated 35 times. The probability to nd a backward path of this length in one trial using a uniform measure is less than 1 10?8 for rule 22, and less than 1 10?10 for the other rules shown.

5.2 Distribution of backward iteration heights

We applied our backward iteration algorithm to sets of 2:5 105 con gurations of length 48 which had been iterated forward in time for 35 iterations ( gure 11). Care was taken to insure that these con gurations were Gardens of Eden, and had not entered a cycle at 35 iterations. An estimate of the natural measure of each rule was obtained by iterating 300 randomly chosen con gurations of size 1200 (open boundary conditions). After 200 time steps, a count was made of the number of occurrences of each possible 8-block in the nal con guration. Normalized, these counts provide an estimate of the invariant 8-block probabilities, from which an estimate of the probabilities of blocks of all size may be calculated. Backward iteration was performed under two conditions: 1) using random choices at algorithmic branch points, or 2) using choices weighted by natural-measure probabilities as described in section 5.1. The abscissa gives the height to which backward iteration was possible, and the ordinate gives the log of the number of trails which reached exactly the given height. Note that the probability to reach a given height decreases exponentially, but much more slowly when natural-measure rather than uniform-measure probabili22

ties are used. These exponents will be denoted respectively rand or nat according to whether random or natural-measure probabilities are used. The di erence between rand and nat is most striking in the case of the complex rules 54 and 110. These rule have strong long-range correlations in their invariant measure. These correlations aid the inverse-iteration algorithm to select "natural" preimages. For rule 30 (not shown) inverse iteration can be continued nearly inde nitely, as most con gurations have but one preimage, which can be easily found. The same is true for rule 4, if choices are made according to the natural measure. By contrast, essentially no backward iteration can be performed for this rule with random choices at the algorithmic branch points. If the topological skeleton hypothesis is correct, one anticipates that the number of backward iterations which can be achieved depends on how many forward iterations are performed before backward iteration is initiated. In this experiment ( gure 12) we begin with a Garden of Eden state and iterate forward to a depth of t = 0 : : : 35 before initiating backward iteration. We check whether the con guration has entered a cycle, in which case it is discarded. For each t, rand and nat are measured as described in the last subsection. Note that on the chaotic end of the spectrum (rules 22 and 18) both exponents rapidly stabilize. We interpret this as evidence for a thin surface of con gurations with short histories which is fully traversed at small t. Once the skeleton has been reached, there begins to exist some long backward paths, so the rate of backward iteration is limited only by the ability of the algorithm to nd these paths. For more complex rules (54 and 110) there is a small t dependence of rand and nat. This indicates a more subtle relationship between the local branching patterns near the surface and the invariant statistical properties of the rule. It appears that an equilibrium branching rate is eventually achieved, but the distinction between surface and skeleton is not as sharp as is the case for chaotic rules.

5.3 Dependence of

rand

and

nat

on depth

Inverse iteration: level curves. In order to investigate more closely the

t dependence of rand and nat we plot the t dependence of each backward iteration height ( gures 13, 14).
23

10

nat 5 rand nat rand

10

nat 5 rand nat rand

0 0

10

20

30 0

10

20

30

Figure 11: Inverse iteration Random choices are compared with natural-measure choices in their ability to drive inverse iteration. The log of the probability to achieve exactly height h before failure of the algorithm is plotted vs. h. This distribution shows an exponential decay for all rules. Inverse iteration is performed under two conditions: 1) driven by random choices at algorithmic branch points (rand), and 2) driven by natural-measure choices (nat).

24

-0.5

-1

-1.5

-2

-2.5 -0.5

-1

-1.5

-2

-2.5 0 10 20 30 0 10 20 30

Figure 12: Decay Exponents as a Function of Depth The inverse iteration decay exponents rand and nat are plotted as function of initial forward iteration. For all rules shown there is an initial increase in the e ectiveness of inverse iteration. For the more chaotic rules, this rate stabilizes, while for more complex rules it continues to increase slowly.

25

Chaotic Complex Statistical Convergence fast slow Skeleton exponentially long di use (power law) surface thin di use (bushy) (t) independent of t slow variation w/ t Table 1: Summary of statistical and topological properties of chaotic and complex rules Statistical Convergence: the rate of convergence to statistical equilibrium. Skeleton: the length of transients on nite systems. Surface: the thickness of the layer of con gurations with only short histories. (t): the backward iteration exponent as a function of the number of preliminary forward iterations t. The interesting measure here is the rate at which the curves stabilize. For rules 22 and 18, the stabilization is rapid, for the more complex rules (54, 110), stabilization is slow. This is true for both natural and uniform measures, indicating that the stabilization rate depends only on the branching structure of the graph. The slow stabilization for complex rules suggests a "bushy" state-transition graph in which the distinction between skeleton and surface is less sharp. One may anticipate, consistent with the critical nature of these rules, that the state-transition graph exhibits a self-similar structure. This possibility is currently under investigation.

6 Discussion
Table 1 summarizes our results. It suggests that chaotic rules may be distinguished from complex rules by features of their state-transition graphs in the limit of large system size. The rapid convergence to statistical equilibrium exhibited by chaotic rules is related to the thickness of the surface. Complex rules have thicker surfaces, shorter skeletons, and slow statistical convergence. Beyond the central distinction between skeleton and surface, one main message here is that using the natural measure to choose small-block preimages dramatically increases the e ciency of inverse iteration, especially in the case of complex rules. Choosing sub-block preimages according to the 26

10

10

0 0

10

20

30

10

20

30

Figure 13: Inverse-Iteration Level Curves: Natural Measure Each curve represents a given backward iteration height. Thus, e.g., an ascending curve indicates that the given height is achieved more frequently as the depth of forward iteration is increased. Thick solid line: the number of times 0 or 1 inverse iterations was achieved. Thin solid lines: the number of times 2 : : : 15 inverse iterations were achieved.

27

10

10

0 0

10

20

30

10

20

30

Figure 14: Inverse-Iteration Level Curves: Uniform Measure Each curve represents a given backward iteration height. Thus, e.g., an ascending curve indicates that the given height is achieved more frequently as the depth of forward iteration is increased. Thick solid line: the number of times 0 or 1 inverse iterations was achieved. Thin solid lines: the number of times 2 : : : 15 inverse iterations were achieved.

28

natural measure does not guarantee that a long history may be constructed. A likely preimage for a sub-block may be incompatible with any preimage of a sub-block further on. The natural measure provides de nite, but generally limited, knowledge about possible con guration histories. In some sense (cf. 1]) the more complex the rule, the more the present state of the system encodes the entire history of its production. Our statistical inverse iteration procedure is one general way to exploit this record, albeit imperfectly. Chaotic, relatively uncomplex, rules forget their history. There is little correlation between the past and present state of the system, resulting in a close backward horizon. The inverse iteration procedure has implications with respect to a number of outstanding problems in mathematical physics, as does the skeletal structure of state-transition graphs it has revealed.

Reversibility. The distinction between reversible and irreversible processes

is central in physics. Fundamental physical laws governing microscopic physics are reversible, while macroscopic physics is irreversible. Cellular automata have been used to model physics at both microscopic and macroscopic levels. Among the most successful cellular automaton models at the microscopic level is the class of lattice-gas automata in which the collision of particles conserves particle number and momentum, and is time-reversible. The NavierStokes equations, which describe uid motion at the microscopic level, can be derived from the cellular automaton rules which govern the motion of particles in a lattice gas. On the other hand, cellular automaton models of macroscopic phenomena are typically irreversible. We have seen here that the distinction between reversible and irreversible is not absolute, and that even these models could be potentially projected backward in time.

state-transition graph scaling structure such as performed here could be undertaken for other types of spatially extended systems such as boolean or neural nets. It remains to be seen to what extent the surface/skeleton structure we have identi ed is general, and to what extent it depends on the uniformity and locality of cellular automata. It would be fairly immediate to extend this analysis to random boolean nets. Wuensche 39] has developed an inverse iteration algorithm for boolean nets which could be used to ex29

Connections with other spatially extended systems. An analysis of

plore surface and skeleton structure in this systems along the lines developed here.

Connections with smooth dynamical systems. We suggest that the

topological skeleton could play a similar role in the analysis of the dynamics of cellular automata to that played by the skeleton of unstable periodic orbits in the cycle-expansion approach of Cvitanovic 6] for the analysis of smooth dynamical systems. The direct cycle-expansion approach to cellular automata is blocked by several technical challenges, including: 1) While for low-dimensional, smooth dynamical systems, statistical properties are well-approximated using a few, short, cycles, for cellular automata, there may be an in nite number of short cycles. Currently there are no methods for summing the contributions of these. 2) While the stability of periodic orbits is a crucial ingredient in the cycle-expansion, there is no satisfactory way to evaluate the stability of a spatio-temporal cycle in a cellular automaton. 3) While the cycle-expansion is appropriate for temporal periodicities, spatio-temporal cycles in cellular automata encode both spatial and dynamical information. The temporal closure of a trajectory may be an artifact of spatial correlation. Temporal cycles simply do not mean the same thing in the spatially extended context. Our claim here is that despite these di culties one central insight can be salvaged from the Cvitanovic approach and brought to bear on cellular automata: the existence of a "topological skeleton", a small collection of states which form a sca olding for the physically signi cant dynamical behavior of the system. For cellular automata, this topological skeleton includes the spatio-temporal cycles, but is not limited to these. Included in the skeleton as well is a subset of the transient tree. While elements of cycles are strictly members of the attractor or non-wandering set of the cellular automaton, elements of the skeleton in the transient tree are nearly so. They have images arbitrarily far forward in time, and preimages over a very long time. This loosening of the notion of attracting set, required in the case of cellular automata, may be useful as well in the study of smooth dynamical systems. When dynamical notions developed for low-dimensional dynamical systems are applied without care to spatially extended dynamical systems, certain paradoxes may arise. Such was signaled by Crutch eld and Kaneko in their 30

study of transient and invariant behavior of coupled-map lattices 5]. Their results show that to overly restrict focus on invariant properties is to fail to comprehend the physical character of these systems. M. Nordahl and A. Wuensche for discussions, and the LMD laboratory of the ENS for their generosity with computer time.

Acknowledgments We would like to thank P. Grassberger, C. Langton,

References
1] C.H. Bennett, in Emerging Syntheses in the Sciences, Ed: D. Pines. (Addison-Wesley, Redwood City, 1988) S. Lloyd and H. Pagels, Ann. Phys. (N.Y.) 188:186 (1988) 2] N. Boccara, J. Nasser, and M. Roger. Annihilation of defects during the evolution of some one-dimensional class-3 deterministic cellular automata, Europhysics letters., 13(6):489, (1990) 3] N. Boccara, J. Nasser, and M. Roger. Particle-like structures and interactions in spatio-temporal patterns generated by one-dimensional deterministic cellular automaton rules, Phys. Rev. A, 44, July (1991) 4] E. R. Berlekamp, J. H. Conway and R. K. Guy, Winning Ways for your Mathematical Plays, vol. 2, chap. 25, Academic Press, (1982) 5] J. P. Crutch eld and K. Kaneko. Are attractors relevant to turbulence? Phys. Rev. Lett., 60:2715, (1988) 6] P. Cvitanovic, Invariant measurement of strange sets in terms of cycles Phys. Rev. Lett. 61:2729 (1988) 7] M. Denker, Ergodic theory on compact spaces, Lect. Not. Math. 527, (1976) 8] J-P. Eckmann et D. Ruelle, Ergodic theory of chaos and strange attractors, Rev. Mod. Phys. 57:617, (1985) 9] M. Eisele, Long-range correlations in chaotic cellular automata, Physica D 48:295-310 (1991) 31

10] K. Eloranta, The dynamics of defect ensembles in one-dimensional cellular automata, J. Stat. Phys. 76:1377 (1994) 11] H. A. Gutowitz, J. D. Victor, and B. W. Knight. Local structure theory for cellular automata. Physica D, 28:18{48, (1987) 12] H.A. Gutowitz, Transients, Cycles and complexity in cellular automata, Physical Review A 44:7881-7884, (1991) 13] H.A. Gutowitz, Cryptography with dynamical systems, In: Cellular Automata and Cooperative Phenomena, Eds: E. Goles and N. Boccara, Kluwer Academic Press, (1993) 14] P. Grassberger, Some more exact enumeration results for 1D cellular automata, J. Phys. A, 20:4039-4046 (1987) 15] P. Grassberger, New mechanism for deterministic di usion, Phys. Rev A 28:3666-3667 (1983) 16] P. Grassberger, Chaos and di usion in deterministic cellular automata, Physica D 10:52-58 (1984) 17] P. Grassberger. appendix, In Stephan Wolfram, editor, Theory and Applications of Cellular Automata. World Scienti c, (1986) 18] P. Grassberger. Long-range e ects in an elementary cellular automaton. J. Stat. Phys, 45:27{39, (1986) 19] P. Grassberger. Problems in quantifying self-generated complexity. Helvetica Physica Acta, 62:489, (1989) 20] J.P. Crutch eld and J.E. Hanson, Turbulent pattern bases for cellular automata Physica D 69:279-301 (1993) 21] J. E. Hanson and J. P. Crutch eld, Computational Mechanics of Cellular Automata: An Example, these proceedings. 22] A. Sinha and C. Jayaprakash, A numerical study of one-dimensional cellular automata, Physica D:39:352-364 (1989) 32

23] E. Jen, Cylindrical cellular automata, Communications in Mathematical Physics, 118:569{590, (1988) 24] E. Jen, Aperiodicity in one-dimensional CA, Physica D, 45:3, (1990) 25] E Jen, Exact solvability and quasiperiodicity of one-dimensional cellular automata, Nonlinearity, 4(2):251, (1991) 26] K. Kaneko, Attractors, basin structures and information processing in cellular automata, In S. Wolfram, editor, Theory and applications of cellular automata. World Scienti c, Singapore, (pp. 367) (1986) 27] W. Li and M. Nordahl, Transient behavior of cellular automata rule 110, Phys. Lett. A, 166(5/6):335{339, (1992) 28] D.A. Lind, Applications of ergodic theory and so c systems to cellular automata, Physica D 10: 36-44 (1984) 29] O. Martin, A. Odlyzko, and S. Wolfram. Algebraic properties of cellular automata, Commun. Math. Phys., 93:219, (1984) 30] H. V. McIntosh. Wolfram's class IV automata and a good life, Physica D, 45:105, (1990) 31] W. Meier and O. Sta elbach, Analysis of pseudo random sequences generated by cellular automata Proceedings of Eurocrypt '91, 186-199 (1991) 32] C. C. Walker. Attractor dominance patterns in sparsely connected boolean nets, Physica D, 45:441{451, (1990) 33] S. Wolfram, Undecidability and intractability in theoretical physics Phys. Rev. Lett. 54:735 (1985) 34] S. Wolfram, Ed: Theory and applications of cellular automata, (World Scienti c, Singapore, (1986) 35] S. Wolfram, Random sequence generation by cellular automata, Adv. Appl. Math, 7:123, (1984) 36] S. Wolfram. Universality and complexity in cellular automata, Physica D, 10:1{35, (1984) 33

37] S. Wolfram Cryptography with cellular automata, (Proceedings of Crypto '85, pp. 429-432, (1985) 38] A. Wuensche and M. Lesser, The global dynamics of cellular automata, volume Reference Vol 1 of Santa Fe Institute Studies in the Sciences of Complexity. Addison-Wesley, (1992) IBSN 0-201-55740-1. 39] A. Wuensche, The ghost in the machine:basins of attraction of random boolean networks, Cognitive Science Research Paper 281, University of Sussex, 1993, (1993) In Arti cial Life III, Santa Fe Institute Studies in the Sciences of Complexity, Addison-Wesley. 40] A. Wuensche, Discrete Dynamics Lab, available from: http://alife.santafe.edu/alife/software/ddlab.html or
ftp://alife.santafe.edu/pub/SOFTWARE/ddlab

41] A. Wuensche. Complexity in one-D cellular automata: gliders, basins of attraction and the Z parameter Santa Fe Institute working paper 94-04025. (1994) 42] B. Voorhees and S. Bradshaw, Predecessors of cellular automata states III. Garden of Eden classi cation of cellular automata, Physica D 73:152-167 (1994) 43] J. Zabolitzky, Critical properties of rule 22 elementary cellular automata, J. Stat. Phys. 50:1255 (1988)

34