Computational Models - Lecture 11

Handout Mode
Iftach Haitner.
Tel Aviv University.

October 31, 2016

Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University.
Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

1 / 41

Talk Outline
I

Languages, words and alphabets

Finite automata and regular languages

Regular operations

Sipsers book, chapter 1.1

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

2 / 41

Part I
Languages, words and alphabets

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

3 / 41

Languages, words and alphabets

Definition 1
An alphabet is a finite set of letters.
I
I
I
I

= {a, b, c, . . . , z} the English alphabet.

= {, , , . . . , } the Greek alphabet.
= {0, 1} the binary alphabet.
= {0, 1, . . . , 9} the digital alphabet.

Definition 2
A word (i.e., string) over , is a finite sequence of letters from .
The collection of all strings over is denoted by .
For {0, 1}, the binary alphabet, , 1, 0, 000000000, 1111111000 are all
members of .
Definition 3
A language over is a (possibly infinite) subset of .
Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

4 / 41

Language Examples
I

Modern English.

Ancient Greek.

All prime numbers, written using digits.

A = {w {0, 1} : w has at most seventeen 0s}.

B = {0n 1n : n 0}.

C = {w {0, 1} : w has an equal number of 0s and 1s}.

Make sure you understand what the above notions stand for...

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

5 / 41

Part II
Finite Automata

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

6 / 41

Example: A One-Way Automatic Door

front
pad

rear
pad

door
I

open when person approaches

hold open until person clears

dont open when someone standing behind door

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

7 / 41

The Automatic Door as DFA

REAR
BOTH
NEITHER

FRONT

closed

FRONT
REAR
BOTH
open

NEITHER
I

States:
I
I

OPEN
CLOSED

Sensor:
I
I
I
I

FRONT: someone on front pad

REAR: someone on rear pad
BOTH: someone(s) on both pads
NEITHER no one on either pad.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

8 / 41

The Automatic Door as DFA

A DFA is Deterministic Finite Automata

REAR
BOTH
NEITHER

FRONT

closed

FRONT
REAR
BOTH
open

NEITHER

closed
open

neither
closed
closed

Iftach Haitner (TAU)

front
open
open

rear
closed
open

both
closed
open

Computational Models, Lecture 1

October 31, 2016

9 / 41

DFA: Informal definition

The machine M1 :
0

q1

q
2

q3
0,1

States: q1 , q2 , and q3 .

Start state: q1 (arrow from outside).

Accept state: q2 (double circle).

State transitions: arrows tagged with letters.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

10 / 41

DFA: Informal definition (cont.)

0

q
2

q1

q3
0,1

On an input string
I
I

DFA begins in start state q1

after reading each symbol, DFA makes
state transition with matching label.

After reading last symbol, DFA produces" output:

I
I

accept if DFA is an accepting state.

reject otherwise.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

11 / 41

DFA: Informal definition (cont..)

0
q1

q
2

q3
0,1

What happens on the following input strings:

I

1101

0010

01100

In general?!

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

12 / 41

DFA: Informal definition (cont...)

0
q1

q
2

q3
0,1

This DFA accepts

I

All input strings that end with a 1

All input strings that contain at least one 1, and end with an even
number of 0s

No other strings

Proof: ?

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

13 / 41

DFA - Formal Definition

Definition 4
A deterministic finite automaton (DFA) is a 5-tuple (Q, , , q0 , F ),
where
I

Q is a finite set called the states

is a finite set called the alphabet

: Q Q is the transition function

q0 Q is the start state

F Q is the set of accept states

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

14 / 41

Back to M1
0

q1

q
2

q3
0,1

M1 = (Q, , , q1 , F ) where
I

Q = {q1 , q2 , q3 }, = {0, 1},

the transition function is

q1 is the start state

F = {q2 }.

Iftach Haitner (TAU)

q1
q2
q3

0
q1
q3
q2

1
q2
q2
q2

Computational Models, Lecture 1

October 31, 2016

15 / 41

Another Example

s
a
q
1

b
r

q
2

Iftach Haitner (TAU)

r2

Computational Models, Lecture 1

October 31, 2016

16 / 41

Formal Model of Computation

Definition 5
M = (Q, , , q0 , F ) accepts w if bM (q0 , w) F .
b
Definition 6 ()
bM : Q
7 Q is defined by

b w1,...,n1 ), wn ), n = |w| 1
((q,
bM (q, w) =
.
q,
w = .
I

w1,...,k stands for the word w1 , . . . , wk the k -letter prefix of w

(w1,...,0 = w = )

I
I

Note that bM (q, ) = (q, ) for .

We write b when M is clear from the context.
Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

17 / 41

An equivalent definition
Definition 7 (Equivalent definition)
M = (Q, , , q0 , F ) accepts w = w1 w2 . . . wn , if r0 , . . . , rn Q s.t.,
I

r0 = q0 .

(ri , wi+1 ) = ri+1 , for all 0 i < n.

rn F .

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

18 / 41

The language of a DFA

Definition 8
L(M), the language of a DFA M, is the set of strings that M accepts.
We sometime say that M accepts the language L(M).
I

M may accept many strings

M accepts only one language.

What language does M accept if it accepts no strings?

Definition 9
A language is called regular, if some deterministic finite automaton
accepts it.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

19 / 41

Example: DFA M2

I
I

Q = {q1 , q2 }, = {0, 1}, F = {q2 }, =?

b 1 , w) = q2 }?
What is L(M2 ) := {w {0, 1} : (q

Theorem 10
L(M2 ) = {w {0, 1} : #1 (w) is odd}.
w.
I
I

#1 (w) number of ones in

Proof by induction on the word length

What does it mean?
The jth assumption: w {0, 1}j is in L(M) iff #1 (w) is odd
Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

20 / 41

Proving L(M2 ) = L0 := {w : #1 (w) is odd}

Basis (length 0):

/ L0

Step: assume hypothesis holds for words of length j 0.

Let x = y {0, 1}j+1 for {0, 1}

b 1 , y ) = q1 .
If #1 (y ) is even. By assumption (q

b 1 , ) = q1 = 
and (q
/ L(M2 ).
(hence, |y | = j).

= 1 =
1. #1 (x) is odd = x L0
b 1 , x) := ((q
b 1 , y ), ) = (q1 , 1) = q2 = x L(M2 ).
2. (q

= 0 =
1. #1 (x) is even = x
/ L0
b
b
2. (q1 , x) := ((q1 , y ), ) = (q1 , 0) = q1 = x
/ L(M2 ).

I
I

If #1 (y ) is odd...
We proved for x {0, 1}j+1 :
Iftach Haitner (TAU)

x L(M2 ) x L0

Computational Models, Lecture 1

October 31, 2016

21 / 41

The language of M1
0

q
2

q1

q3
0,1

Theorem 11
L(M1 ) = {w102k : k 0, w {0, 1} }
Proof:
Claim 12 (implies the theorem)
b 1 , x) = qi } and let
Let L0i = {x {0, 1} : (q
I

L1 = {0k : k 0}

L2 = {w102k : k 0, w {0, 1} }

L3 = {w102k +1 : k 0, w {0, 1} }

Then, L0i = Li for every i {1, 2, 3}

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

22 / 41

Proving Claim 12
We need to prove that i {1, 2, 3}:
x Li x L0i .
Recall that
b
x L0i (x)
= qi

Proof by induction on word length.

Induction basis: Easy to see that hypothesis holds for .

Induction step: Assume hypothesis holds for words of length j 0.

Let x = y {0, 1}j+1 for {0, 1}.

We prove the hypothesis for x, separately for each i {1, 2, 3}

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

23 / 41

L1 = {0k : k 0}
Recall x = y {0, 1}j+1 for {0, 1}.
b 1 , x) = q1
Proving: x L1 = (q

( = x L01 ).

x = 0j+1 , y = 0j and = 0.
b 1 , y ) = q1
Since y L1 , by i.h. (q

b 1 , x) = ((q,
b y ), ) = (q1 , 0) = q1 .
Therefore, (q

b 1 , x) = q1 = x L1 .
Proving: (q
I

b 1, y )
Let qy = (q

qy = q1 and = 0. (?)

By i.h. y = 0j .

Hence, x = y = 0j 0 = 0j+1 L1 .

Iftach Haitner (TAU)

b 1 , x) = (qy , ) = q1 )
(hence, (q

Computational Models, Lecture 1

October 31, 2016

24 / 41

L2 = {w102k : k 0, w {0, 1} }
Recall x = y {0, 1}j+1 for {0, 1}.
b 1 , x) = q2 .
Proving x L2 = (q
I

Assume = 1
I

b 1 , x) = q2 .
Since (qi , 1) = q2 for any i = (q

Assume x = w102k for k > 0

I
I
I

( = y = w102k 1 and = 0)

Hence, y L3 .
b 1 , y ) = q3
By i.h. (q
b 1 , x) = (q3 , 0) = q2 .
Thus, (q

b 1 , x) = q2 = x L2 .
Proving (q
I
I

Assume = 1 = x L2 . (?)
Assume = 0
I
I
I

b 1 , y ) = q3
qy := (q
By i.h. y = w102k +1 for some k 0
Therefore x = y = w102k +1 0 L2 .

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

25 / 41

L3 = {w102k +1 : k 0, w {0, 1} }
Recall x = y {0, 1}j+1 for {0, 1}.
b 1 , x) = q3 .
Proving x L3 = (q
I

x = w102k +1 , y = w102k and = 0

y L2

b 1 , y ) = q2 .
By i.h. (q
b 1 , x) = (q2 , 0) = q3 .
Therefore, (q

b 1 , x) = q3 = x L3 .
Proving (q
I

b 1, y )
Let qy = (q

Hence, qy = q2 and = 0 (?)

By i.h. y = w102k

Therefore, x = y = w102k 0 L3 .

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

26 / 41

Part III
Regular Operations

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

27 / 41

Additional examples of regular languages

Let = {0, 1}.
I

{w {0, 1} : #1 (w) 0 mod 7}.

Sequence of 0 followed by sequence of 1, i.e., {0m 1n : m, n 0}.

Any finite language.

All the above languages are regular

Is there a simple proof?

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

28 / 41

The regular operations

Let A and B be languages.
The union operation:
AB = {x : x A x B}
The concatenation operation:
AkB = {xy : x A y B}
The star operation:
A = {x1 x2 . . . xk : k 0 and each xi A}

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

29 / 41

The regular operations Examples

Let A= {good, bad} and B = {boy, girl}.
Union
A B = {good, bad, boy, girl}
Concatenation
AkB = {goodboy, goodgirl, badboy, badgirl}
Star
A = {, good, bad, goodgood, goodbad, badbad, badgood, . . .}

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

30 / 41

Closure under union

Theorem 13
If L1 and L2 are regular languages, then so is L1 L2 .
Approach to Proof:
I

Some DFA M1 accepts L1

Some DFA M2 accepts L2

Construct DFA M that accepts L1 L2 .

Attempted Proof Idea:

I

first emulate M1 , and

if M1 doesnt accept, then emulate M2 .

Whats wrong with this?

Fix: Emulate both machines simultaneously.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

31 / 41

Closure Under Union: Correct Proof

Suppose
I

M1 = (Q1 , , 1 , q1 , F1 ) accepts L1 ,

M2 = (Q2 , , 2 , q2 , F2 ) accepts L2 .

Define M as follows (M will accept L1 L2 ):

I

Q = Q1 Q2 .

is the same.

For each (r1 , r2 ) Q and a ,

((r1 , r2 ), a) = (1 (r1 , a), 2 (r2 , a))

q0 = (q1 , q2 )

F = {(r1 , r2 ) : r1 F1 or r2 F2 }.

Formal proof (next slide)

(hey, why not choose F = F1 F2 ?)

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

32 / 41

Correctness of the construction

Claim 14
L(M) = L(M1 ) L(M2 ).
Follows by the next claim.
Claim 15
bM ((q1 , q2 ), x) = (b1 (q1 , x), b2 (q2 , x)).
Proof: By induction on word length. DIY ...
Proving Claim 14:
I
I
I
I
I

x L(M1 ) = b1 (q1 , x) = r1 F1 .
(similar if x L(M2 ).)
By Claim 15, bM ((q1 , q2 ), x) = (r1 , ) F = x L(M).
x L(M) = bM ((q1 , q2 ), x) = (r1 , r2 ) F .
By Claim 15, (r1 , r2 ) = (b1 (q1 , x), b2 (q2 , x)).
Hence, either r1 F1 or r2 F2 = x L(M1 ) L(M2 ).
Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

33 / 41

What about concatenation?

Theorem 16
If L1 , L2 are regular languages, then so is L1 kL2 .
Example: L1 = {good, bad} and L2 = {boy, girl}.
L1 kL2 = {goodboy, goodgirl, badboy, badgirl}
This is much harder to prove.
Idea: Simulate M1 for a while, then switch to M2 .
Problem: But when do you switch?
This leads us into non-determinism, wait for next class...

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

34 / 41

Part IV
Non-deterministic Finite Automata

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

35 / 41

NFA non-deterministic Finite Automata

0,1

q1

0,1

q
2

q3

q
4

May have more than one transition labeled with the same symbol,

May have no transitions labeled with a certain symbol,

May have transitions labeled with , the symbol of the empty

string. Will deal with this latter

Every DFA is also an NFA.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

36 / 41

Non-deterministic computation
0,1

q1

0,1

q
2

q3

q
4

What happens when more than one transition is possible?

I

The machine splits into multiple copies

Each branch follows one possibility

Together, branches follow all possibilities.

If the input doesnt appear, that branch dies.

Automaton accepts if some branch accepts.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

37 / 41

Computation on 1001
0,1

q1

0,1

q
2

q3

q
4

symbol
1

q1

q1

q1

q1

q
2

q
2

q3

q
4

q1

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

38 / 41

Why non-determinism?
Theorem 17 (Informal, to be proved soon)
Deterministic and non-deterministic finite automata, accept exactly the
same set of languages.
Q.: So why do we need NFAs?
Design a finite automaton for the language L all binary strings with
a 1 in their third-to-the-last position?

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

39 / 41

NFA for L
0,1

q1

q
2

0,1

q3

0,1

q
4

Guesses which symbol is third from the last, and

checks that indeed it is a 1.

If guess is premature, that branch dies, and no harm occurs.

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

40 / 41

DFA for L
I

Have 8 states, encoding the last three observed letters.

A state for each string in {0, 1}3 .

Add transitions on modifying the suffix, give the new letter.

Mark as accepting, the strings 1

0
0

q
000

q001

0
0

q
100

q101

q
010

q
110

q011

q111
1

DFA has few bugs...

Iftach Haitner (TAU)

Computational Models, Lecture 1

October 31, 2016

41 / 41