Review

•  Languages and Grammars

–  Alphabets, strings, languages
•  Regular Languages
–  Deterministic Finite and Nondeterministic Automata

CS 301 - Lecture 23 – 
– 
– 
Equivalence of NFA and DFA Regular Expressions
Regular Grammars
Properties of Regular Languages

Recursive and Recursively • 

–  Languages that are not regular and the pumping lemma
Context Free Languages
–  Context Free Grammars

Enumerable Languages – 
– 
Derivations: leftmost, rightmost and derivation trees
Parsing and ambiguity
–  Simplifications and Normal Forms
–  Nondeterministic Pushdown Automata
–  Pushdown Automata and Context Free Grammars
Fall 2008 –  Deterministic Pushdown Automata
–  Pumping Lemma for context free grammars
–  Properties of Context Free Grammars
•  Turing Machines
–  Definition, Accepting Languages, and Computing Functions
–  Combining Turing Machines and Turing’s Thesis
–  Turing Machine Variations
–  Universal Turing Machine and Linear Bounded Automata
–  Today: Recursive and Recursively Enumerable Languages

Definition:
A language is recursively enumerable
if some Turing machine accepts it
Recursively Enumerable
and
Recursive Languages

1
Let L be a recursively enumerable language Definition:
A language is recursive
and M the Turing Machine that accepts it
if some Turing machine accepts it
and halts on any input string
For string w:
if w∈ L then M halts in a final state
In other words:
if w∉ L then M halts in a non-final state
A language is recursive if there is
or loops forever a membership algorithm for it

Let L be a recursive language

and M the Turing Machine that accepts it
Turing acceptable languages
For string w: and
Enumeration Procedures
if w∈ L then M halts in a final state

if w∉ L then M halts in a non-final state

2
We will prove:
Theorem:
(weak result)
•  If a language is recursive then if a language L is recursive then
there is an enumeration procedure for it there is an enumeration procedure for it

(strong result)
•  A language is recursively enumerable
if and only if
there is an enumeration procedure for it

Proof: If the alphabet is {a, b} then

~
M can enumerate strings as follows:
Enumeration Machine
a
b
~ M aa
M
ab
ba
bb
aaa
Enumerates all
Accepts
aab
L ......
strings of input alphabet

3
 Enumeration procedure Example: L = {b, ab, bb, aaa,....}
~ Enumeration
Repeat: M L(M ) Output
~ a
M generates a string w
b b b
M checks if w∈ L aa
YES: print w to output ab ab ab
ba
NO: ignore w bb bb bb
aaa aaa aaa
aab
End of Proof ...... ...... ......

Theorem: Proof:

if language L is recursively enumerable then Enumeration Machine

there is an enumeration procedure for it

~ M
M

Enumerates all
Accepts L
strings of input alphabet

4
 NAIVE APPROACH
If the alphabet is {a, b} then
~ Enumeration procedure
M can enumerate strings as follows:
Repeat:
~
M generates a string w
a
b M checks if w∈ L
aa
ab YES: print w to output
ba NO: ignore w
bb
aaa
aab Problem: If w∉ L
machine M may loop forever

BETTER APPROACH
~ ~
M Generates first string w1 M Generates third string w3

M executes first step on w1 M executes first step on w3

second step on w2
~
M Generates second string w2 third step on w1
M executes first step on w2
And so on............
second step on w1

5
 w1 w2 w3 w4 
If for any string wi
1 1 1 1 machine M halts in a final state
Step then it prints wi on the output
in 2 2 2 2
string
3 3 3 3

 End of Proof

Theorem: Proof:
Input Tape
w
If for language L
there is an enumeration procedure
Machine that
then L is recursively enumerable
accepts L

Enumerator
Compare
for L

6
Turing machine that accepts L
We have proven:
For input string w
A language is recursively enumerable
Repeat:
if and only if
•  Using the enumerator, there is an enumeration procedure for it
generate the next string of L
•  Compare generated string with w
If same, accept and exit loop

End of Proof

We will prove: Non Recursively Enumerable

1. There is a specific language

which is not recursively enumerable
Recursively Enumerable
(not accepted by any Turing Machine)

2. There is a specific language Recursive

which is recursively enumerable
but not recursive

7
Some Languages Are Not Definition: A set is uncountable
Recursively Enumerable if it is not countable

Proof Using Uncountable

Sets

Theorem: Proof:

Let S be an infinite countable set Since S is countable, we can write

S = {s1, s2 , s3 ,…}
The powerset 2 S of S is uncountable

Elements of S

8
 We encode each element of the power set
Elements of the powerset have the form: with a binary string of 0’s and 1’s

{s1, s3} Encoding

Powerset
element s1 s2 s3 s4 
{s5 , s7 , s9 , s10 }
{s1} 1 0 0 0 
…… {s2 , s 3 } 0 1 1 0 

{s1, s 3 , s4 } 1 0 1 1 

Let’s assume (for contradiction) Powerset

that the powerset is countable. Encoding
element
t1 1 0 0 0 0 

t2 1 1 0 0 0 
Then: we can enumerate
the elements of the powerset t3 1 1 0 1 0 

t4 1 1 0 0 1 


9
 t1 1 0 0 0 0 

t2 1 1 0 0 0 
Take the powerset element
whose bits are the complements
t3 1 1 0 1 0 
in the diagonal

t4 1 1 0 0 1 

New element: 0011…

(birary complement of diagonal)

The new element must be some ti Since we have a contradiction:

of the powerset
The powerset 2 S of S is uncountable
However, that’s impossible:

from definition of ti

the i-th bit of ti must be

the complement of itself

Contradiction!!!

10
 An Application: Languages

Example Alphabet : {a, b} Example Alphabet : {a, b}

The set of all Strings: The set of all Strings:

S = {a, b}* = {λ , a, b, aa, ab, ba, bb, aaa, aab,…} S = {a, b}* = {λ , a, b, aa, ab, ba, bb, aaa, aab,…}
infinite and countable infinite and countable

A language is a subset of S:
L = {aa, ab, aab}

Languages: uncountable
Example Alphabet : {a, b}  
L1 L2 L3 Lk
The set of all Strings:

S = {a, b}* = {λ , a, b, aa, ab, ba, bb, aaa, aab,…} M3 ?

M1 M2
infinite and countable
Turing machines: countable

The powerset of S contains all languages:

S
2 = {{λ},{a},{a, b}{aa, ab, aab},…} There are more languages
L1 L2 L3 L4  than Turing Machines
uncountable

11
Conclusion: Languages not accepted by Turing Machines
Lk
There are some languages not accepted
by Turing Machines
Languages
Accepted by
Turing Machines
(These languages cannot be described
by algorithms)

We want to find a language that

is not Recursively Enumerable

A Language which
is not
Recursively Enumerable This language is not accepted by any
Turing Machine

12
 Consider alphabet {a}
Consider Turing Machines
that accept languages over alphabet {a}
Strings: a, aa, aaa, aaaa, …

a1 a 2 a3 a4  They are countable:

M1, M 2 , M 3 , M 4 , …

Example language accepted by M i

a1 a2 a3 a4 
L( M i ) = {aa, aaaa, aaaaaa}
L ( M1 ) 0 1 0 1 
L( M i ) = {a 2 , a 4 , a 6 }
L( M 2 ) 1 0 0 1 
Alternative representation
L( M 3 ) 0 1 1 1 
1 2 3 4 5 6 7
a a a a a a a 
L( M 4 ) 0 0 0 1 
L( M i ) 0 1 0 1 0 1 0 

13
 a1 a2 a3 a4 
Consider the language
L ( M1 ) 0 1 0 1 

L = {a i : a i ∈ L( M i )} L( M 2 ) 1 0 0 1 

L( M 3 ) 0 1 1 1 

L( M 4 ) 0 0 0 1 
L consists from the 1’s in the diagonal
L = {a 3 , a 4 ,…}

a1 a2 a3 a4 
Consider the language L
L ( M1 ) 0 1 0 1 

L = {a i : a i ∈ L( M i )} L( M 2 ) 1 0 0 1 

L( M 3 ) 0 1 1 1 
L = {a i : a i ∉ L( M i )}
L( M 4 ) 0 0 0 1 
L consists of the 0’s in the diagonal
L = {a1, a 2 ,…}

14
Theorem: Proof:
Language L is not recursively enumerable Assume for contradiction that
L is recursively enumerable

There must exist some machine Mk

that accepts L

L( M k ) = L

a1 a2 a3 a4  a1 a2 a3 a4 

L ( M1 ) 0 1 0 1  L ( M1 ) 0 1 0 1 

L( M 2 ) 1 0 0 1  L( M 2 ) 1 0 0 1 

L( M 3 ) 0 1 1 1  L( M 3 ) 0 1 1 1 

L( M 4 ) 0 0 0 1  L( M 4 ) 0 0 0 1 
a1 ∈ L( M k )
Question: M k = M1 ? Answer: M k ≠ M1
a1 ∉ L( M1)

15
 a1 a2 a3 a4  a1 a2 a3 a4 

L ( M1 ) 0 1 0 1  L ( M1 ) 0 1 0 1 

L( M 2 ) 1 0 0 1  L( M 2 ) 1 0 0 1 

L( M 3 ) 0 1 1 1  L( M 3 ) 0 1 1 1 

L( M 4 ) 0 0 0 1  L( M 4 ) 0 0 0 1 
a 2 ∈ L( M k )
Question: M k = M 2 ? Answer: Mk ≠ M2
a 2 ∉ L( M 2 )

a1 a2 a3 a4  a1 a2 a3 a4 

L ( M1 ) 0 1 0 1  L ( M1 ) 0 1 0 1 

L( M 2 ) 1 0 0 1  L( M 2 ) 1 0 0 1 

L( M 3 ) 0 1 1 1  L( M 3 ) 0 1 1 1 

L( M 4 ) 0 0 0 1  L( M 4 ) 0 0 0 1 
a3 ∉ L( M k )
Question: M k = M 3 ? Answer: M k ≠ M 3
a3 ∈ L( M 3 )

16
 Therefore, the machine M k cannot exist
Similarly: Mk ≠ Mi for any i

Because either:
Therefore, the language L
is not recursively enumerable
a i ∈ L( M k ) a i ∉ L( M k )
or
a i ∉ L( M i ) a i ∈ L( M i )

End of Proof

Observation: Non Recursively Enumerable

L
There is no algorithm that describes L
Recursively Enumerable

(otherwise L would be accepted by Recursive

some Turing Machine)

17
 What’s Next
•  Read
–  Linz Chapter 1,2.1, 2.2, 2.3, (skip 2.4), 3, 4, 5, 6.1, 6.2, (skip 6.3), 7.1, 7.2, 7.3, (skip
7.4), 8, 9, 10, 11
–  JFLAP Chapter 1, 2.1, (skip 2.2), 3, 4, 5, 6, 7, (skip 8), 9, (skip 10), 11
•  Next Lecture Topics From 11.1, 11.2, 11.3, and 11.4
–  More Recursive Languages, Unrestricted Grammars, Context Sensitive Grammars,
and the Chomsky Hierarchy
•  Quiz 3 in Recitation on Wednesday 11/12
–  Covers Linz 7.1, 7.2, 7.3, (skip 7.4), 8, and JFLAP 5,6,7
–  Closed book, but you may bring one sheet of 8.5 x 11 inch paper with any notes you
like.
–  Quiz will take the full hour
•  Homework
–  Homework Due Thursday

18