N.C.A.

da Costa
F.A. Doria
ON THE EXISTENCE OF VERY DIFFICULT
SATISFIABILITY PROBLEMS
1
Our arguments take place within ordinary informal mathematics. is
the set of natural numbers. We recall that an algorithm is a computational
procedure that can be formalized through Turing machines, partial recur-
sive functions and the like; a computable function f : is a function
that can be computed with the help of an algorithm. Chief results will
be presented in the language of onetape Turing machines over a binary
alphabet A
2
= 0, 1; the set of all nite binary words is noted A

2
. Turing
machine results needed in this paper can be found in either [6] or [9]. Our
binary Turing machines are dened by tables [6] [9] and a Turing machine
cycle is the set of operations described by a line in the machines table. For
an arbitrary real number x, ]x[ is the integral part of x. [x[ will always de-
note the length of x, that is, the number of letters from A
2
in x, even in the
case when x can be taken to be a number. To do so we x an enumeration
for the binary words given by a map : A

2
dened as:
, 0, 1, 00, 01, 10, 11, 000, . . . 0, 1, 2, 3, 4, 5, . . .,
such that if x A

2
(x) , then
[x[ =]log((x) + 1)[,
where log denotes a base 2 logarithm. Therefore our algorithms can repre-
sent maps from to , form to A

2
, and so on.
Definition 1.1. The identity machine is the (trivial) Turing machine
with zero states and an empty set of instructions; it (trivially) holds on
every input. We agree that, if x is its input, then x = x. 2
1
Partially supported by grants from FAPESP and from CNPq (Brazil), Philosophy
Section.
122
Definition 1.2.
1. The function s(M, x), given by:
s(M, x) = #( range in tape squares used when M computes over x),
if M stops over x, and s(M, x) = , if M doesnt stop over x, is the
space budget of M over x.
2. The function t(M, x), dened by:
t(M, x) = [x[ + # (cycles performed by M over x),
if M stops over x, and t(M, x) = , if M doesnt stop over x, is the
operation time of M over x.
3. If M stops over x, the function t

(M, x), dened by:

t

(M, x) = # (cycles performed by M over x),

is the strict time measure of M over x. 2
For the denition of acceptable universal programming systems P see
[6], p.95. Two programming systems P, P

are polynomially related if there

is a 11 total recursive map that sends one P over P

and if s, s

and t, t

are the space budget and operation time functions in the programming
systems P, P

, then s

= p(s, t), t

= q(s, t), where p, q are polynomials over

the integers. In this note we restrict our attention to polynomially related
programming systems.
Let [x] be a measure of the size of x as an input for the Turing machine
M. Let x
n
, n be a sequence of inputs for M such that [x
n
] < [x
n+1
]. (To
clarify matters, we notice that one usually takes [x] = [x[, or if meaningful,
[x] = v(x) = number of variables in x. In the last case, it may be the
number of cities in an instance of the traveling salesman problem, or the
number of propositional variables in the satisability problem.) Finally, let
F : be a function:
123
Definition 1.3. (Complexity of computational resources)
1. Let S
M
([x]) s(M, x). Then S
M
([]) is a space bound for the
machine M.
2. Let T
M
([x]) t(M, x). Then T
M
([]) is a time bound for the
machine M.
3. G
M, J
([x]) = min all space (or time) bounds. J = S or T, is the
space (or time) complexity of computational resources when M
acts upon a set of inputs of values [x]. 2
KolmogorovChaitinManin complexity
Main references here are [1] [5] [7] [8] [10].
Remark 1.4. We are going to require a pairing function :
2
which
is a total recursive embedding with a recursive inverse that satises the
following condition:
(k, j) k(j),
for all k, j , a prescribed function. For examples of functions that
satisfy that condition, see [7], p.226 and [10]. nples are coded by functions
dened recursively as follows:

2
(k, j) = (k, j);

n
(k
1
, . . . , k
n
) = (
n1
(k
1
, . . . , k
n1
), k
n
).
Therefore, if M
f
is a Turing machine, the corresponding partial recur-
sive 1function is given by f(x) = M
f
(x). For an mfunction.
f(x
1
, . . . , x
n
) = M
f
(
n
(x
1
, . . . , x
n
)).
2
Definition 1.5. (KolmogorovChaitin) The complexity of y with respect
to an algorithm f is given by K
f
(y) = min
y=f(x)
[x[, if there is one such x,
or otherwise. 2
124
Corollary 1.6. If y = f(x), then K
f
(y) [x[. 2
The main result on K
f
is:
Proposition 1.7. There is an optimal algorithm u such that for any other
algorithm v, K
u
(f) K
v
(f) + c(u, v). 2
Remark 1.8. We now x for once and all an optimal algorithm u which
will be our reference in all further calculations and proofs dealing with the
concept of complexity. 2
Definition 1.9.
1. K
h
(y) _ K
f
(y) if and only if K
h
(y) K
f
(y) + c(f, h).
2. K
h
K
k
if and only if K
h
_ K
k
and K
k
_ K
h
. 2
If u is our xed optimal family, then:
Definition 1.10. K(y) = K
u
(y). 2
KCMincompressibility
Definition 1.11. A binary sequence x over the alphabet A
2
is KCM
incompressible if:
K

(x) K(x)
where is given in Denition 1.1. 2
Remark 1.12. The idea is that we cannot recursively compress x. 2
From Denitions 1.9. and 1.11. we get:
Corollary 1.13. For KCMincompressible binary sequences x,
125
[x[ + c(g, ) K(x) [x[ c(, g)
or
K(x) + c(, g) [x[ K(x) c(g, ).
Corollary 1.14. For large [x[, the ratio between all KCMincompressible
sequences of length [x[ and the remaining sequences of the same length
exceeds 2
c(,g)
[10]. 2
Corollary 1.15. There are KCMincompressible sequences. 2
Posts theorem on truthtables
Let L be a classical propositional calculus. Let F(p
1
, . . . , p
n
) be a propo-
sitional function (=any expression in L) on the propositional variables
p
1
, . . . , p
n
. Let be the Turing machine program over an adequate bi-
nary coding that out of any such F computes the truth table for F, which
is here seen as a binary sequence of length 2
n
. We restrict our attention
to propositional functions in either conjunctive normal form (cnf) or in
disjunctive normal form (dnf).
Proposition 1.16. (Post) Let
n
be any binary sequence of length 2
n
.
Then we can algorithmically obtain a propositional function F
n
= Fp
1
, . . . , p
n
in cnf (or in dnf ) within L such that
(Fp
1
, . . . , p
n
) =
n
.
Corollary 1.17. If
n
is KCMincompressible, then
K(Fp
1
, . . . , p
n
) K(
n
).
126
Main results on the diculty of computations
We suppose that we have translated propositional expressions in cnf or dnf
into binary sequences (which represent natural numbers) that satisfy the
requirement in Remark 1.4. We will be interested in sequences of propo-
sitional functions F
n
= F(p
1
, . . . , p
n
), n in cnf (or in dnf), where n
denotes both the index for the formula in the succession and the num-
ber of dierent propositional variables in each expression F
n
. As already
emphasized, we suppose that the corresponding truth tables
n
are KCM
incompressible. Let S be any Turing machine that takes one such proposi-
tional function F
n
= F(p
1
, . . . , p
n
) as its input and decides whether or not
F
n
is satisable [6]. Then:
Proposition 1.18. (Exponential dependence on n) If
n
is KCM
incompressible, and if F
n
=
n
, then:
1. t(S, Fp
1
, . . . , p
n
) 2
v(F
n
)
c, c a constant which doesnt depend on
F
n
.
2. For large n = v(F
n
), there is a Turing machine S that checks the sat-
isability of F
n
such that t(S, F
n
) p([F
n
[), p a polynomial function
of the length [F
n
[.
In order to prove the rst statement we proceed as follows: if is the
truthtable generating Turing machine, then we have that F(p
1
, . . . , p
n
) =

n
. Now, from Corollary 1.5:
K

(
n
) = min

n
=F
[F

(p
1
, . . . , p
n
)[. (1)
Therefore we can write:
K

(
n
) [F
n
[, (2)
where F
n
= F(p
1
, . . . , p
n
).
From the fundamental theorem on the logarithmic complexity (Propo-
sition 1.7.), if g is our xed optimal algorithm,
127
K(
n
) = K
g
(
n
) K

(
n
) + c(g, ). (3)
From Corollary 1.13. we have that:
K(
n
) [
n
[ c(, g) = 2
n
c(, g). (4)
If we combine equation (4) with the previous expression, that results from
the fundamental theorem (3) plus equation (2), we have:
2
n
(c(g, ) + c(, g)) K

(
n
) [F
n
[. (5)
Therefore we get an estimate for [F
n
[:
[F(p
1
, . . . , p
n
)[ 2
v(F
n
)
c, (6)
where the parameter c = c(g, ) +c(, g) is independent of both F
n
and n.
Now:
t(S, F
n
) = [F
n
[ + # (cycles performed by S over F
n
). (7)
But from equation (6),
[F
n
[ 2
n
c
given that estimate, from (7)
t(S, F
n
) 2
v(F
n
)
c + # (cycles performed by S over F
n
),
then:
t(S, F
n
) 2
n
c. (8)
For the second statement, if n = v(F
n
) is large, since
n
is KCM
incompressible, we can use Corollary 1.17. Thus, an optimal binary coding
for F
n
would give, at last,
128
[F
n
[ = 2
n
c. (9)
Suppose the usual, direct construction of the truthtable. For each
line in the truthtable we have that the time t

spent in its verication is

approximately
t

(S, F
n
) (2
n
c). (10)
(Here is the average time it takes S to check each item in F
n
when
producing a truthvalue in the truthtable line.) As there are 2
n
lines in
that table, from (7),
t(S, F
n
) (2
n
c) + 2
n
(2
n
c), (11)
whereas one gets
t(S, F
n
) [F
n
[(([F
n
[ + c) + 1), (12)
which, for any nonoptimal binary coding F
n
| for F
n
such that
[F
n
[ [F
n
|[,
gives the desired polynomial relation. 2
Corollary 1.19.
1. For an arbitrary family F
n
, the strict time measure
t

(S, F
n
)
is at least linear on the length [F
n
[.
2. For an arbitrary family F
n
whose truthtables are KCMincompressible,
the strict time measure t

(S, F
n
) is at least exponential on n.
Proof. For the rst assertion: in order to compute the number of variables
in F
n
(a necessary step in the construction of a truthtable) we will have to
129
scan at least the whole length of F
n
since that information is not written
out on the machine tape. The second assertion follows from the rst one.
2
Density, undecidability and incompleteness
Definition 1.20. Given a function s : R (R are the reals), we say
that s(n) is a subexponential if there exists an n
0
such that for all n > n
0
s(n) < e(n), for all e(n) of the form e(n) = a
bn
a > 1 and b > 0. 2
Definition 1.21. A set F
n
of expressions in cnf (or in dnf) is tractable
provided that there is an n
0
so that for every n > n
0
.
[F
n
[ < s(n),
where s(n) is a subexponential. 2
Remark 1.22. Notice what goes on here: for each new propositional
variable added to the expressions in the set F
n
, the corresponding lengths
do not blow up in an exponential way. The polynomial case is strictly
contained in our denition of tractability. 2
Proposition 1.23.
lim
n
# of tractable F
n
# of KCMincompressible F
n
= 0
We can give a more elegant interpretation of the previous result: let
B Ir [0, 1] be the set of binary irrationals in the unit interval. Code
the sequences
n
by clopen basis sets in B Ir dened by the
n
as initial
segments. Proposition 1.16. allows us to uniquely code expressions in cnf in
L by the sequences
n
, so that we map all such expressions in L over clopen
basis sets in B Ir. The density result follows in a more elegant formulation.
Granted that coding plus the preceding denitions and results:
Proposition 1.24. The set of tractable satisability problems for cnf for-
mulae in L has zero measure. 2
130
Proposition 1.25. (Undecidability) Given an arbitrary F
n
of maximal
length, there is no general algorithm to decide whether or not F
n
is in-
tractable. 2
Let us now embed computation theory into an adequately large arith-
metically consistent theory T. (For that concept and related techniques see
[2] [3].) We then assert:
Proposition 1.26. (Incompleteness) If T contains arithmeric and is arith-
metically consistent, there is an expression in T that translates a formula in
cnf in L which is KCMincompressible in a model where T is arithmetically
consistent, but such that we cannot prove or disprove that fact within T. 2
Comment
The whole argument in the present note can be summarized in a single
statement: we can build an nvariable satisability problem which is so
complex that just to import the relevant data about Fp
1
, . . . , p
n
any algo-
rithm that solves it will spend a time interval which grows as an exponential
on n.
The construction of such an example is allowed by Posts algorithm
(Proposition 1.16.). The whole thing is crystal clear; we can actually see
why we cannot compress in less that 2
n
c bits the nvariable propositional
function F
n
. The actual arrangement of the conjuncts in F
n
and the distri-
bution of operators like are coded by an incompressible random sequence.
So, any description of F
n
will have to include that 2
n
digit binary sequence
or another sequence with an equivalent size as a source of information for
the problem data.
The present note was supported by grants from FAPESP and
CNPq/Brazil. The authors wish to thank several colleagues and friends
who received previous versions of this paper and criticized it in detail. The
rst author thanks Prof. H. Enderton for his careful criticisms of one of
131
its early versions; both authors wish to thank Prof. P. Suppes who helped
and guided them through the conceptual selva selvaggia of Kolmogorovs
complexity with many references on algorithmic randomness theory; and
Prof. David Miller, for his kindness in having a preliminary sketch of this
work aptly reviewed.
References
[1] G.J. Chaitin, Information, Randomness and Incompleteness,
World Scientic, 1987.
[2] N.C.A. da Costa and F.A. Doria, Int. J. Theor. Phys 30, 1041
(1991).
[3] N.C.A. da Costa and F.A. Doria, Suppes predicates and the con-
struction of unsolvable problems in the axiomatized sciences, to appear in
Patrick Suppes, Mathematician, Philosopher, P. Humphreys, ed.,
Kluwer, 1993.
[4] N.C.A. da Costa and F.A. Doria, Very hard satisability problems,
preprint CETMC/ECO, 1992.
[5] A.N. Kolmogorov, Problemy Peredachi Informatsii 1, 3 (1965).
[6] M. Machtey, P. Young, An Introduction to the General The-
ory of Algorithms, NorthHolland, 1979.
[7] Yu.I. Manin, A Course in Mathematical Logic, Springer, 1977.
[8] P. MartinLof, Inf. and Control 9, 602 (1966).
[9] H. Rogers Jr., Theory of Recursive Functions and Eective
Computability, McGrawHill, 1967.
[10] A.K. Zvonkin and L.A. Levin, Uspekhi Matem. Nauk 25, 1
(1970).
132
University of Sao Paulo
Research Group on Logic and Foundations
Institute for Advanced Studies and Department of Philosophy
05508 Sao Paulo SP
Brazil
Federal University at Rio de Janeiro
Research Center on Mathematical Theories of Communication
School of Communications
Av. Pasteur 250
22290 Rio de Janeiro RJ
Brazil
133