CS125 Lecture 16 Fall 2014
16.1 Countability
Proving the non-existence of algorithms for computational problems can be very difficult. Indeed, we do not know how to prove P ̸= NP. So a natural question is whether can show that there are any problems outside of P. Here we will show that in fact that “most” problems are outside P. In fact, most problems have no algorithm whatsoever (regardless of efficiency). We will show this by a counting argument, arguing that there are (many) “more” languages than there are algorithms. To do this, we need be able to distinguish the sizes of infinite sets (as there are infinitely many languages and infinitely many algorithms).
Definition 16.1 A set S is countably infinite iff there is a bijection f : N → S.1 S is countable iff it is either finite or countably infinite. A set S is uncountable if it is not countable.
Countably infinite sets are the “smallest” infinite sets. (Every infinite set contains a countably infinite set.) Proposition 16.2 A set is countable iff there is a surjection f : N → S.2
That is, a set S is countable iff we can “enumerate” the elements of S in a sequence f (0), f (1), f (2), . . . (which may have repetitions). The proof of this proposition is a good exercise.
Examples of countable sets:
•N
•Z
• N×N
•Q
• Σ∗ for any finite, nonempty alphabet Σ.
1A bijection is a function that is one-to-one and onto. 2A surjection is an onto function.
16-1
Lecture 16 16-2
• The class R of recursive (=decidable) languages over an alphabet Σ. (Recall that a language L is decidable iff there is an algorithm M such that for every input x, M(x) halts and outputs 1 or 0 according to whether or not x∈L.)
• The class R.E. of recursively enumerable (=recognizable) languages over an alphabet Σ. (Recall that a lan- def
guage L is recognizable iff there is an algorithm M such that L = L(M) = {x : M(x) accepts x}.
We record some useful properties of countable sets (proving them is a good exercise).
Theorem 16.3 1. Every subset of a countable set is countable. 2. The union of countably many countable sets is countable.
Countability 3 3. The direct product of two countable sets is countable.
4. The image of a countable set under any function is countable.
Now, we need some uncountable sets.
YES! Theorem 16.4 If S is infinite, then P(S) is uncountable. Proof:
Theorem: P(N) is uncountable
(The set of all sets of natural numbers)
• Suffices to prove for S = N (why?). Proof by contradiction:
Are there uncountable sets? (Infinite but not countably infinite)
• Suppose for contradiction that P(N ) were countable.
(i.e. assume that P (N) is countable and show that this results in a contradiction)
• Then there is an enumeration of all subsets of N say P(N ) = {S0,S1,…}
• Suppose that P(N) were countable.
• Then there is an enumeration of all subsets of N say P (N) = {S0, S1, . . .}
j=01234 Si
S0
S1
S2
S3
. D
… … … …
YNNYN NNNNN YYNYY NNNYN
“Y ” in cell (i, j) means j ∈ Si
• LetD={i∈N:i∈Si}bethediagonal.
Diagonalization
“Y”inrowi,column jmeans j∈Si
D=YNNY …={0,3,…}
Lecture 16 16-3
• LetD={i∈N:i∈Si}bethediagonal. D = Y N NY . . . = {0, 3, . . .}
• Let D = N−D be its complement. D=NYYN…={1,2,…}
• Claim: D is omitted from the enumeration, contradicting the assumption that every set of natural numbers is one of the Sis.
Pf: D is different from each row because they differ at the diagonal.
Corollary 16.5 For every nonempty finite alphabet Σ, there are uncountably many languages over Σ. In particular, there are uncountably many non-r.e. languages.
16.2 Universal Algorithms
The argument in the previous section is nonconstructive — it tells us that there many undecidable languages, but doesn’t immediately tell us how to get our hands on any of them. In the next section, we will see how to make it more constructive and deduce that specific problems are undecidable. These will initially be problems where the input itself includes the description of an algorithm (much like the languages like ALLDFA you considered on HW8, where the input is a finite automaton). So first we should discuss what can be done for such problems.
Theorem 16.6 (Universal Word RAM) There is a single Word-RAM program UWR such that for every Word-RAM program P and input x ∈ N∗ and k ≥ maxi xi, UWR(⟨P,x,k⟩) has the same behavior as P(x;k). [Here k represents the bound on the size of numbers given to P.]
That is, if P(x;k) halts with an output of y, so does UWR(⟨P,x,k⟩). If P does not halt, neither does UWR. Moreover, if P(x,k) halts in TP(x,k) steps, then UWR(⟨P,x,k⟩) halts in O(TP(x,k)+O(|⟨P,x,k⟩|).
Proof:[sketch] Registers of UWR:
• qP = number of lines in P
• rP = number of registers used by P
• program counter lP = current line number in simulation of P • 2wP where wP is current word size of P
• SP current space usage of P
• Scratch registers
Memory of UWR:
Lecture 16 16-4
• first 4qP locations = lines of the program P, with each line being described by 4 values: – Instruction type (0–11, indicating +, -, ·, MALLOC, HALT, etc.)
– register numbers i, j,k, constants m, or line number l used in instruction
• next rP locations = registers of P
• then we put the memory of P (one location of UWR’s memory per location of P’s memory)
Then each step of P can be simulated using O(1) steps of UWR. Need O(|⟨P,x,k⟩|) steps at beginning to format initial configuration and O(TP(x,k)) steps at end to format output configuration.
Corollary 16.7 (Universal TM) There is a single (multitape) Turing Machine UTM such that for every (multitape) Turing Machine M and input x ∈ Σ∗M, U(⟨M,x⟩) has the same behavior as M(x).
Moreover, if M(x) halts in TM(x) steps, then UTM(⟨M,x⟩) halts in O ̃(TM(x)+|⟨M,x⟩|)2 steps, where O ̃(f) is shorthand for f · poly(log f ).
Proof:
The Universal TM (introduced by Turing) had important technological significance. It meant that computers could be programmable. Instead of building special-purpose hardware for each algorithm you want to implement (what people had done before — essentially constructing calculators), you could build one general-purpose machine that can implement any desired algorithm. Universality also has significance for software. For example, each new generation of C compilers is written using the previous generation of C compilers!
We are interested in the following theoretical implication of the Universal Machines:
Corollary 16.8 The ACCEPTANCE PROBLEMS for Turing machines and Words-RAMs are recognizable (a.k.a. re-
cursively enumerable):
1. ATM = {⟨M,x⟩ : M at TM that accepts x}.
2. AWR = {⟨P, x, k⟩ : P a Word-RAM such that P(x; k) accepts}.
A natural question is whether these languages are decidable (a.ka. recursive).
16.3 Undecidability of the Acceptance Problem
ATM is a natural choice for showing that undecidability because it is r.e.-complete. That is, every r.e. language L = L(M) reduces to ATM:
Before proving its undecidability, it’s useful to make a simplifying detail — that every string represents a TM.
Lecture 16 16-5
• Let Σ be the alphabet over which TMs are represented (that is, ⟨M⟩ ∈ Σ∗ for any TM M)
• Letw∈Σ∗
• ifw=⟨M⟩forsomeTMMthenwrepresentsM
• Otherwise w represents some fixed TM M0 (say the simplest possible TM).
Theorem 16.9 ATM is not recursive. Proof:
• Look at ATM as a table answering every question: w0 w1 w2 w3
M0 Y N N Y M1YY NN M2NN NN M3YY YY
(WLOG assume every string wi encodes a TM Mi)
• Entry matching (Mi, wj) is Y iff Mi accepts wj
• If ATM were recursive, then so would be the diagonal D and its complement.
– D = {wi : Mi accepts wi}.
– D={wi :Mi doesnotacceptwi}.
• But D differs from every row, i.e. it differs from every r.e. language. ⇒⇐.
Unfolding the Diagonalization
• Suppose for contradiction that ATM were recursive.
• Then there is a TM M∗ that decides D = {⟨M⟩ : M does not accept ⟨M⟩}:
• Run M∗ on its own description ⟨M∗⟩. • Does it accept?
M∗ accepts ⟨M∗⟩
⇔ ⟨M∗⟩ ∈ D
⇔ M∗ does not accept ⟨M∗⟩.
Lecture 16 16-6
• Contradiction!
Corollary 16.10 AWR is undecidable. (And same for the Acceptance Problem in any universal model of computa-
tion.)
turing.jpg 170!201 pixels
Alan Mathison Turing (1912-1954)
24 years old when he published On computable numbers . . .
16.4 Non-Recognizable Languages
Theorem 16.11 A language L is decidable iff L is recognizable and co-recognizable. That is, R = R.E. ∩ co-R.E..
Proof: ⇒: R is a subset of R.E. and is closed under complement. ⇐:
Corollary 16.12 ATM and AWR are not recognizable.
http://www.galha.org/image/turing.jpg