BU CS 332 – Theory of Computation
Lecture 10:
• Turing Machines
Reading:
Sipser Ch 3.1-3.2
• TM Variants
Mark Bun February 26, 2020
Turing Machines – Motivation
So far in this class we’ve seen several limited models of computation
Finite Automata / Regular Expressions
• Can do simple pattern matching (e.g., substrings), check parity, addition
• Can’t recognize palindromes
Pushdown Automata / Context-Free Grammars
• Can count and compare, parse math expressions • Can’t recognize 𝑎𝑎𝑛𝑛𝑏𝑏𝑛𝑛𝑐𝑐𝑛𝑛 𝑛𝑛 ≥ 0}
2/26/2020 CS332 – Theory of Computation 2
Turing Machines – Motivation
Goal:
Define a model of computation that is
1) General purpose. Captures all algorithms that can be implemented in any programming language.
2) Mathematically simple. We can hope to prove that things are not computable in this model.
2/26/2020 CS332 – Theory of Computation 3
A Brief History
2/26/2020 CS332 – Theory of Computation 4
1900 – Hilbert’s Tenth Problem
Given a Diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: To devise a process according to which it can be determined in a finite number of operations whether the equation is solvable in rational integers.
2/26/2020 CS332 – Theory of Computation
5
David Hilbert
1862-1943
1928 – The Entscheidungsproblem The “Decision Problem”
Wilhelm Ackermann 1896-1962
David Hilbert
1862-1943
2/26/2020
CS332 – Theory of Computation
6
Is there an algorithm which takes as input a formula (in first-order logic) and decides whether it’s logically valid?
1936 – Solution to the Entscheidungsproblem “An unsolvable problem of elementary
number theory“
Model of computation: 𝜆𝜆-calculus (CS 320)
“On computable numbers, with an application to the Entscheidungsproblem”
Model of computation: Turing Machine CS332 – Theory of Computation 7
Alonzo Church 1903-1995
Alan Turing 1912-1954
2/26/2020
Turing Machines
2/26/2020 CS332 – Theory of Computation 8
The Basic Turing Machine (TM)
Tape 𝑎𝑎𝑏𝑏𝑎𝑎𝑎𝑎 Finite
…
Input
control
• Input is written on an infinitely long tape
• Head can both read and write, and move in both
directions
• Computation halts when control reaches
“accept” or “reject” state
2/26/2020 CS332 – Theory of Computation 9
Example0 → 0,𝑅𝑅 𝑞𝑞0
⊔ → ⊔, 𝑅𝑅
⊔→⊔,𝑅𝑅
𝑞𝑞1 0 → 0,𝑅𝑅 𝑞𝑞accept
𝑞𝑞reject
Example0 → 0,𝑅𝑅 𝑞𝑞0
⊔ → ⊔, 𝑅𝑅
𝑞𝑞reject 𝑞𝑞accept
𝑞𝑞1 0 → 0,𝑅𝑅
⊔ → ⊔, 𝑅𝑅
3 ⊔ → ⊔, 𝐿𝐿
𝑞𝑞 0 → 0,𝑅𝑅
TMs vs. Finite / Pushdown Automata
2/26/2020 CS332 – Theory of Computation 12
Three Levels of Abstraction
High-Level Description
An algorithm (like CS 330)
Implementation-Level Description
Describe (in English) the instructions for a TM • How to move the head
• What to write on the tape
Low-Level Description
State diagram or formal specification
2/26/2020 CS332 – Theory of Computation 13
Example
Decideif𝑤𝑤∈𝐴𝐴= 02𝑛𝑛 𝑛𝑛≥0}
High-Level Description
Repeat the following:
• If there is exactly one 0 in 𝑤𝑤, accept
• If there is an odd number of 0s in 𝑤𝑤 (> 1), reject • Delete half of the 0s in 𝑤𝑤
2/26/2020 CS332 – Theory of Computation 14
Example
Decideif𝑤𝑤∈𝐴𝐴= 02𝑛𝑛 𝑛𝑛≥0} Implementation-Level Description
1.
a) Cross off every other 0
2. 3.
c) If there is an odd number of 0s when we reach the right end of the tape, reject
While moving the tape head left-to-right:
b) If there is exactly one 0 when we reach the right end of the tape, accept
Return the head to the left end of the tape Go back to step 1
2/26/2020 CS332 – Theory of Computation 15
Example
Decideif𝑤𝑤∈𝐴𝐴= 02𝑛𝑛 𝑛𝑛≥0} Low-Level Description
2/26/2020 CS332 – Theory of Computation 16
Formal Definition of a TM
A TM is a 7-tuple 𝑀𝑀 = (𝑄𝑄, Σ, Γ, 𝛿𝛿, 𝑞𝑞0, 𝑞𝑞accept, 𝑞𝑞reject) • 𝑄𝑄 is a finite set of states
• Σ is the input alphabet (does not include ⊔) • Γ is the tape alphabet (contains ⊔ and Σ)
• 𝛿𝛿 is the transition function
…more on this later • 𝑞𝑞 ∈𝑄𝑄isthestartstate
0
• 𝑞𝑞accept ∈ 𝑄𝑄 is the accept state
• 𝑞𝑞reject ∈ 𝑄𝑄 is the reject state (𝑞𝑞reject ≠ 𝑞𝑞accept)
2/26/2020 CS332 – Theory of Computation 17
𝛿𝛿 ∶ 𝑄𝑄 × Γ → 𝑄𝑄 × Γ × {𝐿𝐿, 𝑅𝑅}
𝐿𝐿 means “move left” and 𝑅𝑅 means “move right”
TM Transition Function
𝛿𝛿 𝑝𝑝,𝑎𝑎 = (𝑞𝑞,𝑏𝑏,𝑅𝑅) means:
• Replace 𝑎𝑎 with 𝑏𝑏 in current cell
• Transition from state 𝑝𝑝 to state 𝑞𝑞 • Move tape head right
𝛿𝛿 𝑝𝑝,𝑎𝑎 = (𝑞𝑞,𝑏𝑏,𝐿𝐿) means:
• Replace 𝑎𝑎 with 𝑏𝑏 in current cell
• Transition from state 𝑝𝑝 to state 𝑞𝑞
• Move tape head left UNLESS we are at left end of tape, in which case don’t move
2/26/2020 CS332 – Theory of Computation 18
Configuration of a TM
A string with captures the state of a TM together with the contents of the tape
1010111⊔
𝑞𝑞5
…
2/26/2020 CS332 – Theory of Computation
19
Configuration of a TM: Formally
A configuration is a string 𝑢𝑢𝑞𝑞𝑢𝑢 where 𝑞𝑞 ∈ 𝑄𝑄 and 𝑢𝑢, 𝑢𝑢 ∈ Γ∗ • Tape contents = 𝑢𝑢𝑢𝑢 (followed by blanks ⊔)
• Current state = 𝑞𝑞
• Tape head on first symbol of 𝑢𝑢
…
Example: 101𝑞𝑞5 0111 1010111⊔
𝑞𝑞5
2/26/2020 CS332 – Theory of Computation
20
How a TM Computes
Start configuration: 𝑞𝑞0𝑤𝑤
One step of computation: •𝑢𝑢𝑎𝑎𝑞𝑞𝑏𝑏𝑢𝑢yields𝑢𝑢𝑎𝑎𝑐𝑐𝑞𝑞𝑞𝑢𝑢if 𝛿𝛿 𝑞𝑞,𝑏𝑏 =(𝑞𝑞𝑞,𝑐𝑐,𝑅𝑅) •𝑢𝑢𝑎𝑎𝑞𝑞𝑏𝑏𝑢𝑢yields𝑢𝑢𝑞𝑞𝑞𝑎𝑎𝑐𝑐𝑢𝑢if 𝛿𝛿 𝑞𝑞,𝑏𝑏 =(𝑞𝑞𝑞,𝑐𝑐,𝐿𝐿) • 𝑞𝑞𝑏𝑏𝑢𝑢yields𝑞𝑞𝑞𝑐𝑐𝑢𝑢if 𝛿𝛿 𝑞𝑞,𝑏𝑏 =(𝑞𝑞𝑞,𝑐𝑐,𝐿𝐿)
Accepting configuration: 𝑞𝑞 = 𝑞𝑞accept Rejecting configuration: 𝑞𝑞 = 𝑞𝑞reject
2/26/2020 CS332 – Theory of Computation 21
How a TM Computes
𝑀𝑀 accepts input 𝑤𝑤 if there is a sequence of configurations 𝐶𝐶1,…,𝐶𝐶𝑘𝑘 suchthat:
• 𝐶𝐶1 = 𝑞𝑞0𝑤𝑤
• 𝐶𝐶𝑖𝑖 yields 𝐶𝐶𝑖𝑖+1 for every 𝑖𝑖
• 𝐶𝐶𝑘𝑘 is an accepting configuration
𝐿𝐿(𝑀𝑀) = the set of all strings 𝑤𝑤 which 𝑀𝑀 accepts
𝐴𝐴 is Turing-recognizable if 𝐴𝐴 = 𝐿𝐿(𝑀𝑀) for some TM 𝑀𝑀:
•𝑤𝑤∈𝐴𝐴 ⟹ 𝑀𝑀haltson𝑤𝑤instate𝑞𝑞accept
•𝑤𝑤∉𝐴𝐴 ⟹ 𝑀𝑀haltson𝑤𝑤instate𝑞𝑞reject OR 𝑀𝑀 runs forever
2/26/2020 CS332 – Theory of Computation 22
Recognizers vs. Deciders
𝐿𝐿(𝑀𝑀) = the set of all strings 𝑤𝑤 which 𝑀𝑀 accepts
𝐴𝐴 is Turing-recognizable if 𝐴𝐴 = 𝐿𝐿(𝑀𝑀) for some TM 𝑀𝑀:
•𝑤𝑤∈𝐴𝐴 ⟹ 𝑀𝑀haltson𝑤𝑤instate𝑞𝑞accept
•𝑤𝑤∉𝐴𝐴 ⟹ 𝑀𝑀haltson𝑤𝑤instate𝑞𝑞rejectOR 𝑀𝑀 runs forever
𝐴𝐴 is (Turing-)decidable if 𝐴𝐴 = 𝐿𝐿(𝑀𝑀) for some TM 𝑀𝑀
which halts on every input
•𝑤𝑤∈𝐴𝐴 ⟹ 𝑀𝑀haltson𝑤𝑤instate𝑞𝑞 accept
•𝑤𝑤∉𝐴𝐴 ⟹ 𝑀𝑀haltson𝑤𝑤instate𝑞𝑞reject
2/26/2020 CS332 – Theory of Computation 23
Back to Hilbert’s Tenth Problem
Computational Problem: Given a Diophantine equation, does it have a solution over the integers?
𝐿𝐿=
• 𝐿𝐿 is Turing-recognizable
• 𝐿𝐿 is not decidable (1949-70)
2/26/2020 CS332 – Theory of Computation 24
TM Variants
2/26/2020 CS332 – Theory of Computation 25
How Robust is the TM Model?
Does changing the model result in different languages being recognizable / decidable?
So far we’ve seen…
– We can require that FAs/PDAs have a single accept state
– (CFGs can always be put in Chomsky Normal Form)
– Adding nondeterminism does not change the languages recognized by finite automata
Turing machines have an astonishing level of robustness 2/26/2020 CS332 – Theory of Computation 26
Extensions that do not increase the power of the TM model
• TMs that are allowed to “stay put” instead of moving
left or right 𝛿𝛿 ∶ 𝑄𝑄 × Γ → 𝑄𝑄 × Γ × 𝐿𝐿, 𝑅𝑅, 𝑆𝑆
Proof that TMs with “stay put” are no more powerful:
Simulation: Convert any TM 𝑀𝑀 with “stay put” into an equivalent TM 𝑀𝑀𝑞 without
Replace every “stay put” instruction in 𝑀𝑀 with a move right instruction, followed by a move left instruction in 𝑀𝑀’
2/26/2020 CS332 – Theory of Computation 27
Extensions that do not increase the power of
the TM model
• TMs with a 2-way infinite tape, unbounded left to right Input
Tape …𝑎𝑎𝑏𝑏𝑎𝑎 …
Proof that TMs with 2-way infinite tapes are no more
powerful:
Simulation: Convert any TM 𝑀𝑀 with 2-way infinite tape into
a 1-way infinite TM 𝑀𝑀𝑞 with a “two-track tape”
2/26/2020 CS332 – Theory of Computation 28
TMs are equivalent to…
• TMs with “stay put”
• TMs with 2-way infinite tapes
• Multi-tape TMs
• Nondeterministic TMs
• Enumerators
• Finite automata with access to an unbounded queue = 2-stack PDAs
• Primitive recursive functions
• Cellular automata …
2/26/2020 CS332 – Theory of Computation 29
⊔→⊔,𝑅𝑅 𝑞𝑞2 0 → 0,𝐿𝐿 𝑥𝑥 → 𝑥𝑥,𝑅𝑅 ⊔→⊔,𝐿𝐿
𝑥𝑥 → 𝑥𝑥,𝐿𝐿
𝑥𝑥 → 𝑥𝑥,𝑅𝑅𝑞𝑞0 0 →⊔,𝑅𝑅 𝑞𝑞1
0 → 𝑥𝑥,𝑅𝑅 𝑞𝑞3 𝑥𝑥 → 𝑥𝑥,𝑅𝑅 0 → 𝑥𝑥,𝑅𝑅 𝑞𝑞4 0 → 0,𝑅𝑅
⊔ → ⊔,𝑅𝑅𝑞𝑞reject ⊔ → ⊔,𝑅𝑅𝑞𝑞accept
2/26/2020 CS332 – Theory of Computation⊔ → ⊔, 𝑅𝑅 30
𝑥𝑥 → 𝑥𝑥,𝑅𝑅
𝑞𝑞0 0 → 0,𝑅𝑅
⊔ → ⊔, 𝑅𝑅
𝑞𝑞reject 𝑞𝑞accept
𝑞𝑞1 0 → 0,𝑅𝑅
⊔ → ⊔, 𝑅𝑅
3 ⊔ → ⊔, 𝐿𝐿
𝑞𝑞 0 → 0,𝑅𝑅
2/26/2020
CS332 – Theory of Computation
31