EECS 70 Discrete Mathematics and Probability Theory Fall 2020
1 Mathematical Induction
Note 3
Introduction. In this note, we introduce the proof technique of mathematical induction. Induction is a powerful tool which is used to establish that a statement holds for all natural numbers. Of course, there are infinitely many natural numbers — induction provides a way to reason about them by finite means.
Let us demonstrate the intuition behind induction with an example. Suppose we wish to prove the statement:
For all natural numbers n, 0 + 1 + 2 + 3 + · · · + n = n(n+1) . More formally, using the universal quantifier 2
from Note 1, we can write this as:
n n(n+1)
∀n∈N, ∑i= 2 . (1)
i=0
How would you prove this? Well, you could begin by checking that it holds for n = 0, 1, 2, and so forth. But there are an infinite number of values of n for which it needs to be checked! Moreover, checking just the first few values of n does not suffice to conclude the statement holds for all n ∈ N, as the following example demonstrates:
Sanity check! Consider the statement: ∀n ∈ N, n2 − n + 41 is a prime number. Check that it holds for the first few natural numbers. (In fact, you could check all the way up to n = 40 and not find a counterexample!) Now check the case of n = 41.
2
In mathematical induction, we circumvent this problem by making an interesting observation: Suppose the statement holds for some value n = k, i.e., ∑k i = k(k+1) . (This is called the induction hypothesis.) Then:
i=0
k k(k+1)
(k+1)(k+2)
∑i +(k+1)= 2 +(k+1)= 2 , (2)
i=0
i.e., the claim also holds for n = k + 1! In other words, if the statement holds for some k, then it must also hold for k + 1. Let us call the argument above the inductive step. The inductive step is a very powerful tool: If we can show the statement holds for k, then the inductive step allows us to conclude that it also holds for k+1; but now that it holds for k+1, the inductive step implies that it holds for k+2; and so on. In fact, we can repeat this argument indefinitely for all n ≥ k. So is that it? Have we proven Equation (1)? Almost!
The problem is that in order to apply the inductive step, we first have to establish that Equation (1) holds for some initial value of k. Since our aim is to prove the statement for all natural numbers, the obvious choice is k = 0. We call this choice of k the base case. Then, if the base case holds, the axiom of mathematical induction says that the inductive step allows us to conclude that Equation (1) indeed holds for all n ∈ N.
Let us now formally re-write this proof.
EECS 70, Fall 2020, Note 3 1
n n(n+1) Theorem3.1. ∀n∈N,∑i= 2
.
i=0
Proof. We proceed by induction on the variable n.
i=0
0 0(0+1) Base case (n = 0): Here, we have ∑i = 0 = 2
. Thus, the base case is correct.
Induction Hypothesis: For arbitrary n = k ≥ 0, assume that ∑k i = k(k+1) . In words, the Induction Hypoth-
i=0 2
esis says “let’s assume we have proved the statement for an arbitrary value of n = k ≥ 0”.
Inductive Step: Prove the statement for n = (k + 1), i.e., show that ∑k+1 i = (k+1)(k+2) : i=0 2
k+1 k k(k+1) k(k+1)+2(k+1) ∑i=∑i+(k+1)=2+(k+1)= 2 =2,(3)
i=0 i=0
where the second equality follows from the Induction Hypothesis. By the principle of mathematical induc-
tion, the claim follows.
Sanity check! How exactly did the Induction Hypothesis help us show the second equality in Equation (3)? (Hint: We used it to replace ∑ki=0 i with something useful.)
Recap. What have we learned so far? Letting P(n) denote the statement ∑n i = n(n+1) , our goal was to i=0 2
prove that ∀n ∈ N, P(n). The principle of induction asserts that to prove this requires three simple steps:
1. Base Case: Prove that P(0) is true.
2. Induction Hypothesis: For arbitrary k ≥ 0, assume that P(k) is true.
3. Inductive Step: With the assumption of the Induction Hypothesis in hand, show that P(k + 1) is true.
Let us visualize how these three steps fit together using dominoes. Let the statements P(i) be represented by a sequence of dominoes, numbered 0,1,2,…,n, such that P(0) corresponds to the 0th domino, P(1) corresponds to the 1st domino, and so on. The dominoes are lined up so that if the kth domino is knocked over, then it in turn knocks over the k+1st. Knocking over the kth domino corresponds to proving P(k) is true. And the induction step corresponds to the placement of the dominoes to ensure that if the kth domino falls, it in turn knocks over the k + 1st domino. The base case (n = 0) knocks over the 0t h domino, setting off a chain reaction that knocks down all the dominoes!
Finally, a word about choosing an appropriate base case — in this example we chose k = 0, but in general the choice of base case will naturally depend on the claim you wish to prove.
(k+1)(k+2)
EECS 70, Fall 2020, Note 3 2
Another proof by induction. Let us do another proof by induction. Recall from Note 1 that for integers a and b, we say that a divides b, denoted a|b, if and only if there exists an integer q satisfying b = aq.
Theorem 3.2. For all n ∈ N, n3 − n is divisible by 3.
Proof. We proceed by induction over n. Let P(n) denote the statement that n3 − n is divisible by 3.
Base case (n = 0): P(0) asserts that 3|(03 − 0) or 3|0, which is true since any non-zero integer divides 0.
Induction Hypothesis: Assume for n = k ≥ 0 that P(k) is true. That is, 3|(k3 − k), or equivalently k3 − k = 3q for some integer q.
Inductive Step: We show that P(k + 1) is true, i.e., that 3|((k + 1)3 − (k + 1)). To show this, we expand the number (k + 1)3 − (k + 1) as follows:
(k+1)3 −(k+1) =
= (k3 −k)+3k2 +3k
k3 +3k2 +3k+1−(k+1)
= 3q + 3(k2 + k) for some q ∈ Z (by the Induction Hypothesis)
= 3(q+k2+k).
We conclude that 3|((k + 1)3 − (k + 1)). Thus, by the principle of induction, ∀n ∈ N, 3|(n3 − n).
Sanity check! How exactly did the Induction Hypothesis help us show the third equality in the displayed derivation above?
Two Color Theorem. We now consider a more advanced proof by induction, which establishes a simpli- fied version of the famous four color theorem. The four color theorem states that any map can be colored with four colors such that any two adjacent countries1 have different colors. The four color theorem is very difficult to prove, and several bogus proofs were claimed since the problem was first posed in 1852. In fact, it was not until 1976 that a computer-assisted proof of the theorem was finally given by Appel and Haken. (For an interesting history of the problem and a state-of-the-art proof, which is nonetheless still very challenging, see www.math.gatech.edu/~thomas/FC/fourcolor.html.)
In this note, we consider a simpler version of the theorem in which our “map” is given by a rectangle which is divided into regions by drawing straight lines, such that each line divides the rectangle into two regions. Can we color such a simplified map using no more than two colors (say, red and blue) such that no two bordering regions have the same color? To illustrate, here is an example of a two-colored map:
1Countries are defined to be adjacent if they share any non-trivial part of a border, i.e., anything more than a point.
EECS 70, Fall 2020, Note 3 3
R
B
R
B
Theorem 3.3. Let P(n) denote the statement “Any map of the above form with n lines is two-colorable”. Then, it holds that ∀n ∈ N P(n).
Proof. We proceed by induction on n.
Base Case (n = 0): Clearly P(0) holds, since if we have n = 0 lines we can color the entire map using a
single color.
Induction Hypothesis: For some arbitrary n = k ≥ 0, assume P(k).
Inductive Step: We prove P(k + 1). Specifically, we are given a map with k + 1 lines and wish to show that it can be two-colored. Let’s see what happens if we remove a line. With only k lines on the map, the Induction Hypothesis says we can two-color the map. Let us make the following observation: Given a valid coloring, if we swap red ↔ blue, we still have a valid coloring. With this in mind, let us place back the line we removed, and leave the colors on one side of the line unchanged. On the other side of the line, swap red ↔ blue. This is illustrated in Figure 1. We claim that this is a valid two-coloring of the map with k + 1 lines.
Figure 1: (Left) The map with the (k + 1)st line removed. (Right) The map with the (k + 1)st line shown as a dashed line.
Why does this work? Consider any two regions separated by a shared border. Then, one of two cases must hold: Case 1 is when the shared border is the line that was removed and replaced, i.e., line k + 1. But by construction, we flipped the colors on one side of this line so that any two regions separated by it have distinct colors. Case 2 is when the shared border is one of the first k lines; here, the Induction Hypothesis guarantees that two regions separated by this border have different colors. Thus, in both cases, the regions separated by the shared border have distinct colors, as required.
2 Strengthening the Induction Hypothesis
When using induction, it can be very important to choose the correct statement to prove. For example, suppose we wish to prove the statement: for all n ≥ 1, the sum of the first n odd numbers is a perfect square. Here is a first proof attempt via induction.
Proof attempt. We proceed by induction on n.
Base Case (n = 1): The first odd number is 1, which is a perfect square.
Induction Hypothesis: Assume that the sum of the first k odd numbers is a perfect square, say m2.
Inductive Step: The (k + 1)st odd number is 2k + 1. Thus, by the Induction Hypothesis, the sum of the first k+1 odd numbers is m2 +2k+1. But now we are stuck. Why should m2 +2k+1 be a perfect square?
R
B
R
B
EECS 70, Fall 2020, Note 3 4
R
R
B
B R
B
R
It seems our Induction Hypothesis is too “weak”; it does not give us enough structure to say anything meaningful about the (k + 1) case.
So let’s take a step back for a moment, and do a preliminary check to ensure our claim isn’t obviously false: Let’s compute the values of the first few cases. Perhaps in the process, we can also uncover some hidden structure we have not yet identified.
• n=1:1=12 isaperfectsquare.
• n=2:1+3=4=22 isaperfectsquare.
• n=3:1+3+5=9=32 isaperfectsquare.
• n=4:1+3+5+7=16=42 isaperfectsquare.
It looks like we have good news and even better news: The good news is that we have not yet found a counterexample to our claim. The even better news is that there is a surprising pattern emerging: The sum of the first n odd numbers is not just a perfect square, but is equal precisely to n2! Motivated by this discovery, let’s try something counterintuitive: Let us try to show the following stronger claim.
Theorem 3.4. For all n ≥ 1, the sum of the first n odd numbers is n2.
Proof. We proceed by induction on n.
Base Case (n = 1): The first odd number is 1, which is equal to 12.
Induction Hypothesis: Assume that the sum of the first k odd numbers is k2.
Inductive Step: The (k + 1)-st odd number is 2k + 1. Applying the Induction Hypothesis, the sum of the first k + 1 odd numbers is k2 + (2k + 1) = (k + 1)2. Thus, by the principle of induction the theorem holds.
So let’s get this straight — we couldn’t prove our original statement, so instead we hypothesized a stronger one and managed to prove that. Why on earth did this work? The reason is that our original claim did not capture the true structure of the underlying fact we were trying to prove — it was too vague. As a result, our Induction Hypothesis wasn’t strong enough to prove our desired result. In contrast, although our second claim is a priori stronger, it also has more structure to it; this, in turn, makes our Induction Hypothesis stronger — we can use the fact that not only is the sum of the first k odd numbers a perfect square, but that it in fact equals k2. This additional structure is exactly what we needed to complete the proof. In summary, we have demonstrated an example in which, although a claim was true, the precise formulation of the Induction Hypothesis made the difference between a failing and a successful proof.
Example. Let us now try a second example; this time, we’ll let you do some of the work! Suppose we wishtoprovetheclaim: foralln≥1,∑n 1 ≤2.
Sanity check! The “obvious” choice of Induction Hypothesis says the following: Assume that for n = k, k1 k11
i=1 i2
∑i=1 i2 ≤ 2. Why does this not suffice to prove the claim for n = k+1, i.e., to show that ∑i=1 i2 + (k+1)2 ≤ 2?
(Hint:Isitpossiblethat∑k 1 actuallyequals2?) i=1 i2
EECS 70, Fall 2020, Note 3
5
Now let’s again do something counterintuitive — let’s prove the following stronger statement, i.e., let’s strengthen our induction hypothesis.
n11 Theorem3.5. Foralln≥1,∑i2 ≤2−n.
i=1 Proof. We proceed by induction on n.
111 BaseCase(n=1):Wehave∑i2 =1≤2−1,asrequired.
i=1
Induction Hypothesis: Assume that the claim holds for n = k.
Inductive Step: By the Induction Hypothesis, we have that k+11k11 11
∑i2 =∑i2 +(k+1)2 ≤2−k+(k+1)2. i=1 i=1
Thus, to prove our claim, it suffices to show that
−1+ 1 ≤−1. (4)
k (k+1)2 k+1
Exercise. Show that Equation (4) holds. (Hint: Multiply both sides of the inequality by (k + 1).)
By the principle of mathematical induction, the claim holds.
3 Simple Induction vs. Strong Induction
Thus far, we have been using a notion of induction known as simple or weak induction. There is another notion of induction, which we now discuss, called strong induction. The latter is similar to simple induction, except we have a slightly different induction hypothesis: Instead of just assuming P(k) is true (as was the case with simple induction), we assume the stronger statement that P(0), P(1), . . . , and P(k) are all true (i.e., that ki=0 P(i) is true).
Is there a difference between the power of strong and weak induction, i.e., can strong induction prove statements which weak induction cannot? No! Intuitively, this can be seen by returning to our domino analogy from Section 1. In this picture, weak induction says that if the kth domino falls, so does the (k + 1)st one, whereas strong induction says that if dominos 1 through k fall, then so does the (k + 1)st one. But these scenarios are equivalent: If the first domino falls, then applying weak induction repeatedly we conclude that the first domino in turn topples dominos 2 through k, setting up the case of k + 1, just as is the case with strong induction. With that said, strong induction does have an appealing advantage — it can make proofs easier, since we get to assume a stronger hypothesis. How should we understand this? Consider the analogy of a screwdriver and a power screwdriver — they both accomplish the same task, but the latter is much easier to use!
EECS 70, Fall 2020, Note 3 6
Let’s try a simple example2 of strong induction. As a bonus, this is our first induction proof requiring multiple base cases.
Theorem 3.6. For every natural number n ≥ 12, it holds that n = 4x + 5y for some x, y ∈ N.
Proof. We proceed by induction on n.
Base Case (n = 12): We have 12 = 4·3+5·0.
Base Case (n = 13): We have 13 = 4·2+5·1.
Base Case (n = 14): We have 14 = 4·1+5·2.
Base Case (n = 15): We have 15 = 4·0+5·3.
Induction Hypothesis: Assume that the claim holds for all 12 ≤ n ≤ k for k ≥ 15.
Inductive Step: We prove the claim for n = k + 1 ≥ 16. Specifically, note that (k + 1) − 4 ≥ 12; thus, the Induction Hypothesis implies that (k+1)−4 = 4x′ +5y′ for some x′,y′ ∈ N. Setting x = x′ +1 and y = y′ completes the proof.
Sanity check! Why would the proof above fail if we used weak induction instead of strong induction?
Let’s try a second, more advanced, example. For this, recall that a number n ≥ 2 is prime if 1 and n are its
only divisors.
Theorem 3.7. Every natural number n > 1 can be written as a product of one or more primes.
Proof. We proceed by induction on n. Let P(n) be the proposition that n can be written as a product of primes. We will prove that P(n) is true for all n ≥ 2.
Base Case (n = 2): We start at n = 2. Clearly P(2) holds, since 2 is a prime number.
Induction Hypothesis: Assume P(n) is true for all 2 ≤ n ≤ k.
Inductive Step: Prove that n = k + 1 can be written as a product of primes. We have two cases: either k + 1 is a prime number, or it is not. For the first case, if k+1 is a prime number, then we are done since k+1 is trivially the product of one prime (itself). For the second case, if k + 1 is not a prime number, then by definition k+1 = xy for some x,y ∈ Z+ satisfying 1 < x,y < k+1. By the Induction Hypothesis, x and y can each be written as a product of primes (since x, y ≤ n). But this implies that k + 1 can also be written as a product of primes.
Sanity check! Why does this proof fail if we instead use simple induction? (Hint: Suppose k = 42. Since 42 = 6 × 7, in the Inductive Step we wish to use the facts that P(6) and P(7) are true. But weak induction does not give us this — which value of P does weak induction allow us to assume is true in this case?)
2This problem also goes under the name of the Postage Stamp Problem, which states that every amount of postage of 12 cents or more can be paid using only 4- and 5-cent stamps.
EECS 70, Fall 2020, Note 3 7
Finally, for those who wish to have a more formal understanding of why weak and strong induction are equivalent,considerthefollowing. LetQ(n)=P(0)∧P(1)∧···∧P(n). Then,stronginductiononPis equivalent to weak induction on Q. Try to convince yourself of this claim.
4 Recursion, programming and induction
There is an intimate connection between induction and recursion, which we explore here via two examples.
Example 1: Fibonacci’s rabbits. In the 13th century there lived a famous Italian mathematician known as Fibonacci3, who in 1202 considered the following puzzle: Starting with a pair of rabbits, how many rabbits are there after a year, if each month each pair begets a new pair which from the second month on becomes productive? This model of population growth can be modeled by recursively defining a function; the resulting sequence of numbers is nowadays referred to as the Fibonacci numbers.
To recursively model our rabbit population, let F(n) denote the number of pairs of rabbits in month n. By the rules above, our initial conditions are as follows: Clearly F(0) = 0. In month 1, a single pair of rabbits is introduced, implying F(1) = 1. Finally, since it takes a month for new rabbits to become productive, we also have F(2) = 1.
In month 3, the fun begins. For example, F(3) = 2, since the original pair breeds to produce a new pair. But how about F(n) for a general value of n? It seems difficult to give an explicit formula for F(n). However, we can define F (n) recursively as follows. In month n − 1, by definition we have F (n − 1) pairs. How many of these pairs were productive? Well, only those that were already alive in the previous month, i.e., F (n − 2) of them. Thus, we have F(n−2) new pairs in the nth month in addition to our existing F(n−1) pairs. Hence, we have F(n) = F(n−1)+F(n−2). To summarize:
• F(0)=0.
• F(1)=1.
• For n ≥ 2, F(n) = F(n−1)+F(n−2).
Pretty neat, except you might wonder why we care about rabbit reproduction in the first place! It turns out this simplified model of population growth illustrates a fundamental principle: Left unchecked, populations grow exponentially over time. The following exercise gives a lower bound on this exponential rate of growth. (The true rate of growth is rather larger, around 1.6n.)
Exercise. Use induction to show that the Fibonacci numbers satisfy F(n) ≥ 2(n−1)/2 for all n ≥ 3. Note that you will need two base cases: n = 3 and n = 4. (Why?)
In fact, understanding the significance of this unchecked exponential population growth was a key step that led Darwin to formulate his theory of evolution. To quote Darwin: “There is no exception to the rule that every organic being increases at so high a rate, that if not destroyed, the earth would soon be covered by the progeny of a single pair."
3Fibonacci was also known as Leonardo of Pisa. If you ever find yourself in Pisa, Italy, you can visit his grave; his remains are buried at Campo Santo.
EECS 70, Fall 2020, Note 3 8
Now, we promised you programming in the title of this section, so here it is — a trivial recursive program to evaluate F(n):
function F(n)
if n=0 then return 0
if n=1 then return 1
else return F(n-1) + F(n-2)
Exercise. How long does it take this program to compute F(n), i.e., how many calls to F(n) are needed? You should be able to show by induction that the number of calls is at least F(n) itself!
The exercise above should convince you that this is a very inefficient way to compute the nth Fibonacci number. Here is a much faster iterative algorithm to accomplish the same task (this should be a familiar example of turning a tail-recursion into an iterative algorithm):
function F2(n)
if n=0 then return 0 if n=1 then return 1 a=1
b=0
for k = 2 to n do
temp = a a=a+b b = temp
return a
Exercise. How long does this iterative algorithm take to compute F2(n), i.e., how many iterations of its loop are needed? Can you show by induction that this new function F2(n) = F(n)?
Example 2: Binary search. We next use induction to analyze a recursive algorithm which even your grandmother is likely familiar with — binary search! Let us discuss binary search in the context of finding a word in the dictionary: To find word W , we open the dictionary to its middle page. If the first word of that page comes after W , we recurse on the first half of the dictionary (before the middle page); else, we recurse on the second half of the dictionary (from the middle page onwards). (When the dictionary has an even number 2m of pages, we define the middle page to be the (m + 1)st page.) Once we’ve narrowed down the dictionary to a single page, we resort to a brute force search to find W on that page. In pseudocode, we have:
EECS 70, Fall 2020, Note 3 9
1. //precondition: W is a word and D is a subset of the dictionary with at least 1 page.
2. //postcondition: Either the definition of W is returned, or “W not found” is returned.
3. findWord(W,D){
4. //Base case
5. If (D has precisely one page)
6. Look for W in D by brute force. If found, return its definition; else, return “W not found”.
7.
8. //Recursive case
9. Let W ′ be the first word on the middle page of D.
10. If(W comesbeforeW′ )
11. return findWord( first half of D )
12. Else
13. return findWord( second half of D )
14. }
Let us prove using induction that findWord() is correct, i.e., it returns the definition of W if W is in the dictionary D. As a bonus, the proof requires us to use strong induction instead of simple induction!
Proof of correctness of findWord(). We proceed by strong induction on the number of pages, n, in D.
Base Case (n = 1): If D has one page, line 6 searches via brute force for W . Thus, if W is present, it is found
and returned; else, we return “W not found”, as desired.
Induction Hypothesis: Assume that findWord() is correct for all 1 ≤ n ≤ k.
Inductive Step: We prove findWord() is correct for n = k+1. After Step 9, we know W′; thus we can determine whether W must be in the first or second half of D. In the first case, we recurse on the first half of D in line 11, and in the second case we recurse on the second half of D in line 13. By the Induction Hypothesis, the recursive call correctly finds W in the first or second half, or returns “W not found”, since both of these two halves contain at most k pages. Since we return the recursive call’s answer, we conclude that findWord() correctly finds W in D of size n = k + 1. By induction, findWord() is therefore correct.
Sanity check! Why did we require strong induction in the proof above?
5 False proofs
It is very easy to prove false things if your proof is incorrect! Let’s illustrate with a famous example. In the middle of the last century, a colloquial expression in common use was “that is a horse of a different color", referring to something that is quite different from normal or common expectation. The reknowned mathematician George Pólya, who was also a great expositor of mathematics for the lay public, gave the following proof to show that there is no horse of a different color!
Theorem 3.8. All horses are the same color.
Proof. We proceed by induction on the number of horses, n. Let P(n) denote the statement of the claim. EECS 70, Fall 2020, Note 3 10
Base Case (n = 1): P(1) is certainly true, since if you have a set containing just one horse, all horses in the set have the same color.
Induction Hypothesis: Assume P(n) holds for some arbitrary n ≥ 1.
Inductive Step: Given a set of n+1 horses {h1,h2,...,hn+1}, we can exclude the last horse in the set and apply the induction hypothesis just to the first n horses {h1,...,hn}, deducing that they all have the same color. Similarly, we can conclude that the last n horses {h2,...,hn+1} all have the same color. But now the “middle” horses {h2,...,hn} (i.e., all but the first and the last) belong to both of these sets, so they have the same color as horse h1 and horse hn+1. It follows, therefore, that all n+1 horses have the same color. We conclude, by the principle of induction, that all horses have the same color.
Clearly, it is not true that all horses are of the same color! So, where did we go wrong? Recall that in order for the principle of mathematical induction to apply, the induction step must show the statement: ∀n ≥ 1, P(n) =⇒ P(n + 1). We claim that in the proof above, this statement is false — in particular, there exists a choice of n which yields a counterexample to this statement.
Exercise. For which value of n is it not true that P(n) =⇒ P(n + 1)? (Hint: Think about the key property in the proof of having “middle” horses.)
6
Practice Problems
1. Prove for any natural number n that 12 +22 +32 +...+n2 = 1n(n+1)(2n+1). 6
2. In real analysis, Bernoulli’s Inequality is an inequality which approximates the exponentiations of 1+x. Prove this inequality, which states that (1+x)n ≥ 1+nx if n is a natural number and 1+x > 0.
3. A common recursively defined function is the factorial, defined for a nonnegative number n as n! = n(n − 1)(n − 2)…1, with base case 0! = 1. Let us reinforce our understanding of the connection between recursion and induction by considering the following theorem involving factorials.
Theorem: Foralln∈N,n>1=⇒n!