��������� ������ �� ����������
������� � ���� �
������ ����
������� ��
�������� �� ����
�
�������� ������
� � ��� �� ��������� ���� ��� ���� � �������� ����
�
�������� ������
� � ��� �� ��������� ���� ��� ���� � �������� ���� � ������� ��� � ��� ������� �� ��� ������
�
�������� ������
� � ��� �� ��������� ���� ��� ���� � �������� ����
� ������� ��� � ��� ������� �� ��� ������
� ������ ��� ��� ������� ���� ��� ������� �������� ��� ������ �� ���� ��� ������
�
�������� ������
� � ��� �� ��������� ���� ��� ���� � �������� ����
� ������� ��� � ��� ������� �� ��� ������
� ������ ��� ��� ������� ���� ��� ������� �������� ��� ������ �� ���� ��� ������
� ���� �� ���� �� �� ��������� ���� ������������
�
�������� ������
� � ��� �� ��������� ���� ��� ���� � �������� ����
� ������� ��� � ��� ������� �� ��� ������
� ������ ��� ��� ������� ���� ��� ������� �������� ��� ������ �� ���� ��� ������
� ���� �� ���� �� �� ��������� ���� ������������
� ����� �� ��� ��� ���� �� ��� ������������
�
������� ����� � �������� �����
� �������� ��������������� ������ �� �������� ������� �����������
�
������� ����� � �������� �����
� �������� ��������������� ������ �� �������� ������� �����������
� �� ����� �� �������� ����������� �� ������ ��������� �����
�
������� ����� � �������� �����
� �������� ��������������� ������ �� �������� ������� �����������
� �� ����� �� �������� ����������� �� ������ ��������� ����� � �� ����� �� ����� �� ������ ���������
�
��� ����
���� � ���� ���� � ��� �����������
�
��� ����
���� � ���� ���� � ��� �����������
� ������ ��� ���� ��� �������� ������� ��� �����
�
��� ����
���� � ���� ���� � ��� �����������
� ������ ��� ���� ��� �������� ������� ��� �����
� �������� ���� ��������� �� ������ ������� ��� ���� ����� �� ����� ������ ���� ��
�
��� ����
���� � ���� ���� � ��� �����������
� ������ ��� ���� ��� �������� ������� ��� �����
� �������� ���� ��������� �� ������ ������� ��� ���� ����� �� ����� ������ ���� ��
� �������� ���� ������ ��� ���� ������ ��� ��� ����� ����� ���� ��������� ������ ��� �� �������� �������� ���������
�
��� ����
���� � ���� ���� � ��� �����������
� ������ ��� ���� ��� �������� ������� ��� �����
� �������� ���� ��������� �� ������ ������� ��� ���� ����� �� ����� ������ ���� ��
� �������� ���� ������ ��� ���� ������ ��� ��� ����� ����� ���� ��������� ������ ��� �� �������� �������� ���������
� �������� ��������� ���� ��� �� � ������ ���� �� ������ ������ ���� ��� ��� �� ��� ���������
�
����� ��� ���������
����� �� ����� ��� ������ ✎
�� �� ��
� �� �� �� �� ��
� � �� �� � �� � � � �� �
�� �� ��
� �� �� � �� ��
��� ��� ������
�
����� �� ������
�� ��� ������� ��� ������������ �� ��� ���� ��� ����� ��� �������� �� ����������� �� �� ����� ��
���� ���� ��� �������� �� ���� � ���� �� ����� �� ��� �� + ��
�
�
� �� ��
����
���� �����
� � �� �� ��
�
�
�
�
�
��
� � � � � � � � � � �� �� ��
�
�
�
�
�
�
�
�
����� �� ������
�
�
� �� ��
����
���� ���� ��� ���� ��������� �� �������
∀� ∈ {�,�,…,�}� �� ���� ����
�[�] ≤ �[��/��]�
��
� � � � � � � � � � �� �� ��
���� �����
� � �� �� ��
�
�
�
�
�
�
�
�
�
�
�
�
�
��������
�������� ����������[�..�]� � ������ �[�] �� � �������� ���������[�..�]�
��� � ← � �� � ��
�������[�..�]�
�
�������
�������� �����������������[�..�]� ��� � ← ��/�� ������ � ��
�←�
� ← �[�]
���� ← �����
����� ��� ���� ��� � × � ≤ � ��
�←�×�
�� � < � ����
� ��� �������� �� �[�] < �[� + �] ���� � ← � + �
�� � ≥ �[�] ���� ���� ← ���� �����[�] ← �[�]� � ← �
�[�] ← �
�
Analysis of Bottom-Up Heapify
Assume the heap is a full binary tree: n = 2h+1 − 1.
10
Analysis of Bottom-Up Heapify
Assume the heap is a full binary tree: n = 2h+1 − 1.
Here is an upper bound on the number of “down-sifts” needed
h−1 i
2(h−i) =
The last equation can be proved by mathematical induction.
h−1
i=0 nodes at level i
i=0
2(h−i)2 = 2(n−log2(n+1))
10
∑∑∑
Analysis of Bottom-Up Heapify
Assume the heap is a full binary tree: n = 2h+1 − 1.
Here is an upper bound on the number of “down-sifts” needed
h−1 i
2(h−i) =
The last equation can be proved by mathematical induction.
Note that 2(n − log2(n + 1)) ∈ Θ(n), hence we have a linear-time algorithm for heap creation.
h−1
i=0 nodes at level i
i=0
2(h−i)2 = 2(n−log2(n+1))
10
∑∑∑
Eject
function Eject(A[1..i]) Swap(A[i], A[1]) k←1
v ← A[k]
heap ← False
while not heap and 2 × k ≤ i − 1 do
j←2×k
if j < i − 1 then
if A[j] < A[j + 1] then j ← j + 1 if v ≥ A[j] then heap ← True
elseA[k] ← A[j]; k ← j A[k] ← v
▷ two children
11
Heapsort - Properties
Transform-and-Conquer paradigm
12
Heapsort - Properties
Transform-and-Conquer paradigm
• Exactly as Selection Sort, but with a preprocessing step.
12
Heapsort - Properties
Transform-and-Conquer paradigm
• Exactly as Selection Sort, but with a preprocessing step.
Questions!
12
Heapsort - Properties
Transform-and-Conquer paradigm
• Exactly as Selection Sort, but with a preprocessing step.
Questions!
• In-place?
12
Heapsort - Properties
Transform-and-Conquer paradigm
• Exactly as Selection Sort, but with a preprocessing step.
Questions!
• In-place? • Stable?
12
Heapsort - Properties
• In-place?
13
Heapsort - Properties
• In-place? Yes! Only additional O(1) memory for auxiliary variables.
13
Heapsort - Properties
• In-place? Yes! Only additional O(1) memory for auxiliary variables.
• Stable?
13
Heapsort - Properties
• In-place? Yes! Only additional O(1) memory for auxiliary variables.
• Stable? No. Non-local swaps break stability.
13
Heapsort - Complexity
• We already saw in the videos that the complexity of Heapsort in the worst case is Θ(n log n).
14
Heapsort - Complexity
• We already saw in the videos that the complexity of Heapsort in the worst case is Θ(n log n).
• What’s the complexity in the best case?
14
Heapsort - Complexity
• Heapify is Θ(n) in the worst case.
15
Heapsort - Complexity
• Heapify is Θ(n) in the worst case. • Eject is Θ(log n) in the worst case.
15
Heapsort - Complexity
• Heapify is Θ(n) in the worst case.
• Eject is Θ(log n) in the worst case.
• In the worst case, heapsort is Θ(n) + n × Θ(log n) ∈ Θ(n log n).
15
Heapsort - Complexity (best case)
16
Heapsort - In practice
17
Heapsort - In practice
• Vs. Mergesort: fully in-place but not stable
17
Heapsort - In practice
• Vs. Mergesort: fully in-place but not stable
• Vs. Quicksort: guaranteed Θ(n log n) performance but empirically slower.
17
Heapsort - In practice
• Vs. Mergesort: fully in-place but not stable
• Vs. Quicksort: guaranteed Θ(n log n) performance but empirically slower.
• Used in the Linux kernel.
17
Heapsort - In practice
• Vs. Mergesort: fully in-place but not stable
• Vs. Quicksort: guaranteed Θ(n log n) performance but empirically slower.
• Used in the Linux kernel.
Take-home message: Heapsort is the best choice when low-memory footprint is required and guaranteed
Θ(n log n) performance is needed (for example, security reasons).
17
Sorting - Practical Summary
• Selection Sort: slow, but O(n) swaps.
18
Sorting - Practical Summary
• Selection Sort: slow, but O(n) swaps.
• Insertion Sort: low-memory, stable, excellent for small and almost sorted data.
18
Sorting - Practical Summary
• Selection Sort: slow, but O(n) swaps.
• Insertion Sort: low-memory, stable, excellent for small
and almost sorted data.
• Mergesort: good for mid-size data and when stability is required. Also good with secondary memory devices. Extra O(n) memory cost can be a barrier.
18
Sorting - Practical Summary
• Selection Sort: slow, but O(n) swaps.
• Insertion Sort: low-memory, stable, excellent for small
and almost sorted data.
• Mergesort: good for mid-size data and when stability is required. Also good with secondary memory devices. Extra O(n) memory cost can be a barrier.
• Quicksort: best for more general cases and large amounts of data. Not stable.
18
Sorting - Practical Summary
• Selection Sort: slow, but O(n) swaps.
• Insertion Sort: low-memory, stable, excellent for small
and almost sorted data.
• Mergesort: good for mid-size data and when stability is required. Also good with secondary memory devices. Extra O(n) memory cost can be a barrier.
• Quicksort: best for more general cases and large amounts of data. Not stable.
• Heapsort: slower in practice but low-memory and guaranteed Θ(n log n) performance.
18
Sorting - Paradigm Summary
• Selection Sort: brute force
19
Sorting - Paradigm Summary
• Selection Sort: brute force
• Insertion Sort: decrease-and-conquer
19
Sorting - Paradigm Summary
• Selection Sort: brute force
• Insertion Sort: decrease-and-conquer
• Mergesort, Quicksort: divide-and-conquer
19
Sorting - Paradigm Summary
• Selection Sort: brute force
• Insertion Sort: decrease-and-conquer
• Mergesort, Quicksort: divide-and-conquer • Heapsort: transform-and-conquer
19
Sorting - Paradigm Summary
• Selection Sort: brute force
• Insertion Sort: decrease-and-conquer
• Mergesort, Quicksort: divide-and-conquer • Heapsort: transform-and-conquer
There are all comparison-based sorting algorithms.
19
Sorting - Paradigm Summary
• Selection Sort: brute force
• Insertion Sort: decrease-and-conquer
• Mergesort, Quicksort: divide-and-conquer • Heapsort: transform-and-conquer
There are all comparison-based sorting algorithms.
Next lecture: Ok, my data is sorted. Now how do I keep it sorted?
19