Non-linear data structures, 2

layout:true
template: default
name: section
class: inverse, middle, center

---
layout:true
template: default
name: challenge
class: challenge

---
layout:true
template: default
name: poll
class: inverse, full-height, center, middle

---
layout:true
template: default
name: breakout
class: breakout

---

layout:true
template:default
name:slide
class: slide

---

## Challenge

Given `N` integers in an arbitrary order find `k` largest ones.

---
class: center, title-slide

## CSCI-UA 480: APS
## Algorithmic Problem Solving
 
## Non-Linear Data Structures
### Priority Queue

.license[
Copyright 2020 Joanna Klukowska. Unless noted otherwise all content is released under a 
[Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/). 
Background image by Stewart Weiss ]

]

.license[
Unless noted otherwise all content is released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
]

---

## Priority Queue

A __priority queue__ is a data structure that supports
- element insertion
- retrieval/access and removal of either minimum or maximum element
(depending on the type of the queue)

Functionally, it is similar to an ordinary queue in which the priorities
are determined based on the time of arrival of an element (first in first out).

In a priority queue, priorities associated with the elements determine the
next minimum or maximum element.

Multiple elements in a priority queue can have the same priority associated with them.

Priority queues are typically implemented using a _binary heap_ structure.

---
## Binary Heap / Priority Queue

A _binary heap_ is binary tree with two additional properties:

- shape property: it is a complete tree  (i.e., all levels are filled except for the lowest one, the
lowest one is filled from left to right)

- order property:  for each node the values of its children are
    - smaller than or equal to the one in the node (max-heap)
    - larger than or equal to the one in the node (min-heap)

Performance of the operations:
  - add $O(log N)$
  - remove $O(log N)$
  - top  $O(1)$

Building a heap from a collection of values is $O(N)$ !!!  Why?

---

## Building a Binary Heap in O(N)

Visualizations:

- [VisuAlgo](https://visualgo.net/bn/heap)
  - allows you to customize the elements
  - has both $O(n \log n)$ and $O(N)$ processes illustrated

- [USFCA visualization](https://www.cs.usfca.edu/~galles/visualization/Heap.html)
    - illustrates it with the worst case when the array is reverse sorted

---
## Examples:

.left-column2-smaller[
in C++'s' `priority_queue` the defaul order of elements is "decreasing" (max-heap)

```C++
priority_queue<int> q;

q.push(3);
q.push(5);
q.push(7);
q.push(2);
cout << q.top() << "\n"; // 7
q.pop();
cout << q.top() << "\n"; // 5
```
To reverse the ordering of elements, instantiate `q` as

```C++
priority_queue<int,
 vector<int>,
 greater<int> > q;
```

]
.right-column2-larger[
in Java's `PriorityQueue<>` the defaul order of elements is "increasing" (min-heap)

```Java
PriorityQueue<Integer> q =
 new PriorityQueue<Integer>();
q.add(3);
q.add(5);
q.add(7);
q.add(2);
System.out.println( q.peek() ); // 2
q.remove();
System.out.println( q.peek() ); // 3
```
.smaller[
To reverse the order of elements in `q`, define a `Comparator`

```java
class Comp implements Comparator<Integer>{
 public int compare (Integer a, Integer b) {
 return b-a;
 }
}
```

and then instantiate `q` as

```java
PriorityQueue<Integer> q =
 new PriorityQueue<Integer>(new Comp());
```
]
]
---

## Challenge

- Given `n` integers in arbitrary order find `k` largest ones.

- A basic max-heap  only supports element addition and removal of the max
element. How would you implement the following two operations:
  - `update(index, new_value)` - modify the value at a specified index to the new_value
  - `delete(index)` - deletes the value at a specified index

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 1__:

- sort the array
- extract k largest values from one of the ends

__Performance__:

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 1__:

- sort the array
- extract k largest values from one of the ends

__Performance__: $O(n \log n)$

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 2__:

- repeat `findMax` algorithm on an unsorted array `k` times

__Performance__:

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 2__:

- repeat `findMax` algorithm on an unsorted array `k` times

__Performance__: $O(k n)$

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 3__:

- build a max-heap from the array elements
- repeat `removeMax` `k` times

__Performance__:

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 3__:

- build a max-heap from the array elements
- repeat `removeMax` `k` times

__Performance__: $O(n + k \log n)$

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 4__:

- for i: 1 to k
  - add the i'th element to a min-heap

- for i: k+1 to n
  - add the i'th element to a min-heap
  - remove the smallest element from the min-heap

- the values in the min-heap are the largest ones from the data set

__Performance__:

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 4__:

- for i: 1 to k
  - add the i'th element to a min-heap

- for i: k+1 to n
  - add the i'th element to a min-heap
  - remove the smallest element from the min-heap

- the values in the min-heap are the largest ones from the data set

__Performance__: $O(n \log k)$

---
## k-largest values

__Problem__:  Given $n$ integers in arbitrary order find `k` largest ones.

__Solution 5__:

- use a modification of quicksort to obtain partition that contains the `k`
elements we are interested in (related to selection rank algorithm)

__Performance__: average $O(n)$, worst case $O(n^2)$

We'll come back to this solution later.

---
template: challenge

## Median

__Problem 1__: Find the median

Given $n$ integers in arbitrary order find their median.

__Problem 2__: Find the running median

You are given a total of $n$ integers in arbitrary order. As you are reading
the values one by one, we want to know the value of the median for the values read in so far.

---

## Median - solutions

- use any of the k-largest values approach, but set k = n/2 (or adjust accordingly
for the even n) and only use the k'th value

- use two heaps: min-heap (stores values larger than the median) and max-heap
(stores values smaller than the median) -> median is always at the top of one
of the heaps (or the average of the two tops)

- use the median of median's algorithm (see [Wikipedia](https://en.wikipedia.org/wiki/Median_of_medians) or
an algorithms textbook)

---

## Challenge

- Is the structure represented by a 1-based compact array (ignoring
index 0) sorted in descending order a Max Heap?

- Given a max-heap represented by a 1-based compact array (ignoring index 0)
what are the possible indexes at which the second largest element is located?

- Given a max-heap represented by a 1-based compact array (ignoring index 0)
what are the possible indexes at which the third largest element is located?

- Given a max-heap represented by a 1-based compact array (ignoring index 0)
what are the possible indexes at which the smallest element is located?

</optgroup>