Dynamic Programming

class: center, title-slide

## CSCI-UA 480: APS
## Algorithmic Problem Solving
 
## Dynamic Programming
### Wedding Shopping

.author[
Instructor: Joanna Klukowska 
]

.license[
Copyright 2020 Joanna Klukowska. Unless noted otherwise all content is released under a 
[Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/). 
Background image by Stewart Weiss ]

---
layout:true
template: default
name: section
class: inverse, middle, center

---
layout:true
template: default
name: challenge
class: challenge

---
layout:true
template: default
name: poll
class: inverse, full-height, center, middle

---
layout:true
template: default
name: breakout
class: breakout

---

layout:true
template:default
name:slide
class: slide

.bottom-left[© Joanna Klukowska. CC-BY-SA.]

---
class:middle, center

## Wedding Shopping

---
## Challenge: Wedding Shopping

Given different options for each garment (e.g. 3 shirt models, 2 belt models,
4 shoe models, . . . ) and a certain limited budget,
our task is to buy one model of each garment.

We cannot spend more money than
the given budget, but we want to spend the maximum possible amount.

The __input__ consists of two integers 1 ≤ M ≤ 200 and 1 ≤ C ≤ 20, where M is the budget
and C is the number of garments that you have to buy, followed by some information about
the C garments.
For the garment g ∈ [0..C-1], we will receive an integer 1 ≤ K ≤ 20
which indicates the number of different models there are for that garment g, followed by K
integers indicating the price of each model ∈ [1..K] of that garment g.

The __output__ is one integer that indicates the maximum amount of money we can spend
purchasing one of each garment without exceeding the budget. If there is no solution due to
the small budget given to us, then simply print “no solution”.

---
## Challenge: Wedding Shopping

__Example 1__

M = 20, C = 3:

```
3:   6 4 8
2:   5 10
4:   1 5 3 5
```

Solution: 19

.left-column2[
3: 6 4 __8__ 
2: 5 __10__ 
4: __1__ 5 3 5 
]
.right-column2[
3: __6__ 4 8 
2: 5 __10__ 
4: 1 5 __3__ 5 
]

---
## Challenge: Wedding Shopping

### Greedy Attempt Fails

__Idea__: Go through each garment and pick the most expensive item within the
remaining budget.

__Problem__: Each decision influences the possible future decisions, making some of them impossible.
In many cases, this approach will lead to wrong answers (sub-optimal solution or no solution).

__Example__:

M = 12, C = 3:

```
3:   6 4 8
2:   5 10
4:   1 5 3 5
```

Greedy approach picks 8 for the first garment, and returns no solution since
the remaining budget of 4 does not allow us to pick anything else.

Optimal solution picks 6, 5, 1 and uses the entire budget.

__BUT__ greedy approach works for the original example when the budget was 20.

---
## Challenge: Wedding Shopping

### Divide and Conquer

Subproblems (selecting of each type of the garment) are not independent - the
solution depends on the choices made for other subproblems.

The divide and conquer approach is not suitable here.

---
## Challenge: Wedding Shopping

### Complete Search / Recursive Backtracking

Algorithm:

- start with money $m=M$ and garment category $i=0$,
- $shop( m, i )$ - returns the largest amount that can be spent assuming we have the available
amount of money $m$ and categories $i..C$ to chose from

```
shop( m, i )

if m < 0 return -INF //run out of money
 if i == C return (M - m) //finished and spent M-m

best = -1
    for g in  0 .. count[i]
        best  =  max ( best,  shop(m - price[i][g], i+1) )

return best
```

Produces correct answer, but slow
- in the largest case, each garment category has 20 different items and
there are 20 garment categories
- this gives $20^{20}$ recursive calls to solve the sub-problems
in the worst case - too many

---
## Challenge: Wedding Shopping

But, the good news is that some of those $20^{20}$ subproblems are overlapping and
we really have many fewer unique subproblems to solve.
--

- if any garment category has more than one item with the same price, then the two
subproblems (selecting one or the other of the garments) are the same, example:
    ```
    C = 3:

3:   6 1 8
    3:   5 10 5
    4:   1 5 3 5
    ```
    selecting either of the 5's in the second row, will lead to the same solutions
--

-  at the time that we look at garment cagory i, we might be starting with the same
amount of money even though we made different choices in the previous categories,  example:
    ```
    C = 3:

3:   6 1 8
    3:   5 10 5
    4:   1 5 3 5
    ```
    selecting either 6 and 5, or 1 and 10 will lead to the same _best_ option for
    the last row (in both cases, we already spend 11 out of the budget)

---
## Challenge: Wedding Shopping

.small[
consider the following functions and quantities
- $shop(m, i)$, $0 <= m <= M$ and $0 <= i <= C$ 
returns the amount of money we can spend using `m` dollars for the
garments in categories `i, i+1, ..., C-1`

- $p[i][g]$ is the price of garment g in garment category i

- $count[i]$ is the number of garments in garment category i
]

then the following are true

- if $m < 0$, then $shop(m, i) = -\infty$

- if $i == C$ (an item in last garment category has been purchased), then
$shop(m,i) = M-m$

- for all other cases,

$$shop(m, i) = \max_{g=0}^{count[i]}{ shop(m - p[i][g], i+1)}$$

In this problem, there are
- 201 options for M (since `0 <= M <= 200`)
- 20 options for item category (since `1 <= C <= 20`)

so there are 201*20 = 4020 possible subproblems (in the worst case).

---
## Wedding Shopping with DP (top-down)

```
answer = an M+1 by C 2D array initialized to -1

shop( m, i )

if m < 0 return -INF //run out of money
 if i == C return (M - m) //finished and spent M-m

// !!! without this line below, we are simply doing backtracking
    if answer[m][i] != -1   return answer[m][i]

best = -1
    for g in  0 .. count[i]
        best  =  max ( best,  shop(m - price[i][g], i+1) )

answer[m][i] = best

return best

```
---
.left-column2-small[
```
M = 20
C = 3
c0: 6, 1, 8
c1: 5, 10, 5
c2: 1, 5, 3, 5
```
]
.right-column2-large[
```
shop( m, i )
 if m < 0 return -INF
 if i == C return (M - m)
 if answer[m][i] != -1 return answer[m][i]
 best = -1
 for g in 0 .. count[i]
 best = max ( best, shop(m - price[i][g], i+1) )
 answer[m][i] = best
 return best
```
]
.below-column2[
answer table:

| | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 |
|---|---|---|---|---|---|---|---|---|---|----|----|----|----|----|----|----|----|----|----|----|
| 0 | | | | | | | | | | | | | | | | | | | | |
| 1 | | | | | | | | | | | | | | | | | | | | |
| 2 | | | | | | | | | | | | | | | | | | | | |
```
shop(20, 0)
```
]
---
.left-column2-small[
```
M = 20
C = 3
c0: 6, 1, 8
c1: 5, 10, 5
c2: 1, 5, 3, 5
```
]
.right-column2-large[
```
shop( m, i )
 if m < 0 return -INF
 if i == C return (M - m)
 if answer[m][i] != -1 return answer[m][i]
 best = -1
 for g in 0 .. count[i]
 best = max ( best, shop(m - price[i][g], i+1) )
 answer[m][i] = best
 return best
```
]
.below-column2[
answer table:

```
shop(20, 0)
  shop(14, 1)
  shop(19, 1)
  shop(12, 1)

```
]

---
.left-column2-small[
```
M = 20
C = 3
c0: 6, 1, 8
c1: 5, 10, 5
c2: 1, 5, 3, 5
```
]
.right-column2-large[
```
shop( m, i )
 if m < 0 return -INF
 if i == C return (M - m)
 if answer[m][i] != -1 return answer[m][i]
 best = -1
 for g in 0 .. count[i]
 best = max ( best, shop(m - price[i][g], i+1) )
 answer[m][i] = best
 return best
```
]
.below-column2[
answer table:

```
shop(20, 0)
  shop(14, 1)  --- shop(9, 2)
  shop(19, 1)      shop(4, 2)
  shop(12, 1)      shop(9, 2)

```
]

```
shop(20, 0)
  shop(14, 1)  --- shop(9, 2)  ---  shop(8, 3)
  shop(19, 1)      shop(4, 2)       shop(4, 3)
  shop(12, 1)      shop(9, 2)       shop(6, 3)
                                    shop(4, 3)

```
]

```
shop(20, 0)
  shop(14, 1)  --- shop(9, 2)  ---  shop(8, 3)  --- return 12
  shop(19, 1)      shop(4, 2)       shop(4, 3)  --- return 16
  shop(12, 1)      shop(9, 2)       shop(6, 3)  --- return 14
                                    shop(4, 3)  --- return 16

```
]

```
shop(20, 0)
  shop(14, 1)  --- shop(9, 2)  ---  16
  shop(19, 1)      shop(4, 2)  ---  shop(3, 3)
  shop(12, 1)      shop(9, 2)       shop(-1, 3)
                                    shop(1, 3)
                                    shop(-1, 3)

```
]

```
shop(20, 0)
  shop(14, 1)  --- shop(9, 2)  ---  16
  shop(19, 1)      shop(4, 2)  ---  shop(3, 3)  --- 17
  shop(12, 1)      shop(9, 2)       shop(-1, 3) --- -INF
                                    shop(1, 3)  --- 19
                                    shop(-1, 3) --- -INF

```
]

```
shop(20, 0)
  shop(14, 1)  --- shop(9, 2)  ---  16
  shop(19, 1)      shop(4, 2)  ---  19
  shop(12, 1)      shop(9, 2)  ---  16 (table lookup)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- shop(14, 2)
  shop(12, 1)      shop(9, 2)
                   shop(14, 2)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- shop(14, 2)   ---  shop(13, 3)
  shop(12, 1)      shop(9, 2)         shop(9, 3)
                   shop(14, 2)        shop(11, 3)
                                      shop(9, 3)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- shop(14, 2)   ---  shop(13, 3)  ---  7
  shop(12, 1)      shop(9, 2)         shop(9, 3)   ---  11
                   shop(14, 2)        shop(11, 3)  ---  9
                                      shop(9, 3)   ---  11

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- shop(14, 2)   ---  11
  shop(12, 1)      shop(9, 2)
                   shop(14, 2)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- shop(14, 2)   ---  11
  shop(12, 1)      shop(9, 2)    ---  16  (table lookup)
                   shop(14, 2)   ---  11  (table lookup)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- shop(7, 2)
                   shop(2, 2)
                   shop(7, 2)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- shop(7, 2)  ---  shop(6, 3)
                   shop(2, 2)       shop(2, 3)
                   shop(7, 2)       shop(4, 3)
                                    shop(2, 3)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- shop(7, 2)  ---  shop(6, 3)  ---  14
                   shop(2, 2)       shop(2, 3)  ---  18
                   shop(7, 2)       shop(4, 3)  ---  16
                                    shop(2, 3)  ---  18

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- shop(7, 2)  ---  18
                   shop(2, 2)
                   shop(7, 2)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- shop(7, 2)  ---  18
                   shop(2, 2)  ---  shop(1, 3)  --- 19
                   shop(7, 2)       shop(-3, 3) --- -INF
                                    shop(-8, 3) --- -INF
                                    shop(-3, 3) --- -INF

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- shop(7, 2)  ---  18
                   shop(2, 2)  ---  19
                   shop(7, 2)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- shop(7, 2)  ---  18
                   shop(2, 2)  ---  19
                   shop(7, 2)  ---  18 (table lookup)

```
]

```
shop(20, 0)
  shop(14, 1)  --- 19
  shop(19, 1)  --- 16
  shop(12, 1)  --- 19

```
]

```
shop(20, 0) --- 19

```
]

---
## Wedding Shopping with DP (bottom-up)

How do we solve this problem using bottom-up approach?

.center[.huge[??? ]]
--

- there are different approaches to this, so there are several possible answers

---

## Wedding Shopping with DP (bottom-up)

```
answer = an M+1 by C 2D array initialized to -1

shop( M )

//populate the table with values from the first category
    for g in  0 .. count[0]
        answer[ price[0][g] ][i] = price[0][g]

//populate the remaining categories
 for i in 1 .. C-1
 for m in 0 .. M
 if answer[m][i-1] != -1
 for g in 0 .. count[i]
 answer[ m + price[i][g] ][i] = m + price[i][g] <<<<

// find the highest price in the bottom row of the table
    for m in M .. 1
        if answer[m][C-1] != -1
            return m
```

`<<<< ` be careful with array out of bounds access here

---

.left-column2-small[
```
M = 20
C = 3
c0: 6, 1, 8
c1: 5, 10, 5
c2: 1, 5, 3, 5
```
]
.right-column2-large[
```
shop( M )
    for g in  0 .. count[0]
        answer[ price[0][g] ][i] = price[0][g]

for i in 1 .. C-1
        for m  in  0 .. M
            if answer[m][i-1] != -1
                for g in 0 .. count[i]
                    answer[ m + price[i][g] ][i] = m + price[i][g]