This is code for the paper

Deep Kernel Learning
Andrew Gordon Wilson*, Zhiting Hu*, Ruslan Salakhutdinov, and Eric P. Xing (*equal contributions)
Artificial Intelligence and Statistics, 2016
http://jmlr.org/proceedings/papers/v51/wilson1i6.pdf

This code is copyright (2016) by the authors.  

Users of this code for research publications must cite the above paper.  Permission to use this code for commerical purposes must be obtained from the authors.
Any use or modification of this code must include this original license.


## Overview ##

This software depends on the (modified) [CAFFE](http://caffe.berkeleyvision.org/) DL library (`caffe/`), and the [GPML](http://www.gaussianprocess.org/gpml/code/matlab/doc/) library (`gpml/`). We provide DKL codes (`examples/`) on 3 example datasets: 

* [UCI](https://archive.ics.uci.edu/ml/datasets.html) Kin40K regression (section 5.1 of the ppaer)
* MNIST digit magnitude extraction (section 5.3)
* Olivetti face orientation extraction (section 5.2)

## Installation ##

The modified caffe library is under `caffe/`. Please refer to the [installation instructions](http://caffe.berkeleyvision.org/installation.html) for prerequisites and configurations. Run the following commands for installation:
```
make
make matcaffe
```

## DKL Example: Olivetti Face ##

We take the olivetti face task for example.

### Preparing data ###

```
#!bash
cd caffe
unzip examples/face/data/face_images.zip
sh examples/face/create_face.sh  # create databases for caffe CNN training and DKL training
```
Databases will be created under `caffe/examples/face/data`

### Pre-training CNN ###

Under `caffe/`:
```
#!bash
sh examples/face/train_face.sh
```
Please refer to the [tutorial](http://caffe.berkeleyvision.org/gathered/examples/mnist.html) for CNN configurations, etc.

After training you will see the trained CNN model `caffe/examples/face/face_iter_6000.caffemodel`. We have also provided a pre-trained model `caffe/examples/face/face_iter_6000.caffemodel.example`, so that you can skip this step and go straight to DKL model training.

### Training DKL models ###

The DKL codes are under `examples/face`, where `dkl_rbf_face.m` uses the RBF kernel as the base kernel, and `dkl_sm_face.m` uses the spectral mixture kernel as the base kernel. Training parameters are configured at the beginning of the respective files, including the pre-trained CNN model for initialization, number of training iterations, learning rate, and grid size, etc.

Under `examples/face`, run the models using matlab
```
#!matlab
run ../../gpml/startup.m  % setup gpml
run dkl_rbf_face.m  % or dkl_sm_face.m
```

### Notes ###

* For preparing the MNIST dataset, run

```
#!bash
cd caffe
sh examples/mnist/data/get_mnist.sh
sh examples/mnist/create_mnist.sh
```

* For preparing the Kin40k dataset, simply run `sh examples/kin40k/create_kin40k.sh` under `caffe/`
* Use GPU for the DNN part to speedup training
