MATLAB GPU Tutorial

Welcome the the MATLAB GPU Tutorial page! Your CIMS Linux machine may have come installed with a GPU (Graphical Processing Unit) already installed. With a GPU-enabled device, you can perform powerful, advanced calculations in only a fraction of the time when compared to utilizing just your machine's CPU. While using a GPU can make complex calculations a quick and easy process, actually utiliizing your GPU in MATLAB isn't necessarily intuitive. As a matter of fact, even though it has GPU resources available, MATLAB will automatically use your CPU when performing calculations. Or that is to say, we must explicitly tell MATLAB to utilize a GPU, otherwise it will use your CPU by default. 

Based on MATLAB's official GPU usage guide, this page aims to familiarize you with utilizing your GPU-enabled device with the MATLAB application, in addition to answering some common GPU-based questions.  

Table of Contents:

Of course, should you have any additional GPU-related questions, please feel free to contact the NYU Courant Helpdesk at helpdesk@cims.nyu.edu. 

Why use a GPU? 

MATLAB provides built-in support for GPU computing through the Parallel Computing Toolbox. By using a GPU, one can speed up operations that involve large datasets and/or complex computations via parallel processing. Basically, instead of having your CPU handle one large block of computation, the GPU breaks up that large, complex computation into a large number of proverbially "bite-sized" computations that can be performed simulataneously. 

How do I know if my computer has a GPU? 

Before we get started on learning how to use a GPU with MATLAB, it is probably a good idea to ensure that you are in fact utilizing a GPU-enabled device. Through MATLAB, we can quickly discover if your machine has a GPU by using the gpuDevice() function. Here is an example of how to use this function:

gpuInfo = gpuDevice();

disp(gpuInfo);

These commands will return information about the available GPU on your machine. If a GPU is not available, however, then an error message will inform you that MATLAB is using the CPU instead.

Incidentally, MATLAB only offers GPU accelerated computers to GPUs created by NVIDIA. GPUs that have not been created by NVIDIA -- such as AMD GPU devices -- will not work with MATLAB. There are currently no workarounds (that we are aware of) to bypass this GPU-manufacturer limitation.

Granted, all of the NYU-supplied GPUs are of the NVIDIA brand, but please keep this limitation in mind in the event that you are considering investing or acquiring a different GPU device for your machine.

How powerful is my GPU? 

You can perform a benchmark test of your GPU in MATLAB by entering "benchmark" at the command prompt (>>).

Incidentally, this benchmark tool is one of the few built-in MATLAB features that will actually automatically utilize your machine's GPU. 

How do I use my GPU? 

Now that we have confirmed that your machine is equipped with a GPU that can be used by MATLAB, let's actually perform some GPU-accelerated commands.

When you perform any calculation in MATLAB, the application will instinctly use your machine's CPU by default. Therefore, we must explicitly tell MATLAB to use your machine's GPU via the command prompt interface. Specifically, you will want to use the gpuArray() function in order to utilize your GPU in MATLAB. Specifically, you are going to want to pass the data that you would like to be handled by the GPU to this gpuArray() function. When we use the gpuArray() function, we are explicitly passing data from our CPU to the GPU.

Here is the most basic example of passing data from the CPU to the GPU in MATLAB:

A = rand(1000); % Create a random 1000x1000 matrix on the CPU

A_gpu = gpuArray(A); % Transfer the matrix to the GPU 

A_gpu is now on the GPU. Any subsequent operations on this A_gpu variable will be performed on the GPU as well. 

Alternatively, this matrix multiplication example will not utilize the GPU, as the data -- "A" -- has not been passed to it: 

A = rand(10); % Here "A" represents our data

d = 2; % "d" represents the dimension for summation

sum(A,gpuArray(d)); % This operation will be performed on the CPU, as our data has not been passed to the GPU.

Even though the second argument, d, is passed to the sum function as a GPU array,  MATLAB will not move the data (A) to the GPU just because one of the arguments (d) is on the GPU. Therefore, MATLAB will perform the summation on the CPU because the matrix A is still stored on the CPU.

What basic operations can be performed on a GPU? 

Now that we have confirmed that your machine is equipped with a GPU that can be used by MATLAB, let's actually perform some common GPU-accelerated operations.

A proverbial "bread and butter" GPU operation is multiplying matrices, so let's walk through some GPU-accelerated matrix multiplication:

A = rand(1000); % create a randomized 1000x1000 matrix

B = rand(1000); $ create a second, randomized 1000x1000 matrix

A_gpu = gpuArray(A); 

B_gpu = gpuArray(B);

C_gpu = A_gpu * B_gpu; % Matrix multiplication on the GPU

C = gather(C_gpu); % Bring the result back to the CPU

Notice the gather() method in particular, which transfers your data from the GPU back to the CPU -- essentially the inverse of the gpuArray() function. 

Likewise, basic arithmetic-based operations on the GPU, such as addition, subtraction, and exponentiation, can benefit greatly from GPU-acceleration: 

C_gpu = A_gpu + B_gpu; % recall that A_gpu and B_gpu were set up in our Matrix Multiplication example. 

Furthermore, mathemtatical functions such as sin, cos, log, and sqrt, are also great candidates for GPU calculation: 

Y_gpu = sin(A_gpu); % Compute sine of each element in A on the GPU

Of course, you can also mix these functions and operations together as well. 

What GPU functions are included with MATLAB? 

While the gpuArray() function explicitly passes data from the CPU to the GPU, there exists a beevy of other GPU-based built-in functions in MATLAB. 

Some common MATLAB GPU functions include, but are not limited to: 

  • gpuArray(): Transfers data from CPU to GPU.
  • gather(): Transfers data back from GPU to CPU.
  • fft(), ifft(): Perform fast Fourier transforms on the GPU.
  • sin(), cos(), exp(), log(): Element-wise mathematical functions on the GPU.
  • mldivide() (\): Solves linear systems on the GPU.
  • conv2(): 2D convolution on the GPU.
  • eig(), svd(): Eigenvalue and singular value decomposition on the GPU.

Please note the syntax for the mldivide() function, as these two following examples are equivalent to one another: 

x = A \ B;

x = mldivide(A, B);

Additional gpuArray-supported functions in MATLAB can be found on the official MathWorks site for running MATLAB functions on a GPU. 

Are there any drawbacks or limitations to using a GPU?

While GPU-accelerated calculations are on whole faster than their CPU counterparts, there are still some limitations that one must consider when using a GPU. Generally speaking, there exists a data overhead -- the amount of time it takes to move data between the GPU and the CPU. Typically, transferring data between the two is a slow process.

Therefore, it is always a good idea to minimize the total number of CPU-GPU transfers in your calculations. One solid approach to minimize transfers is to perform as many operations as possible on the GPU before ultimately moving your result back to the CPU via the gather() function. 

Furthermore, the benefits of GPU-acceleration are better seen when working with particularly large datasets. In fact, there exists a proverbial "data overhead" threshold wherein calculations are actually performed faster on the CPU if the datasets are not large enough. If you need to perform a small calculation for instance, then using just the CPU will be a much faster process.