Euler Cluster Computing
About the cluster
The EULER cluster is an Opteron Linux cluster with 8GB memory and 8 CPU cores on each node. This cluster is setup to only submit jobs through SGE. To use the cluster, log on to the head node euler01.cims.nyu.edu and use Sun Grid Engine to submit/monitor/administer your compute jobs. This is the only authorized way to execute your jobs on the cluster.
Execution nodes for this cluster are:
- euler02.cims.nyu.edu
- euler03.cims.nyu.edu
- euler04.cims.nyu.edu
- euler05.cims.nyu.edu
Note: Please do not run intensive jobs on euler01.cims.nyu.edu as it is only for job submission and is not an execution node.
Sun Grid Engine (SGE) Commands
- qsub - submit a batch job to Grid Engine
- qstat - show the status of Grid Engine jobs and queues
- qconf -sel - displays the Grid Engine execution host list
How to submit a simple job
Make sure that your shell environment is set up correctly. If you use csh/tcsh and have not removed the line that sources the system-wide cshrc, your environment should already be correct. If you have completely replaced the initial .cshrc provided by CIMS, or if you use another shell, you need to execute the following command before running any SGE commands:
csh/tcsh:
source /opt/sge/euler/common/settings.csh
sh/ksh/bash:
. /opt/sge/euler/common/settings.sh
Create a shell script that contains the commands you need to execute, e.g.
#!/bin/sh
# This is a sample script file for
# running a sample program under SGE
cd /home/user135/expt
./myprog
Now you can submit your job with qsub:
% qsub -m beas myjob.sh
Your job 321 ("myjob.sh") has been submitted.
The -m option sends you mail when your job (b)egins, (e)nds, (a)borts, or (s)uspends.
You can use qstat to see the status of submitted jobs:
% qstat
By default, qstat will only display the status of jobs you submitted. To see the status of any user's job, you can use the argument -u '*':
% qstat -u '*'
Since your SGE batch jobs do not have a connected terminal, standard output and standard error will be redirected to job-name.ojob-id and job-name.ejob-id in your home directory respectively. In the above example, where your job-id was 321, the files containing output from your script would be myjob.sh.o321 and myjob.sh.e321.
If you would like to run your job on only a subset of nodes for any reasons, you can use the -q option followed by the queue list, e.g. the following command will submit your job to be run on only euler02 or euler03:
% qsub -q all.q@euler02.cims.nyu.edu,all.q@euler03.cims.nyu.edu myjob.sh
Script-embedded qsub options
qsub takes various options (see man page for details), and you can embed them in your job script using lines that begin with #$, e.g.
#!/bin/sh
# Ask for Korn Shell to execute job
#$ -S /bin/ksh
# Email me when job is done
#$ -M user135@nyu.edu
#$ -m e
# Specify the file for standard output and
# merge stdout and stderr into single file
#$ -o myprog.out -j y
cd /home/user135/expt
./myprog
Please refer to the man pages of qsub and qstat, and the Grid Engine 6 User's Guide for further details.