Crontab

The crontab command, found in Unix and Unix-like operating systems, is used to schedule commands to be executed periodically. It reads a series of commands from standard input and collects them into a file known as a "crontab" which is later read and whose instructions are carried out.

Generally, crontab uses a daemon, crond, which runs constantly in the background and checks once a minute to see if any of the scheduled jobs need to be executed. If so, it executes them. These jobs are generally referred to as cron jobs. Cron jobs run as the user who set them up, as though you typed this command into your shell, except with more limited environment settings.

You'll need to enable shell access before you can modify your crontab unless you set up your crontab via the control panel.

More information is available at the shell prompt, with the following commands (hit q to exit):

man cron
man -a crontab

About cron on DreamHost

Setting up cron via the DH Control Panel

As most users will be looking for information on using the Control Panel to create cron scripts, you should look for the CRON menu under the GOODIES menu in your Control Panel

Please be aware that the command you enter IS NOT the command the Control Panel will place in the crontab. Something like this is what will appear:

###--- BEGIN DREAMHOST BLOCK
###--- Changes made to this part of the file WILL be destroyed!
# Moodle Cron Process
MAILTO="user@yourdoamin.com"
*/10 * * * * /usr/local/bin/setlock -n /tmp/cronlock.1234567.12345 sh -c $'wget -q -O /dev/null http\072//yourdreamhosturl.com/thepathtoyourapp/cron.php'
###--- You can make changes below the next line and they will be preserved!
###--- END DREAMHOST BLOCK

You can see that the Control Panel invokes "setlock". If you have shell access configured you can look at the man page or you can view a typical setlock manpage by clicking here.

The use of setlock will result in error messages if your cron job is invoked before the prior invocation releases your lockfile. This is often due to your script running longer than the time between job executions.

Execution Environment

Due to security concerns, DreamHost cron does NOT execute with a normal environment. This means you cannot use bare commands. The following will not work:

* * * * * perl myscript.pl
* * * * * perl /home/myuser/mydomain.com/myscript.pl

This is because the PATH environment variable is simply /usr/bin:/bin, and the SHELL is set to /bin/sh. The cron daemon doesn't know where the perl command is located. You must specify the full path to all commands not in the default PATH, and always specify the full path for files, like so:

* * * * * /usr/local/bin/perl /home/myuser/mydomain.com/myscript.pl

If you have any trouble locating where a command is, you can use the following command:

which command-name

Example:

which perl

Will print:

/usr/local/bin/perl

(However, these days we do indeed find perl in /usr/bin,

$ PATH=/usr/bin:/bin which perl|xargs ls -og
-rwxr-xr-x  2 1057324 2007-11-04 23:52 /usr/bin/perl

so maybe the above perl example is dated.)

NOTE: You CANNOT run cron jobs in the admin account. It must be under a non-admin shell account only.

MAILTO variable requirement

For performance reasons, we are now requiring that all crontabs have a MAILTO variable set. This is because by default cron will email output of your cronjobs to your user, and based on testing and research we've found that most people who have cronjobs set never actually check these emails, causing lots of wasted disk space and also some performance issues under certain conditions.

The MAILTO variable can be set to anything. If you leave it blank it will not email output of your cronjobs. You can set it to your username to have it delivered to your user as usual, or you can specify an external email address. Please note that cron emails *do* factor in to your email sending quota if they are delivered to an external email address.

If you do not have a MAILTO variable set when you go to edit or replace your crontab, our system will ask you where you would like it sent.

man 5 crontab says the value string is not parsed for environmental substitutions, thus MAILTO=$LOGNAME will not work, sorry.

Examples:

# The following line will disable sending cronjob output via email
MAILTO=""

# The following line will send cronjob output to someuser@example.com
MAILTO=someuser@example.com

# The following line will send cronjob output to your user
MAILTO=youruser

Cronjob character limit

Dreamhost's cron web interface states a limit of 1000 characters in a cronjob. What's not explained is that is a limit for the whole file, not just the user input. The real limit is somewhere between 600 and 800 characters. Violating the limit results in this warning:

/bin/sh: -c: line 1: unexpected EOF while looking for matching `''
/bin/sh: -c: line 2: syntax error: unexpected end of file

Crontab files

Basics

The crontab files are where the lists of jobs and other instructions to the cron daemon are kept. Each user at dreamhost has their own individual crontab files and (often there is a systemwide crontab file at /etc/crontab which is also used but can only be edited by the system administrators).

Crontab files are simple text files that have a particular format. Each line of a crontab file follows a particular format as a series of fields, separated by spaces and/or tabs. Each field can have a single value or a series of values. A single cron job should take up exactly one line, but this can be a long line (more than 80 characters).

Each line has five time and date fields, followed by a command, followed by a newline character (’\n’). A common problem is not including a newline, so hit carriage return (a time or three) at the end. Another common problem is automatic word-wrap breaking up a long line into multiple lines, so make sure your editor doesn't. The file must end with a newline character.

Blank lines and leading spaces and tabs are ignored. Lines whose first non-space character is a hash-sign (#) are comments, and are ignored. Note that comments are not allowed on the same line as cron commands, since they will be taken to be part of the command. Similarly, comments are not allowed on the same line as environment variable settings.

Operators

There are several ways of specifying multiple values in a field:

• The comma (',') operator specifies a list of values, for example: "1,3,4,7,8"
• The dash ('-') operator specifies a range of values, for example: "1-6", which is equivalent to "1,2,3,4,5,6"
• The asterisk ('*') operator specifies all possible values for a field. For example, an asterisk in the hour time field would be equivalent to 'every hour'.

The slash ('/') operator can be used to skip a given number of values. For example, "*/3" means to skip to every third value. So for example "*/3" in the hour time field is equivalent to "0,3,6,9,12,15,18,21"; "*" specifies 'every hour' but the "/3" means that only the first, fourth, seventh...and such values given by "*" are used.

Fields

The first five fields of the line are the date and time field which specify how frequently and when to execute a command.

Note: For day of the week, both 0 and 7 are considered Sunday. The time is based on that of the server running cron.

The sixth and subsequent fields (i.e., the rest of the line) specify the command to be run.

Output

The output from commands is determined by the last entries on the line. By default (i.e. if you don't specify anything at all) regular and error output will be e-mailed to you. According to "man cron," when executing commands, any output is mailed to the owner of the crontab (or to the user named in the MAILTO environment variable in the crontab, if such exists). The MAILTO variable may be set in the crontab file. See Handy Crontab Header below for an example.

cron(8) will look at MAILTO if it has any reason to send mail as a result of running commands in ‘‘this’’ crontab. If MAILTO is defined (and non-empty), mail is sent to the user so named. If MAILTO is defined but empty (MAILTO=""), no mail will be sent. Otherwise mail is sent to the owner of the crontab (assuming it has mail delivery or forwarding).

You can redirect the standard output and the errors wherever you like by using ">". When you use ">" without a number before it, it defaults to "1>", and this the 1st output is the standard (non-error) output. When you use "2>" you are saying what to do with the error output. So, for example, ">/my_file.txt" would redirect standard output to a file called "my_file.txt", and "2>/my_errors.txt" would redirect the errors to a file called "my_errors.txt".

Permissions

If you choose to run a script via cron, you may need to set the permissions to 744 using chmod in order to allow it to execute properly.

Examples

Example 1, this will run a command at 4:10pm, and email you the regular and error output:

10 16 * * * /home/someone/bin/mycoolscript.pl

Example 2, this will run a command at 2:00am on saturday, and only email you errors:

0 2 * * 6 /home/me/weekly/weekly-pruning.sh >/dev/null

Example 3, this will run at midnight on newyears, and not tell you anything:

0 0 1 1 0 /home/you/happy.new.years.pyc >/dev/null 2>&1

In that example I used a special redirection. '2>&1' sends the second output (error output, stderr) to the same place that regular output goes (standard out, stdout). In place of that you could have given anything, such as a error log: 2>/home/me/cron.errors.log, or the bit bucket again: 2>/dev/null.

Note: It may be necessary for you to use two digit minute and hour fields. One user reported as having to do this to get cron to accecpt the entry. For example:

01 02 * * * perl /home/me/awstats

Example 4, this will run a php page called cron.php every hour:

0 * * * * wget -O /dev/null www.somedrupaldomain.com/cron.php

Example 5, this will run a script every 15 minutes:

*/15 * * * * /usr/local/bin/perl /home/myuser/mydomain.com/myscript.pl

Ecample 6, this will run a script every 30 minutes using curl:

*/30 * * * * /usr/bin/curl -s domain.com/send.php 1>&2 &>/dev/null

Additional Perl cron operational notes

Executing a PHP Script with Crontabs

To execute a PHP script using crontabs, you must add this line to the top of the PHP script you want to execute (before the opening <?php declaration), or it will not work:

#!/usr/local/bin/php -q

Note: When using the DreamHost Web Panel Cron Jobs, it appears that it is *not* necessary to include the shebang (#!/usr/local/bin/php -q -- or similar line beginning in #!) line in the PHP script itself! (at least not with PHP5 running as a CGI)

Alternately, you can set up your cron entry like this (pay attention to which version of PHP you're running):

Example: This runs p.php every 10 minutes with 0 output, it is shared-server friendly by utilizing the nice command.

*/10 * * * * /usr/bin/nice -n 19 /dh/cgi-system/php5.cgi /home/user/p.php 1>&2 &>/dev/null

* * * * * /usr/local/php5/bin/php /home/youruser/yourdomain.com/path-to-php5-script.php
* * * * * /usr/local/bin/php /home/youruser/yourdomain.com/path-to-php4-script.php

* * * * * /dh/cgi-system/php5.cgi /home/youruser/yourdomain.com/path-to-php5-script.php
* * * * * /dh/cgi-system/php.cgi /home/youruser/yourdomain.com/path-to-php4-script.php

Perhaps an even better way is simply requesting the public url of the php with wget, curl, GET, lwp-request, links, lynx, etc. A benefit to doing this is your php files won't have to be modified and they will be executed using the same environment and permissions as usual.

* * * * * wget -q --delete-after yourdomain.com/path-to-php-script.php

The below example (links -dump) will actually email the output if setup through the web panel. Furthermore, wget downloads to /home/username/ a copy of the output, thus making a mess of the folder (even making empty files if there is no output)

* * * * * links -dump yourdomain.com/path-to-php-script.php

Executing a PHP script with Crontab through the control panel

Please read the section above to initially familiarize yourself with the process for executing PHP scripts through Crontab. Come back and read this section after you have a basic understanding of how this works, as you'll need to modify it slightly for it to work as expected through the control panel.

Cron jobs can now be set up directly through your DreamHost control panel here.

The process for executing a PHP script (and others, too!) is slightly different than what you would use if you were manually editing your cron jobs through the shell.

Instead of having to specify how often you'd like the script to be executed in the script line itself like */10 * * * * you would leave that out and just enter the command you'd like to be executed, like /usr/local/php5/bin/php /home/youruser/yourdomain.com/path-to-php5-script.php. You specify how often you'd like it to run on the form for editing the cron job itself instead.

You also need to leave out the trailing portion of the cron job that normally tells the server to squelch email related to the job. Basically, don't add 1>&2 &>/dev/null to the end as you'll determine what happens to the email through the control panel.

Handy Crontab Header

MAILTO=mycronic@example.com
# minute (0-59),
# |      hour (0-23),
# |      |       day of the month (1-31),
# |      |       |       month of the year (1-12),
# |      |       |       |       day of the week (0-6 with 0=Sunday).
# |      |       |       |       |       commands
3       2       *       *       0,6     /some/command/to/run
3       2       *       *       1-5     /another/command/to/run

Shortcut Crontab Header

Instead of the first five fields as above, one of these special strings may be used:

string          meaning
------          -------
@yearly         Run once a year, "0 0 1 1 *".
@annually       (same as @yearly)
@monthly        Run once a month, "0 0 1 * *".
@weekly         Run once a week, "0 0 * * 0".
@daily          Run once a day, "0 0 * * *".
@midnight       (same as @daily)
@hourly         Run once an hour, "0 * * * *".

Editing Crontab

To edit the your crontab you'll have to log in to your server with SSH (read the SSH article to learn how to do this). Once you've sucessfully logged in you can begin issuing commands.

From a file

Create a crontab file using your favorite text editor. Take care to follow the syntax above, one entry per line and end with an (empty) new line.

If you prefer, the text file may be uploaded. Just make sure to set your FTP client to do an ASCII transfer.

crontab filename

command line text editor

You can also directly edit your crontab by executing the following command:

crontab -e

This pulls up your crontab file with the default text editor, Nano. You can type away, making sure to properly format your command(s). When you're done press ctrl and o at the same time to save, or write out, the file. Now press ctrl and x at the same time to exit. You should recieve a success message, or you may recieve an error message. It should be noted that all commands must appear on one line. If the command wraps onto a second line cron will think it's two seperate commands.

To check that the change was successful:

crontab -l

To remove your crontab and start fresh:

crontab -r

(note: if you attempt to edit your crontab outside your home directory the save will fail. Go home!)

direct editing

If you are logged in as a VPS admin user, you can edit the crontab file directly. It is stored at

/var/spool/cron/crontabs/myuser

Having nano read your uploaded text file

First you must upload your text file with all the command lines meeting all the guidelines above within yourdomian.com, then connect to your server shell and once nano starts type Ctrl + R then type the URL that points to your uploaded file:

/home/myuser/mydomain.com/mytextfile.txt

save your file and exit, you should get a confirmation of success.

Changing your Default editor

If your command is wrapping, or if you'd rather work with a different text editor you can do that. You'll have to edit your .bash_profile file. At the command line type:

nano .bash_profile

This is a hidden file so it won't normally show up in your file list. To set your default text editor to not wrap lines, add the following line to the end of your .bash_profile file.

export VISUAL="nano -w"

Now press ctrl and o at the same time to save, or write out, the file. Then press ctrl and x at the same time to exit.

If you're used to pico, use the following in your .bash_profile script to keep it from wrapping lines:

alias pico='pico -w'

The following sets the default editor to vim:

export EDITOR="/usr/bin/vim"

Notes

If you are using the whenever ruby gem and need to write the crontab dynamically to override the dreamhost "we need to input an email for the app's output" script dont forget to set this variable in your schedule.rb file:

 env :MAILTO, 'some@example.com'  

and if you are not using this gem and just need to get pass this (using php or something else) this is the way to do it:

 (crontab -l 2>/dev/null; echo "*/5 * * * * /path/to/job -with args") | crontab - 

See also

• Run php from cron

External Links

• Unix crontab - Full Reference at Freebsd.org
• builtin nice command reference
• - Helpful article for people using Drupal CMS
• Crontab Command Examples in Unix