Flock for Cron jobs

On almost every project I’ve worked on, I’ve ended up running some background tasks via cron. While some tasks run only once a day, occasionally I’ll find a task that needs to run much more often. For example, on Tunnlr, we have a cron job that runs every minute to set up new accounts and change passwords on the shell server.

Normally, the task completes in just a few seconds. If the task takes longer than a minute, however, we end up with multiple copies running at once. While this doesn’t cause problems for Tunnlr, it can in other cases. It might also lead to the server slowing down which can lead to even more copies of the job backing up. Clearly this is something we want to avoid.

A few years ago, Patrick Jaberg at Appolicious taught me a neat trick. Many Linux distributions have a command called flock which will run a command only if it can obtain a lock on a specific file. To keep background jobs from piling up, you can simply execute using flock as follows

flock -n /tmp/path.to.lockfile -c command with args

There are only a few parts to this command. The -n option tells flock not to wait for the lock, but to instead exit. The lockfile path can be any writable file. Finally, the -c and command are the command to be executed. With just this simple change, I no longer have to worry about jobs backing up again.