On almost every project I’ve worked on, I’ve ended up running some background tasks via cron. While some tasks run only once a day, occasionally I’ll find a task that needs to run much more often. For example, on Tunnlr, we have a cron job that runs every minute to set up new accounts and change passwords on the shell server.
Normally, the task completes in just a few seconds. If the task takes longer than a minute, however, we end up with multiple copies running at once. While this doesn’t cause problems for Tunnlr, it can in other cases. It might also lead to the server slowing down which can lead to even more copies of the job backing up. Clearly this is something we want to avoid.
A few years ago, Patrick Jaberg at Appolicious taught me a neat trick. Many Linux distributions have a command called
flock which will run a command only if it can obtain a lock on a specific file. To keep background jobs from piling up, you can simply execute using flock as follows
flock -n /tmp/path.to.lockfile -c command with args
There are only a few parts to this command. The
-n option tells
flock not to wait for the lock, but to instead exit. The lockfile path can be any writable file. Finally, the
-c and command are the command to be executed. With just this simple change, I no longer have to worry about jobs backing up again.