File locking: what if two copies of my program run at the same time?

Contributors:

• Stas sbekman@hotmail.com
• boutell boutell@boutell.com
• Stas sbekman@hotmail.com

---

• 1. FileLocking Issues. WWW Guestbook Case.
• 1.1 Locking with flock and modifying the file by opening it only once
• 1.2 File Locking when flock(3) or lockf(3) is unavailable
• 1.3 File Locking Issues : Summary

1. File Locking: the WWW Guestbook Example

File locking is a must when you program a cgi script in any language. Don't cry if your database or some file is corrupted or broken and you didn't think about file locking! Before you go any father learn how to protect yourself from these annoying disasters.

How it can happen? Very simple: let's say you wrote a guestbook, which works by reading the file in modifying it and writing it back. Now imagine that TWO users were delighted to submit their appreciation of your site using your guestbook, and it happens that they press the submit button at the same time. As a result, 2 occurences of your guestbook are starting to run. Both read the original file, modify it, and write it back. What happens next? There are several possibilities:

1. The first copy of the script finished writing before the second started to. Result: one of the users will not find her submitted entry, because the second user (the second copy of the guestbook) overwrote the changes made by the first one. Bad news!
2. Both copies tried to write at once (not completely true since only one process can use a CPU in any given time, but they take turns by getting a very short CPU time slice; if the file to write is a very big one, 1 time slice of CPU will not be sufficient to accomplish the writing). Result - You get a corrupted file!

There are few other scenarios to describe, but this one is the most important. So how do you solve it? Easy (if you have a flock(3) working). Use flock (100% clean) or emulate it(not 100% clean). Lets rewrite the guestbook:

1. Flock the file
     Critical section start
2. Read the file into the memory
3. Modify it
4. Write it to the disk
     Critical section end
5. Unlock (just close the file; this automatically unlocks the file as soon as it is safe to do so)

The Critical Section> is the section of the code where you don't want other programs (processes) to use your resource (the guestbook file) while you are modifying it.

What does the other process do if you lock the file? It will wait patiently until you release it! So it's important to make all the critical sections as short as possible (imagine a situation where 100 users submit to your guestbook every second). So, do all the input parsing and guestbook new entry preparing before you open the guestbook file. Once you open and lock it, just locate the place you want to write to, write there and close it!

Two Ways to Lock Files

Here are two ways to lock files: first the preferred method, using the flock function, and second the alternative method using external file locking which is not always effective.

[Back to the TOC]

---

1.1 Locking with flock

Notes:

This Perl solution uses the flock function which must be available on your system. Check whether it actually works for you before you trust it on a particular operating system. Flock is reported not to work on NFS (you need a flock daemon for that, and it's bad on some systems such as AIX).

  my $file = "filename";       # specify the filename to open
  open FH, "+<$file" or die "Cannot open $file: $!";
  flock FH, 2;                 # wait for exclusive lock
  ## from here to the close FH, we have critical region
  ## be sure to minimize this time

  ## get current content
  seek FH, 0, 0;               # rewind to beginning
  my @content = <FH>;          # get current content

  <Change your @content lines here>

  ## write new list, and release file
  seek FH, 0, 0;               # rewind again
  truncate $file, 0;           # empty the file
  print FH @content;           # print the new content

  ## release file
  close FH;
  # Note that close frees flock as well !

If you need to append, do this:

  open FH, "+<$file" or die "Cannot open $file: $!";
  flock FH, 2;                 # wait for exclusive lock

  seek FH, 0, 2;                # end of file
  print FH "Anything you want to append";
  close FH;

[Back to the TOC]

1.2 File Locking when flock(3) or lockf(3) is unavailable

If you have the flock(3) function working, use the method described in Locking with flock.

If not, you need to create a lockfile when you enter a critical section, and remove it when you leave it. You also have to handle the case where your cgi is getting killed in the middle of the critical section, preventing the lockfile from being removed. So, another instance of the program should be able to do the cleanup. Here is the code, which is far from perfect since there is actually no guarantee that the CPU will not let another copy of the program run during the check for the lock file; however, making this test as fast as possible is better than having no locking at all. flock(3) is a better solution.

sub get_file_lock
  {
  local ($lock_file) = @_;
  local ($timeout);

  $timeout=20;    # in seconds
  while (-e $lock_file && (stat($lock_file))[9]+$timeout>time)
  { sleep(1);}
 
  open(LOCK_FILE, ">$lock_file") || die "Can't open $lock_file:$!);
}

sub release_file_lock
  {
  local ($lock_file) = @_;

  close(LOCK_FILE);
  unlink($lock_file);
  }

So how do you use it? Like flock(), but you need to specify a lockfile name to the function.
Important: it should be a static name like /tmp/myapplication.lock. If you use something like $$.lock ($$ is the process number), you fail to accomplish your goal because two copies of the program won't look for the same lock file!

Why is this code not perfect? In the test and write operation. There is a chance (you decide if you can afford it) that 2 programs will try to get the lock file at the time (lets name them A and B) . Now imagine this:

A executes the code and arrives to the line:

while (-e $lock_file) {sleep 1);
open(LOCK_FILE, ">$lock_file") || die "Can't open $lock_file:$!);

A tests for the lock ( -e $lock_file ), and finds that there is none (or it has to kick the dead one). The moment A is finished with the test and knows that the way is clear, it is interrupted since its CPU time slice is over!

Now B gets the CPU do the same thing: checks for the lock file, and doesn't find one since A didn't have a chance to create one yet. So B writes the lockfile, believing it has a lock. And now B is interrupted.

A wakes up. The first thing it does is overwrite B's lockfile (it already tested that there is no lockfile, a moment before it went to sleep) and both programs happily overwrite each other changes, probably ruining something on the way. To make things worse, B finishes much earlier than A since it had a short task to do, so it deletes the lockfile. And A is continuing to do a long task without any lockfile! Now process C enters to run, doesn't find the lockfile, creates one and keeps ruining your files. Now A deletes the lockfile (it thinks it owns the lock, remember?) and on and on. Hope you understand the scenario!

This is why it is definitely better to use flock if at all possible.

[Back to the TOC]

1.3 File Locking Issues : Summary

1. Use flock when more then one program can write on the same time and it has to read/write from the same file.

2. Remember that if you use external locking , the program can be stopped in the middle of critical section (User presses a Stop button ). The lockfile isn't getting deleted, so you have to handle that case as well (in the case of flock it's not a problem since when file is getting closed it's geitting unlocked as well).

3. Be careful when you use scripts someone else wrote. They can be badly written (in the case of file locking and other cases as well). The script which is reported to work by many people can fail for you if it's not properly written. So this is a second important issue to check when using someone else's code, almost as important as taint checking (some poorly written scripts pass your input to the Unix command line, allowing users to trick the shell into deleting your files and so on).

---

Previous | Next | Table of Contents

---