I was implementing a file watch service using apache common io framework. Using this framework we can monitor a particular directory (in my case). Any file created or deleted in this directory will be notified to the callback registered. The call-back is invoked in a background thread that is started when we start monitoring. We need to provide a periodic check interval to look for the changes. I think it takes the snapshot of current and compares with the previous to detect the changes.
During the initialization of my server, I need to check if any files are there in those directories. I have to treat them as newly created. So I had code like
- Check for exiting files.
- Register for any file changes.
The problem with the above order is that between 1 and 2 there is a chance that some files may get created. This is a race condition. To fix this issue, we have to flip the order. It is fine to process the same file multiple times. (Duplicates are possible with the reverse order)
During the discussion of a design of workflow manager, one of the problems mentioned was: there is a completion queue and there is a submission queue. The work item is submitted by the subscriber, we should be able to avoid duplicate work submissions from the subscribers. To implement this we have decided to keep the work item in the submission queue until completed. Once completed we move it to the completion queue. when moving the item from submission queue to completion queue we should do that atomically ( we should not see the item in both the queues or we should not see the item in none of the queues). We can use transactions to do this move. The problem is that the submitter/subscriber checks these queues before submitting the item to the submission queue (it is neither in submission nor in completion queues). Looking at both the queues cannot be done using transactions. Because of this, there are races possible which can lead to the submission of duplicate items to the queue. One solution I proposed is
- Look into the submission Queue
- Look into the Completion Queue
- Look into the submission Queue again to confirm there are no changes done between 1 and 3.