Half-written files and fs.watch on Windows: what I measured, and why rename fixes it

A lot of small desktop tools talk to other programs through a drop folder: the writer puts a JSON file in a directory, and the app watches the directory with fs.watch. It is simple and needs no ports or sockets. It also has a trap: the watcher can see the file before the writer has finished writing it.

I wanted to know how real that trap is on Windows, so I measured it instead of guessing. Everything below was run on Windows 11 with Node 24. I did not benchmark speed, only which events arrive and what a reader can see.

What fs.watch reports for a plain write

In a single process I wrote a 200 KB JSON file with fs.writeFileSync straight into the watched folder:

rename  a.json
change  a.json

So one write gives two events: rename (the file appeared) and change (the content was modified). Windows reports a file creation as rename, which surprises people who expect create. If the handler assumes one event per file, it runs twice.

Now the same payload written to b.json.tmp and then moved with fs.renameSync to b.json:

rename  b.json.tmp
change  b.json.tmp
rename  b.json.tmp
rename  b.json

The temp name produces its own events (it appears, is modified, then disappears), and the final name gets exactly one rename. When the consumer later deletes b.json, that arrives as one more rename, now with the file gone. Two lessons: filter by the final extension so .tmp events are ignored, and when you get a rename, check whether the file still exists. A rename event does not tell you whether the file was created or removed.

The torn read

A same-process test can't show a half-written file, because writeFileSync has finished before the watcher callback runs. The realistic case is two processes. I made a child process write a 2 MB JSON file in 50 KB chunks with a few milliseconds between them, and read the file from the watcher callback in the parent. Real event files are tiny, so this exaggerates the window. The point is only to show that the failure exists.

Plain write, straight to e.json:

rename e.json  bytes=50021    parses=false
change e.json  bytes=50021    parses=false
change e.json  bytes=2000023  parses=true

The first event fired after the first chunk, and the reader got 50 KB of invalid JSON. Write to e.json.tmp and rename when finished:

rename e.json  bytes=2000023  parses=true

One event, complete file. On the same volume, rename into place is the step that makes the file appear whole, which is why the pattern works. (I am describing what I observed on NTFS here; cross-volume moves are a copy and lose this property.)

Why a delay is not a fix

A common patch is "wait 150 ms, then read". In the plain-write run above, a reader that waits a fixed time is right whenever the writer finishes inside that time and wrong otherwise. The tiny files a hook writes usually finish in well under a millisecond, so it works nearly always, and "nearly always" is the dangerous kind of working. The bigger problem is what happens next. If the reader treats an unparsable file as garbage and deletes it, the event is lost for good: the writer later finishes the file, but it was already removed, and the follow-up change finds nothing.

The pattern I use now

Writer:

const tmp = path.join(dir, `${id}.json.tmp`);
fs.writeFileSync(tmp, JSON.stringify(event));
fs.renameSync(tmp, path.join(dir, `${id}.json`));

Reader:

fs.watch(dir, (_e, file) => {
  if (!file || !file.endsWith('.json')) return;      // ignores *.tmp
  const full = path.join(dir, file);
  let text;
  try { text = fs.readFileSync(full, 'utf8'); }
  catch (e) { if (e.code === 'ENOENT') return; throw e; } // delete event or already handled
  let event;
  try { event = JSON.parse(text); }
  catch { return; }                                  // leave it; do not delete
  fs.unlinkSync(full);
  handle(event);
});

Three details matter. First, ignore names that don't end in .json. Second, treat ENOENT as normal, because the rename event for your own unlink comes back through the same callback. Third, don't delete a file you could not parse. Quarantine it by renaming it to .bad, or leave it, so a bug in a writer leaves evidence.

Also scan the folder once at startup. Events that arrived while the app was closed produce no watcher events at all.

What this does not cover

Rename makes the file appear whole, but it does not make the event unique. Two writers choosing the same name will overwrite each other, so put a timestamp and a random suffix in the name. It also does nothing about a writer that crashes before the rename: you will be left with stale .tmp files, which are worth cleaning up on startup.

My measurements come from one machine, one Node version and one filesystem. A network drive, antivirus filters or a different Windows build may behave differently, so run your own test before relying on any of this.

If you are building a small Windows tool that listens to hooks from other programs, the one rule that held up for me: write elsewhere, rename into place, and never delete what you cannot read.

Context: I hit this in a tray app I sell, CatCode. I'm the author. The details above apply to any Node watcher, with or without that app.