Description
The analyzer currently uses the watcher package to watch the filesystem for changes to files and directories, so that it knows when to reanalyze files. A major benefit of this approach is that it is platform-agnostic, allowing the analyzer to use the same file-watching API regardless of whether the system is Linux, MacOS, or Windows.
But this benefit comes with a potentially serious drawback: the API provided by the watcher package is quite detailed (firing add/modify/remove events for each modified file within the directories being watched), whereas in some platforms, the events natively supplied by the OS are much less detailed. As a result, the watcher may have to recursively examine all files in a directory (and its subdirectories) both when watching begins and when an event is received, and then "diff" them in order to produce watch events. This is a lot of extra work, and it's potentially very time consuming, especially if the directories being watched are not local to the machine. Since a lot of this work is done when the analyzer first starts watching a directory, it potentially may have a big influence in initial analysis time.
And the extra work is redundant, because the analyzer already has its own model of the directories being watched, and their subdirectories, and the analyzer's model is more compact, because it ignores excluded directories and non-Dart files. This is potentially a huge savings in situations where a subdirectory that the analyzer knows to ignore may contain a lot of files (e.g. a .git directory). In fact, I believe that the analyzer doesn't even care about the details of the events fired by the watcher package; last time I dug into the design (which is admittedly many years ago), the analyzer merely used the watcher events as a signal to re-examine the directory in question and deterimine for itself what had changed.
I think we should consider re-tooling the way the analyzer watches files so that it just uses the FileSystemEntity.watch method directly. Although this method exposes the analyzer to more platform-specific details, I think that's ok because the analyzer's needs are so modest. Specifically:
- On Windows, individual files can't be watched, only directories. That's fine; what the analyzer wants to watch are analysis roots and package roots, which are directories.
- On Mac, strict event ordering isn't guaranteed, some events might be coalesced, and events that occurred before the watch was started might still appear on the stream. That's fine; all the analyzer needs to know is when it should re-examine the directory.
- On Linux, recursive directory watching is not supported. This is annoying, but it should be easy enough to deal with; the analyzer already recursively walks all the subdirectories it cares about looking for
.dartfiles to analyze; it simply needs to start watching each subdirectory as it encounters it.