Search code examples
gitgruntjsgrunt-contrib-watch

GIT: how can I watch for local changes in a performant way?

Short description

I want to detect local changes in my local git repository in a performant way. This means detecting which files are modified locally, but also the unversioned ones. Any uncommited change / file addition has to be picked up.

My attempt

What I've gotten so far (gist): https://gist.github.com/kevin-smets/02bda148a5330f350e6b

This is a script which can be run indepentently or as a git plugin. What it does is basically the following, launch a git status --porcelain every two seconds, interpret this to check which files are changed. However, when it's running, it takes about 8% CPU constantly. I tried git --diff-files and all kinds of other commands, but all resulted in a high CPU usage. :(

Why?

I would like to feed this list of files to a grunt watch process. Currently my codebase is around 15k files and growing. Watching all these files will simply become impossible. I already apply a static filter to limit the amount of files watched, but it is getting cumbersome to change this by hand every time I want to edit a file outside of the defined filter.

A thought

A thought I had was, how does IntelliJ / WebStorm (my current IDE) monitor the GIT repository? I tried going into the source code of the community edition but well, I can't figure that one out :). There must be some way WebStorm is doing this without taxing the CPU, it seems to pick up these changes on a timeout of a second or less... Even if they were made outside of the IDE. I was wondering if there is some eventing system in GIT, but I can't find anything along those lines.

Thanks for reading

Any help, is always highly appreciated!

Solution

  • I eventually went with Chokidar to monitor my source folder. It's now at around 20k files and as long as I watch the folder itself and no globs like **/*.scss, it works like a charm.

    I do the filtering on filetype myself by checking the extname of the files and feeding it to the correct build target. This way I eliminated git from the process. You could still run any git command when Chokidar notifies you of a file change, this way the ugly arbitrary timeout is eliminated and your CPU will not get taxed constantly.