Search code examples
watchman

Using socket interface keeps sending overflow warnings


I'm establishing a watch via the socket interface, and then subscribing to changes.

For each incoming PDU, if the map has a "warning" key, I output the warning to the console/user, as the docs suggest.

However, when an overflow happens, it looks like I don't get the "warning" key just once, but instead every incoming PDU has the same warning ("recrawl happened 1 time") over and over (AFAICT?), so I end up spamming the console with the same error message.

For me, it'd be preferable if Watchman only sent the "warning" key once per overflow event. Otherwise I'm looking at having to cache the "warnings already shown to the user" to avoid spamming the console.

Also, in terms of overflow behavior in general, the warning says:

To resolve, please review the information on
https://facebook.github.io/watchman/docs/troubleshooting.html#recrawl
To clear this warning, run:
`watchman watch-del ... ; watchman watch-project ...`

But I'd prefer to have a way to reset the warning without having to cancel and resubscribe my subscription. E.g. right now I have to control-c kill my program, run the watchman watch-del command, then restart my program.

Which I could automate internally, e.g. have my program detect the "overflow happened" warning message, kill it's subscription, issue a watch-del, and then re-issue the watch.

But, even if I could reset the warning via the socket interface, or do this internal watch-del, I'm wondering why the warning needs reset at all--e.g. in theory if watchman has already done a recrawl, and I told the user it happened (by logging it to the console), shouldn't things be fine now? Why is the watch-del + re-watch required in the first place?

E.g. as long as overflows are not happening constantly, it seems like watchman doing the recrawl (so gets back in sync with the file system) + issuing one warning PDU should mean everything is back to normal, and my user program could, ideally, stay dumb/simple and just keep getting the post-overflow/post-recrawl PDUs on it's same/existing subscription.


Solution

  • Sorry that this isn't as clear as it could be.

    First: you don't strictly have to take action here, as the watchman service already recovered from the overflow.

    It's just advising you that you may have a local configuration problem; if you are on Linux you might consider increasing the various inotify sysctl parameters. If you are on the mac there is very little you can do about this. The warning is sticky so that it is forced in the face of the user. We later had users request a way to suppress it, so the suggestion for deleting and restarting the watch was added.

    The result was a pretty noisy warning that caused more confusion.

    In watchman 4.7 we added a configuration option to turn off this warning: https://facebook.github.io/watchman/docs/config.html#suppress_recrawl_warnings The intent here is to hide the warning from users that don't know how (or don't have permission) to remediate the system configuration. It works well in conjunction with the (undocumented) perf sampling configuration options that can record and report the volume of recrawls to a site-specific reporting system.