> From: sbaugh@catern.com > Date: Sat, 22 Jul 2023 10:38:37 +0000 (UTC) > Cc: Spencer Baugh <sbaugh@janestreet.com>, dmitry@gutov.dev, > yantar92@posteo.net, michael.albinus@gmx.de, rms@gnu.org, > 64735@debbugs.gnu.org > > Eli Zaretskii <eliz@gnu.org> writes: > > No, the first step is to use in Emacs what Find does today, because it > > will already be a significant speedup. > > Why bother? directory-files-recursively is a rarely used API, as you > have mentioned before in this thread. Because we could then use it much more (assuming the result will be performant enough -- this remains to be seen). > And there is a way to speed it up which will have a performance boost > which is unbeatable any other way: Use find instead of > directory-files-recursively, and operate on files as they find prints > them. Not every command can operate on the output sequentially: some need to see all of the output, others will need to be redesigned and reimplemented to support such sequential mode. Moreover, piping from Find incurs overhead: data is broken into blocks by the pipe or PTY, reading the data can be slowed down if Emacs is busy processing something, etc. So I think a primitive that traverses the tree and produces file names with or without attributes, and can call some callback if needed, still has its place. > Since this runs the directory traversal in parallel with Emacs, it > has a speed advantage that is impossible to match in > directory-files-recursively. See above: you have an optimistic view of what actually happens in the relevant use cases. > We can fall back to directory-files-recursively when find is not > available. Find is already available today on many platforms, and we are evidently not happy enough with the results. That is the trigger for this discussion, isn't it? We are talking about ways to improve the performance, and I think having our own primitive that can do it is one such way, or at least it is not clear that it cannot be such a way. > > Optimizing the case of a long > > list of omissions should come later, as it is a minor optimization. > > This seems wrong. directory-files-recursively is rarely used, and rgrep > is a very popular command, and this problem with find makes rgrep around > ~10x slower by default. How in any world is that a minor optimization? > Most Emacs users will never realize that they can speed up rgrep > massively by setting grep-find-ignored-files to nil. Indeed, no-one > realized that until I just pointed it out. In my experience, they just > stop using rgrep in favor of other third-party packages like ripgrep, > because "grep is slow". Making grep-find-ignored-files smaller is independent of this particular issue. If we can make it shorter, we should.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4