How to limit Unix find number of results to handle directories with many files
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------
Music by Eric Matyas
https://www.soundimage.org
Track title: Puzzle Meditation
--
Chapters
00:00 How To Limit Unix Find Number Of Results To Handle Directories With Many Files
00:24 Answer 1 Score 31
01:54 Accepted Answer Score 7
02:06 Answer 3 Score 0
02:39 Answer 4 Score 0
03:20 Thank you
--
Full question
https://superuser.com/questions/112144/h...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#unix #find
#avk47
ANSWER 1
Score 31
You could try something like find [...] |head -[NUMBER]
. This will send a SIGPIPE
to find
when head
outputs its however-many lines so that find
doesn't continue its search.
Caveat: find
outputs files in the order they appear in the directory structure. Most *NIX file systems do not order directories by entry name. This means the results are given in an unpredictable order. find |sort
will put the list in the sort order defined by your LC_COLLATE
setting -- in most cases, ASCIIbetical order.
Another caveat: It's exceedingly rare to see in the wild, but *NIX filenames can contain newline characters. Many programs get around this by optionally using a NUL byte (\0
) as the record separator.
Most *nix text-processing utilities have the option to use a NUL as a record separator instead of a newline. Some examples:
grep -z
xargs -0
find -print0
sort -z
head -z
perl -0
Putting this all together, to safely remove the first 5000 files, in alphabetical order:
find /some/log -type f -name '*.log' -print0 |
sort -z |
head -5000 -z |
xargs -0 rm
* Line breaks here are added for clarity, though either syntax is valid and works the same; you could execute this all on one line (foo | bar | baz
) provided you make sure to not delete the |
(vertical pipe) separating the commands.
ACCEPTED ANSWER
Score 7
It sounds like you're looking for xargs, but don't know it yet.
find /some/log/dir -type f -name "*.log" | xargs rm
ANSWER 3
Score 0
find /some/log -type f -name *.log -exec rm {} ; | limit 5000
Well, the command as quoted will not work, of course (limit
isn't even a valid command).
But if you run something similar to the find command above, it's probably
a classic problem. You're probably having performance problems because find
runs rm
once for every file.
You want to use xargs
, it can combine several files into one command line, so it will invoke rm
a limited times for many files at once, which is much faster.
ANSWER 4
Score 0
Just |head
didn't work for me:
root@static2 [/home/dir]# find . -uid 501 -exec ls -l {} \; | head 2>/dev/null
total 620
-rw-r--r-- 1 root root 55 Sep 8 15:22 08E7384AE2.txt
drwxr-xr-x 3 lamav statlus 4096 Apr 22 2015 1701A_new_email
drwxr-xr-x 3 lamav statlus 4096 Apr 22 2015 1701B_new_email
drwxr-xr-x 3 lamav statlus 4096 May 11 2015 1701C_new_email
drwxr-xr-x 2 lamav statlus 4096 Sep 24 18:58 20150924_test
drwxr-xr-x 3 lamav statlus 4096 Jun 4 2013 23141_welcome_newsletter
drwxr-xr-x 3 lamav statlus 4096 Oct 31 2012 23861_welcome_email
drwxr-xr-x 3 lamav statlus 4096 Sep 19 2013 24176_welco
drwxr-xr-x 3 lamav statlus 4096 Jan 11 2013 24290_convel
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
(...etc...)
My (definitely not the best) solution:
find . -uid 501 -exec ls -l {} \; 2>/dev/null | head
The disadvantage is that the 'find' itself isn't terminated after required number of lines, and run in background until ^C or end, therefore ideas are welcomed.