The Computer Oracle

How to limit Unix find number of results to handle directories with many files

--------------------------------------------------
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Puzzle Meditation

--

Chapters
00:00 How To Limit Unix Find Number Of Results To Handle Directories With Many Files
00:24 Answer 1 Score 31
01:54 Accepted Answer Score 7
02:06 Answer 3 Score 0
02:39 Answer 4 Score 0
03:20 Thank you

--

Full question
https://superuser.com/questions/112144/h...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#unix #find

#avk47



ANSWER 1

Score 31


You could try something like find [...] |head -[NUMBER]. This will send a SIGPIPE to find when head outputs its however-many lines so that find doesn't continue its search.

Caveat: find outputs files in the order they appear in the directory structure. Most *NIX file systems do not order directories by entry name. This means the results are given in an unpredictable order. find |sort will put the list in the sort order defined by your LC_COLLATE setting -- in most cases, ASCIIbetical order.

Another caveat: It's exceedingly rare to see in the wild, but *NIX filenames can contain newline characters. Many programs get around this by optionally using a NUL byte (\0) as the record separator.

Most *nix text-processing utilities have the option to use a NUL as a record separator instead of a newline. Some examples:

  • grep -z
  • xargs -0
  • find -print0
  • sort -z
  • head -z
  • perl -0

Putting this all together, to safely remove the first 5000 files, in alphabetical order:

find /some/log -type f -name '*.log' -print0 |
sort -z |
head -5000 -z |
xargs -0 rm

* Line breaks here are added for clarity, though either syntax is valid and works the same; you could execute this all on one line (foo | bar | baz) provided you make sure to not delete the | (vertical pipe) separating the commands.




ACCEPTED ANSWER

Score 7


It sounds like you're looking for xargs, but don't know it yet.

find /some/log/dir -type f -name "*.log" | xargs rm



ANSWER 3

Score 0


find /some/log -type f -name *.log -exec rm {} ; | limit 5000

Well, the command as quoted will not work, of course (limit isn't even a valid command).

But if you run something similar to the find command above, it's probably a classic problem. You're probably having performance problems because find runs rm once for every file.

You want to use xargs, it can combine several files into one command line, so it will invoke rm a limited times for many files at once, which is much faster.




ANSWER 4

Score 0


Just |head didn't work for me:

root@static2 [/home/dir]# find . -uid 501 -exec ls -l {} \; | head 2>/dev/null
total 620
-rw-r--r--  1 root   root           55 Sep  8 15:22 08E7384AE2.txt
drwxr-xr-x  3 lamav statlus 4096 Apr 22  2015 1701A_new_email
drwxr-xr-x  3 lamav statlus 4096 Apr 22  2015 1701B_new_email
drwxr-xr-x  3 lamav statlus 4096 May 11  2015 1701C_new_email
drwxr-xr-x  2 lamav statlus 4096 Sep 24 18:58 20150924_test
drwxr-xr-x  3 lamav statlus 4096 Jun  4  2013 23141_welcome_newsletter
drwxr-xr-x  3 lamav statlus 4096 Oct 31  2012 23861_welcome_email
drwxr-xr-x  3 lamav statlus 4096 Sep 19  2013 24176_welco
drwxr-xr-x  3 lamav statlus 4096 Jan 11  2013 24290_convel
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13
find: `ls' terminated by signal 13

(...etc...)

My (definitely not the best) solution:

find . -uid 501 -exec ls -l {} \; 2>/dev/null | head

The disadvantage is that the 'find' itself isn't terminated after required number of lines, and run in background until ^C or end, therefore ideas are welcomed.