The Computer Oracle

RSYNC Does Not Delete Source Directories

--------------------------------------------------
Hire the world's top talent on demand or became one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Techno Intrigue Looping

--

Chapters
00:00 Rsync Does Not Delete Source Directories
00:41 Accepted Answer Score 22
01:32 Answer 2 Score 54
02:13 Answer 3 Score 7
02:39 Answer 4 Score 1
03:06 Thank you

--

Full question
https://superuser.com/questions/676671/r...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#linux #rsync

#avk47



ANSWER 1

Score 54


The manpage mentions that it only removes non-dirs:

--remove-source-files   sender removes synchronized files (non-dirs)

This is expected behavior.

If you want to remove empty directories in your source, if there are still files left, do a:

find . -type d -empty -delete

This will recurse, because it is using find, and only apply to´ empty directories (via the -empty option). -delete deletes those directories. (You cannot delete non-empty directories this way.)

If it's just one empty source directory, a rmdir <directory> will of course suffice.




ACCEPTED ANSWER

Score 22


The behavior of --remove-source-files that you observe is exactly that specified by man rsync:

--remove-source-files

   This tells rsync to remove from the sending side the files (meaning non-directories) that are a part of the transfer and have been successfully duplicated on the receiving side.

There is no specific command to remove the directories, as these two discussions in StackExchange and ServerFault clearly show. The solution suggested there is to issue two separate commands:

 rsync -av --ignore-existing --remove-source-files source/ destination/ && \
 rsync -av --delete `mktemp -d`/ source/ 

The last piece of the command suggested in these two posts,

 rmdir source/

which is needed to remove the (now emptied) source directory has this form in these posts because the OPs and the answers are using rsync to move large amounts of files within the same machine. In your case you will have to do this manually.




ANSWER 3

Score 7


Using "rm -rf" has an inherent race condition, you could namely delete files that were just created between the rsync and the rm invocations.

I prefer to use:

rsync --remove-source-files -a server:incoming/ incoming/ &&

ssh server find incoming -type d -delete

This will NOT remove the directories if they are not empty.




ANSWER 4

Score 1


Remove source files, then remove directories to be safe.

# given this scenario where you generate folders 2014-01-01 etc.. that have an archive myfile.tar.gz
pushd $(mktemp -d)
mkdir 201{4..6}-{01..12}-{01..31}
for i in $(ls); do; touch $i/myfile.tar.gz;done;
# find and rsync on 10 CPU threads directories that match ./2015-*
find /tmp/tmp.yjDyF1jN70/src -type d -name '2015-*' | \
parallel \
--jobs 10 \
--progress \
--eta \
--round-robin \
rsync \
--hard-links \
--archive --verbose --protect-args \
--remove-source-files \
{} /tmp/tmp.yjDyF1jN70/dest
# now safely remove empty directories only
for i in $(ls /tmp/tmp.yjDyF1jN70/src); do; rmdir /tmp/tmp.yjDyF1jN70/src/$i; done;

More on GNU Parallel