The Computer Oracle

Is there a filesystem that keeps only one copy of a file, and other copies are just references?

--------------------------------------------------
Hire the world's top talent on demand or became one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Mysterious Puzzle

--

Chapters
00:00 Is There A Filesystem That Keeps Only One Copy Of A File, And Other Copies Are Just References?
00:49 Accepted Answer Score 18
01:18 Answer 2 Score 10
01:48 Answer 3 Score 4
01:59 Answer 4 Score 0
02:12 Thank you

--

Full question
https://superuser.com/questions/506542/i...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#linux #filesystems

#avk47



ACCEPTED ANSWER

Score 18


This feature is called deduplication. None of the popular Linux filesystems (ext*) support it, but apparently, ZFS supports it partially. There is also a table of filesystems listing, among others, deduplication, but there don’t appear to be any popular choices - it is a planned feature for Btrfs, though.

I would guess that periodically checking your filesystem and creating appropriate hard links is the best you can do at the moment, although that does not imply copy-on-write.




ANSWER 2

Score 10


The primary keyword you want to look for is "copy on write." BTRFS does have a clone operation that does exactly what you want, and cp --reflink will do what you're looking for, provided your system has a modern enough kernel and coreutils 7.5. Wiki Source Also, bedup is a tool that will merge duplicates over an entire volume. CoW is also the driving feature underneath btrfs's snapshotting technology, IIRC.




ANSWER 3

Score 4


There is an online file system S3QL designed for backups with great capacity of deduplication.




ANSWER 4

Score 0


Zfs, btrfs, ext3cow, bcachefs (afaik, but there is a chance it's not yet implemented). Microsoft had one in development but they stopped for unknown reasons.