The Computer Oracle

Are files that use NTFS compression decompressed onto disk or into memory?

--------------------------------------------------
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Switch On Looping

--

Chapters
00:00 Are Files That Use Ntfs Compression Decompressed Onto Disk Or Into Memory?
00:50 Accepted Answer Score 21
02:35 Thank you

--

Full question
https://superuser.com/questions/1096885/...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#windows #ntfs

#avk47



ACCEPTED ANSWER

Score 21


Windows decompresses files into memory. Doing it onto disk would completely obliterate any speed improvements and would cause a lot of unnecessary disk writing. See the end of this Microsoft blog article on NTFS sparse files and compression:

  1. NTFS determines which compression unit is being accessed.
  2. The compression unit’s entire allocated range is read.
  3. If the unit is not compressed, then we skip to step 5. Otherwise, NTFS would attempt to reserve (but not allocate) the space required to write the decompressed CU back to disk. If insufficient free space exists on the disk, then the application might get an ERROR_DISK_FULL during the read.
  4. The CU would be decompressed in memory.
  5. The decompressed byte range would be mapped into cache and returned to the requesting application.
  6. ...

Of course, if you're low on memory, the memory used by the decompression process could cause other memory be paged out and written to disk in the page file. Fortunately, only the chunks containing sections that your programs actually read will be decompressed; NTFS doesn't have to decompress the whole thing if you only need a few bytes.

If your SSD is fast, you're probably not going to get speed improvements from NTFS compression. It's conceivable that the time your processor spends decompressing data plus the time your disk spends reading the compressed data could add to be more than the time your SSD takes to read the uncompressed data. It also depends on the size of the files you work with. The minimum size of a compressible file ranges from 8 KB to 64 KB, depending on your cluster size. Any files less than that size won't be compressed at all, but a tiny amount of bookkeeping would be added.

If you do a lot of writing to compressed files, you could see a lot of variance in speed due to the compression algorithm used (LZ).

Further reading: How does NTFS compression affect performance?