ReFS
Developer(s)Microsoft
Full nameResilient File System
Introduced1 August 2012; 11 years ago (2012-08-01)[1] with Windows Server 2012
Structures
Directory contentsB+ tree[2]
Limits
Max volume size35 petabytes[3]
Max file size35 petabytes[3]
Features
AttributesYes
Transparent
compression		No
Data deduplicationYes, since v3.2 debuting in 2016 v1709[4]
Copy-on-writeYes
Other
Supported
operating systems
Websitelearn.microsoft.com/en-US/windows-server/storage/refs/refs-overview

Resilient File System (ReFS),[6] codenamed "Protogon",[7] is a Microsoft proprietary file system introduced with Windows Server 2012 with the intent of becoming the "next generation" file system after NTFS.

ReFS was designed to overcome problems that had become significant over the years since NTFS was conceived, which are related to how data storage requirements have changed. These requirements arose from two major changes in storage systems and usage – the size of storage in use (large or massive arrays of multi-terabyte drives now being fairly common), and the need for continual reliability. As a result, the file system needs to be self-repairing (to prevent disk checking from being impractically slow or disruptive), along with abstraction or virtualization between physical disks and logical volumes.

The key design advantages of ReFS include automatic integrity checking and data scrubbing, elimination of the need for running chkdsk, protection against data degradation, built-in handling of hard disk drive failure and redundancy, integration of RAID functionality, a switch to copy/allocate on write for data and metadata updates, handling of very long paths and filenames, and storage virtualization and pooling, including almost arbitrarily sized logical volumes (unrelated to the physical sizes of the used drives).

Comparison with NTFS

Major new features

Improved reliability for on-disk structures

ReFS uses B+ trees for all on-disk structures, including all metadata and file data.[2][8] Metadata and file data are organized into tables similar to a relational database. The file size, number of files in a folder, total volume size, and number of folders in a volume are limited by 64-bit numbers; as a result, ReFS supports a maximum file size of 35 petabytes, and a maximum volume size of 35 petabytes.[3]

Built-in resilience

ReFS employs an allocation-on-write update strategy for metadata,[2] which allocates new chunks for every update transaction and uses large IO batches. All ReFS metadata have 64-bit checksums which are stored independently. The file data can have an optional checksum in a separate "integrity stream", in which case the file update strategy also implements allocation-on-write for file data; this is controlled by a new "integrity" attribute applicable to both files and directories. If file data or metadata become corrupt, the file can be deleted without taking the whole volume offline for maintenance, and then be restored from the backup. As a result of built-in resiliency, administrators do not need to periodically run error-checking tools such as CHKDSK when using ReFS.

Compatibility with existing APIs and technologies

ReFS supports only a subset of NTFS features, and only supports Win32 APIs that are "widely adopted". It does not require new system APIs, and most file system filters continue to work with ReFS volumes.[2] ReFS supports many existing Windows and NTFS features such as BitLocker encryption, Access Control Lists, USN Journal, change notifications,[9] symbolic links, junction points, mount points, reparse points, volume snapshots, file IDs, and oplock. ReFS seamlessly integrates with Storage Spaces,[2] a storage virtualization layer that allows data mirroring and striping, as well as sharing storage pools between machines.[10] ReFS resiliency features enhance the mirroring feature provided by Storage Spaces and can detect whether any mirrored copies of files become corrupt using a data scrubbing process,[8] which periodically reads all mirror copies and verifies their checksums, then replaces bad copies with good ones.

Microsoft Windows and Windows Server include ReFSUtil, a command-line utility that can be used to diagnose heavily damaged ReFS volumes, identify remaining files, and copy those files to another volume.[11]

Removed features

Some NTFS features are not implemented in ReFS. These include object IDs, 8.3 filename, NTFS compression, Encrypting File System (EFS), transactional NTFS, extended attributes, and disk quotas.[7][2][12] Dynamic disks with mirrored or striped volumes are replaced with mirrored or striped storage pools provided by Storage Spaces; however, automated error-correction is only supported on mirrored spaces. Data deduplication was missing in early versions of ReFS.[2] It was implemented in v3.2, debuting in Windows Server v1709.[4]

Support for alternate data streams and hard links was initially not implemented in ReFS. In Windows 8.1 64-bit and Server 2012 R2, the file system reacquired support for alternate data streams, with lengths of up to 128K, and automatic correction of corruption when integrity streams are used on parity spaces.[13] ReFS had initially been unsuitable for Microsoft SQL Server instance allocation due to the absence of alternate data streams.[14] Hard links were introduced with preview versions of Windows Server 2022 but are not yet available in Windows 11.

Implementations

ReFS was initially added to Windows Server 2012 only, with the aim of gradual migration to consumer systems in future versions; this was achieved as of Windows 8.1.[3] The initial versions removed some NTFS features, such as disk quotas, alternate data streams, and extended attributes. Some of these were re-implemented in later versions of ReFS.

In early versions (2012–2013), ReFS was similar to or slightly faster than NTFS in most tests,[15] but far slower when full integrity checking was enabled, a result attributed to the relative newness of ReFS.[16][self-published source][17][self-published source]

The ability to create ReFS volumes was removed in Windows 10's 2017 Fall Creators Update for all editions except Enterprise and Pro for Workstations.[5][why?]

Starting with Windows Server 2022 and Windows 11 21H2, the boot environment natively supports ReFS, allowing the system to be installed and run in a special way on a volume formatted with ReFS v3. If it is a volume formatted with ReFS v1, it cannot be booted with ReFS.[18]

Starting with Windows 11 build 22621.2338, ReFS is re-introduced via a Dev Drive feature; allowing fixed storage drives and VHDs to be formatted as ReFS, with special file and Microsoft Defender policies added during use.

The cluster size of a ReFS volume is either 4 KB or 64 KB.[19]

History

Server 2016

At the Storage Developer Conference 2015, a Microsoft developer presented enhancements of ReFS expected to be released with Windows Server 2016 and included in Technical Preview 4, titled "ReFS v2".[20] It highlighted that ReFS now included capabilities for very high speed moving, reordering, and cloning of blocks between files[21] (which can be done for all blocks of a file). This is particularly needed for virtualization, and is stated to allow fast provisioning, diff merging, and tiering. Other enhancements cover the redo log (for synchronous disk writes), parallelization, efficient tracking of uninitialized sparse data and files, and efficient 4k I/O.[20]

Server 2022

Windows Server 2022 (using ReFS version 3.7) supports file-level snapshots.[3]

Windows Insider Preview 22H2 and 23H2 (builds 226** and 25***) support ReFS volume compression using LZ4 and zstd algorithms.[22]

Versions

This article may be confusing or unclear to readers. Please help clarify the article. There might be a discussion about this on the talk page. (July 2017) (Learn how and when to remove this message)

ReFS has some different versions, with various degrees of compatibility between operating system versions. Aside from development versions of the filesystem, usually, later operating system versions can mount filesystems created with earlier OS versions (backwards compatibility). Some features may not be compatible with the feature set of the OS. The version, cluster size and other features of the filesystem can be queried with the command fsutil fsinfo refsinfo volumename.

Known problems

This article needs to be updated. Please help update this article to reflect recent events or newly available information. (February 2019)

Issues identified or suggested for ReFS, when running on Storage Spaces, include:

Comparison with other file systems

Like ReFS, ZFS and Btrfs are designed to integrate data protection, snapshots, and background error correction.

In 2012, Phoronix wrote an analysis[25] of ReFS vs Btrfs. At the time, their features were similar, with both supporting checksums, RAID-like use of multiple disks, and error correction. However, ReFS lacked copy-on-write snapshots and compression, both found in Btrfs and ZFS.

In 2014, BetaNews wrote a review of ReFS and assessed its readiness for production use.[26] The review concluded that ReFS had at least some advantages over two of its main file system competitors.

Reverse engineering and internals

As of November 2019, Microsoft has not published any specifications for ReFS, nor have any working open-source drivers been made. A third-party open-source project to document ReFS is on GitHub.[28][29]

Paragon Software Group provides a closed-source driver for Windows and Linux.

See also

References

  1. ^ Snover, Jeffrey (1 August 2012). "Windows Server 2012 released to manufacturing!". Windows Server Blog. Microsoft Corporation. Archived from the original on 22 December 2015. Retrieved 2 September 2014.
  2. ^ a b c d e f g Verma, Surendra (16 January 2012). Sinofsky, Steven (ed.). "Building the next generation file system for Windows: ReFS". Building Windows 8. Microsoft. MSDN blogs. Archived from the original on 31 July 2012. Retrieved 20 January 2013.
  3. ^ a b c d e f "Resilient File System (ReFS) overview". Microsoft. Archived from the original on 25 September 2020. Retrieved 22 September 2020.
  4. ^ a b Sarkar, Dona (13 July 2017). "Announcing Windows Server Insider Preview Build 16237". Windows Experience Blog. Microsoft. Archived from the original on 23 July 2017. Retrieved 20 July 2017.
  5. ^ a b c "Windows 10 features and functionality removed in Windows 10". docs.microsoft.com. 30 January 2020. Archived from the original on 18 November 2019. Retrieved 23 February 2020. Creation ability will be available in the following editions only: Windows 10 Enterprise and Windows 10 Pro for Workstations. Creation ability will be removed from all other editions. All other editions will have Read and Write ability.
  6. ^ Lucas, Martin (1 January 2013). "Windows Server 2012: Does ReFS replace NTFS? When should I use it?". Ask Premier Field Engineering (PFE) Platforms. Microsoft. TechNet blogs. Archived from the original on 2 June 2013. Retrieved 20 January 2013.
  7. ^ a b Foley, Mary Jo (16 January 2012). "Microsoft goes public with plans for its new Windows 8 file system". ZDNet. CBS Interactive. Archived from the original on 21 May 2013. Retrieved 21 January 2013.
  8. ^ a b Meyer, David (17 January 2012). "Microsoft takes wraps off new Windows file system". ZDNet. CBS Interactive. Archived from the original on 21 May 2013. Retrieved 20 January 2013.
  9. ^ "ReadDirectoryChangesW function". MSDN Library. Microsoft. 8 November 2012. Archived from the original on 4 May 2013. Retrieved 21 January 2013.
  10. ^ Nagar, Rajeev (5 January 2012). Sinofsky, Steven (ed.). "Virtualizing storage for scale, resiliency, and efficiency". Building Windows 8. Microsoft. MSDN blogs. Archived from the original on 6 January 2012. Retrieved 18 January 2013.
  11. ^ "ReFSUtil | Microsoft Docs". Archived from the original on 27 August 2021. Retrieved 13 July 2021.
  12. ^ Windows Server 2012: Does ReFS replace NTFS? When should I use it? Archived 2 June 2013 at the Wayback Machine – Martin Lucas, TechNet
  13. ^ "Resilient File System Overview". Microsoft Docs. Microsoft. 29 February 2012. Retrieved 24 August 2022.
  14. ^ Cherry, Denny (21 August 2012). "SQL Server and Windows Server 2012's ReFS File System | Troubleshooting SQL Server Storage Problems". SQL Server Pro magazine. Penton Media. Archived from the original on 14 July 2018. Retrieved 9 May 2020.
  15. ^ "ReFS vs NTFS Benchmark – First Ever – Exclusive Technology News". FirstEver.eu. Archived from the original on 8 July 2014.
  16. ^ "Relative Performance of Parity .vs. Mirror in Storage Spaces". Baris Eris's Blog. 22 October 2012. Archived from the original on 22 June 2014. Retrieved 17 June 2014.
  17. ^ Eckert, Jason (25 February 2014). "Microsoft ReFS vs Oracle ZFS – Fight!". Archived from the original on 13 January 2017. Retrieved 12 January 2017.
  18. ^ Windows 11 on ReFS, archived from the original on 10 July 2022, retrieved 10 July 2022
  19. ^ "Storage at Microsoft: Cluster size recommendations for ReFS and NTFS". 13 January 2017. Retrieved 22 August 2022.
  20. ^ a b J. R. Tipton. "ReFS v2 – Cloning, projecting, and moving data" (PDF). Retrieved 20 May 2023.
  21. ^ "Block Cloning". Microsoft Docs. Microsoft. 7 January 2021. Retrieved 20 May 2023.
  22. ^ REFS_COMPRESSION_FORMATS - NTIFS.h Windows SDK Insider Preview Archived 14 May 2022 at the Wayback Machine build 22621
  23. ^ a b Pakhomova, Elena (8 August 2012). "Data Recovery Tales: When Windows Storage Spaces Go Bad". Archived from the original on 30 July 2016. Retrieved 17 June 2014.
  24. ^ "What I learned after 30 minutes with ReFS". Ctrl blog. 18 May 2017. Archived from the original on 19 May 2017. Retrieved 30 May 2017.
  25. ^ "Microsoft's ReFS File-System: Competitor To Btrfs?". phoronix.com. Archived from the original on 8 September 2016. Retrieved 14 September 2016.
  26. ^ "Windows Storage Spaces and ReFS: Is it time to ditch RAID for good?". BetaNews. 15 January 2014. Archived from the original on 31 January 2014. Retrieved 17 June 2014.
  27. ^ "How To Size Main Memory for ZFS Deduplication". www.oracle.com. Retrieved 24 August 2022.
  28. ^ "libyal/libfsrefs". GitHub. 18 October 2019. Archived from the original on 9 February 2019. Retrieved 11 November 2019.
  29. ^ "ReFS documentation project - PDF document of the ReFS filing system (direct link)". GitHub. Archived from the original on 10 July 2022. Retrieved 28 June 2020.

External links

File systems
Microsoft Windows components
Categories