Paragon is working to get its ntfs3 filesystem into the Linux kernel

 

In March of last year, proprietary filesystem vendor Paragon Software unleashed a stream of anti-open source FUD about a Samsung-derived exFAT implementation headed into the Linux kernel. Several months later, Paragon seemed to have seen the error of its ways and began the arduous process of getting its own implementation of Microsoft's NTFS (the default filesystem for all Windows machines) into the kernel as well.

 

Although Paragon is still clearly struggling to get its processes and practices aligned to open source-friendly ones, Linux kernel BDFL Linus Torvalds seems to have taken a personal interest in the process. After nearly a year of effort by Paragon, Torvalds continues to gently nudge both it and skeptical Linux devs in order to keep the project moving forward.

Why Paragon?

To those familiar with daily Linux use, the utility of Paragon's version of NTFS might not be immediately obvious. The Linux kernel already has one implementation of NTFS, and most distributions make it incredibly easy to install and use another FUSE-based implementation (ntfs-3g) beyond that.

 

Both existing implementations have problems, however. The in-kernel implementation of NTFS is extremely old, poorly maintained, and should only be used read-only. As a result, most people who actually need to mount NTFS filesystems on Linux use the ntfs-3g driver instead.

 

Ntfs-3g is in reasonably good shape—it's much newer than the in-kernel ntfs implementation, and as Linux filesystem guru Ted Ts'o points out, it actually passes more automated filesystem tests than Paragon's own ntfs3 does.

 

Unfortunately, due to operating in userspace rather than in-kernel, ntfs-3g's performance is abysmal. In Ts'o's testing, Paragon's ntfs3 completed automated testing in 8,106 seconds—but the FUSE-based ntfs-3g required a whopping 34,783 seconds.

 

Bugs and performance aside, ongoing maintenance is a key aspect to Paragon's ntfs3 making it in-kernel. Torvalds opined that "Paragon should just make a pull request for [ntfs3]"—but he did so after noting that the code should get OKs from current maintainers and that Paragon itself should maintain the code going forward. (Paragon developer Konstantin Komarov quickly replied that the company intended to continue maintaining the code, once accepted.)

Why not Paragon?

Although Torvalds himself seems positive about getting Paragon's ntfs3 driver mainlined, as do several other users and developers, there are still some concerns about getting Paragon and its workflow properly integrated into the kernel dev community and up to that community's standards.

 

Ted Ts'o—core maintainer of Linux's ext3/ext4 filesystems, and the e2fsprogs userspace utilities used to manage them—seems to be the most critical. In addition to the slightly higher number of failed automated tests he found in Paragon's code, he notes other issues such as whole-system deadlocks that pop up if ntfs3 is stressed too hard. (This is an issue that we have heard over the years from people who've purchased Paragon's ntfs3, as well.)

 

Ts'o also raises questions about maintenance and communication, saying, "I'd feel better if *someone* at Paragon Software responded to Darrick [Wong] and my queries about their quality assurance, and/or made commitments that they would at least *try* to fix the problems that about 5 minutes of testing using fstests turned up trivially."

 

Fellow developer Darrick Wong added that he wants to make sure Paragon was invested in maintenance moving forward, so that ntfs3 wouldn't "become one of the shabby Linux filesystem drivers, like [the current in-kernel ntfs]."

The path forward

Despite skepticism from Ts'o and Wong, we broadly expect that inclusion of Paragon's ntfs3 will happen eventually. The company has worked for a year so far to take its code from 27,000 lines tossed over the wall into a Linux-ready patch set—and although primary developer Komarov may not have always replied as quickly or thoroughly as Ts'o and Wong prefer, he does continue to respond.

 

For his own part, Torvalds seems determined to find a performant, modern, maintainable replacement for the ancient (2001-era) and seldom-used ntfs implementation in the kernel now. As long as Paragon remains willing to keep playing, it seems likely to get there eventually—perhaps even in time for the 5.15 kernel.