"NTFS really isn't that bad" - Robert Collins (LCA 2020)
Why Hacker News recommends it
The go-to reference for why file operations are slow on Windows and how antivirus filters, not NTFS itself, are the cause.
What it is
Robert Collins explains why rustup installs were painfully slow on Windows and how changes to rustup and Windows cut install time from 3m30s to 14s. The talk covers NTFS, small files and the cost of antivirus hooks on file close.
Recommended by 5 HN commenters across 3 different months, 2020 to 2025.
What Hacker News said
Recommends jmmv · 2022-09 · on “When to use Bazel?”
Add your npm directories to the virus file exclusion lists of Windows Defender. Note that some of the directories may not be obvious (those under AppData for example). The speed improvements are massive for tools like these that touch lots of small files. Also take a look at this talk on NTFS, file system speed, and how the presenter optimized the Rust installer: [this video] . There is an awful lot that can be done to tools that deal with lots of files to perform well under Windows too.
Recommends ablob · 2024-07 · on “Show HN: Dut – a fast Linux disk usage calculator”
I want to take this opportunity to recommend the talk "NTFS isn't that bad" ( [this video] ). NTFS prefers a different access pattern than most usual file systems. I remember that a part of the talk was about speed-ups on Linux as well. So even if it doesn't sway your opinion it should enhance your perspective on how file systems work.
Recommends unchar1 · 2024-07 · on “SQLite: 35% Faster Than the Filesystem”
That may be due to a combination of Malware detection + most unix programs not really written to take advantage of the features NTFS has to offer This is a great talk on the topic [this video]
Recommends amiga386 · 2024-07 · on “SQLite: 35% Faster Than the Filesystem”
NTFS can be especially awful if you're used to Linux behaviour, port code over to Windows and expect the same performance. Here's a talk on porting rustup to Windows: [this video] To begin with, it takes rustup 3m30s to install on Windows. After rejigging the code again and again away from its naive implementation which works fine on Linux, to perform sympathetically towards NTFS, it takes the same rustup 14s to install. That's quite a performance gain! But it needed a lot of changes to rustup, and to Windows itself.
Recommends miki123211 · 2025-01 · on “Wild – A fast linker for Linux”
And as a result of this, if you want to optimize your program to be fast on Windows, you will make very different optimization decisions than on other platforms. For example, Windows runs virus scans on close(), which makes it very slow. This means that sometimes it makes sense to have one or more background threads exclusively dedicated to closing files. There's a good talk on this at [this video]
Cites for_xyz · 2020-11 · on “Windows Subsystem for Linux: The lost potential”
> the bottleneck is that NTFS is a crappy filesystem. Care to explain why do you think NTFS is crap? From my experience it's usually bad assumptions about files under windows and every application or library tries to stick to posix interface when dealing with files (open, read/write, close) which tends to block for longer periods on windows than on linux counterparts which results in significant perfomance loss . Linux first software will always outperform windows implementation and Windows first software will outperform Linux implementations unless you provide separate code paths to properly handle underlying OS architecture and assumptions. On Windows closing the file handle is extremely c
Cites d2wa · 2022-07 · on “TeamViewer installs suspicious font only useful for web fingerprinting”
> Even if windows indexing aint malicious, it certainly behaves as if it is. The problem is that closing files takes a lot of time on Windows. [this video]
Cites CoolCold · 2023-08 · on “Microsoft's backwards compatibility is insane”
to my understanding it's not really IO (reading/writing bytes), but overhead on OpenFile/CloseFile being under AV/Defender inspection. I'm taking that understanding basing on [this video] with " "NTFS really isn't that bad" - Robert Collins (LCA 2020) " > Why was rustup slow (3m30s to install (not including download time)) in early 2019 on Windows, and why isn't it slow (14s to install) now? >Early in 2019 I was developing some things in Rustlang on Windows, got frustrated at the performance of rustup (the installer for rust) and looked for bug. I found one which blamed a combination of NTFS, small files and anti-virus software for the problem. This wasn't an entirely satisfactory answer to my mi