How to find duplicate files on Windows for free

Why matching by name does not work

The same photo can be saved as IMG_2034.jpg, IMG_2034 (1).jpg and vacation-cover.jpg. Matching on filenames misses all three as duplicates, and at the same time it flags genuinely different files that happen to share a name. The only reliable way to know two files are identical is to compare their contents.

Hashing is the reliable way

A content hash reads a file and produces a short fingerprint. Two files with the same fingerprint are the same file, byte for byte, regardless of name, date or location. Sifty uses SHA-256 for this, and it does it efficiently:

• It groups by size first. Files of different sizes cannot be identical, so only same-size candidates ever get hashed. That skips the vast majority of the work.
• It hashes in parallel. The size-matched candidates are fingerprinted across multiple threads, so large folders go quickly.
• It is hardlink-aware. On NTFS, two paths can point at the same physical data through a hardlink. Those occupy one block on disk, so Sifty counts them once and never reports them as wasted space.

Run the scan

Point it at any folder or whole drive. This only reads, it changes nothing:

PS C:\> sifty disk duplicates D:\Photos
Found 316 duplicate groups.

Reclaimable by de-duplicating: 6.4 GB

A "group" is a set of identical files. The reclaimable figure is what you would save by keeping one copy from each group and removing the rest.

Remove the extras safely

When you want to act on it, the cleanup command de-duplicates a folder, keeping one copy of each file and sending the rest to the Recycle Bin. It previews by default:

PS C:\> sifty cleanup duplicates D:\Photos          # preview (dry-run)
PS C:\> sifty cleanup duplicates D:\Photos --apply  # asks, then trashes extras