How to Use PowerShell to Find Duplicate Files in Folders: Detailed Guide

Duplicate files slow down your computer, and make it harder to locate the files you need. There are several ways you can try to find duplicate files on your computer. PowerShell is one of them. In this article, I’ll show you detailed steps on how to use PowerShell to find duplicate files in folders. I’ve tested this method and it worked. I also want to mention that it requires some technical and programming knowledge, otherwise you might find it a bit complicated. If you only have basic computer skills, or want to save time, there are plenty of free duplicate finder software on the market can better help you.

What is PowerShell

PowerShell is a powerful task automation and configuration management tool developed by Microsoft. It provides a combination of a command-line shell and a scripting language. Through PowerShell, users can write scripts to automate various tasks, such as file management, generate reports, send emails, and more.

You can also use PowerShell to find files in folders with the same hash (A hash value is a unique value that corresponds to the content of the file. Files with the same content will have the same hash value). And take appropriate action as needed, such as letting PowerShell auto-delete duplicates for you or manually select files from folders to delete one by one.

PowerShell is installed on Windows system by default. You can also install and use PowerShell on macOS, Linus.

The pros and cons of using PowerShell to find duplicate files

Using PowerShell to find duplicate files in folders has the following advantages and disadvantages:

How to find and delete duplicate files with PowerShell in folders?

To find duplicate files in folders with PowerShell, follow the steps below:

1. Open PowerShell on your Windows computer. You can search for “PowerShell" in the start menu and run it as administrator.

2. Use the cd (change directory) command to change the current working directory to the folder that you want to search for duplicates. For example, if you want to find duplicate files in the "Downloads" folder, enter the following command:

cd C:\Users\YourUsername\ Downloads

Then press Enter on your keyboard.

3. Run the following command to count the hash value of each files in this folder and find duplicate files:

Get-ChildItem -Recurse | Get-FileHash | Group-Object -Property Hash | Where-Object { $_.Count -gt 1 } | Select-Object -ExpandProperty Group | Select-Object -ExpandProperty Path

Press Enter. Wait for the command execution to complete. This may take some time, depending on the number and size of files in the folder. Then it will group files in this folder by their hash.

4. Now you can run the following commend to remove all duplicate files except one from each group:

Get-ChildItem -Recurse | Get-FileHash | Group-Object -Property Hash | Where-Object { $_.Count -gt 1 } | foreach {

    $group = $_.Group | Select-Object -ExpandProperty Path

    $group | Select-Object -Skip 1 | Remove-Item -Force

}

Press Enter. Go back to the folder that had duplicate files before. You will find that all duplicate files with the same contents (regardless of file name) are removed.

Since PowerShell doesn’t allow you to view the duplicate files it finds, it’s necessary to back up files before removing them.  

Other easier way to find and remove duplicate files in folders

In fact, using PowerShell can help you quickly and accurately find duplicate files in folders, but the premise is that you are familiar with this tool. For most users, directly using third-party duplicate finder tool is the most efficient way. Cisdem Duplicate Finder is a reliable one.

Compared with PowerShell, there are several advantages to find duplicate files in folders with this tool:

• Allow you to scan multiple folders at once.
• Offer 2 scan methods: File content and File name.
• Divide the found duplicate files into categories-images, music, documents, videos, etc.
• Offer 3 preview modes, including side-by-side.
• Enable you to view detailed information of the files, such as path, size.
• Automatically select duplicate files for you to delete with 1-click.
• Provide 8 selection rules to help you mass select duplicates to delete.
• Also allow you to manually choose duplicate files to remove.
• Let you delete the duplicate files to Recycle Bin, remove permanently or move to folder.
• Compatibility: Windows 10/11, macOS 10.11 or later.

Let’s see the easy steps on how to use this tool to find duplicate files in folders:

1. Download and install Cisdem Duplicate Finder on your computer.

Free Download Windows 11/10

2. Click the plus icon on the main window. Add one or more folders to scan.

Or directly drag and drop folders to this tool.

3. It scans fast. And then displays a list of duplicate files that were found. This tool will auto-select duplicates for you to remove. You can directly delete them with 1-click. Or preview the files before removing.

You can also change the preview mode to view files’ details.

4. The selection rules is for helping you mass select duplicate files in folders to remove. Such as select by priority, select oldest, largest, etc.

5. After checking the selected duplicate files, click on Delete on the bottom right corner.

Note: The duplicates will be removed to Recycle Bin/Trash by default. If you’d like to delete permanently or move to a folder, set the Remove method from the Preferences window.

Conclusion

You can use PowerShell to find duplicate files in folders. I’ve showed you detailed steps about how to use it to search and delete duplicate files with different names but same contents in this article. In addition to PowerShell, some other professional duplicate finder tools also allow you to automatically find and remove duplicate files in folders quickly and easily. Whichever method you choose, remember to regularly clean up duplicate files to keep your files organized and free up computer storage space.