fdupes
FDUPES is a program for identifying or deleting duplicate files residing within specified directories.
Top Related Projects
Quick Overview
FDUPES is a program for identifying and/or deleting duplicate files residing within specified directories. It can be used to reclaim disk space by replacing redundant files with hardlinks to a single copy of the file content. FDUPES is designed to be efficient and user-friendly, making it a valuable tool for managing file systems.
Pros
- Fast and efficient in identifying duplicate files
- Supports recursive directory scanning
- Offers various options for handling duplicates (delete, hardlink, symlink)
- Can preserve a specified number of duplicate files
Cons
- Command-line interface may be intimidating for non-technical users
- Limited GUI options available
- May require careful use to avoid accidental data loss
- Does not support cloud storage or network drives directly
Getting Started
To get started with FDUPES, follow these steps:
-
Install FDUPES on your system:
# On Ubuntu/Debian sudo apt-get install fdupes # On macOS with Homebrew brew install fdupes -
Basic usage to find duplicates in a directory:
fdupes /path/to/directory -
To find duplicates recursively in multiple directories:
fdupes -r /path/to/dir1 /path/to/dir2 -
To delete duplicates interactively:
fdupes -d /path/to/directory -
To replace duplicates with hardlinks:
fdupes -rH /path/to/directory
Remember to use FDUPES carefully, especially when deleting files. Always ensure you have backups of important data before making changes to your file system.
Competitor Comparisons
Find duplicate files
Pros of dupeGuru
- Graphical user interface, making it more user-friendly for non-technical users
- Supports multiple file types, including music and pictures, with specialized scanning modes
- Offers more advanced filtering and matching options, including fuzzy matching
Cons of dupeGuru
- Slower performance compared to fdupes, especially for large file sets
- More complex setup and installation process
- Requires more system resources due to its GUI and additional features
Code Comparison
fdupes uses a simple command-line interface:
fdupes -r /path/to/directory
dupeGuru offers a GUI, but can also be used via command-line:
dupeguru -o results.csv /path/to/directory
Key Differences
- fdupes is a lightweight, command-line tool focused on fast duplicate file detection
- dupeGuru provides a more comprehensive solution with a GUI and advanced features
- fdupes is better suited for quick, simple duplicate searches, while dupeGuru excels in complex scenarios and for users who prefer visual interfaces
Use Cases
-
Choose fdupes for:
- Quick duplicate file detection in terminal environments
- Scripting and automation tasks
- Systems with limited resources
-
Choose dupeGuru for:
- Users who prefer graphical interfaces
- Complex duplicate detection scenarios
- Handling various file types, including music and pictures
Extremely fast tool to remove duplicates and other lint from your filesystem
Pros of rmlint
- Faster performance, especially for large datasets
- More comprehensive duplicate detection (includes empty files, non-regular files)
- Advanced features like handling hardlinks and symlinks
Cons of rmlint
- More complex command-line interface
- Steeper learning curve for advanced features
- Potentially overwhelming output for simple use cases
Code Comparison
rmlint:
rmlint /path/to/directory
fdupes:
fdupes -r /path/to/directory
Both tools can be used to find duplicate files, but rmlint offers more options and flexibility in its command-line interface. For example:
rmlint:
rmlint --types=duplicates,emptyfiles,nonstripped /path/to/directory
fdupes doesn't have built-in support for detecting empty files or non-stripped binaries, focusing primarily on finding duplicate files based on content.
rmlint generates a shell script to handle the duplicates, which can be reviewed before execution:
sh -i rmlint.sh
fdupes can directly remove duplicates with the -d option:
fdupes -rd /path/to/directory
Overall, rmlint offers more features and flexibility, while fdupes provides a simpler, more straightforward approach to finding and managing duplicate files.
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Introduction
FDUPES is a program for identifying duplicate files residing within specified directories.
Usage
Usage: fdupes [options] DIRECTORY...
-r --recurse for every directory given follow subdirectories encountered within -R --recurse: for each directory given after this option follow subdirectories encountered within (note the ':' at the end of the option, manpage for more details) -s --symlinks follow symlinks -H --hardlinks normally, when two or more files point to the same disk area they are treated as non-duplicates; this option will change this behavior -G --minsize=SIZE consider only files greater than or equal to SIZE bytes -L --maxsize=SIZE consider only files less than or equal to SIZE bytes -c --cache speed up file comparisons by keeping track of their signatures in a database; additional parameters may be provided using one or more cache parameters (as below) -x cache.OPTION supply an optional cache parameter, where OPTION is one of the keywords below and multiple options may be supplied via successive -x arguments: readonly read but do not update file signatures prune look through entire cache and delete orphaned entries clear clear all entries from cache vacuum reduce size of DB file, if possible (note that the options prune, clear, and vacuum may be employed without supplying a DIRECTORY argument, and will take effect even if readonly is also specified) -n --noempty exclude zero-length files from consideration -A --nohidden exclude hidden files from consideration -f --omitfirst omit the first file in each set of matches -1 --sameline list each set of matches on a single line -S --size show size of duplicate files -t --time show modification time of duplicate files -m --summarize summarize dupe information -q --quiet hide progress indicator -d --delete prompt user for files to preserve and delete all others; important: under particular circumstances, data may be lost when using this option together with -s or --symlinks, or when specifying a particular directory more than once; refer to the fdupes documentation for additional information -D --deferconfirmation in interactive mode, defer byte-for-byte confirmation of duplicates until just before file deletion -P --plain with --delete, use line-based prompt (as with older versions of fdupes) instead of screen-mode interface -N --noprompt together with --delete, preserve the first file in each set of duplicates and delete the rest without prompting the user -I --immediate delete duplicates as they are encountered, without grouping into sets; implies --noprompt -p --permissions don't consider files with different owner/group or permission bits as duplicates -o --order=BY select sort order for output and deleting; by file modification time (BY='time'; default), status change time (BY='ctime'), or filename (BY='name') -i --reverse reverse order while sorting -l --log=LOGFILE log file deletion choices to LOGFILE -v --version display fdupes version -h --help display this help message
Unless -1 or --sameline is specified, duplicate files are listed together in groups, each file displayed on a separate line. The groups are then separated from each other by blank lines.
When -1 or --sameline is specified, spaces and backslash characters () appearing in a filename are preceded by a backslash character. For instance, "with spaces" becomes "with\ spaces".
When using -d or --delete, care should be taken to insure against accidental data loss. While no information will be immediately lost, using this option together with -s or --symlink can lead to confusing information being presented to the user when prompted for files to preserve. Specifically, a user could accidentally preserve a symlink while deleting the file it points to. A similar problem arises when specifying a particular directory more than once. All files within that directory will be listed as their own duplicates, leading to data loss should a user preserve a file without its "duplicate" (the file itself!).
Contact Information for Adrian Lopez
email: adrianlopezroche@gmail.com
Legal Information
FDUPES Copyright (c) 1999-2019 Adrian Lopez
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Top Related Projects
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot