fdupes
FDUPES is a program for identifying or deleting duplicate files residing within specified directories.
Top Related Projects
Quick Overview
FDUPES is a program for identifying and/or deleting duplicate files residing within specified directories. It can be used to reclaim disk space by replacing redundant files with hardlinks to a single copy of the file content. FDUPES is designed to be efficient and user-friendly, making it a valuable tool for managing file systems.
Pros
- Fast and efficient in identifying duplicate files
- Supports recursive directory scanning
- Offers various options for handling duplicates (delete, hardlink, symlink)
- Can preserve a specified number of duplicate files
Cons
- Command-line interface may be intimidating for non-technical users
- Limited GUI options available
- May require careful use to avoid accidental data loss
- Does not support cloud storage or network drives directly
Getting Started
To get started with FDUPES, follow these steps:
-
Install FDUPES on your system:
# On Ubuntu/Debian sudo apt-get install fdupes # On macOS with Homebrew brew install fdupes
-
Basic usage to find duplicates in a directory:
fdupes /path/to/directory
-
To find duplicates recursively in multiple directories:
fdupes -r /path/to/dir1 /path/to/dir2
-
To delete duplicates interactively:
fdupes -d /path/to/directory
-
To replace duplicates with hardlinks:
fdupes -rH /path/to/directory
Remember to use FDUPES carefully, especially when deleting files. Always ensure you have backups of important data before making changes to your file system.
Competitor Comparisons
Find duplicate files
Pros of dupeGuru
- Graphical user interface, making it more user-friendly for non-technical users
- Supports multiple file types, including music and pictures, with specialized scanning modes
- Offers more advanced filtering and matching options, including fuzzy matching
Cons of dupeGuru
- Slower performance compared to fdupes, especially for large file sets
- More complex setup and installation process
- Requires more system resources due to its GUI and additional features
Code Comparison
fdupes uses a simple command-line interface:
fdupes -r /path/to/directory
dupeGuru offers a GUI, but can also be used via command-line:
dupeguru -o results.csv /path/to/directory
Key Differences
- fdupes is a lightweight, command-line tool focused on fast duplicate file detection
- dupeGuru provides a more comprehensive solution with a GUI and advanced features
- fdupes is better suited for quick, simple duplicate searches, while dupeGuru excels in complex scenarios and for users who prefer visual interfaces
Use Cases
-
Choose fdupes for:
- Quick duplicate file detection in terminal environments
- Scripting and automation tasks
- Systems with limited resources
-
Choose dupeGuru for:
- Users who prefer graphical interfaces
- Complex duplicate detection scenarios
- Handling various file types, including music and pictures
Extremely fast tool to remove duplicates and other lint from your filesystem
Pros of rmlint
- Faster performance, especially for large datasets
- More comprehensive duplicate detection (includes empty files, non-regular files)
- Advanced features like handling hardlinks and symlinks
Cons of rmlint
- More complex command-line interface
- Steeper learning curve for advanced features
- Potentially overwhelming output for simple use cases
Code Comparison
rmlint:
rmlint /path/to/directory
fdupes:
fdupes -r /path/to/directory
Both tools can be used to find duplicate files, but rmlint offers more options and flexibility in its command-line interface. For example:
rmlint:
rmlint --types=duplicates,emptyfiles,nonstripped /path/to/directory
fdupes doesn't have built-in support for detecting empty files or non-stripped binaries, focusing primarily on finding duplicate files based on content.
rmlint generates a shell script to handle the duplicates, which can be reviewed before execution:
sh -i rmlint.sh
fdupes can directly remove duplicates with the -d
option:
fdupes -rd /path/to/directory
Overall, rmlint offers more features and flexibility, while fdupes provides a simpler, more straightforward approach to finding and managing duplicate files.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Introduction
FDUPES is a program for identifying duplicate files residing within specified directories.
Usage
Usage: fdupes [options] DIRECTORY...
-r --recurse for every directory given follow subdirectories
encountered within
-R --recurse: for each directory given after this option follow
subdirectories encountered within (note the ':' at the
end of the option, manpage for more details)
-s --symlinks follow symlinks
-H --hardlinks normally, when two or more files point to the same
disk area they are treated as non-duplicates; this
option will change this behavior
-G --minsize=SIZE consider only files greater than or equal to SIZE bytes
-L --maxsize=SIZE consider only files less than or equal to SIZE bytes
-c --cache speed up file comparisons by keeping track of their
signatures in a database; additional parameters may be
provided using one or more cache parameters (as below)
-x cache.OPTION supply an optional cache parameter, where OPTION is one
of the keywords below and multiple options may be
supplied via successive -x arguments:
readonly read but do not update file signatures
prune look through entire cache and delete orphaned entries
clear clear all entries from cache
vacuum reduce size of DB file, if possible
(note that the options prune, clear, and vacuum may be
employed without supplying a DIRECTORY argument, and
will take effect even if readonly is also specified)
-n --noempty exclude zero-length files from consideration
-A --nohidden exclude hidden files from consideration
-f --omitfirst omit the first file in each set of matches
-1 --sameline list each set of matches on a single line
-S --size show size of duplicate files
-t --time show modification time of duplicate files
-m --summarize summarize dupe information
-M --quicksummary summarize dupe information quickly, skipping the
slower byte-for-byte match confirmation
-q --quiet hide progress indicator
-d --delete prompt user for files to preserve and delete all
others; important: under particular circumstances,
data may be lost when using this option together
with -s or --symlinks, or when specifying a
particular directory more than once; refer to the
fdupes documentation for additional information
-D --deferconfirmation in interactive mode, defer byte-for-byte confirmation
of duplicates until just before file deletion
-P --plain with --delete, use line-based prompt (as with older
versions of fdupes) instead of screen-mode interface
-N --noprompt together with --delete, preserve the first file in
each set of duplicates and delete the rest without
prompting the user
-I --immediate delete duplicates as they are encountered, without
grouping into sets; implies --noprompt
-p --permissions don't consider files with different owner/group or
permission bits as duplicates
-o --order=BY select sort order for output and deleting; by file
modification time (BY='time'; default), status
change time (BY='ctime'), or filename (BY='name')
-i --reverse reverse order while sorting
-l --log=LOGFILE log file deletion choices to LOGFILE
-v --version display fdupes version
-h --help display this help message
Unless -1 or --sameline is specified, duplicate files are listed
together in groups, each file displayed on a separate line. The
groups are then separated from each other by blank lines.
When -1 or --sameline is specified, spaces and backslash characters (\)
appearing in a filename are preceded by a backslash character. For
instance, "with spaces" becomes "with\ spaces".
When using -d or --delete, care should be taken to insure against
accidental data loss. While no information will be immediately
lost, using this option together with -s or --symlink can lead
to confusing information being presented to the user when prompted
for files to preserve. Specifically, a user could accidentally
preserve a symlink while deleting the file it points to. A similar
problem arises when specifying a particular directory more than
once. All files within that directory will be listed as their own
duplicates, leading to data loss should a user preserve a file
without its "duplicate" (the file itself!).
Contact Information for Adrian Lopez
email: adrianlopezroche@gmail.com
Top Related Projects
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot