Imagine that we are writing a backup application used to backup files from your laptop to a remote server. To save on network bandwidth, we want to identify duplicate files (i.e., files with the same contents). This way, we only need to upload duplicate files once.
Write a function that identifies sets of files with identical contents.
find_dupes(root_path) → sets/lists of 2 or more file paths that have identical contents
find_dupes("/home/airtable") → [
[".bashrc", "Backups/2017_bashrc"],
["Photos/Vacation/DSC1234.JPG", "profile.jpeg", ".trash/lej2dp28/87msnlgyr"],
]For scale, imagine running this on a computer with at most 2 TB of data and at most 1 million files.
For traversing the filesystem, use these library functions: