@DATABASE EqFiles.guide @AUTHOR Fulvio Peruggi @(C) 1998 Fulvio Peruggi @$VER: EqFiles.guide V1.7 @FONT Xhelvetica.font 13 @SMARTWRAP @NODE MAIN "EqFiles.rexx © 1998 Fulvio Peruggi" @NEXT MAIN @FONT Xcourier.font 13 @{CODE} @{JCENTER} @{bg highlight} @{b}EqFiles.rexx@{ub} V1.8 written by @{b}Fulvio Peruggi@{ub} in 1998 @{bg background} @{" Introduction " link E.intro} @{" Installation " link E.insta} @{" Copyright " link E.copyr} @{" Usage " link E.usage} @{" Examples " link E.examp} @{" History " link E.histo} @{" To do " link E.todo} @{" Thanks " link E.thank} @{" Author " link E.autho} @{" Italian users " link I.intro} @{fg highlight} // Only \\X/ Amiga makes it possible! @{fg text} Please visit: Home Of MadOne - The WWW Klondike collectors site @{"http://hem.passagen.se/madone/" SYSTEM "AWeb-II:AWeb-II URL http://hem.passagen.se/madone/"} (AWeb-II} @{APEN 0}This guide is designed for V40. @ENDNODE @NODE E.intro "Introduction" @{b}EqFiles.rexx@{ub} is a program which compares files in one or two directories (or directory trees) and searches for files having the same contents and (possibly) distinct filenames. You know the problem. You have collections of files (pictures, music modules, etc.) downloaded from different sites or extracted from distinct CD-ROMs. Often you have one or more duplicates of certain files, because distinct sources used different filenames for the same file. You can find duplicate files manually, but actually this is convenient only in a small collection. When your collection is formed by hundreds or thousands of files the only reasonable way to find duplicates is a program that does the job for you. As far as I know, all usual @{i}compare@{ui} programs make comparisons between files having the same filename, and so they are useless in the present case. Here is the point where @{b}EqFiles.rexx@{ub} comes in handy. With @{b}EqFiles.rexx@{ub} it is also possible to delete automatically all duplicate files found in a search. @{b}EqFiles.rexx@{ub} is optimized for maximum speed. @{b}EqFiles.rexx@{ub} can be used from within Directory Opus. @{FG Highlight}Operation details:@{FG Text} @{b}EqFiles.rexx@{ub} scans the directories involved calling the C:List command, then sorts the output with the C:Sort command, and finally operates in RAM: to find all pairs of files with equal size. Only these pairs are actually compared to ascertain if they are really equal. (The worst case occurs when a great number of files have the same size: then there will be many explicit comparisons among them that can take a long time.) @{FG Highlight}External requirements:@{FG Text} @{b}EqFiles.rexx@{ub} makes use of @{b}cmp@{ub} by Alex Kazik (AmiNet:util/cli/ALeXcompare.lha), i.e. in my opinion the best @{i}compare@{ui} program ever written for the Amiga computer. @{FG Highlight}System requirements:@{FG Text} AmigaOS 2.04 or better, RexxMast running. @{b}EqFiles.rexx@{ub} uses the C:List, C:Sort, and C:Delete commands. Be sure you have the original OS commands or absolutely equivalent substitutes! @ENDNODE @NODE E.insta "Installation" Copy @{b}EqFiles.rexx@{ub} into your AREXX: directory and the documentation wherever you want. Download AmiNet:util/cli/ALeXcompare.lha and install @{b}cmp@{ub} following its instructions. @{fg highlight}NOTE:@{fg text} @{b}cmp@{ub} must be in C: or in your command path! Elsewhere the correct path should be notified to @{b}EqFiles.rexx @{ub} by inserting it at the beginning of the code (instructions are given in the code). @ENDNODE @NODE E.copyr "Copyright" @{FG Highlight}Copyright © 1998 by Fulvio Peruggi@{FG Text} @{b}EqFiles.rexx@{ub} is an EmailWare program. If you use it, please send @{"me" LINK E.autho} a message and let me know. Thank you. The package may not be altered in any way and cannot be used for commercial purposes without the prior written permission of the author. The copyright message should be preserved. @{b}EqFiles.rexx@{ub} makes use of @{b}cmp@{ub}, a FreeWare program copyright © 1997 by ALeX Kazik. @{FG Highlight}No warranty:@{FG Text} No responsibility or liability will be accepted for any damage that may appear to have resulted from use of this program. All use is at your own risk. The software is provided "as is" without any warranty implied or otherwise to the fitness or accuracy of the software and documentation. The documentation is believed to be correct but the author reserves the right to update the software and/or documentation without notice. @ENDNODE @NODE E.usage "Usage" @{b}EqFiles.rexx@{ub} @{i}is an ARexx application that can be started by typing @{ui} rx EqFiles.rexx ALL DELETE @{i}in a shell window.@{ui} @{fg highlight}rx@{fg text} @{i}The starter program @{ui}@{b}rx@{ub} @{i}can be omitted if you activate the script bit of @{ui}@{b}EqFiles.rexx@{ub} @{i}(to do this just type @{ui}Protect AREXX:EqFiles.rexx +s @{i}in a shell window).@{ui} @{fg highlight}.rexx@{fg text} @{i}This extension is optional for @{ui}WShell @{i}users (like me). They can omit both the starter program @{ui}@{b}rx @{ub}@{i}(don't worry about the script bit!) and the @{ui}@{b}.rexx @{ub}@{i}extension.@{ui} @{fg highlight} @{fg text} @{b}EqFiles.rexx@{ub} @{i}scans all files in directories @{ui}@{b} @{ub}@{i}and @{ui}@{b}@{ub}@{i}, compares all pairs of files having the same size, and lists all pairs of equal files detected. When @{ui}@{b}@{ub} @{i}and @{ui}@{b}@{ub} @{i}are the same directory, or when @{ui}@{b}@{ub} @{i}is not specified, @{ui}@{b}EqFiles.rexx@{ub} @{i}scans and compares the files in @{ui}@{b}@{ub}@{i}, and lists all pairs of equal files detected.@{ui} @{b}EqFiles.rexx@{ub} @{i}acts on the current directory if you don't specify neither @{ui}@{b}@{ub} @{i}nor @{ui}@{b}@{ub}. @{fg highlight}ALL@{fg text} @{i}The search for equal files is done in all the subdirectories of @{ui}@{b}@{ub} @{i}(and @{ui}@{b}@{ub}@{i}) if the optional parameter @{ui}@{b}ALL @{ub}@{i}is specified.@{ui} @{fg highlight}DELETE@{fg text} @{b}EqFiles.rexx@{ub} @{i}also deletes all duplicates if the optional parameter @{ui}@{b}DELETE@{ub} @{i}is specified. When two directories are searched, duplicates are always deleted in @{ui}@{b}@{ub}. @{i}To obtain the template and a brief help text you can type @{ui} rx EqFiles.rexx ? @{i}in a shell window: @{ui} 1.RD0:> rx EqFiles.rexx ? @{bg highlight}EqFiles V1.8 by Fulvio Peruggi - October 11, 1998 @{bg background} Usage: [rx] EqFiles[.rexx] [] [] [ALL] [DELETE] Scans all files in directories and , compares all pairs of files having the same size, and lists all pairs of equal files. Scans, compares, and lists all pairs of equal files in , if = or if is not specified. Command operations act on all subdirectories and their files if the optional parameter ALL is specified. Equal files found in are deleted if the optional parameter DELETE is specified. @{i}The attempt has been done to make fool-proof the parsing routine that interprets the arguments you type. A number of explanatory comments are printed in case of errors. Please let me know (eh eh...) if you are so fool to obtain an improper operation.@{ui} @ENDNODE @NODE E.examp "Examples" @{i}Assume we have the following directory trees in @{ui}RD0:@{i} and @{ui}RD1:. 1.RD0:> list RD0: all Directory "RD0:" on Lunedì 05-Ott-98 subdir1 Dir ----rwed Oggi 18:04:49 random00.jpg 57305 ----rwed 25-Apr-98 01:18:46 random01.jpg 39548 ----rwed 25-Apr-98 01:18:18 random10.jpg 57305 ----rwed 25-Apr-98 01:18:46 random05.jpg 51689 ----rwed 25-Apr-98 01:19:33 4 files - 1 directory - 409 blocks used Directory "RD0:subdir1" on Lunedì 05-Ott-98 random07.jpg 51689 ----rwed 25-Apr-98 01:19:33 random05.jpg 51689 ----rwed 25-Apr-98 01:19:33 random24.jpg 46808 ----rwed 22-Apr-98 23:55:18 random01.jpg 39548 ----rwed 25-Apr-98 01:18:18 4 files - 376 blocks used TOTAL: 8 files - 1 directory - 785 blocks used 1.RD0:> list RD1: all Directory "RD1:" on Lunedì 05-Ott-98 subdir2 Dir ----rwed Oggi 18:04:01 random13.jpg 43433 ----rwed 25-Apr-98 01:18:57 random00.jpg 57305 ----rwed 25-Apr-98 01:18:46 random10.jpg 57305 ----rwed 25-Apr-98 01:18:46 random25.jpg 29363 ----rwed 22-Apr-98 23:55:41 4 files - 1 directory - 373 blocks used Directory "RD1:subdir2" on Lunedì 05-Ott-98 random07.jpg 51689 ----rwed 25-Apr-98 01:19:33 random00.jpg 57305 ----rwed 25-Apr-98 01:18:46 random24.jpg 46808 ----rwed 22-Apr-98 23:55:18 random25.jpg 29363 ----rwed 22-Apr-98 23:55:41 4 files - 367 blocks used TOTAL: 8 files - 1 directory - 740 blocks used @{i}All pairs of files having the same filesize (among the files listed above) are equal. Let us apply @{ui}@{b}EqFiles.rexx @{ub}@{i}in various ways to search for duplicates.@{ui} 1.RD0:> rx EqFiles.rexx @{fg highlight}Scanning directory @{b}"RD0:"@{ub}@{fg text} random00.jpg == random10.jpg @{i}The current directory (@{ui} RD0:@{i}) is scanned to search for equal files. Possible subdirectories (@{ui}subdir1 @{i}in this case) are ignored. Stupid or useless comparisons are avoided: for example comparing a file with itself, or comparing file B with file A after file A was already compared with file B. The real thing makes use of ANSI codes to obtain more readable colour and/or bold outputs. [NOTE: You may change the colours used by the program. Search the code for expressions of the type @{ui}[0;36m. @{i}Then replace 36 with 31, 32, ..., 35, or 37 to otain other workbench colours (respectively colour 2, 3, ..., 6, or 8 of the workbench palette).]@{ui} 1.RD0:> rx EqFiles.rexx all @{b}@{ub} "ALL" @{b}does not exist!@{ub} @{i}Options @{ui}ALL@{i} and @{ui}DELETE@{i} will be executed only if you explicitly specify a directory. Options can be keyed in using small or capital letters. [NOTE: In such a way an improbable directory named @{ui}all@{i} or @{ui}delete@{i} can be scanned if you have one.]@{ui} 1.RD0:> rx EqFiles.rexx "" all @{fg highlight}Scanning directory @{b}"RD0:"@{ub} with option(s) ALL@{fg text} random00.jpg == random10.jpg random01.jpg == subdir1/random01.jpg random05.jpg == subdir1/random05.jpg random05.jpg == subdir1/random07.jpg subdir1/random05.jpg == subdir1/random07.jpg @{i}The current directory and all its subdirectories are scanned to search for equal files. The output is sorted.@{ui} 1.RD0:> rx EqFiles.rexx RD0: RD1: delete @{fg highlight}Scanning directories @{b}"RD0:"@{ub} and @{b}"RD1:"@{ub} with option(s) DELETE@{fg text} random00.jpg == random00.jpg @{fg highlight}RD1:random00.jpg Deleted@{fg text} random00.jpg == random10.jpg @{fg highlight}RD1:random10.jpg Deleted@{fg text} @{i}Directories @{ui}RD0:@{i} and @{ui}RD1:@{i} are scanned to search for equal files. All duplicates found in @{ui}RD1:@{i} are deleted and deletion is notified. Subdirectories in both @{ui}RD0:@{i} and @{ui}RD1:@{i} are ignored.@{ui} 1.RD0:> rx EqFiles.rexx RD0: RD1:subdir2 all @{fg highlight}Scanning directories @{b}"RD0:"@{ub} and @{b}"RD1:subdir2/"@{ub} with option(s) ALL@{fg text} random00.jpg == random00.jpg random05.jpg == random07.jpg random10.jpg == random00.jpg subdir1/random05.jpg == random07.jpg subdir1/random07.jpg == random07.jpg subdir1/random24.jpg == random24.jpg @{i}The directory paths scanned are printed only in the first line of output, then only residual paths are printed. The first column refers to the first directory path, and the second column refers to the second directory path.@{ui} @endnode @NODE E.histo "History" @{u}Version 0.1-0.9@{uu} (01.04.98-30.04.98) · Actually... prehistory. The prototype of all further versions was a program named SearchEqFiles.rexx. It was subsequently replaced by two distinct programs (FindEqFiles.rexx and DeleteEqFiles.rexx), that at last were integrated into a single program. @{u}Version 1.0@{uu} (02.05.98) · First fully working and integrated program. · Internal release. @{u}Version 1.1@{uu} (09.05.98) · Improved parsing routine. · Internal release. @{u}Version 1.2@{uu} (16.05.98) · Optimized comparisons. · Internal release. @{u}Version 1.3@{uu} (30.05.98) · Parsing routine rewritten from scratch. · Internal release. @{u}Version 1.4@{uu} (31.05.98) · Reworked comparison code to minimize hard disk accesses. · Internal release. @{u}Version 1.5@{uu} (29.07.98) · Modified file calling (filenames with spaces caused errors). · Internal release. @{u}Version 1.6@{uu} (03.09.98) · Modified deletion routine: in some (rare) cases files with equal length and distinct contents were deleted. · Internal release. @{u}Version 1.7@{uu} (10.09.98) · Improved parsing routine: now really fool-proof? · Improved comparison code: speed gain. · Created document file in AmigaGuide format. · Beta-tester release. @{u}Version 1.8@{uu} (11.10.98) · Removed residual erroneous handling of filenames with spaces in case of deletion. · Major changes and additions in the document file. · First Aminet release. @ENDNODE @NODE E.todo "Things to do" · Remove residual bugs, hopefully none. (Please let me know if you find one.) · Insert new options required by a reasonable number of users. · Compile the program. (I must purchase the compiler...) @ENDNODE @NODE E.thank "Thanks to..." @{JCENTER} @{b}@{u}Thanks to:@{ub}@{uu} @{b}Alex Kazik@{ub} beta testing; the friend without whom this program would never be written. @{b}Alberto Mangini@{ub} beta testing; solicitor of improved performances. @{b}Constantinos Nicolakakis@{ub} beta testing and useful suggestions. @{b}Salvatore Stilo@{ub} proof-reading, correction of the guide and beta testing. @ENDNODE @NODE E.autho "Author" @NEXT E.autho @FONT Xcourier.font 13 @{CODE} @{JCENTER} If you have any questions, suggestions, wishes or bug reports please contact me: (I like receiving e-mail!) Fulvio Peruggi Via Consalvo 150, isolato 5 80126 Napoli Italy E-Mail: @{FG Highlight}peruggi@unina.it@{FG Text} @ENDNODE @NODE I.intro "Sorry..." Sono spiacente, ma ho scritto la documentazione direttamente in inglese e non ho tempo per riscriverla in italiano. Forse nella prossima versione... @ENDNODE