****************************************************************************

DoubleTake V2.1 01-10-97 by Olly Koenders (A.R.G BBS +61-3-9870-4608 In Oz)


Below are the docs for V2.0, which had a slight code malfunction that caused
DoubleTake to read AmiFileSafe and FFS directory cached disks incorrectly.

Why?  Unknown really.  But rest-assured it now works on all DosTypes,
Kickstarts and Processors etc.  Some code has been optimized and memory
handling is better.

Many thanks to Hans de Groot (hansg@3wis.nl) for bringing this to my
attention as soon as it appeared on Aminet.  Sorry fellas, didn't have AFS
or anything with directory caching to test on when I was with KS 1.3
(yeah..), and now it should also work with FastSmoothDirs (old proggy)!

Anyhoo - enjoy!


Olly :)

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

...
...
...address changed from V2.0 docs!
...
...


Why V2.0?  Well, it started something like this...



Brother: "Excuse me Mister `Expert Programmer'..."

Me: "Eh?" (Here we go - Another Chrissy wish-list taped to the fridge!)

Brother: "I need a program that will find all the doubled-up files an'
	 stuff on me hard disk." (New A4000T with 1.2 Giggles o' stuff)

Me: "Uh." (Shit! - old A500 with WB 1.3 and still scratching floppies
          because I never needed to upgrade)

Brother: "'Cos I found a few files that were doubled-up in different
	 directories an' I reckon there'd be HEAPS more layin' around
	 wastin' space..." (Space?!? - What?!?)

Me: "Yyyyyyyehhh... Umm... I could access the filenames using DOS an'
    then hash 'em around a bit... This might need some lengthy
    experimentation..."

Meanwhile `Brother dearest' leaves me to ponder said intricacies and,
appearing to be satisfied with his inspirational address, goes back to his
`Playstation' to continue cursing at an unspecified application that
"...won't @#?*&! handle @#?*&! brushmaps properly!!!".  Complaints I reckon
are unfounded considering he got the thing from a coverdisk.

While fumbling around trying to put out a fire somewhere in my overstuffed
ashtray, I was thinking that this shouldn't be too hard...



The rest of this story can be found in the `TECHIE SECTION' if required.

**** NON-TECHIE SECTION ****

The above gives a basic outline of its purpose.  The results can be well
worth it or not depending on the number of useless duplicate files/libraries
etc. - Considering the 17000-odd file & directory names dragged from
the abovementioned machine (not to mention the hundreds of duplicates!).

To get it running, click its icon from WorkBench or enter its name in the
CLI, then click on one of the appropriate buttons.  Time to completion
will be dependant on the amount of stuff contained on the device, the
level of directory fragmentation, whether its a floppy or HD and also
the speed of your computer.

DoubleTake first searches the entire selected device, retrieves all the
file and directory names and writes each entry name to `RAM:DirList'.

There's enough memory allocated to go 30 directory levels deep
(Assembler:/devs/printers/etc./etc.) so if you have a directory with 31
entries behind it then either you're really dumb and like wasting 512 bytes
every time you `Makedir' another one or you try to be too organised -
Get over it!

Pass 1 sorts them into some logical order deeming which files belong in
what directory etc.  This is necessary due to the DOS exnext() function
returning the next FileInfoBlock that's chained to the previous block which
could belong to either a file or directory.  All directory programs attempt
to sort their directory output if specified, and that's exactly what they
have to stumble through, albeit this one isn't done alphabetically.

During sorting, each entry is written back to the file `RAM:Dirlist' which
is necessary for the next function but not deleted so the user can do with
this file as they please.

This file (somewhat shorter than the first) is then loaded back into the
same allocated memoryblock and Pass 2 begins where the duplicate files
are matched in their respective directories and output in `RAM:Multilist'
(Which is the free gift you should be most interested in and is yours to
keep whether you return the Ginsu Salami Slicer or not...).

Note that this was about the easiest way I could devise to get the results
and also create the Dirlist which can have its uses (Quick! - Think of
something!  ...Anything!).

Although Norton Utilities for the PC have something similar and is probably
many, MANY killerbytes huge, it's bloody useless on the Amiga.  Besides
that, So far I've not seen any program that does this on the Amiga but I'm
sure there are a few.  Some might even be found in applications such as
`Pagesetter' or even large and specialised database programs.  But buying
a massive (read expensive) application for this small but handy purpose
doesn't have many merits apart from some of the extra options that may be
available, like Datestamping, Comments or filesize output.

You don't get much for nothing (usually even less) - version 1.0 was Shell
use only, and the first version that worked fully from start to finish
with its original purpose in mind.  During its construction, many
(Well - about a hundred) versions were built and tested just to get the
damn thing to work (All this kind of stuff is very new to me).  If I
named it version 107.24, even I'd pass it by with scant regard (Remember
Cache's X-Copy and the plethora of versions following it?).  Version 1.0
was far from satisfactorily complete so a user interface was implemented.

When in operation, the numbers you see are vital to the internal workings
of the program but also serve as a good progress indicator.  With the Amiga
and IBM (Idiot Breed Mentality) world in a storm at the moment about the
ubiquitous `Progress Indicator' where every damn application is thrusting
this thing in our face, I thought I'd get my own back.

DoubleTake is solid, fast and there are no real bugs.  If a disk or device
isn't accessible then this error will be detected and dealt with
appropriately.  If however a problem develops during the disk reading phase
(Corrupt disk?) then the operating system will more than likely pipe up and
tell you that something is really, REALLY wrong and either hang up or let
you do it yourself.  This won't cause any further damage to your disk
though.  While on the subject...

**** DISCLAIMER ****

I will not be held responsible for damage caused by DoubleTake under any
circumstances (Blah, Blah, Blah...).

Note that damage would be completely impossible due to the program only
writing to the RAM: disk - unless you already have a file in RAM: called
`Dirlist' (Gee-whiz! The shields are down to 17 percent AGAIN, Captain! I
shoulda foreseen that fantastic probability...).

The reading process will only damage a disk if you've been storing it in a
sand pit or if you HAMMER THE BLOODY THING INTO THE DRIVE UPSIDE DOWN!

**** FREE GIFT! *  FREE GIFT! * FREE GIFT! * FREE GIFT! * FREE GIFT! ****  

The icon is free.

This program and its associated files (This one and the icon) are free.
This program and its associated files may be distributed anywhere on
anything in any usable form including Belfast, Norway, Russia and
UpperKumbucta West if so desired because this program and its associated
files - are free.

A small donation wouldn't hurt though...  Enjoy.

**** TECHIE SECTION ****

... Anyway - four days, many headaches, some (almost) sleepless nights, an
incalculable number of coffees and a couple slices of cheese later...

Here 'tis! - Sorta.

At the time, the only way I was able to get the entire directory of a disk
was to use the original `c/Dir' command and type the string:

dir >ram:dirlist <nil: df3: opt ai

Remove the characters between `dir' and `df3:' and you'll see what I mean.
the created file was full of spaces, these `?' bloody things and `(dir)'
kinds of text and required considerable processing before I could load the
file and find all the duplicates.  Listing the duplicates with their
directories was not on my mind at the time and would've proved very
difficult.  The `Dir' command doesn't use the DOS lock() subroutine and
therefore confused the hell outta me.  It probably rakes through the
Hash Chains and goes from there.  The first day was wasted attempting to
utilise the lock() function and no matter what I did it was virtually all
for nothing.

So, forging ahead with great gusto (?), the third phase of the original
concept was taking shape (Full of holes, processor traps and gurus, but a
shape nonetheless) and after those four days was implemented.

The results spoke for themselves.  Sure it could find duplicates, but it
took 10-15 minutes on the A4000 just to output filenames and a number
beside each depicting how many times it was found overall.  This isn't
useful to anyone.  Besides, the numbers on the first attempt were mostly
wrong so after another 20 minutes of fiddling that was fixed.

My Brother loaded the file and, going by all the duplicate filenames,
must've been scared off as he said he'd go through the list some other
time.  The only way to accomplish that would've been using DirectoryOpus to
`Hunt' for that particular file that number of times and find out where
they all are and why the hell they're in there.

A solution started the next day when I got the idea (I should've realised
it from the start!) that the FileInfoBlock and lock() are related - to use
exnext() with a different lock would cause exnext() to look for the next
entry relevant only to that lock and not the lock I needed.  So a memory
buffer of 7920 bytes is allocated and used to store all the previous locks
and InfoBlocks.  When a directory entry is found, I copy the lock and
InfoBlock from my work area to this buffer and lock() onto the new
directory name.  Once there are no more entries under this directory lock,
I then unlock() the one I was using and copy back the last saved lock and
InfoBlock to my work area again and continue searching with exnext().  This
will go on ad-nausaeum until the root directory was once again reached and
exnext() returns a no more entries signal.

The only way to lock onto another directory was to use the directory names
I came across and form them into a string (Assembler:devs/printers/etc.),
which I would then pass to lock() and the lock for that new directory would
be returned for me to use with exnext().  That string would then be written
to RAM: and all filenames found within that lock would be written after it
until a directory entry was found etc.

The theory doesn't sound too complicated but in practice it took some time
for the gurus to go away.

The reason I write all the file/dirnames to the RAM: disk is because you
don't know how many files etc. the thing has until you hack through it,
which means a second pass to get the names into memory - too much time
wasting.

If you have 17800 files and tried to save every infoblock (260 bytes for
each block) then the amount of memory needed would be a staggering 4.6 Meg!
The filenames themselves are different lengths so calculating on filename
length means a second pass anyway.  Not good enough.

With that section up and running faultlessly (More on that later!) it was
time to move on to sorting.  While reading all these entries that come up
in any order (dir-file-file-dir-dir-file etc.), when the directory entry
copied back to the work area was complete, the pathname string used for
lock() was reformatted to contain up to that last directory entry and
written to RAM: so that if any more file entries were found under that
directory name then their origin could be determined during sorting.

With all the above the nature of the sorting process can be determined
in a hopefully uncomplicated manner so I'll leave it alone.

The duplicates are found in much the same manner but it's slightly more
complicated because I try to make fast and efficient programs and that's
one reason I use Assembler.  (A program in `C' that prints `Hello World!'
to the Shell can reach a few thousand bytes depending on the programmer,
due to C's `printf()' function containing mostly useless code that either
never gets executed or runs through stacking and unstacking - wasting heaps
of time in the process whereas assembler code can accomplish this in as
little as a hundred bytes).  To ensure the sorting and duplicate finding
routines are as fast as possible, it only makes sense not to hash through
garbage that's already been processed.  You might notice that the number
display will speed up as the operation gets closer to zero.

Although the main problem was the DOS lock() funtion (that I rarely need to
use for anything) not working as planned originally, the second was that
assembler code doesn't have an easy (None at all!) way of working with
strings - you have to make your own routine.  Whereas basic is hugely
simplified in this area:

	input #1, new$

In assembler this translates to something much larger but in reality is
loading an entire string first through DOS read() and then into the address
labelled `new'.  All this translates into a MASSIVE loss in speed so
something much simpler was required:

	bsr.b loadfile		;this line just an example
	lea string(pc),a4
	move.l buffer(pc),a5
	bsr.b getstring
	...

getstring:
	move.b (a5)+,(a4)+
	cmp.b #10,(a5)
	bne.b getstring
	move.b (a5)+,(a4)+
	rts

The entire file is loaded once only into the buffer with read() and then the
address where the string should be copied to is put in a4.  The address of
the buffers' starting point is loaded into a5 and subsequently the
subroutine is called where each byte is copied from the address in a5 to
the address in a4 (with automatic address incrementation on each).  If a
returncode is found (#10), then that byte as well is copied to the string
area to be used as an end marker and a5 is now pointing to the start of the
next string in the buffer.  This getstring subroutine takes barely a
nanosecond to execute and is executed every time a new string is required
from the buffer (file or dir name) where the string after it can be
compared to the one already got:

	if next$ = new$ then
	...

Translated to assembler again:

nextstring:
	lea string(pc),a4
	bsr.b compare
	tst.l d0		;see what d0 has
	bne.b nextstring	;if d0 non-null, get next one (simplified!)
	bra.b found

compare:
	tst.b (a5)		;if null byte then skip to not-equal
	beq.b nequal
	cmp.b (a5)+,(a4)+
	bne.b nequal
	cmp.b #10,(a5)
	beq.b verify
	bra.b compare

verify:
	cmp.b #10,(a4)
	bne.b nequal
	moveq #0,d0		;clear d0 in 4 clock cycles (fast!)

comprts:
	addq.l #1,a5		;add 1 to a5 - line up to next string
	rts

nequal:
	moveq #1,d0		;make d0 non-null (just as fast)
	bra.b comprts

If the byte in address a5 doesn't equal the byte in address a4 then a `1'
is put in d0 to tell me that the two strings don't compare.  Otherwise
it goes back to compare the next byte at each address until a returncode
is located in a5.  To verify the entire string, the returncode must also be
found in a4.  If so, then d0 is neutralised so I know beyond a shadow of a
doubt that the entire string has been processed to the end and all bytes
(including the end marker) are the same.

Throughout the entire process the address in a5 is kept track of
automatically and doesn't require storing anywhere until the end of the
buffer is reached, where the address of the next directory is loaded into
a5 as the next `old' start point:

°Assembler:	<-- old start point after the #10 in this one
DISKDIR		<-- getstring loaded this one
IMPLODER	<-- after getstring, a5 stored in `next'
SPEED
FIXDISK
°Assembler:c/	<-- new start point stored as `start' after the #10
DIR		<-- compare checked this one
FORMAT		<-- after compare, a5 is at the start of this string
PP
CMQ
FBL
FF
AVAILMEM
HEATSYNC
DISKCOPY
<-- end of buffer right here (last byte is the #10 after `Y' on `DISKCOPY')

`DISKDIR' is compared with `DIR' then `FORMAT' then `PP' until the end
of the buffer is reached.  Then `IMPLODER' is compared with `DIR' etc.
Once next = start then getstring loads `DIR' and stores a5 in `next'.
The next directory is found with `°' and then after the #10 a5 is stored
as `start'.  If the last directory entry = start then we know not to bother
processing any further because you can't have two exact filenames in the
same directory.  Any duplicates found are tagged by changing the first byte
to a null (`F' from `FBL') so I know that filename has already been noted
in Multilist as duplicate and will subsequently be skipped.

This process requires bugger-all execution time and is probably the best
way I can explain without another 20 KBytes of text and `Mavis Beacon'
typing skills.  The numbers on the display actually decrement with each
new `start' postion (Ahhhhhhh!).  Very crude - but effective.

Three days later again and everything finally seeming to function as it
should I decided to play around and check for bugs.  It worked fine on
df4: (In my startup-sequence I've set up two RAD's on a permanent basis,
the other called df5:.  My system disk is diskcopied to df4: and df5: is
formatted.  Reboot times are MUCH faster than a hard drive and my original
system disk is removed from the drive and safe from virus attacks etc.
Any disk can be diskcopied to df5: and buggerised around with safely) so
I tried it on one of the externals - BOOM!  All the wrong directories were
being accessed (attempted anyway) and naturally lock failures were
occurring all over the place.  Damn!  I thought I'd fixed that routine.
A little experimentation found that `FastSmoothDirs' was cocking up the
lock() function for the external drives.  Still unknown why as other
directory programs don't seem to have that problem.  Meanwhile, don't run
FSD before DoubleTake.

Being it now on version hundred-and-something (1.0) and still slightly
incomplete (No user guidance), I decided to make a GUI for it but didn't
want to bugger around with structures relating to drawborder() and then if
I didn't like it or wanted to change it later I'd have to change ALL the
numbers in the structure just to make it look different.  So I decided on
a real `graphic' user interface.  Using Dpaint 4, an hour later the brushes
were done and saved.  I then used BitPack (Another of my creations) to
strip the `FORM' headers off the brushes and compress them in my own format
that compresses much better than normal ByteRun 1 (IFF) and decompresses
heaps faster.  These were then assembled into the program and a couple
of hours later the whole thing was complete.  DoubleTake contains 10892
bytes worth of graphics (The progress indicator is the only real `text').
DoubleTake without the graphics would be only about 4000 bytes.  Only 3740
bytes worth of chip mem is required as a buffer to display them all.

When a new image is required for display, it's unpacked to the chip buffer
on the fly, if WB 2.0 or higher is available, then the bitplanes (Which
are sequentially placed and required for drawimage()) are swapped in this
buffer so the colours for differing versions of WorkBench will always be
right.  This happens super-quick and even on a bog standard A500 there'll
be no time lag:

	...
	lea frontpanel(pc),a0	;address of this packed image data to a0
	move.l chbuffer(pc),a1	;address of the chip buffer to a1
	bsr.b decrunch		;unpack it (done when null-byte found)
	move.l #1870,d0		;half of req'd image size (in bytes) to d0
	bsr.b draw		;do the nasty...
	...			;continue processing (gadget checking etc.)

draw:
	tst.w wb2		;test if variable `wb2' is null
	beq.b display		;if so then display immediately
	bsr.b swapmem		;otherwise do the swap then display

display:
	...			;drawimage() routine
	rts			;return to instruction after `bsr.b draw'

swapmem:
	move.l cbuffer(pc),a0	;address of the chip buffer to a0
	move.l a0,a1		;copy it to a1
	add.l d0,a1		;add size of 1 bitplane from d0 to a1
	divu #2,d0		;divide loopnumber by 2 (moving `wordwise')
	subq.w #1,d0		;subtract quick 1 for `dbra'

cmloop:
	move.w (a0),d1		;copy two bytes (word) into d1 (spare)
	move.w (a1),d2		;same for d2
	move.w d2,(a0)+		;copy the word into a0's address then add 2
	move.w d1,(a1)+		;same for a1
	dbra d0,cmloop		;continue until d0 = -1 (reason for `subq')
	rts

If you do this without storing the bytes in a couple of spare registers
first then you'll only serve to overwrite the bytes you're trying to copy
to the other address (Does this make sense?).

I'm not going to document the unpacking routine because the damn thing is
too complex to explain without documenting and explaining the packing
routine (The entire source code is available for anyone forwarding a small
donation in undocumented form - I never waste time inserting comments in my
source code).

Due to the graphics, if I want to make any serious changes to the look of
DoubleTake, then a few minutes redrawing the brushes, compressing them,
and then changing the coordinates of the gadgets (If I move them) is easily
done.

This time the results were heaps better and now it doesn't require days
of `Hunting' with Opus.  `Brother dearest' said this was very useful and
promptly copied the new Multilist over the old.  (Happy birthday to yoooo!)

I can't foresee any major updates, although suppression of `.info' files
might be a luxury, I don't think it's a necessary option because you can
simply delete these from the Multilist yourself with any good editor.

If you would like a personalised GUI then you can draw the brushes and send
them to me in either IFF or paper format.  A few dollars donation and
shipping funds will cover everything (including source code if you want it)
and I'll be happy to accommodate.

Brush dimensions must be:

	Front panel    : 272 x 55

	Info panel     : same

	Suggestive text: 192 x 7

If you want them bigger, then the `x' dimension must be divisible by 16 into
a WHOLE number - I.E: 272 / 16 = 17 (Not 17.5 or 23.58 please).  Note that
this is a hardware limitation and has always been that way.  The `y'
doesn't matter for bigger pictures.  Remember that bigger pictures mean a
bigger program.

Any ideas or correspondence can be forwarded to the address at the very,
VERY top of this text and I'll be glad to hear.

If you're reading this - then at least I've had a captive audience.  Thanks
for your attention.

	Olly.