From zerkle Tue Jan 17 08:10:49 1995 Received: by toadflax.cs.ucdavis.edu (4.1/UCD.CS.2.6) id AA13037; Tue, 17 Jan 95 08:10:48 PST Received: from ice.cs.ucdavis.edu by toadflax.cs.ucdavis.edu (4.1/UCD.CS.2.6) id AA13030; Tue, 17 Jan 95 08:10:41 PST Received: from dialup-30.werple.mira.net.au by ice.cs.ucdavis.edu (5.65/UCD.CS.2.6) id AA29834; Tue, 17 Jan 1995 08:10:22 -0800 Received: by ArtWorks.mira.net.au (Amiga SMTPpost 1.04 December 9, 1994) id AA01; Wed, 18 Jan 95 03:10:14 +1000 Received: by ArtWorks.mira.net.au (Amiga SMTPpost 1.04 December 9, 1994) id AA01; Wed, 18 Jan 95 03:02:15 +1000 Message-Id: <201045b2.5cc60-JamesB@ArtWorks.mira.net.au> X-Mailer: //\\miga Electronic Mail (AmiElm 4.159) From: JamesB@werple.mira.net.au Reply-To: JamesB@werple.mira.net.au Date: Wed, 18 Jan 95 03:02:15 +1000 Organization: Melbourne ArtWorks To: announce@cs.ucdavis.edu Subject: SUBMIT ??? Errors-To: zerkle@cs.ucdavis.edu X-Server: Zserver v0.90beta Status: RO submit Short: Fetch Web source trees, save to file. Uploader: James Burton (burton@cs.latrobe.edu.au) Author: James Burton (burton@cs.latrobe.edu.au) Type: comm/tcp TITLE GetURL.rexx VERSION 1.0 AUTHOR James Burton c/o Department of Computer Science & Computer Engineering Latrobe University Bundoora, Victoria, 3083 Australia EMail: burton@cs.latrobe.edu.au Web: http://www.cs.latrobe.edu.au/~burton/ DESCRIPTION -- Script to download HTML systems across the network -- GetURL.rexx is an ARexx script which will download World-Wide Web pages. With a simple command line it will download a specific page, and with more complex command lines it will be able to download specific sets of documents. The intention was to create a tool that allowed local caching of important web pages and a flexible way of specifying what pages are important. The script has no GUI as of yet but may have at some stage in the future. If you have ever tried to download and save to disc a 200 page document using Mosaic, then you know what this script is for. Mosaic will only let you load a page, then load it to disc, then load another page etc. This is a very frustrating process. GetURL automates this process and will run in batch mode without user intervention. The major features of GetURL.rexx are as follows: * doesn't require AMosaic, you you can be browsing something else with AMosaic whilst this is running * save pages to your hard disc so that they can be read offline and you can also give them to friends on a floppy disc. Who know, you may even be able to sell discs containing web pages :-) * flexible set of command line switches that allow you to restrict the type of pages that it downloads * ability to specify files for the lists of URLs that it keeps so that any search for pages can be stopped and restarted at a later date. i.e. you could run GetURL for 2 hours a day whilst you are online and gradually download everything in the entire universe and it won't repeat itself. * includes the ability to download itself when there are new versions. * will use a proxy if you have access to one, in order to both speed up access to pages and also to reduce network load. * will download binary files (*.gif, *.lha) as easily as text and html files. * documentation is in the top of the script file. NEW FEATURES SPECIAL REQUIREMENTS * Until somebody writes a TCP: device for the AS225 TCP/IP protocol stack unfortunately this script will require AmiTCP. * requires the TCP: device be mounted * either restraint, or an extremely large hard disc - your choice :-) - This script is no use at all unless you have AmiTCP set up and running. If you don't know what this means then please ask me (burton@cs.latrobe.edu.au). HOST NAME DIRECTORY This script is available via Anonymous FTP to AmiNet wuarchive.wustl.edu (128.252.135.4) /pub/aminet/comm/tcp/GetURL-1.0.rexx and all of it's mirrors. Please check the closest mirror FIRST. HTTP from my university account http://www.cs.latrobe.edu.au/~burton/Public/GetURL.rexx (this URL will always point to the newest version) FILE NAMES GetURL-1.0.rexx 41770 Bytes GetURL-1.0.readme 3874 Bytes PRICE Absolutely free to humans DISTRIBUTABILITY Public domain. But so that a hundred different versions of this don't appear, please send corrections, new features, bug fixes etc. to me and I will coordinate. End of File -- James Burton | EMail: JamesB@werple.mira.net.au | Latrobe University WWW : http://www.cs.latrobe.edu.au/~burton/ | Melbourne, Australia