Short: CERN/ASERVE HTTPD Logfile Analyzer by PirlAGA Uploader: tekne@cmns.mnegri.it (PirlAGA) Author: tekne@cmns.mnegri.it (PirlAGA) Type: comm/net Replace: HTTPirLOG_v04b.lha HTTPirLOG v0.5 beta =================== WHAT IS IT? ----------- This is a New Beta Version of the HTTPD Logfile Analyzer & Web Connections Statistics Maker, written by PirlAGA. This Program allows you to process the logfile of the World Wide Web servers programs (httpd = HTTP Daemons) and to get Stats. WHY? ---- I've decided to write this program because i was curious to know Stats about my Web site, but there are no Amiga Programs that do It. I've decided to release this beta version so you can help me to find bugs (if they are), to test it with different logfiles and to suggest me new functions to add. I think to release a MS-DOS version too, to get more users, but AmigaOS is The Best Operating System Ever. WHAT KIND OF DATA? ------------------ I've tested it with the Logfile of the CERN httpd, and the program seems to work correctly. Now the program works with AServe/AWS logfiles too !! I haven't found logfiles of other daemons, so i cannot test my program with this. If you have other types of HTTPD-logfile, like that of NCSA httpd, please contact me SOON, so i can learn about it and teach to my progam to recognize and process it. FUNCTIONS --------- At the actual state, this program allows you to know: - How many requests for each page, - How many connections from each domanin, - How many connections in each time band, - How many connections in each day, month or year. The program shows stats on the screen, and produces an output file by default (in ASCII, ANSI or HTML format) too, so you can read the stats later too. The program shows how many pages are listed in the logfiles, and how many of these are processed for creating stats, as the user told. The program can also calculates percentages for each information, and the time it taked to get all operations. Now you can SAVE the current config, so you will able to run the program fully congifured with just 1 parameter (that loads saved config). However, the program is fully configurable and customizable by parameters, so you can do exactly everything you want, and know anything you need. TO DO ----- - NCSA HTTPD log processing is quite done, i'm just waiting for a logfile to test. - I'll implement the function to specify a wildcard extension of pagenames to process. - Next thing to do, is to write a more friendly user interface. - I hope to be able to sort and tabulate output stats of pages, domains and time bands. - Catalogs support. - CGI-BIN support. - I'll add all functions you will suggest to me, if possible, so write to me soon! THE ARCHIVE ----------- The Distribution Package is one LHA archive with: HTTPirLOGv05 ---> The Main Executable HTTPirLOGwb ---> A Simple WB Interface (written in AmigaDOS) HTTPirLOGv05b.config ---> The Config File HTTPirLOG.doc ---> English ASCII Documentation HTTPirLOG.guide ---> English AmigaGuide Documentation HTTPirLOG_ITA.doc ---> Italian ASCII Documentation HTTPirLOG_ITA.guide ---> Italias AmigaGuide Documentation and lots of Original MagicWB Icons. I've tested the program on standard A1200 and A3000 with 2, 3 & 4Mb Cern-httpd logfiles, and it seems to work correctly. On A1200 there are some memory problems with all functions enabled. USAGE ----- The program is CLI only; if you run it without parameters, it shows you a detailed "Usage parameters" list. As FIRST parameter, if you enter "-h" or "-H" you will get the Online step-by-step Help; if you enter "-l" or "-L" you will LOAD saved config. The first parameter have to be the filename, the others may have random order. Parameters (except the PageBase field), are case unsensitive. You can also use the HTTPirLOGwb script, that provides to open window requesters and to give a littlebit more friendly interface; but it not allows you to enter pagebase. Usage: HTTPirLOG FILENAME/A File To Check NOELAPSED/S Don't Show Elapsed Time PROGRESS/S Show Progress JUSTHTML/S Stats for .HTML Pages Only PERCENT/S Calculate Percentages NOSEE/S Don't Show Stats on Screen NUMTOO/S Show Stats of Numerical Domains OUT=N/S Output File Type CLASS=N/S Domain Level DATE=N/S Date Depth BAND=NN/S Time Band Depth PAGE=XXX/S Page or Directory Name SAVE/S Save Config HTTPirLOG -H For The OnLine Help HTTPirLOG -L LOAD and use saved config FILENAME The only necessary parameter is the FILENAME of the logfile to process, who have to be the first entry after the programname. NOELAPSED This is a Flag. Tells to the Program to not calculate the elapsed time in the process. PROGRESS Flag to enable the progress showing. JUSTHTML Flag to tell to the program to process .HTML files only. PERCENT With this flag, you ask to program to report percentages count in stats. NOSEE This flag disables the video output. If both video and file outputs are disabled, the program don't starts. NUMTOO By default The Program Put all numerical domains in a single field called NUMERICAL. If you wanna show stats for each numerical domain too you have to enable this flag. OUT=N Specifies the Output File Type. 0= No Output File. 1= ASCII File (Default), 2= ANSII, 3= HTML !! CLASS=N this parameter needs a numerical value after the = . It represents the class of domains to process (of the caller machine), and maybe a number from 0 to 9. The 0 disables this function. The default value is 1. EG: class=1 processes .it domains class=2 processes .cmns.it domanis class=2 processes .mnegri.cmns.it domains etc. DATE=N this contains a numerical value that tells to the program what kind of dates to analyze. The range is 0 .. 3, 0 disables this function. Default is 1. EG: date=1 is for Day/Month/Year date=2 is for Month/Year date=3 id for Year only BAND=NN this flag is for time bands size. It maybe 0, 15, 30 or 60. 0 disable this function, default is 60. EG: band=60 for 1 hour bands band=30 for 30 mins bands band=15 for 15 mind bands PAGE=XXX this is the indicator of the most important function, the "pages to process" indicator. 0 disables the pagenames processing (but NOT other functions). Any other string will be accepted as a pagename or a base-directory name: the program will process only pages in specified directory, or just the specified page. (However, the program will process all page names starting with that string) All other Stats will refer to This pages. Default is all pages. For HomeUsers directory, you can entry both ~ or %7E qualifiers. EG: page=/~tekne/welcome.html - program will make stats for this file page=/ridere/ \__ program will process the files in page=ridere / ridere directory only. SAVE Adding this flag to other parameters, you can save the current config in a file. You can edit this file, so you can confgure the program via file, so it's easier. -L to Load saved config. EXAMPLES -------- HTTPirLOG httpd.log save Output file (httpd.log.PIR) in ASCII, TimeCount Enabled, time bands of 1 hour, date type is Day/Month/Year, domains type is .it , do not show progress, assuming all pages, display on video, don't calculate percentages, all numerical domains in a NUMERICAL field. Saves the DEFAULT config. HTTPirLOG -l Same as above. HTTPirLOG httpd2.log class=2 band=15 noelapsed justhtml progress page=/ridere/ nosee percent numtoo out=3 Output file (httpd2.log.html) in HTML, domains type is cmns.it time bands are 15 mins long, do not calculate time elapsed, date type is still Day/Month/Year. Show progress, process just .HTML pages from the /ridere/ directory, don't display on video, calculate percentages, each numerical domain have his own stat. Dont saves config. THE CONFIG FILE --------------- The config file is very easy to modify: just change the fields values. The file is structured to have rilevant data in "pair" lines, and garbage and comment text in others. The Comments starts with "#", but the important is their line positions. You CANNOT delete this lines !!! Because the program reads just PAIR lines and SKIPS the others. You also CANNOT delete first 6 comment lines. Here's a sample (comments are between * ) : ################################################################### ## HTTPirLOG v05b Config File -- (c)1995 PirlAGA ## ## Dont Change Lines Starting with "#" - Dont modify Lines Order ## ## For Value Parameters, next line contains the VALUE ## ## For Flags, next line contains Y or N ## ################################################################### # Next---> FILENAME * you cannot remove this lines * httpd.log * you can modify this line * # Next---> CALCULATE ELAPSED ? N * you can subsitute N with Y * # Next---> SHOW PROGRESS ? Y # Next---> JUST HTML ? Y # Next---> CALCULATE PERCENTAGES ? n # Next---> SHOW ON SCREEN ? y # Next---> PROCESS NUMERICAL DOMAINS ? n # Next---> OUTPUT TYPE 2 * here you can change value * # Next---> DOMAINS CLASS DEPTH 1 * according with program parameters * # Next---> DATE DEPH 2 * IMPORTANT: all values have to start at * # Next---> TIME BANDS DEPTH * the BEGINNING of the line * 30 # Next---> PAGE BASE /ridere/ BUGS ---- - For numerical domains it's not so nice to see just numbers, but it comes from the logfile and from the caller's machine. Maybe i'll implement an AmiTCP based solve-names, but i think it will take too many machine resources. - My bad english. - Sometimes, with small and dirty memory, something not goes on. REFERENCE --------- This is the Logfiles Reference: CERN ------------------------- Type: Site - - [Day/Month/Year:Hours(24):Min:Sec +GMT_Hours] "Method URL Protocol" #1 #2 Es.: a1200.tekne.it - - [28/Sep/1995:13:15:35 +0000] "GET /~tekne/ridere/home.html HTTP/1.0" 304 0 NCSA ? ------------------------- Type: Site [Weekday Month Day Hours(24):Min:Sec Year] Method URL Protocol Es.: a1200.tekne.it [Sat Sep 28 13:15:35 1995] GET /~tekne/ridere/home.html HTTP/1.0 AServe ------------------------- Type: [Weekday, Day Month Year Hours(24):Min:Sec GMT] Site - - "Method URL Protocol" #1 #2 <- Browser> Es.: [Sat, 28 Sep 1995 13:15:35 GMT] a1200.tekne.it - - "GET / HTTP/1.0" 301 398 <- Voyager/AmigaOS/0.29> The AServe Logfile contains more information than others, but at the actual state it's not so nice to decode. LEGAL & AUTHOR -------------- This program is provided "as is", you use it at your own risk. This Program is EveryThingYouWant-Ware, that means you can send me Every Thing You Want to get Registered In Future Versions. To refer bugs and suggestions, you can contact me via e.Mail: tekne@cmns.mnegri.it alessandro@mca.shiny.it i hope to hear from you soon! Look'up my Web at http://www.cmns.mnegri.it/~tekne/ See ya, aMiGa RulEz! .oO PirlAGA Oo.