 |
Using MS Access to Evaluate Your Web Site Logs
Scrolling through the gobbledegook of a typical Web site log file, the average manufacturer may think they need expensive analysis software. Fortunately, an inexpensive alternative exists. Just convert your Apache (UNIX) Web site logs (the most common format on the Web) into a text file and import the file into MS Access where it can be manipulated to generate a range of reports.
To start, download any small Web site log file (usually by FTP) from your ISP to a directory on your computer. Open the file with your text editor. Use "cut and paste" and copy the first 10 to 20 lines of data to the clipboard and save the new file as weblog.txt (or similar name). Compare your data with the example in link (1) (the example shows only one line/row of data). Next, use the "search and replace" function to search --[ and replace with no characters; then search 2003: and replace with 2003 followed by a space as shown in the example in links (2) and (3).
-
Components of a basic Apache Web log (pop-up window)
-
Modifying logs with search and replace (pop-up window)
-
Typical log data after search and replace (pop-up window)
Next, create a new MS Access database.
- Start MS Access
- Click New (toolbar left), select "Database" - click OK
- Name your new .mdb database and click Create
- Select "Get External Data" - "Import" from the "File" menu
- Click on your weblog.txt text file in the "Import" window
- Select "Delimited - Characters . . ." - click Next
- Select "Space" for delimiter and Text Qualifier "{none}" - click Next
- Select "New Table" - click Next
- Click on Advanced and check fields in "Skip" boxes numbered 4, 5, 7, 11 to 17
- Click on Save As . . . and save specification as "ImportLogFile" - click OK
- Click OK - click Next
- Select "Let Access Add Primary Key" - click Next
- Name your table and click Finish
Next, open your MS Access table in "Design View" and rename the fields (ignore the first ID field created automatically by MS Access) HOST, DATE, TIME, PAGE, CODE, SIZE, REFERRER.
Next, check your PAGE field data and note you have files with numerous extensions. Graphic files are usually ignored in Web log analysis - only .htm or home directory files / are important. (If you are running scripts, you may wish to include .cgi, .pl, etc., files.)
- Open your table in "Design View," click on PAGE in the list of Field Names
- Find "Validation Rules" under Field Properties (left side, toward bottom)
- Enter Like "*.htm" Or Like "/" (every character, including quotes)
- (If using scripts, add Or Like "*.cgi*" Or Like "*.pl*" etc.)
Next, create a macro to import your Web site log data on a regular basis.
- Click on Macros in the "Objects" window
- Click on New
- Select "Transfer Text" from "Action" drop-down window
- Complete the following in the fields in the lower input box: (MS Access supplies help info on the right)
- Click on Specification Name and select "ImportLogFile" from drop-down window
- In "Table Name" enter your table name
- In "File Name" enter the path to the directory for weblog.txt
- In "Has Field Names" enter "No" (no quotes)
- Close the Macro window and name the macro in the box displayed
Next, download your first large Web site log for analysis, save as weblog.txt, search and replace the two character strings (as explained above) and save. To import the file, open your MS Access database and run the macro you created. You now have a table in your database containing the most relevant log fields for analysis and only HTML files (no graphics). Delete the first few logs you used to set up the import criteria.
Finally, you can now apply queries and run reports on your data, using MS Access "Wizards" or applying expressions (data filters) directly in the "Criteria" grid of a "Select Query" in "Design View." For example, apply a specific day expression to the DATE field and a Like "*google*" expression to the REFERRER field to retrieve the number of hits from Google on that day. The usability of this method of analysis depends on your skill at writing queries and knowing which data you wish to isolate. The MS Access manual sections on generating different types of queries will help.
Addendum When importing data, MS Access may report errors and produce an "Error" table. The number of errors (caused by corrupt data) is usually very small. Open the Error table - if it is small, delete and ignore; if it is large, you probably made an error in the set-up for importing external data.
|