Site Index 1.01

The script will spider website and extract all anchor links. Optionally, it can extract titles and plain text for all webpages.

The script will spider website and extract all anchor links and webpage titles. All extracted URLs are divided into 4 categories: internal and external webpages, internal and internal downloadable files. If you store session ids or some other random values in the query string the script will optionally remove them from query string. You can use this resource as a Service to index your website.