System: Searchable Directory of User Agents: Link Checking Utilities
The following is a directory of user agents, including their source and general purpose as far as we can determine. Most entries link to an "official" site containing more detailed information. You can also paste a UA from your logs into the form below, hit [Go!] and see a list the relevant agents.
We currently have 946 distinct user agents in our database representing everything from search engines to software components and spambots. These have been collected from our log files over a number of years and researched manually.
Search for User Agent
To use this form just copy and paste an entire User-Agent string from your server log file into the input box and then submit the form. The search is case-sensitive so "nokia" will not match "Nokia".
Most user agent strings now contain a number of separate components so the search will return a list of everything that has a match in the database.
Link Checking Utilities
This is software that conducts remote or local link checking.
- 1) AJLinkCheck - www.ajconsulting.com/services.html
- 2) Astra
- National Archives of Australia ?!?
- 3) AWeber - www.aweber.com/faq/questions/526/
- AWeber Communications Inc. Link Validator
- 4) Big Brother - pauillac.inria.fr/~fpottier/bb.html.en
- Big Brother is a management tool for hypertext (HTML) documents.
- 5) BLT - www.braxtech.com/blt/
- BLT (Braxton's Link Tester) is a freeware link checking program for Mac OS X
- 6) Checkbot - degraaff.org/checkbot/
- Checkbot is a tool to verify links on a set of HTML pages. Checkbot can check a single document, or a set of documents on one or more servers. Checkbot creates a report which summarizes all links which caused some kind of warning or error.
- 7) CheckURL
- 8) CheckWeb - p.duby.free.fr/chkweb.htm
- Ce logiciel permet d'analyser les pages d'un site web, et de repérer d'éventuels liens qui ne fonctionnent pas.
- 9) CyberSpyder - www.cyberspyder.com/cslnkts1.html
- CyberSpyder Link Test is a Web site management program to be used for verifying that the URLs on a site are not broken and for analyzing site content.
- 10) FARK - www.fark.com/
- 11) FavOrg
- 12) Flinch - www.remote.org/jochen/software/flinch/
- Flinch is a powerful and flexible web link checker that will make your life as a web designer or web server admin easier.
- 13) HHG Robot - www.hirners.com/robot.php
- HHG Robot is a web robot which checks the listed hotels' home pages and email addresses continously.
- 14) ht://check - htcheck.sourceforge.net/
- ht://Check is more than a link checker. It's a console application written for GNU/Linux systems in C++ and derived from the best search engine available on the Internet for free: ht://Dig.
- 15) hverify - www.stonehenge.com/merlyn/WebTechniques/col07.html
- hierarchical verify
- 16) IDS LinkChecker - www.csa.com/csa/ids/ids-main.shtml
- 17) InfoLink - www.biggbyte.com/infolink/
- 18) JRTS Check Favorites - www.jrtwine.com/Products/CheckFavs/
- Check Favorites is a full-featured solution for maintaining all of the Internet-based links in your Favorites list (bookmarks
- 19) JRTwine - www.jrtwine.com/Products/CheckFavs/
- Check Favorites is a full-featured solution for maintaining all of the Internet-based links in your Favorites list (bookmarks).
- 20) Larry's Link Checker
- 21) Linbot
- 22) Linkbot - www.watchfire.com/products/
- 23) LinkBot - www.watchfire.com/products/desktop/webqa/
- Watchfire WebQA, the next generation of Linkbot, is a website quality testing tool designed for small, medium, and departmental websites.
- 24) Link Checker - www.dotsw.com/linkchecker.html
- Link Checker can check all your web sites if they are online or stored locally on your hard drive (Mac OS)
- 25) LinkChecker - packages.debian.org/stable/web/linkchecker
- 26) Link Checker Pro - www.link-checker-pro.com/
- Link Checker Pro is an essential utility for anybody wishing to check internal and external links within their Internet site.
- 27) LinkLint - www.linklint.org/
- Linklint is an Open Source Perl program that checks links on web sites. It is licensed under the Gnu General Public License.
- 28) LinkScan - www.elsop.com/linkscan/
- LinkScan is an industrial-strength link checking and website management tool.
- 29) LinkSweeper - www.lss.com.au/
- Linksweeper will search through Netscape bookmarks and Internet Explorer favorites and find all of the bookmarks that no longer work.
- 30) Link Valet - www.htmlhelp.com/tools/valet/
- Link Valet is a WWW Link checker
- 31) MTCheckLinks - www.staggernation.com/mtplugins/CheckLinksReadMe.html
- CheckLinks Plugin for Movable Type
- 32) online link validator - www.dead-links.com/
- Put your website or weblog home page URL in the input box and a little spider will read the html code and check for the broken links.
- 33) Ontolica - www.navigosystems.com/english/
- 34) RDS URL Checker - www.dytech.com.au/projects/RDS.asp
- Information Resource Management Tool/Web Portal
- 35) REL Link Checker - www.relsoftware.com/rlc/
- REL Link Checker Lite is a free easy-to-use link checker tool to help you maintain the accuracy of links on your web site. The program will help you easily locate broken links and links containing syntactic errors.
- 36) W3C-checklink - validator.w3.org/checklink
- Enter the address (URL) of a document that you would like to check.
- 37) Web Link Validator - www.relsoftware.com/wlv/
- 38) WebTrends - www.webtrends.com/
- 39) wein - www.wein.cc/wein/linkchecker.php
- In regelmäßigen Abständen prüft unser Crawler, ob Ihre Website noch aktuell ist.
- 40) Xenu - home.snafu.de/tilman/xenulink.html
- Xenu's Link Sleuth (TM) checks Web sites for broken links. Link verification is done on "normal" links, images, frames, plug-ins, backgrounds, local image maps, style sheets, scripts and java applets. It displays a continously updated list of URLs which you can sort by different criteria. A report can be produced at any time.
- 41) Z-Add Link Checker - w3.z-add.co.uk/linkcheck/
For more information on the user agents listed you can click on the associated link. If you think any of the information here is incorrect or misleading please let us know using the Feedback link below.
Please be aware that we do not add user agents to the database on request, but rather wait to see them in our log files.
Browse User Agents by Category
- Browser Extensions (42)
- Browser extensions are programs that change or enhnace your web browser. Some of them also collect data by sending information on your browsing habits back to a central server.
- Content Management (13)
- Data Collection - Commercial (47)
- These are sites that collect information for commercial benefit. As far as we are aware no useful information or reports are provided to the public.
- Data Collection - Research (29)
- These agents are conducting research on the WWW. They may also offer commercial services.
- Devices (23)
- Mobile phones and other gadgets with browser technology.
- Download Managers (39)
- Programs that enable users to download or extract information from a website or web server.
- Indexing Tools (50)
- This is software that enables local or remote indexing of web pages and other content for the purposes of setting up a search engine.
- Link Checking Utilities (41)
- This is software that conducts remote or local link checking.
- Media Players (5)
- Applications for playing music, video and other media over the Internet.
- Other Resources (12)
- Links to online resources relating to robots and spiders.
- Proxies (7)
- If several clients request the same content, the proxy can deliver that content from its cache, rather than requesting it from the origin server each time.
- RSS/Atom Aggregators (43)
- These are browser extensions or search spiders that focus on indexing or aggregating RSS and Atom feeds.
- Search Engine Spiders (220)
- These agents conduct Internet-wide indexing for various search engines.
- Server Platforms (6)
- Server Software (31)
- Site Monitoring Services (15)
- Software Components (58)
- These are code libraries or application development packages that can be used to build Internet-related applications. How they are used depends on the developer.
- Spambots? (45)
- These are programs that are used predominately to harvest email addresses, find open guestbooks to post to, etc. They may also have legitimate uses.
- Unclassified (174)
- The following user agents have either not been identified or do not fit neatly into other categories. New agents appear every day that have limited lifespans. Most (but not all) legitimate user agents identify themselves with a URI or email address.
- Validation Tools (10)
- These are programs and sites that can be used to validate various aspects of your site: HTML, CSS, META tags, etc.
- Web Browsers (36)