System: Searchable Directory of User Agents: Download Managers
The following is a directory of user agents, including their source and general purpose as far as we can determine. Most entries link to an "official" site containing more detailed information. You can also paste a UA from your logs into the form below, hit [Go!] and see a list the relevant agents.
We currently have 946 distinct user agents in our database representing everything from search engines to software components and spambots. These have been collected from our log files over a number of years and researched manually.
Search for User Agent
To use this form just copy and paste an entire User-Agent string from your server log file into the input box and then submit the form. The search is case-sensitive so "nokia" will not match "Nokia".
Most user agent strings now contain a number of separate components so the search will return a list of everything that has a match in the database.
Download Managers
Programs that enable users to download or extract information from a website or web server.
- 1) curl - curl.haxx.se/
- Curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE and LDAP. Curl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading, kerberos, HTTP form based upload, proxies, cookies, user+password authentication, file transfer resume, http proxy tunneling and a busload of other useful tricks.
- 2) Custo - www.netwu.com/
- Capable of reading HTML, CSS, JavaScript, and Shockwave Flash, Custo allows you to quickly retrieve information about the structure of a Web site.
- 3) DA - www.speedbit.com/
- Download Accelerator Plus
- 4) Download Master
- 5) Extreme Picture Finder - www.exisoftware.com/picture_finder/
- Download from free and password protected websites
- 6) FDM - www.freedownloadmanager.org/
- Free Download Manager is a free download accelerator and manager.
- 7) FlashGet - www.amazesoft.com/
- FlashGet can split downloaded files into sections, downloading each section simultaneously.
- 8) GetRight - www.getright.com/
- GetRight works with your web browser to help you download files from the Internet.
- 9) Gnus - www.gnus.org/about.html
- 10) HTTrack - www.httrack.com/
- HTTrack is a free (GPL, libre/open source) and easy-to-use offline browser utility.
- 11) iGetter - www.igetter.net/
- iGetter is a full featured download manager and accelerator.
- 12) JoBo - www.matuschek.net/software/jobo/
- JoBo is a simple program to download complete websites to your local computer
- 13) JOC Web Spider - www.jocsoft.com/jws/
- 14) Kontiki Client - www.kontiki.com/
- 15) LeechGet - www.leechget.net/
- LeechGet is a download manager that integrates into Internet Explorer, allowing you to start and manage your downloads.
- 16) NetAnts - www.netants.com/
- NetAnts allows you to easily perform batch downloads from the Internet, either totally unattended, or while you're surfing with your favorite web browser.
- 17) Offline Explorer - www.metaproducts.com/
- Using Offline Explorer Enterprise, you can download an unlimited number of your favorite Web, FTP, HTTPS, MMS and RTSP sites for later offline viewing, editing or browsing.
- 18) PICgrabber - www.movies-free.net/
- 19) puf - puf.sourceforge.net/
- puf is a download tool for UNIX-like systems. You may use it to download single files or to mirror entire servers.
- 20) RealDownload - www.real.com/download/features/
- RealDownload is the most efficient way to download files and applications over the Internet.
- 21) SiteCopy - www.sitecopy.dk/
- SiteCopy arkiverer din hjemmeside og sikrer virksomhedens digitale historie.
- 22) SiteSnagger - www.pcmag.com/article2/0,4149,20161,00.asp
- SiteSnagger lets you download the contents of a Web site to your PC's hard disk.
- 23) SiteSucker - www.sitesucker.us/
- SiteSucker can be used to make local copies of your web sites for easy maintenance. SiteSucker can "localize" the files it downloads, allowing you to browse a site off-line.
- 24) SmartDownload
- 25) Star Downloader - www.stardownloader.com/
- Star Downloader is a download accelerator that can increase download speeds by up to 400 percent. Unlike similar applications, it is completely free. It does not contain any banner ads or spy-ware.
- 26) Teleport Pro - www.tenmax.com/teleport/pro/
- 27) WebCopier - www.maximumsoft.com/
- Use our products to record websites and store them locally until you are ready to view them.
- 28) Web Downloader
- 29) WebDownloader for X
- 30) Webdup
- 31) WebMiner - www.tribolic.com/webminer/about.html
- WebMiner finds specific Internet content online and transfers the files to your local hard drive.
- 32) WebPix - www.netwu.com/webpix/
- WebPix is several tools in one. A fully automated, multithreaded webspider will retrieve all the images from any Web site. A full-featured image Viewer quickly generates a high quality display of your image.
- 33) WebReaper - www.webreaper.net/
- WebReaper is web crawler or spider, which can work its way through a website, downloading pages, pictures and objects that it finds
- 34) WebStripper - www.webstripper.net/
- Use WebStripper offline browser to quickly download web sites and pages onto your hard disk.
- 35) WebZIP - www.spidersoft.com/
- Save reference material, online books, e-zines, news articles and other Web content to your hard drive and browse them anywhere, anytime - fast!
- 36) Wget - www.gnu.org/software/wget/wget.html
- GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without Xsupport, etc.
- 37) WinHTTP Robot
- 38) Xaldon WebSpider - www.xaldon.de/
- Xaldon WebSpider 2 durchsucht automatisch gemäß den Vorgaben des Benutzers das World-Wide-Web und speichert die gesammelten Informationen lokal ab.
- 39) x-Tractor - www.asona.org/
- Website Extractor gives you complete control over including and excluding downloads from individual servers, folders, URLs, and files using keywords.
For more information on the user agents listed you can click on the associated link. If you think any of the information here is incorrect or misleading please let us know using the Feedback link below.
Please be aware that we do not add user agents to the database on request, but rather wait to see them in our log files.
Browse User Agents by Category
- Browser Extensions (42)
- Browser extensions are programs that change or enhnace your web browser. Some of them also collect data by sending information on your browsing habits back to a central server.
- Content Management (13)
- Data Collection - Commercial (47)
- These are sites that collect information for commercial benefit. As far as we are aware no useful information or reports are provided to the public.
- Data Collection - Research (29)
- These agents are conducting research on the WWW. They may also offer commercial services.
- Devices (23)
- Mobile phones and other gadgets with browser technology.
- Download Managers (39)
- Programs that enable users to download or extract information from a website or web server.
- Indexing Tools (50)
- This is software that enables local or remote indexing of web pages and other content for the purposes of setting up a search engine.
- Link Checking Utilities (41)
- This is software that conducts remote or local link checking.
- Media Players (5)
- Applications for playing music, video and other media over the Internet.
- Other Resources (12)
- Links to online resources relating to robots and spiders.
- Proxies (7)
- If several clients request the same content, the proxy can deliver that content from its cache, rather than requesting it from the origin server each time.
- RSS/Atom Aggregators (43)
- These are browser extensions or search spiders that focus on indexing or aggregating RSS and Atom feeds.
- Search Engine Spiders (220)
- These agents conduct Internet-wide indexing for various search engines.
- Server Platforms (6)
- Server Software (31)
- Site Monitoring Services (15)
- Software Components (58)
- These are code libraries or application development packages that can be used to build Internet-related applications. How they are used depends on the developer.
- Spambots? (45)
- These are programs that are used predominately to harvest email addresses, find open guestbooks to post to, etc. They may also have legitimate uses.
- Unclassified (174)
- The following user agents have either not been identified or do not fit neatly into other categories. New agents appear every day that have limited lifespans. Most (but not all) legitimate user agents identify themselves with a URI or email address.
- Validation Tools (10)
- These are programs and sites that can be used to validate various aspects of your site: HTML, CSS, META tags, etc.
- Web Browsers (36)