Article
Some (Non-)universal features of Web robot traffic
2018 52nd Annual Conference on Information Sciences and Systems (CISS)
(2018)
Abstract
Understanding the qualities of Web robot traffic is essential to build mechanisms for mitigating the impact of their traffic on Web systems. This paper presents an updated characterization of the navigational and session patterns of Web robot traffic across three Web servers in the United States, Europe, and the Middle East under 30 different features. The results indicate that some features may be fitted to the same heavy-tailed model across the Web servers, but the best fitting models for other features depend on the Web server. Due to some different tasks of Web robots and security policies set by website administrators, there are thus some features of Web robot traffic that cannot be universally modeled.
Keywords
- Web robot characterization,
- Web robot,
- maximum likelihood estimation,
- Vuong test,
- heavy-tailed distribution
Disciplines
Publication Date
Spring May 24, 2018
DOI
10.1109/CISS.2018.8362266
Citation Information
Mahdieh Zabihimayvan and Derek Doran. "Some (Non-)universal features of Web robot traffic" 2018 52nd Annual Conference on Information Sciences and Systems (CISS) Iss. IEEE (2018) Available at: http://works.bepress.com/mahdieh-zabihimayvan/3/