Developing a Dark Web collection and infrastructure for computational and social sciences

Yulei Zhang, Shuo Zeng, Chun Neng Huang, Li Fan, Ximing Yu, Yan Dang, Catherine A. Larson, Dorothy Denning, Nancy Roberts, Hsinchun Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

In recent years, there have been numerous studies from a variety of perspectives analyzing the Internet presence of hate and extremist groups. Yet the websites and forums of extremist and terrorist groups have long remained an underutilized resource for terrorism researchers due to their ephemeral nature and access and analysis problems. The purpose of the Dark Web archive is to provide a research infrastructure for use by social scientists, computer and information scientists, policy and security analysts, and others studying a wide range of social and organizational phenomena and computational problems. The Dark Web Forum Portal provides web enabled access to critical international jihadist and other extremist web forums. The focus of this paper is on the significant extensions to previous work including: increasing the scope of data collection, adding an incremental spidering component for regular data updates; enhancing the searching and browsing functions; enhancing multilingual machine-translation for Arabic, French, German and Russian; and advanced Social Network Analysis. A case study on identifying active participants is shown at the end.

Original languageEnglish (US)
Title of host publicationISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security
Pages59-64
Number of pages6
DOIs
StatePublished - 2010
Externally publishedYes
Event2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security, ISI 2010 - Vancouver, BC, Canada
Duration: May 23 2010May 26 2010

Other

Other2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security, ISI 2010
CountryCanada
CityVancouver, BC
Period5/23/105/26/10

    Fingerprint

Keywords

  • Dark Web archive
  • Incremental forum spidering
  • Multilingual translation
  • Social Network visualization

ASJC Scopus subject areas

  • Artificial Intelligence
  • Information Systems
  • Safety, Risk, Reliability and Quality

Cite this

Zhang, Y., Zeng, S., Huang, C. N., Fan, L., Yu, X., Dang, Y., Larson, C. A., Denning, D., Roberts, N., & Chen, H. (2010). Developing a Dark Web collection and infrastructure for computational and social sciences. In ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security (pp. 59-64). [5484774] https://doi.org/10.1109/ISI.2010.5484774