Developing a Dark Web collection and infrastructure for computational and social sciences

Yulei Zhang, Shuo Zeng, Chun Neng Huang, Li Fan, Ximing Yu, Yan Dang, Catherine A. Larson, Dorothy Denning, Nancy Roberts, Hsinchun Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Citations (Scopus)

Abstract

In recent years, there have been numerous studies from a variety of perspectives analyzing the Internet presence of hate and extremist groups. Yet the websites and forums of extremist and terrorist groups have long remained an underutilized resource for terrorism researchers due to their ephemeral nature and access and analysis problems. The purpose of the Dark Web archive is to provide a research infrastructure for use by social scientists, computer and information scientists, policy and security analysts, and others studying a wide range of social and organizational phenomena and computational problems. The Dark Web Forum Portal provides web enabled access to critical international jihadist and other extremist web forums. The focus of this paper is on the significant extensions to previous work including: increasing the scope of data collection, adding an incremental spidering component for regular data updates; enhancing the searching and browsing functions; enhancing multilingual machine-translation for Arabic, French, German and Russian; and advanced Social Network Analysis. A case study on identifying active participants is shown at the end.

Original languageEnglish (US)
Title of host publicationISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security
Pages59-64
Number of pages6
DOIs
StatePublished - 2010
Externally publishedYes
Event2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security, ISI 2010 - Vancouver, BC, Canada
Duration: May 23 2010May 26 2010

Other

Other2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security, ISI 2010
CountryCanada
CityVancouver, BC
Period5/23/105/26/10

Fingerprint

Terrorism
Social sciences
Electric network analysis
World Wide Web
Websites
Internet

Keywords

  • Dark Web archive
  • Incremental forum spidering
  • Multilingual translation
  • Social Network visualization

ASJC Scopus subject areas

  • Artificial Intelligence
  • Information Systems
  • Safety, Risk, Reliability and Quality

Cite this

Zhang, Y., Zeng, S., Huang, C. N., Fan, L., Yu, X., Dang, Y., ... Chen, H. (2010). Developing a Dark Web collection and infrastructure for computational and social sciences. In ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security (pp. 59-64). [5484774] https://doi.org/10.1109/ISI.2010.5484774

Developing a Dark Web collection and infrastructure for computational and social sciences. / Zhang, Yulei; Zeng, Shuo; Huang, Chun Neng; Fan, Li; Yu, Ximing; Dang, Yan; Larson, Catherine A.; Denning, Dorothy; Roberts, Nancy; Chen, Hsinchun.

ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security. 2010. p. 59-64 5484774.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhang, Y, Zeng, S, Huang, CN, Fan, L, Yu, X, Dang, Y, Larson, CA, Denning, D, Roberts, N & Chen, H 2010, Developing a Dark Web collection and infrastructure for computational and social sciences. in ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security., 5484774, pp. 59-64, 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security, ISI 2010, Vancouver, BC, Canada, 5/23/10. https://doi.org/10.1109/ISI.2010.5484774
Zhang Y, Zeng S, Huang CN, Fan L, Yu X, Dang Y et al. Developing a Dark Web collection and infrastructure for computational and social sciences. In ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security. 2010. p. 59-64. 5484774 https://doi.org/10.1109/ISI.2010.5484774
Zhang, Yulei ; Zeng, Shuo ; Huang, Chun Neng ; Fan, Li ; Yu, Ximing ; Dang, Yan ; Larson, Catherine A. ; Denning, Dorothy ; Roberts, Nancy ; Chen, Hsinchun. / Developing a Dark Web collection and infrastructure for computational and social sciences. ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security. 2010. pp. 59-64
@inproceedings{768044f9b01042f7be5746b61a1674e3,
title = "Developing a Dark Web collection and infrastructure for computational and social sciences",
abstract = "In recent years, there have been numerous studies from a variety of perspectives analyzing the Internet presence of hate and extremist groups. Yet the websites and forums of extremist and terrorist groups have long remained an underutilized resource for terrorism researchers due to their ephemeral nature and access and analysis problems. The purpose of the Dark Web archive is to provide a research infrastructure for use by social scientists, computer and information scientists, policy and security analysts, and others studying a wide range of social and organizational phenomena and computational problems. The Dark Web Forum Portal provides web enabled access to critical international jihadist and other extremist web forums. The focus of this paper is on the significant extensions to previous work including: increasing the scope of data collection, adding an incremental spidering component for regular data updates; enhancing the searching and browsing functions; enhancing multilingual machine-translation for Arabic, French, German and Russian; and advanced Social Network Analysis. A case study on identifying active participants is shown at the end.",
keywords = "Dark Web archive, Incremental forum spidering, Multilingual translation, Social Network visualization",
author = "Yulei Zhang and Shuo Zeng and Huang, {Chun Neng} and Li Fan and Ximing Yu and Yan Dang and Larson, {Catherine A.} and Dorothy Denning and Nancy Roberts and Hsinchun Chen",
year = "2010",
doi = "10.1109/ISI.2010.5484774",
language = "English (US)",
isbn = "9781424464609",
pages = "59--64",
booktitle = "ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security",

}

TY - GEN

T1 - Developing a Dark Web collection and infrastructure for computational and social sciences

AU - Zhang, Yulei

AU - Zeng, Shuo

AU - Huang, Chun Neng

AU - Fan, Li

AU - Yu, Ximing

AU - Dang, Yan

AU - Larson, Catherine A.

AU - Denning, Dorothy

AU - Roberts, Nancy

AU - Chen, Hsinchun

PY - 2010

Y1 - 2010

N2 - In recent years, there have been numerous studies from a variety of perspectives analyzing the Internet presence of hate and extremist groups. Yet the websites and forums of extremist and terrorist groups have long remained an underutilized resource for terrorism researchers due to their ephemeral nature and access and analysis problems. The purpose of the Dark Web archive is to provide a research infrastructure for use by social scientists, computer and information scientists, policy and security analysts, and others studying a wide range of social and organizational phenomena and computational problems. The Dark Web Forum Portal provides web enabled access to critical international jihadist and other extremist web forums. The focus of this paper is on the significant extensions to previous work including: increasing the scope of data collection, adding an incremental spidering component for regular data updates; enhancing the searching and browsing functions; enhancing multilingual machine-translation for Arabic, French, German and Russian; and advanced Social Network Analysis. A case study on identifying active participants is shown at the end.

AB - In recent years, there have been numerous studies from a variety of perspectives analyzing the Internet presence of hate and extremist groups. Yet the websites and forums of extremist and terrorist groups have long remained an underutilized resource for terrorism researchers due to their ephemeral nature and access and analysis problems. The purpose of the Dark Web archive is to provide a research infrastructure for use by social scientists, computer and information scientists, policy and security analysts, and others studying a wide range of social and organizational phenomena and computational problems. The Dark Web Forum Portal provides web enabled access to critical international jihadist and other extremist web forums. The focus of this paper is on the significant extensions to previous work including: increasing the scope of data collection, adding an incremental spidering component for regular data updates; enhancing the searching and browsing functions; enhancing multilingual machine-translation for Arabic, French, German and Russian; and advanced Social Network Analysis. A case study on identifying active participants is shown at the end.

KW - Dark Web archive

KW - Incremental forum spidering

KW - Multilingual translation

KW - Social Network visualization

UR - http://www.scopus.com/inward/record.url?scp=77954799296&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77954799296&partnerID=8YFLogxK

U2 - 10.1109/ISI.2010.5484774

DO - 10.1109/ISI.2010.5484774

M3 - Conference contribution

SN - 9781424464609

SP - 59

EP - 64

BT - ISI 2010 - 2010 IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security

ER -