Scraping The Deep Web: A 3-Dimensional Framework For Cyber-Threat Intelligence

Adewale, Adewopo Victor; Gonen, Bilal

Cyberspace is one of the most complex systems ever built by humans. The utilization of cybertechnology resources are used ubiquitously by many, but sparsely understood by the majority of the users. In the past, cyberattacks were usually orchestrated in a random pattern of attack to lure unsuspecting targets. However, the cyber virtual environment is an ecosystem that provided a platform for an organized and sophisticated approach to launch an attack against a specific target group or organization by nefarious actors. In 2019, the average cost of cyber-attack in the US was about $1.6 million. This paper proposes a 3D framework to signal new threat alert before the actual occurrence of the threat on the surface web to alert cybersecurity experts and law enforcement agencies in preventive measures or means of mitigating the severity of damage caused by cyberattacks. The methodology combines information extracted from the deep web through a smart web crawler with socio-personal and technical indicators from twitter which is mapped with OTX (Open Threat Exchange). The OTX is an open-source cyber threat platform managed by security experts. The OTX endpoint security tool(OTX python SDK) will be used to identify a new type of cyber threats. The effectiveness of the framework will be tested using the machine learning algorithm precision-recall rate.

Creator

License

Subject