Posted by Admin: System Admin
As cyberattacks become increasingly prevalent globally, there is a need to identify trends in these cyberattacks and take suitable countermeasures quickly. The darknet, an unused IP address space, is relatively conducive to observing and analyzing indiscriminate cyberattacks because of the absence of legitimate communication. Indiscriminate scanning activities by malware to spread their infections often show similar spatiotemporal patterns, and such trends are also observed on the darknet. To address the problem of early detection of malware activities, we focus on anomalous synchronization of spatiotemporal patterns observed in darknet traffic data. Our previous studies proposed algorithms that automatically estimate and detect anomalous spatiotemporal patterns of darknet traffic in real time by employing three independent machine learning methods. In this study, we integrated the previously proposed methods into a single framework, which we refer to as Dark-TRACER, and conducted quantitative experiments to evaluate its ability to detect these malware activities. We used darknet traffic data from October 2018 to October 2020 observed in our large-scale darknet sensors (up to /17 subnet scales). The results demonstrate that the weaknesses of the methods complement each other, and the proposed framework achieves an overall 100% recall rate. In addition, Dark-TRACER detects the average of malware activities 153.6 days earlier than when those malware activities are revealed to the public by reputable third-party security research organizations. Finally, we evaluated the cost of human analysis to implement the proposed system and demonstrated that two analysts can perform the daily operations necessary to operate the framework in approximately 7.3 h. Machine learning is an important component of the growing field of data science. Through the use of statistical methods, different type of algorithms is trained to make classifications or predictions, and to uncover key insights in this project. These insights subsequently drive decision making within applications and businesses, ideally impacting key growth metrics. Machine learning algorithms build a model based on this project data, known as training data, in order to make predictions or decisions without being explicitly programmed to do so. Machine learning algorithms are used in a wide variety of datasets, where it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks.
Dainotti et al. contributed to a census-like analysis of how the IP address space is used by developing malware and evaluating methods to remove spoofed traffic from darknets and live networks [49]. Durumeric et al. analyzed a large-scale darknet to investigate Internet-wide scanning activities and identify patterns of extensive horizontal scanning operations [50]. Fachkha et al. devised an inference and characterization module to identify and analyze the probing activities of cyberphysical systems (CPS) by extracting various features from large amounts of darknet data and performing correlational analyses [51]. Jonker et al. introduced a framework to protect against DoS attacks based on various data sources, including darknet traffic data [52]. They found that one-third of all /24 networks on the Internet had suffered at least one DoS attack in the past two years. Shaikh et al. identified unsolicited IoT devices by collecting IP header information from darknet traffic data and classifying them using several machine learning algorithms [53]. Akiyoshi et al. proposed a method to detect emerging scanning activities and their scale by analyzing the correlation between traffic in honeypots and darknets [54]. Most of the measurement analysis studies using darknets have been applied to understand the general trend of malicious communications observed in darknets. Thus, for detailed analysis, many studies use not only darknet data but also trap-based monitoring systems such as honeypots. Disadvantages ? The system is not implemented a large-scale darknet observation system, the NICTER project and which aims to understand global trends in indiscriminate cyber attacks. ? An existing system attempted to detect potential malware activities by estimating the group of source hosts with high synchronization in their spatiotemporal patterns on a large-scale darknet.
We integrated our three prior methods (modules) into a single framework, Dark-TRACER. To the best of our knowledge, our approach is the first method that focuses on the synchronization of spatiotemporal patterns of the darknet traffic. Dark-TRACER can detect malware activities that show anomalous synchronization. This work is also the most advanced practical study that quantitatively evaluated the detection performance of malware activities and the feasibility of early detection. We found that Dark-TRACER complements the weaknesses of each module, and achieves a 100% recall rate. In addition, the results demonstrate that Dark- TRACER detects threats on average 153.6 days earlier than when the threats are revealed to the public. We also demonstrated that two analysts can conduct the necessary daily operations of the framework in approximately 7.3 h. Advantages 1) The proposed system can reduce the effect of benign noise communication in the darknet traffic and highlight the malicious communication. 2) In addition, malware activities that are difficult to trace by conventional manual operations, such as threats that are small-scale, orchestrated, or have no visible explicit spikes, can be captured before the malware infection becomes widespread by detecting anomalously synchronized spatial features. 3) Finally, if a malware activity is found to be synchronized with other malware activities at a time when the scale of infection is small (i.e., before it spreads in earnest), it can be detected at that early stage.