Adversarial Artificial Intelligence

AI systems deployed in healthcare, security, and finance can be manipulated by adversarial inputs — carefully crafted perturbations that cause confident wrong predictions. We develop defenses, detection systems, and robust architectures that remain reliable even when attacked.

Research Areas

Adversarial CAPTCHAs & Usable Security — Designing CAPTCHAs that are easy for humans but hard for AI using precise noise targeting. Published at AAMAS 2026.
Encrypted Network Traffic — Detecting anomalies and classifying malicious traffic in fully encrypted streams without decryption, using contrastive learning for zero-day attacks.
Android Malware Detection — ML-based classifiers hardened against evasion attacks, maintaining accuracy even when adversaries know the detection method.
Healthcare AI Robustness — Adversarial training and certified defenses for medical diagnosis and patient prediction models under attack.
Social Network Manipulation — Detecting coordinated inauthentic behavior, bot accounts, and disinformation using graph-based anomaly detection.

Technical Approaches

Contrastive & Self-Supervised Learning — SimCSE-based methods that build robust representations effective for zero-shot detection of novel attack patterns.
Adversarial Training & Certified Defenses — Training on adversarial examples with provable robustness guarantees via randomized smoothing for safety-critical applications.
Privacy-Preserving Detection — Federated learning and differential privacy techniques that protect sensitive data while maintaining detection performance.

Related Publications

2026

Uncovering Microservice Faults: A Temporal Graph Approach to Root Cause Analysis

2026

Udi Aharon, Amit Dvir, Ran Dubin, Revital Marbel, and Chen Hajaj

Proceedings of the IEEE International Conference on Communications. ICC 2026

Real-time processing demands for massive IoT sensor data necessitate reliance on distributed microservice systems within edge clusters. However, pinpointing the root cause of anomalies within these edge microservice clusters poses a critical challenge for intelligent IoT operation and maintenance. To address the issue, a spatio-temporal graph propagation model ST-GraphRCA is proposed for root cause analysis in IoT edge environments. Our approach begins by resolving the fundamental issue of time-series asynchrony across distributed multi-source metrics. A PCA-DTW hybrid feature extraction method is introduced with a dynamic alignment strategy to mitigate the effects of random network delays and data deformation without requiring prior synchronization. Subsequently, ST-GraphRCA constructs a stream-based forward propagation graph based on the flow conservation principle. By integrating dynamic edge weights with node-level input–output anomaly scores, ST-GraphRCA precisely infers fault propagation pathways and identifies potential root cause candidates through causal reasoning. Finally, a topology-constrained high-utility mining algorithm filters these candidates. Using a constraint matrix, the algorithm filters out unreachable service combinations to locate low-frequency and high-risk root causes. Experimental results indicate that ST-GraphRCA achieves an F1-Score of 0.89, outperforming existing methods. In resource-constrained edge scenarios, its average localization time is merely 238.8 ms, representing a six-fold improvement over key benchmarks. Thus, ST-GraphRCA not only provides an efficient anomaly fault tracing solution for large-scale IoT systems but also offers technical support for the intelligent operation and maintenance of distributed microservice systems.
Real-Time Network Security: Integrating ANN and Dynamic Graph-Based Clustering

2026

Zohar Simhon, Matan Weiss, Revital Marbel, Chen Hajaj, Amit Dvir, and Ran Dubin

Computer Networks

DOI
★ Cleaner Adversarial CAPTCHAs: Intelligent Targets and Precise Noise for Usable Security

2026

Meir Litman, and Chen Hajaj

Proceedings of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026)

Traditional CAPTCHAs are increasingly vulnerable to deep learning-based solvers that decode text and images with high accuracy. In this work, we propose methods to strengthen adversarial CAPTCHAs without compromising human usability. First, we introduce a Precise Gradient Method (PGM) that preserves gradient magnitude (rather than discarding it via a sign operator), producing adversarial perturbations with significantly lower perceptual noise. Second, we develop intelligent target class selection, using either dataset-level confusion structure (Class Relations Network) or image-specific softmax probabilities (Distance-Based Target), to steer adversarial perturbations more efficiently. Across multiple modern architectures (MobileNets, EfficientNets, ResNet, and Vision Transformer), our framework achieves faster convergence (fewer iterations), reduced visual distortion, and notably greater robustness under iterative adversarial retraining. Experiments show that our methods consistently reduce iteration counts and perceptual distortion while significantly increasing the difficulty for automated attacks. Our results offer a practical, scalable path toward the next generation of CAPTCHA systems and contribute new insights to the adversarial machine learning landscape focused on security and usability.

Abstract

2025

Cloudy with a Chance of Anomalies: Dynamic Graph Neural Network for Early Detection of Cloud Services’ User Anomalies

2025

Revital Marbel, Yanir Cohen, Ran Dubin, Amit Dvir, and Chen Hajaj

Proceedings of the 34th International Conference on Computer Communications and Networks

In today’s digital landscape, ensuring the security of cloud environments is critical for organizational resilience, growth, and operational efficiency. As cloud services become more prevalent, so do sophisticated attacks targeting cloud users, making early detection essential. This paper introduces a novel time-based embedding approach for Cloud Services Graph-based Anomaly Detection (CS-GAD) that leverages a Graph Neural Network (GNN) to detect anomalous user behavior. We propose a dynamic tripartite graph to model interactions among users, actions, and cloud services over time. Using behavioral patterns, our GNN generates user embeddings to enable early detection of anomalies. We evaluate this approach on a novel dataset simulating five real-world attacks: cryptojacking, billing abuse, lateral movement, monitor exploitation, and service targeting. The dataset comprises 107,116 Application Programming Interface (API) calls over 32 days, tracking 79 AWS services, with attacks embedded within legitimate cloud traffic. Our results demonstrate that the proposed method achieves a lower false positive rate and higher detection accuracy than a prevailing method, as evidenced by improved accuracy, precision, recall, and F1-score.

DOI
Optimized File Type Detection and One-Shot Reclassification Model

2025

Simona Lisker, Ayelet Botman, Chen Hajaj, Ran Dubin, and Amit Dvir

Proceedings of the IEEE International Conference on Communications

File type classification is critical in digital forensics, and file carving. However, the increasing diversity of file formats challenges accurate classification. Traditional methods rely on hand-crafted features or compact neural networks but face long training times, limited training data, and lower accuracy. This paper introduces three novel, content-based file-type classification approaches to address these challenges. These approaches improve accuracy and streamline the integration of new file types using pre-trained models, enhancing both speed and reliability. The first approach utilizes Natural Language Processing (NLP) with a transformer architecture, while the second combines statistical features with a pre-trained model via transfer learning. These methods achieved accuracy rates of 72.4 % and 69.2 %, respectively, surpassing state-of-the-art Convolutional Neural Network (CNN) models. The third approach employs one-shot learning, achieving 100 % accuracy in several scenarios, enabling efficient training with minimal data.

DOI
A New D-MAGIC: Dynamic Model for Cybersecurity Attack Detection Using GNNs into Clustering

2025

Zohar Simhon, Matan Weiss, Chen Hajaj, Revital Marbel, Ran Dubin, and Amit Dvir

Proceedings of the IEEE International Conference on Communications

The increasing sophistication and frequency of cyberattacks have made Network Intrusion Detection Systems (NIDS) a critical component of modern cybersecurity. This work presents D-MAGIC, a novel real-time NIDS that leverages zero-shot learning and graph-based dynamic clustering to detect known and unknown threats. Unlike traditional systems that rely on labeled datasets and predefined attack signatures, D-MAGIC operates unsupervised, identifying anomalies by detecting deviations from normal network behavior. By embedding the relationships between network flows into a graph structure and dynamically clustering similar patterns, D-MAGIC can detect coordinated attacks and emerging threats with minimal delay. Experimental results on the CIC-IDS-2017 and CSE-CIC-IDS-2018 datasets demonstrate that D-MAGIC achieves an improvement of up to 12 % based on the standard F1 score compared to state-of-the-art methods, while significantly reducing false positives and ensuring rapid, real-time detection with minimal detection latency.

DOI
PQClass: Classification of Post-Quantum Encryption Applications in Internet Traffic

2025

Angelos Marnerides, Chen Hajaj, Revital Marbel, Ran Dubin, and Amit Dvir

Proceedings of the IEEE International Conference on Communications

Post-quantum cryptography (PQC) is expected to revolutionize secure communications in next-generation digital ecosystems. Previous and ongoing activities demonstrate that different PQC algorithms significantly impact traffic latency, but they do not yet provide a scheme to assess the existence of the PQC algorithm or its identification when encrypted traffic is analyzed for traffic engineering purposes. Hence, this work is the first to propose a novel PQClass pipeline for classifying encrypted Internet traffic of recently NIST-approved PQC algorithms. Hence, it establishes solid grounds for enabling engineers to optimize their networks and, in parallel, for cybersecurity practitioners to familiarise themselves with PQC algorithmic properties for enhancing or devising security architectures in diverse setups. Our pipeline demonstrates impressive performance on real-world data, achieving 86% accuracy in detecting the presence of a PQC algorithm and 91% and 98% accuracy in identifying the browser and OS, respectively, based on PQC-based traffic.

DOI
Leveraging OSINT for Advanced Proactive Cybersecurity: Strategies and Solutions

2025

Zafrir Avrahami, Moti Zwilling, and Chen Hajaj

IEEE Access

The growing complexity of the digital environment has increased the need for proactive and intelligence-based approaches to cybersecurity. This study examines the role of open-source intelligence (OSINT) as a strategic tool in proactive cybersecurity operations. Drawing on a wide range of peer-reviewed literature and professional sources, it reviews definitions, operational processes, areas of application, benefits, and challenges associated with OSINT. The analysis highlights OSINT’s contribution to situational awareness, early threat detection, and cyber threat intelligence (CTI) capabilities. By using publicly accessible data from the internet and social platforms, organizations can strengthen their defensive posture against diverse cyber threats. The study outlines workflows for collecting and analyzing OSINT, with attention to its integration into organizational intelligence frameworks and cybersecurity strategies. Examples from both the public and private sectors demonstrate how OSINT supports decision-making, incident response, and preparedness for emerging threats. The review also considers OSINT’s main advantages, including cost effectiveness, accessibility, and real-time relevance, alongside its main challenges such as data reliability, legal concerns, and information overload. The findings suggest that, despite these limitations, OSINT has significant potential to enhance proactive cybersecurity measures, support compliance with standards, assist law enforcement, prevent terrorism, and contribute to business decision-making. The paper concludes with a call for further research on integration with advanced technologies, real-time data analysis, and effective intelligence collaboration.

DOI

2024

Few-Shot API Attack Detection: Overcoming Data Scarcity with GAN-Inspired Learning

2024

Udi Aharon, Revital Marbel, Ran Dubin, Amit Dvir, and Chen Hajaj

arXiv preprint arXiv:2405.11258

Web applications and APIs face constant threats from malicious actors seeking to exploit vulnerabilities for illicit gains. To defend against these threats, it is essential to have anomaly detection systems that can identify a variety of malicious behaviors. However, a significant challenge in this area is the limited availability of training data. Existing datasets often do not provide sufficient coverage of the diverse API structures, parameter formats, and usage patterns encountered in real-world scenarios. As a result, models trained on these datasets often struggle to generalize and may fail to detect less common or emerging attack vectors. To enhance detection accuracy and robustness, it is crucial to access larger and more representative datasets that capture the true variability of API traffic. To address this, we introduce a GAN-inspired learning framework that extends limited API traffic datasets through targeted, domain-aware synthesis. Drawing on techniques from Natural Language Processing (NLP), our approach leverages Transformer-based architectures, particularly RoBERTa, to enhance the contextual representation of API requests and generate realistic synthetic samples aligned with security-specific semantics. We evaluate our framework on two benchmark datasets, CSIC 2010 and ATRDF 2023, and compare it with a previous data augmentation technique to assess the importance of domain-specific synthesis. In addition, we apply our augmented data to various anomaly detection models to evaluate its impact on classification performance. Our method achieves up to a 4.94% increase in F1 score on CSIC 2010 and up to 21.10% on ATRDF 2023. The source codes of this work are available at this https URL.
Extending Limited Datasets with GAN-Like Self-Supervision for SMS Spam Detection

2024

Or Haim Anidjar, Revital Marbel, Ran Dubin, Amit Dvir, and Chen Hajaj

Computers & Security

DOI

2023

Breaking the Structure of MaMaDroid

2023

Harel Berger, Amit Dvir, Enrico Mariconti, and Chen Hajaj

Expert Systems with Applications

The rise in popularity of the Android platform has resulted in an explosion of malware threats targeting it. As both Android malware and the operating system itself constantly evolve, it is very challenging to design robust malware mitigation techniques that can operate for long periods of time without the need for modifications or costly re-training. In this paper, we present MaMaDroid, an Android malware detection system that relies on app behavior. MaMaDroid builds a behavioral model, in the form of a Markov chain, from the sequence of abstracted API calls performed by an app, and uses it to extract features and perform classification. By abstracting calls to their packages or families, MaMaDroid maintains resilience to API changes and keeps the feature set size manageable. We evaluate its accuracy on a dataset of 8.5K benign and 35.5K malicious apps collected over a period of six years, showing that it not only effectively detects malware (with up to 99% F-measure), but also that the model built by the system keeps its detection capabilities for long periods of time (on average, 86% and 75% F-measure, respectively, one and two years after training). Finally, we compare against DroidAPIMiner, a state-of-the-art system that relies on the frequency of API calls performed by apps, showing that MaMaDroid significantly outperforms it.

DOI
Detecting Parallel Covert Data Transmission Channels in Video Conferencing Using Machine Learning

2023

Ofir Joseph, Avshalom Elmalech, and Chen Hajaj

Electronics

Covert communication channels are a concept in which a policy-breaking method is used in order to covertly transmit data from inside an organization to an external or accessible point. VoIP and Video systems are exposed to such attacks on different layers, such as the underlying real-time transport protocol (RTP) which uses Transmission Control Protocol (TCP) or User Datagram Protocol (UDP) packet streams to punch a hole through Network address translation (NAT). This paper presents different innovative attack methods utilizing covert communication and RTP channels to spread malware or to create a data leak channel between different organizations. The demonstrated attacks are based on a UDP punch hole created using Skype peer-to-peer video conferencing communication. The different attack methods were successfully able to transmit a small text file in an undetectable manner by observing the communication channel, and without causing interruption to the audio/video channels or creating a noticeable disturbance to the quality. While these attacks are hard to detect by the eye, we show that applying classical Machine Learning algorithms to detect these covert channels on statistical features sampled from the communication channel is effective for one type of attack.

DOI

2022

MaMaDroid2.0–The Holes of Control Flow Graphs

2022

Harel Berger, Chen Hajaj, Enrico Mariconti, and Amit Dvir

arXiv preprint arXiv:2202.13922

Android malware is a continuously expanding threat to billions of mobile users around the globe. Detection systems are updated constantly to address these threats. However, a backlash takes the form of evasion attacks, in which an adversary changes malicious samples such that those samples will be misclassified as benign. This paper fully inspects a well-known Android malware detection system, MaMaDroid, which analyzes the control flow graph of the application. Changes to the portion of benign samples in the train set and models are considered to see their effect on the classifier. The changes in the ratio between benign and malicious samples have a clear effect on each one of the models, resulting in a decrease of more than 40% in their detection rate. Moreover, adopted ML models are implemented as well, including 5-NN, Decision Tree, and Adaboost. Exploration of the six models reveals a typical behavior in different cases, of tree-based models and distance-based models. Moreover, three novel attacks that manipulate the CFG and their detection rates are described for each one of the targeted models. The attacks decrease the detection rate of most of the models to 0%, with regards to different ratios of benign to malicious apps. As a result, a new version of MaMaDroid is engineered. This model fuses the CFG of the app and static analysis of features of the app. This improved model is proved to be robust against evasion attacks targeting both CFG-based models and static analysis models, achieving a detection rate of more than 90% against each one of the attacks.
Problem-Space Evasion Attacks in the Android OS: A Survey

2022

Harel Berger, Chen Hajaj, and Amit Dvir

arXiv preprint arXiv:2205.14576

Android is the most popular OS worldwide. Therefore, it is a target for various kinds of malware. As a countermeasure, the security community works day and night to develop appropriate Android malware detection systems, with ML-based or DL-based systems considered as some of the most common types. Against these detection systems, intelligent adversaries develop a wide set of evasion attacks, in which an attacker slightly modifies a malware sample to evade its target detection system. In this survey, we address problem-space evasion attacks in the Android OS, where attackers manipulate actual APKs, rather than their extracted feature vector. We aim to explore this kind of attacks, frequently overlooked by the research community due to a lack of knowledge of the Android domain, or due to focusing on general mathematical evasion attacks - i.e., feature-space evasion attacks. We discuss the different aspects of problem-space evasion attacks, using a new taxonomy, which focuses on key ingredients of each problem-space attack, such as the attacker model, the attacker’s mode of operation, and the functional assessment of post-attack applications.
Do You Think You Can Hold Me? The Real Challenge of Problem-Space Evasion Attacks

2022

Harel Berger, Amit Dvir, Chen Hajaj, and Rony Ronen

arXiv preprint arXiv:2205.04293

Given the continually rising frequency of cyberattacks, the adoption of artificial intelligence methods, particularly Machine Learning (ML), Deep Learning (DL), and Reinforcement Learning (RL), has become essential in the realm of cybersecurity. These techniques have proven to be effective in detecting and mitigating cyberattacks, which can cause significant harm to individuals, organizations, and even countries. Machine learning algorithms use statistical methods to identify patterns and anomalies in large datasets, enabling security analysts to detect previously unknown threats. Deep learning, a subfield of ML, has shown great potential in improving the accuracy and efficiency of cybersecurity systems, particularly in image and speech recognition. On the other hand, RL is again a subfield of machine learning that trains algorithms to learn through trial and error, making it particularly effective in dynamic environments. We also evaluated the usage of ChatGPT-like AI tools in cyber-related problem domains on both sides, positive and negative. This article provides an overview of how ML, DL, and RL are applied in cybersecurity, including their usage in malware detection, intrusion detection, vulnerability assessment, and other areas. The state-of-the-art studies using ML, DL, and RL models are evaluated in each section based on the main idea, techniques, and important findings. It also discusses these techniques’ challenges and limitations, including data quality, interpretability, and adversarial attacks. Overall, the use of ML, DL, and RL in cybersecurity holds great promise for improving the effectiveness of security systems and enhancing our ability to protect against cyberattacks. However, it is essential to continue developing and refining these techniques to address the ever-evolving nature of cyber threats. Besides, some promising solutions that rely on machine learning, deep learning, and reinforcement learning are susceptible to adversarial attacks, underscoring the importance of factoring in this vulnerability when devising countermeasures against sophisticated cyber threats. We also concluded that ChatGPT can be a valuable tool for cybersecurity, but it should be noted that ChatGPT-like tools can also be manipulated to threaten the integrity, confidentiality, and availability of data.
Less Is More: Robust and Novel Features for Malicious Domain Detection

2022

Chen Hajaj, Nitay Hason, and Amit Dvir

Electronics

Malicious domains are increasingly common and pose a severe cybersecurity threat. Specifically, many types of current cyber attacks use URLs for attack communications (e.g., CandC, phishing, and spear-phishing). Despite the continuous progress in detecting cyber attacks, there are still critical weak spots in the structure of defense mechanisms. Since machine learning has become one of the most prominent malware detection methods, a robust feature selection mechanism is proposed that results in malicious domain detection models that are resistant to evasion attacks. This mechanism exhibits a high performance based on empirical data. This paper makes two main contributions: First, it provides an analysis of robust feature selection based on widely used features in the literature. Note that even though the feature set dimensional space is cut by half, the performance of the classifier is still improved (an increase in the model’s F1-score from 92.92% to 95.81%). Second, it introduces novel features that are robust with regard to the adversary’s manipulation. Based on an extensive evaluation of the different feature sets and commonly used classification models, this paper shows that models based on robust features are resistant to malicious perturbations and concurrently are helpful in classifying non-manipulated data.

DOI
MalDIST: From Encrypted Traffic Classification to Malware Traffic Detection and Classification

2022

Ofek Bader, Adi Lichy, Chen Hajaj, Ran Dubin, and Amit Dvir

2022 IEEE 19th annual consumer communications & networking conference (CCNC)

The world of malware is shifting towards using encrypted traffic. While encryption improves the privacy of users, it brings challenges in the fields of QoS, QoE, and cybersecurity. Recent state-of-the-art Deep-Learning architectures for encrypted traffic classifications demonstrated superb results in tasks of traffic categorization over encrypted traffic. In this paper, we leverage the feasibility to use such architectures for the tasks of malware detection and classification to gain insights into how well these architectures perform in the domain of malware traffic. Specifically, we present a Deep-Learning model for malware traffic detection and classification (MalDIST), which outperforms both classical ML and DL malware traffic classification models both in terms of detection and classification.

DOI

2021

Crystal Ball: From Innovative Attacks to Attack Effectiveness Classifier

2021

Harel Berger, Chen Hajaj, Enrico Mariconti, and Amit Dvir

IEEE Access

Android OS is one of the most popular operating systems worldwide, making it a desirable target for malware attacks. Some of the latest and most important defensive systems are based on machine learning (ML) and cybercriminals continuously search for ways to overcome the barriers posed by these systems. Thus, the focus of this work is on evasion attacks in the attempt to show the weaknesses of state of the art research and how more resilient systems can be built. Evasion attacks consist of manipulating either the actual malicious application (problem-based) or its extracted feature vector (feature-based), to avoid being detected by ML systems. This study presents a set of innovative problem-based evasion attacks against well-known Android malware detection systems, which decrease their detection rate by up to 97%. Moreover, an analysis of the effectiveness of these attacks against VirusTotal (VT) scanners was conducted, empirically showing their efficiency against well-known scanners (e.g., McAfee and Comodo) as well. The VT system proved to be a great candidate for the attacks, as in 98% of the apps, less scanners detected the manipulated apps than the original malicious apps. As not all the attacks are effective in the same manner against the VT scanners, the attack efficiency classifiers are advised. Each classifier predicts the applicability of one of the attacks. The set of classifiers creates an ensemble, which shows high success rates, allowing the attacker to decide which attack is best to use for each malicious app and defense system.

DOI
Robust Coordination in Adversarial Social Networks: From Human Behavior to Agent-Based Modeling

2021

Chen Hajaj, Zlatko Joveski, Sixie Yu, and Yevgeniy Vorobeychik

Network Science

AbstractDecentralized coordination is one of the fundamental challenges for societies and organizations. While extensively explored from a variety of perspectives, one issue that has received limited attention is human coordination in the presence of adversarial agents. We study this problem by situating human subjects as nodes on a network, and endowing each with a role, either regular (with the goal of achieving consensus among all regular players), or adversarial (aiming to prevent consensus among regular players). We show that adversarial nodes are, indeed, quite successful in preventing consensus. However, we demonstrate that having the ability to communicate among network neighbors can considerably improve coordination success, as well as resilience to adversarial nodes. Our analysis of communication suggests that adversarial nodes attempt to exploit this capability for their ends, but do so in a somewhat limited way, perhaps to prevent regular nodes from recognizing their intent. In addition, we show that the presence of trusted nodes generally has limited value, but does help when many adversarial nodes are present, and players can communicate. Finally, we use experimental data to develop computational models of human behavior and explore additional parametric variations: features of network topologies and densities, and placement, all using the resulting data-driven agent-based (DDAB) model.

DOI

2020

Encrypted Video Traffic Clustering Demystified

2020

Amit Dvir, Angelos K Marnerides, Ran Dubin, Nehor Golan, and Chen Hajaj

Computers & Security

Cyber threat intelligence officers and forensics investigators often require the behavioural profiling of groups based on their online video viewing activity. It has been demonstrated that encrypted video traffic can be classified under the assumption of using a known subset of video titles based on temporal video viewing trends of particular groups. Nonetheless, composing such a subset is extremely challenging in real situations. Therefore, this work exhibits a novel profiling scheme for encrypted video traffic with no a priori assumption of a known subset of titles. It introduces a seminal synergy of Natural Language Processing (NLP) and Deep Encoder-based feature embedding algorithms with refined clustering schemes from off-the-shelf solutions, in order to group viewing profiles with unknown video streams. This study is the first to highlight the most computationally effective, accurate combinations of feature embedding and clustering using real datasets, thereby, paving the way to future forensics tools for automated behavioural profiling of malicious actors.

DOI
Evasion Is Not Enough: A Case Study of Android Malware

2020

Harel Berger, Chen Hajaj, and Amit Dvir

International symposium on cyber security cryptography and machine learning
Robust Malicious Domain Detection

2020

Nitay Hason, Amit Dvir, and Chen Hajaj

Cyber Security Cryptography and Machine Learning: Fourth International Symposium, CSCML 2020, Be’er Sheva, Israel, July 2–3, 2020, Proceedings 4

Domain name system (DNS) is a crucial part of the Internet, yet has been widely exploited by cyber attackers. Apart from making static methods like blacklists or sinkholes infeasible, some weasel attackers can even bypass detection systems with machine learning based classifiers. As a solution to this problem, we propose a robust domain detection system named HinDom. Instead of relying on manually selected features, HinDom models the DNS scene as a Heterogeneous Information Network (HIN) consist of clients, domains, IP addresses and their diverse relationships. Besides, the metapath-based transductive classification method enables HinDom to detect malicious domains with only a small fraction of labeled samples. So far as we know, this is the first work to apply HIN in DNS analysis. We build a prototype of HinDom and evaluate it in CERNET2 and TUNET. The results reveal that HinDom is accurate, robust and can identify previously unknown malicious domains.

2019

Adversarial Coordination on Social Networks

2019

Chen Hajaj, Sixie Yu, Zlatko Joveski, and Yevgeniy Vorobeychik

Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems

Extensive literature exists studying decentralized coordination and consensus, with considerable attention devoted to ensuring robustness to faults and attacks. However, most of the latter literature assumes that non-malicious agents follow simple stylized rules. In reality, decentralized protocols often involve humans, and understanding how people coordinate in adversarial settings is an open problem. We initiate a study of this problem, starting with a human subjects investigation of human coordination on networks in the presence of adversarial agents, and subsequently using the resulting data to bootstrap the development of a credible agent-based model of adversarial decentralized coordination. In human subjects experiments, we observe that while adversarial nodes can successfully prevent consensus, the ability to communicate can significantly improve robustness, with the impact particularly significant in scale-free networks. On the other hand, and contrary to typical stylized models of behavior, we show that the existence of trusted nodes has limited utility. Next, we use the data collected in human subject experiments to develop a data-driven agent-based model of adversarial coordination. We show that this model successfully reproduces observed behavior in experiments, is robust to small errors in individual agent models, and illustrate its utility by using it to explore the impact of optimizing network location of trusted and adversarial nodes.

DOI
★ Improving Robustness of ML Classifiers Against Realizable Evasion Attacks Using Conserved Features

2019

Liang Tong, Bo Li, Chen Hajaj, Chaowei Xiao, Ning Zhang, and Yevgeniy Vorobeychik

28th USENIX Security Symposium (USENIX Security 19)

Machine learning (ML) techniques are increasingly common in security applications, such as malware and intrusion detection. However, ML models are often susceptible to evasion attacks, in which an adversary makes changes to the input (such as malware) in order to avoid being detected. A conventional approach to evaluate ML robustness to such attacks, as well as to design robust ML, is by considering simplified feature-space models of attacks, where the attacker changes ML features directly to effect evasion, while minimizing or constraining the magnitude of this change. We investigate the effectiveness of this approach to designing robust ML in the face of attacks that can be realized in actual malware (realizable attacks). We demonstrate that in the context of structure-based PDF malware detection, such techniques appear to have limited effectiveness, but they are effective with content-based detectors. In either case, we show that augmenting the feature space models with conserved features (those that cannot be unilaterally modified without compromising malicious functionality) significantly improves performance. Finally, we show that feature space models enable generalized robustness when faced with a variety of realizable attacks, as compared to classifiers which are tuned to be robust to a specific realizable attack.

Abstract

2018

Adversarial task assignment

2018

Chen Hajaj, and Yevgeniy Vorobeychik

International Joint Conference on Artificial Intelligence

The problem of assigning tasks to workers is of long-standing fundamental importance. Examples of this include the classical problem of assigning computing tasks to nodes in a distributed computing environment, assigning jobs to robots, and crowdsourcing. Extensive research into this problem generally addresses important issues such as uncertainty and incentives. However, the problem of adversarial tampering with the task assignment process has not received as much attention. We are concerned with a particular adversarial setting where an attacker may target a set of workers in order to prevent the tasks assigned to these workers from being completed. When all tasks are homogeneous, we provide an efficient algorithm for computing the optimal assignment. When tasks are heterogeneous, we show that the adversarial assignment problem is NP-Hard, and present an algorithm for solving it approximately. Our theoretical results are accompanied by extensive experiments showing the effectiveness of our algorithms.

DOI