WACCO 2023

5th Workshop on Attackers and Cyber-Crime Operations

IEEE European Symposium on Security and Privacy 2023

July 3, 2023 - Delft, Netherlands

The emergence and commoditization of cyber-criminal activities calls for new empirical methods, measures, and technologies to quantify and understand offender operations across all forms of cyber-crime: from malware engineering and attack delivery, to running underground operations trading illegal goods such as drugs and illegal pornography, to spreading disinformation and planning (cyber-)terrorism operations. Without appropriate scientific measures of cyber-offender and attacker operations, capabilities, and resources, it remains impossible to derive sound policies, strategies and technologies that appropriately address realistic and evidence-based attacker and offender models. 

Call for Papers

WACCO 2023 welcomes (full and short) paper submissions, for publication in the EuroSP IEEE proceedings and presentation at WACCO 2023, and Research talks, for presentation at WACCO 2023 (no proceedings). All submissions will go through the same review process, and receive feedback from the PC.

Topics of interest include, but are not limited to:

  • Empirical studies on attacker operations and communities
  • Novel methods to perform attacker measurements at scale across several communities
  • Cooperation and trust as a source of attackers’ effectiveness
  • Attackers’ skill set
  • Attackers’ operational security
  • Measuring the spread of false information campaigns on social media
  • Quantitative and qualitative methods to measure, track, and counter cybercrime
  • Cybercrime measurement and networks
  • Cybercrime policy
  • Economics of cybercrime
  • Profiling of cybercriminals
  • Security metric design and evaluation
  • Security patch measurement
  • Statistical exploration and prediction of security incidents
  • Open Source INTelligence (OSINT) and digital footprints

The workshop is co-located with the 8th IEEE European Symposium on Security and Privacy (EuroS&P 2023).

Important Dates

All deadlines are Anywhere on Earth (AoE = UTC-12h).

Paper submission due March 28, 2023
Acceptance notice to authors April 30, 2023
Publication-ready papers submitted May 15, 2023
Workshop July 3, 2023


Session 1: Cybercrime and offenders

09:00 - 10:40 (Timezone: CEST (UTC+02:00))

Visualizing Cyber-Threats in Underground Forums

09:00 - 09:25

James Burroughs, Michal Tereszkowski-Kaminski and Guillermo Suarez-Tangil

In this paper, we develop a language-agnostic methodology to extract features of interest to an analyst from forum posts and visualize them in a way which facilitates identification and stratification of areas of interest in the forums, as well as further manual analysis of the text. We then apply this methodology to a specific Russian underground forum. The visualization acts as a 'thumbnail' for individual posts, conveying semantic metadata of post contents. By viewing the thumbnail, an analyst is provided with an immediate 'sense' of post length and key features present within a post, as well as their frequency and spatial arrangement. Using the generated visualizations of posts from the underground forum we speed up analyst identification of post subject matter by up to 72%. As a key novelty, we propose that the image output of our method has fractal properties that can be exploited when sorting threats and extracting highly technical posts. Thus, we use a method based on the Minkowski-Bouligand fractal dimension to prioritize analysis of posts which represent more sophisticated threats.

An Argument for Linguistic Expertise in Cyber Threat Analysis: LOLSec in Russian Language eCrime Landscape

09:25 - 09:50

Dalyapraz Manatova, L Jean Camp, Julia R Fox, Sandra Kuebler, Maria A Shardakova and Inna Kouper

In this position paper, we argue for a holistic perspective on threat analysis and other studies of state-sponsored or state-aligned eCrime groups. Specifically, we argue that understanding eCrime requires approaching it as a sociotechnical system and that studying such a system requires combining linguistic, regional, professional, and technical expertise. To illustrate it, we focus on the discourse of the Conti ransomware group in the context of the Russian invasion in Ukraine. We discuss the background of this group and their actions and provide examples of how the discourse and threats from such groups can be easily misunderstood without appropriate linguistic expertise.

Digital Drift and the Evolution of a Large Cybercrime Forum

09:50 - 10:15

Jack Hughes and Alice Hutchings

Cybercrime forum datasets are large and complex. Prior research uses aggregated time series data to create a picture of the whole dataset, or focuses on a smaller sample of cross sectional data, often for a specific subcommunity or crime time. This paper uses the longitudinal time series aspect of cybercrime forums to measure and observe the evolution of forums at a macro scale. Applying the Digital Drift theoretical framework, borrowed from criminology, we find a large amount of churn on the forum, with only a small proportion of users continuing long-term engagement. Measurements show a continual shift in forum activity, with year-based cohorts moving from starting in hacking discussions, towards starting in general discussions, and later towards e-whoring boards. The group of members who are active on the forum for over 12 months, typically have their last post in the marketplace, while other members, who are active for shorter periods of time, have their last post in hacking-related boards. Overall, we see an increasing trend towards financially-driven cybercrime, at both the user and forum level. Users post more in financially-related boards over time, and forum activity has trended away from gaming/social activity, trending towards more activity in market-related boards.

Applying Neutralisation Theory To Better Understand Ransomware Offenders

10:15 - 10:40

Lena Yuryna Connolly, Hervé Borrion, Budi Arief and Sanaa Kaddoura

The work presented in this paper investigates the crime of ransomware from the perspective of neutralisation theory. In particular, this research-in-progress paper aims to explore the feasibility of using neutralisation theory to better understand one of the key stakeholders in ransomware operations: the offenders. Individuals (including offenders) may employ techniques of neutralisation in order to justify their rule-breaking acts, and to diminish both the perceived consequences of their acts and the feeling of guilt. The focus of this work is on highly organised ransomware groups that not only conduct cyber attacks but also operate Ransomware-as-a-Service (RaaS) businesses. Secondary data was used in this research, including media interviews with alleged ransomware offenders. Data analysis is currently ongoing, but preliminary results show that ransomware offenders mainly use six neutralisation techniques to minimise the perceived impact and/or guilty feeling of their actions. These six neutralisation techniques are (1) denial of victim, (2) denial of injury, (3) claim of benefits, (4) claim of entitlement, (5) defence of necessity, and (6) claim of relative acceptability. The findings from this work can shed some light on the ransomware offending pathways, which in turn can be utilised to devise more effective countermeasures for combatting ransomware crime.


10:40 - 11:00 (Timezone: CEST (UTC+02:00))

Session 2: Internet threats and incident reporting

11:00 - 12:40 (Timezone: CEST (UTC+02:00))

The Peculiar Case of Tailored Phishing against SMEs: Detection and Collective Defense Mechanisms at a Small IT Company

11:00 - 11:25

Pavlo Burda, Abdul Malek Altawekji, Nicola Zannone and Luca Allodi

Phishing attacks are increasingly more sophisticated, with attackers exploiting publicly available information on their targets to personalize their attacks. Although an increasing body of research has investigated the effectiveness of tailored phishing campaigns, researchers have primarily focused on large enterprises. Company size, composition, and resource availability (e.g., of security experts or a phishing response team handling incidents) play an important role in the studied dynamics. However, whether the same also applies to small and medium-sized enterprises (SMEs), which typically do not have those resources, is unclear. On the other hand, studying SME security is hard as they generally have no expertise in-house to run the required experiments. This work provides a first study filling this gap by investigating the effectiveness of tailored phishing campaigns against an SME IT company in Europe. To this end, we conducted a field experiment targeting 30 employees at an SME and, subsequently, interviewed nine employees to understand the cognitive processes underlying the detection and response of our phishing campaign as well as the group defense mechanisms at the SME. Our findings show that expectation mismatch was the primary method for detecting our phishing email and that the collective defense mechanism enabled a surprisingly prompt response and containment of the attack, possibly, due to the network dynamics of a small company.

On gaps in enterprise cyber attack reporting

11:25 - 11:50

Abu Hajizada and Tyler Moore

It has long been lamented that firms underreport cyber attacks. In recent years, regulators have begun mandating that certain organizations must publicly report when incidents occur. Adherence to these requirements is an empirical question that has been largely unexamined to date. In this paper, we study regulatory filings by U.S. public companies to the Securities Exchange Commission and to the Department Health and Human Services that discuss cyber attacks. We also compare the findings against crowdsourced reports of cyber incidents appearing in media outlets. We find substantial gaps in coverage, both in terms of attacks that make the news but do not appear in regulatory filings and vice versa. We conclude by discussing the implications for the study of cyber attack and defense as well as for policymakers.

Mapping the Cyberstalking Landscape: An Empirical Analysis of Federal U.S. Crimes

11:50 - 12:15

Sasha Romanosky and Peter Schirmer

Among the new forms of technology-facilitated abuses, cyberstalking has become a growing and important problem. Cyberstalking involves the use of technology to stalk, threaten, or harass one or more individuals. For example, it can include tracking and intimidating a victim over social media, email, or text messages, or threatening to expose someone’s intimate photographs (sextortion). Cyberstalking has become a mechanism used by current or former domestic or intimate-partners, lone perpetrators, individuals targeting victims based on their employment or public image, and members of extremist groups. The innovations of this research are twofold. First, using multiple data sets, we developed an automated capability to identify and collect the full set of all federally prosecuted cyberstalking cases in the U.S.. Second, we employ natural language processing, network analysis, and regression analysis methods to code and analyze these court records. We apply these methods in order to answer three main research questions: how many federal cyberstalking cases are there?; what kinds of stalking behavior are being committed?; and what characteristics are correlated with conviction and severity of punishment?

Enhancing Vulnerability Prioritization: Data-Driven Exploit Predictions with Community-Driven Insights

12:15 - 12:40

Sasha Romanosky, Jay Jacobs, Octavian Suciu, Benjamin Edwards and Armin Sarabi

The number of disclosed vulnerabilities has been steadily increasing over the years. At the same time, organizations face significant challenges patching their systems, leading to a need to prioritize vulnerability remediation in order to reduce the risk of attacks. Unfortunately, existing vulnerability scoring systems are either vendor-specific, proprietary, or are only commercially available. Moreover, these and other prioritization strategies based on vulnerability severity are poor predictors of actual vulnerability exploitation because they do not incorporate new information that might impact the likelihood of exploitation. In this paper we present the efforts behind building a Special Interest Group (SIG) that seeks to develop a completely data-driven exploit scoring system that produces scores for all known vulnerabilities, that is freely available, and which adapts to new information. The Exploit Prediction Scoring System (EPSS) SIG consists of more than 170 experts from around the world and across all industries, providing crowd-sourced expertise and feedback. Based on these collective insights, we describe the design decisions and trade-offs that lead to the development of the next version of EPSS. This new machine learning model provides an 82% performance improvement over past models in distinguishing vulnerabilities that are exploited in the wild and thus may be prioritized for remediation.

Lunch Break

12:40 - 13:30 (Timezone: CEST (UTC+02:00))

Session 3: Cryptocurrency environment

13:30 - 14:20 (Timezone: CEST (UTC+02:00))

How Cryptocurrency Exchange Interruptions Create Arbitrage Opportunities

13:30 - 13:55

Andrew Morin and Tyler Moore

Centralized cryptocurrency exchanges offer users a more convenient platform to trade their digital assets at the cost of reduced control. As a result, when these exchanges suffer interruptions users struggle to access their funds or modify their orders. We investigate 41 events at the popular exchange Bitfinex, and measure the impact these events have on trades, volume, and pricing. We find that the volume to trade ratio increases during events, as fewer traders are moving large amounts of bitcoin. We also find that these interruptions often occur at the same time as arbitrage opportunities, with substantial profit opportunities.

"Not another Ponzi!": Comparing adverts for cryptocurrency investment scams across platforms

13:55 - 14:20

Gilberto Atondo Siu and Alice Hutchings

This work compares machine learning methods using supervised, semi-supervised and unsupervised learning, to classify advertisements for cryptocurrency related investment scams found in the online forum Bitcointalk, and the social media platform Reddit. We extract more than 24.2 million posts from Bitcointalk and use Reddit's API to collect 2,108 submissions. We train and compare several multi-class text classification approaches and use the models with highest accuracy and F-measure to identify cryptocurrency investment scam advertisements found on both platforms. We discover around five percent of all posts collected on both sites are potential scams. We then use another text classifier to identify the scam actors involved in these investment scam advertisements. We also discover the lures used within these fraudulent adverts and find the main differences in luring techniques used between Bitcointalk and Reddit. We identify that the most prevalent lure type uses the financial principle.

Closing keynote

14:20 - 15:00 (Timezone: CEST (UTC+02:00))

Measuring and Monitoring Malicious Social Media Behaviours

14:20 - 15:00

Rebekah Overdorf

From fake accounts to disinformation, malicious behaviour is evidently present on social media sites to anyone who uses them. However, actually measuring the different types of attacks and malicious behaviour presents many challenges. The correct data can be difficult to collect, the ground truth is often not concrete, and basic assumptions that we make about "normal" or "abnormal" behaviour often do not hold up to scrutiny. This keynote will illustrate some of these challenges through three different types of content-inflation attacks on social media that we uncovered over the past few years and discuss how they were overcome.

Closure and break

Review Model

Open reports

WACCO promotes an open and transparent review process. Reviews of accepted papers will be published together with the papers and archived in a public github repository associated with WACCO. A link to that repository must be included in all accepted submissions. The reasons why WACCO implements an open report model are the following:

  • It documents why the paper was considered positively to contribute to the larger scientific domain it pertains to;
  • It provides a critique useful to better delineate research limitations and scope, which can be of particular benefit to young researchers and students alike;
  • It provides a structural incentive for reviewers to write constructive and clear reviews;
  • It provides a structural incentive for authors to implement reviewer recommendations for the camera-ready version of their paper;
  • It provides a critical viewpoint for future work and research follow-ups;
  • It provides additional transparency to the quality of the adopted review process and its outcomes.


WACCO encourages submission of full papers and position papers from academia, industry, and government for appearance in the EuroSP IEEE proceedings. They should present interesting results for both theory and experimentation in the area of attacker and cyber-crime operations. We also particularly welcome independent reproduction of previous studies or experiments or negative results. We expect full papers to be of 10 pages in length (IEEE Format). Longer papers that document extensive experimentation are full in scope (which could be described in annex of the main body of the paper). Position papers of around 4 pages in length should present new open and interesting questions that the community should address or open questions that past research papers have not yet addressed. We expect position papers to be presented in panels or poster-platform sessions.

Additionally, WACCO 2023 welcomes submissions of Research Talks. Research Talk submissions will go through the same review process as full/short papers and will be evaluated on the same criteria of quality, but will not appear in the IEEE proceedings. We especially encourage the submission of multidisciplinary work looking for feedback from qualified experts in the domain. Research Talk submissions can be in any format, and of length commensurate to the contribution. Indicatively, Research Talks submissions are expected to be in the range of 7000-8000 words. To keep review loads acceptable, submissions of more than 10000 words may be desk rejected. Research Talk submissions should clearly state “Research Talk” in the title of the submission.

Anonymous submissions

Papers should be fully anonymized before review: author names or affiliations may not appear or be revealed in the text. Previous work of the authors should be referred to the third person. In the unusual case that an anonymous reference is not possible, the authors should blind the reference (e.g. “[x] Blinded citation to preserve submission anonymity”). Papers that are not properly anonymized may be desk rejected.
Submission of work that has been previously presented at conferences without proceedings, even if that work is associated with the names of the authors, or is published on online repositories such as ArXiv.org or SSRN, is allowed as long as the submission is fully anonymized. PC members that may recognize the work and its authors are asked to declare conflict on that paper and will not be assigned to it.


All papers will be published by IEEE CS and posted on the IEEE digital libraries. All authors of accepted papers are expected to present their paper at the workshop.

Submission site

Please submit your paper through EasyChair here.

Organization Committees

Program Co-chairs

Luca Allodi Eindhoven University of Technology l.allodi@tue.nl
Alice Hutchings University of Cambridge alice.hutchings@cl.cam.ac.uk
Sergio Pastrana University Carlos III of Madrid spastran@inf.uc3m.es

Publicity and Publication Co-chairs

To be announced.

Program Committee


The workshop is co-located with the 8th IEEE European Symposium on Security and Privacy (EuroS&P 2023). To register please visit the registration page of the main event.