OTE, February 2021

Testbeds were always important, as all new services and products always need to be tested thoroughly, especially the ones that operate in complex environments with many prospective users, although this is not always the case. Additionally, proper testing environment and procedure can help avoid later hardships. Nevertheless, this is the case with most of the research projects at EU, where the validity of a project’s outputs are checked and verified.

As the pilots are progressing, we can now discuss the CYBERTRUST testbeds, concerning the requirements and some implementation aspects. The requirements, which came along time, for the testbeds were:

  1. Big enough to host the CYBERTRUST platform on the NP side and a significant number (750) of emulated and simulated Small Offices/Homes (SOHOs).
  2. Tight integration of the SOHO gateways with the networking cloud infrastructure, in order to be able to provide proper routing, packet sniffing, internet and DHCP to the SOHO devices,
  3. Produce and allow significant amounts of normal and malicious traffic to traverse the testbed, without being blocked by the inherent cloud security mechanisms, even perform DDOS attacks while the testbed retains stability
  4. Monitor, capture and reproduce the circulated traffic at various points within the topology for analysis and manipulation
  5. Provide an adequate number and variety of PC and smart home devices and OSes to make for creating realistic SOHOs.
  6. Perform other types of attacks and infections towards the SOHOs and monitor their reactions as well as the CYBERTRUST platform.

Following, we will take the above requirements and provide a short overview on how we addressed them:

  1. It was considered that a private cloud would be better suited to handle the required isolation, versatility and control, while presenting the interoperability and applicability of the CYBERTRUST platform, between different virtualization platfroms, hence Openstack (Queens) at OTE and Hyper-V at KEMEA, were used for this.
  2. Two different classes of VMs are deployed under each smarthome. The Cyber-Trust platform services and the user devices. Initially, PFSENSE (2.4.*), was used as a SOHO-GW but because some important additional configuration was required for the traffic capturing and DHCP provisioning, it was later decided to use Ubuntu 20.4×64
    1. Initially, when trying to provision DHCP from a VM, there was no proper syncing of the leases and MAC addresses between Openstack and VM DHCP server (SOHO-GW). Bypassed by Opestack DHCP provision and static routing through SOHO-GW.
    2. Unfortunately, Netplan had its fair share of networking issues, mostly regarding DNS resolving, but were usually overcome by adding DNS in the tail file.
    3. Usage of address pairs at SOHO-GW to get internet inside SOHOs
    4. Security groups and IPTABLES to limit access to SOHOs
  3. For the traffic generation, there are utilized a number of tools, Cisco TRex TRex7, Ostinato, TCPliveplay, in order to produce and manipulate traffic by injecting infections as well, to create adaptable, realistic attack scenarios that can be easily repeated and adapted in a controlled way.
  4. For the capturing of traffic, there are employed tcpdump and TSHARK. The captures are performed on the compute hosts, on the SOHO-GWs or on specific SOHO devices.
  5. For the SOHO virtualized devices the following OSes were used in VM or dockerized form, Ubuntu 14/16/18.04 , Windows XP-SP3, Windows 7, Android. All these came from different sources with different forms such as QCOW2, OVA, VMDK, VDI, ISO where conversion tools such as qemu-img (a, b, c) and virtual box where used, but unfortunately not before causing some headaches first! These images were shared between the two testbeds, hence another requirement for conversion. Also in order to be able to reach the high number of 750 SOHOs, 10 SOHOs are emulated, while the others will be simulated, by monitoring, reproducing and retransmitting the equivalent amount of traffic corresponding to the rest of 740 SOHOs, in various scenarios within the testbed, for the CYBERTRUST platform to analyze and check the results.
  6. The synthesized attacks [D2.5] are common cybersecurity threats and attacks against IoT devices in smart homes and mobile devices domains. The idea behind these scenarios and the selection of the attacks was to be as realistic as possible given the testbed limitations, both at device and network level. In order to properly demonstrate these attacks, some VMs were created with certain vulnerabilities and deployed in the emulated SOHOs. The attacks are listed below:
    1. Zero-day exploits attacks
    2. DDoS attack with Mirai Botnet
    3. DDoS attack with Blackenergy malware
    4. Steal sensitive data with Zeus malware
    5. Malware Rootkits for Android, Linux and windows
    6. Attacks with replayed PCAP files of malicious network traffic.
Figure 1: CYBERTRUST testbed high level network topology
Figure 2: CYBERTRUST testbed graph topology snapshot

For more information on the above you can also check the below deliverables: