SDN-Sim: Integrating a System-Level Simulator with a Software Defined Network

With the introduction of diverse technology paradigms in next-generation cellular and vehicular networks, design and structural complexity are skyrocketing. The beyond-5G use cases such as mobile broadband, 5G-V2X, and UAV communications require support for ultra-low latency and high throughput and reliability with limited operational complexity and cost. These use cases are being explored in 3GPP Releases 16 and 17. To facilitate end-to-end performance evaluation for these applications, we propose SDN-Sim -- an integration of a system-level simulator (SLS) with a software defined network (SDN) infrastructure. While the SLS models the communication channel and evaluates system performance on the physical and data link layers, the SDN performs network and application tasks including routing, load balancing, and so on. The proposed architecture replicates the SLS-defined topology into an SDN emulator for offloading control operations. It uses link and node information calculated by the SLS to compute routes in SDN and feeds the results back to the SLS. Along with the architecture, data modeling, and processing, replication and route calculation frameworks are proposed.


Introduction
Toward 5G/beyond 5G (B5G), the 3rd Generation Partnership Project (3GPP) is finalizing Release 16 and defining Release 17. 1 . In the area of vehicular networks, 3GPP, in partnership with the Fifth Generation Automotive Association (5GAA), is driving the efforts on 5G-based the vehicle-to-everything (V2X) paradigm, which adds advanced features to the LTE-V2X from Release 14, particularly in the areas of support for ultra-reliable and low-latency communication (URLLC) applications for the future intelligent transport systems (ITS) [1][2][3]. In the evolution path from LTE-V2X to 5G-V2X, the authors in [1] advocated the incorporation of softwaredefined networking (SDN) in the architecture to enhance the system perfromance through SDN's capabilities in facilitating intelligent multihop routing, dynamic resource allocation, and advanced mobility support, among others.
To evaluate the performance of proposed algorithms, techniques, and frameworks for any new era of communication networks, numerical simulations, mathematical analyses, and field trials are the three main approaches being employed. Although analytically tractable, mathematical methods (e.g., stochastic geometry tools) are often constrained with simplifying assumptions that potentially limit their use in modeling largescale, highly complex, and dynamic networks. Realistic performance can be measured in live operating environments. However, the economic and operational requirements are costly and practically infeasible for the early design and development stages. Hence, in the past few decades, simulations have become remarkably important tools for the assessment of networks' performance due to the obvious cost and implementation advantages [4].
Depending on the performance metrics under investigation, simulators can be categorized into three: link-level simulators (LLSs), system-level simulators (SLSs), and network-level simulators (NLSs). An LLS examines detailed, bit-level physical (PHY) layer functionalities of a single link. An SLS evaluates the performance of links involving many base stations (BSs) and user equipments (UEs) at the medium access control (MAC) layer (with the PHY abstracted). It focuses on the radio access network/air interface and facilitates analyses of resource allocation, capacity, coverage, and spectral and energy efficiencies, among others. The NLS, however, assesses the performance of protocols across all layers of the network, including control signaling and backhaul/fronthaul issues. Performance is characterized using metrics including latency, packet loss, and so on [5].
Besides metric-based classification, simulators can also be grouped based on radio access technologies supported (cellular, vehicular, WiFi, etc.), coding language (MATLAB, Python, C++, etc.), licensing option (open source, proprietary, free of charge for academic use) or network scenario capabilities (LTE, 5G, B5G, etc.) [5]. 2 While the SLS does not simulate beyond the MAC layer, the NLS simulates networks up to the application layer. However, the implementation and computational complexity of the NLS becomes very high when a large number of nodes are involved [4].
Another major paradigm shift in network design took place with the advent of SDN [6]. It decouples the control (signaling) plane from the data (forwarding) plane and runs applications in the application plane to manage the network. This brings transparency in network design and lets software developers write applications for man-S. Ghosh, S. A. Busari, T. Dagiuklas, M. Iqbal, R. Mumtaz, J. Gonzalez, S. Stavrou, and L. Kanaris 1 https://www.3gpp.org/ news-events/2058-ran-rel-16progress-and-rel-17-potentialwork-areas. 2  aging the networks, keeping the internal design in abstraction. Each layer uses several interfaces to communicate with each other. The control plane communicates with both the application and data planes using north and southbound interfaces, respectively. In the case of a cluster of controllers, east and westbound interfaces are used for communicating among them.
OpenFlow, as the default southbound protocol for SDN, uses flow tables (FTs) to perform packet forwarding. Each FT entry is a forwarding rule determined by the controller. A forwarding rule has mainly three major fi elds, a "match," an "action," and a "priority." A "match" is some criteria for an inbound packet to be checked. If a packet satisfi es the criteria, it is called a "table hit"; otherwise, it is a "table miss." For each case, an action is defi ned such that the OpenFlow switch executes on the subjected packet. If a packet satisfi es matches from multiple fl ow rules, priority is used to break the tie. Flow entries are populated by the SDN controllers. For every table miss, the OpenFlow switch requests the controller, and the controller replies with a fl ow entry. If the controller cannot resolve an action, it is set as a "drop," and the switch does not process the packet. The decoupled control plane reduces computational cost on forwarding devices by offloading the control packet processing tasks to the controller. Therefore, SDN off ers better modularity, programmability, agility, automation, and load balancing capability than do traditional networks. Also, the SDN-based approach is used in network design practices for cloud computing and 5G.
In this work, we present a novel SDN-based system-level simulator (SDN-Sim) platform where the SLS-Stage runs in MATLAB and the NLSstage is based on python3. By inheriting all the benefits of SDN, the architecture is designed to reduce the overall computational complexity of the system considerably. The computationally demanding upper layer network functions (e.g., inter-cellular routing) are offloaded to the virtualized cloud infrastructure. Low-level network information (e.g,. channel model, topology) are maped from SLS to SDN. Python is used for application development. Virtual infrastructure is set up using VMWare ESXi servers. OpenFlow and RESTConf are used as south and northbound protocols, respectively. Opendaylight is used as the SDN controller, while GNS3 and Mininet-wifi are used for data plane emulation.
This article proposes an architecture to integrate MATLAB-based SLS with SDN using OpenDaylight, Mininet-Wifi, and GNS3. The implementation is described with data modeling of the middleware and sequence modeling of messages exchanged among various entities. Another major paradigm shift in network design took place with the advent of SDN [6]. It decouples the control (signaling) plane from data (forwarding) plane and runs applications in the Application Plane to manage the network. This brings transparency in network design and lets software developers write applications for managing the networks, keeping the internal design in abstraction.
Three applications (TopoBuild, TopoSense, and TopoRoute) are also presented for data fetching, interfacing, and processing. The remainder of this article is organized as follows. We describe the system architecture and implementation. The experimental setup and results are presented, followed by conclusions and future research directions.
System Architecture and Implementation

System-Level Simulator
The tasks of the SLS (described below) are executed in loops of transmission time intervals (TTIs), and the results are averaged over several simulation runs or channel realizations [7]. Scenario Setting: The layout depicts a vehicular network of BSs/roadside units (RSUs) and vehicle mounted radio/UEs, confi gured with 3D locations of the nodes. UEs and vehicles are mobile and attached to their serving RSU, which further interfaces with the cellular networks through BSs with parameters such as line-of-sight probability, distance, and signal-to-noise ratio (SNR).
Channel Modeling: For all links, the path loss (PL), shadow fading (SF), transmit and receive and antenna gains, and fast fading are calculated to estimate the channel of each user for both desired and interfering links.
Scheduling: Radio resources are allocated to users based on the scheduling algorithm. Resource blocks (bandwidth) and power are allocated either in quasi-random fashion (for open loop configuration) or based on feedback from the users in closed loop systems. The channel state information (CSI) feedback, together with other factors such as the traffi c type of users, link adaptation strategy employed, and quality of service (QoS) demands, are used as decision determinants at the scheduling stage.
Link Quality and Performance Estimation: The signal-to-interference-plus-noise ratio (SINR) of the links are then estimated. Using the SINRs and the link abstraction model (for the block error rate, BLER), the throughput of the users and the capacities of the cells are calculated.  Recent developments in SLSs are mainly focused on solving problems on channel modeling [8], high-frequency communication [9,10], coexistence and performance optimization [11], energy efficiency, latency, scheduling, and load balancing over a heterogeneous network [12], among others. The 5G Public-Private Partnership (5GPPP) project has described several aspects in softwarization, service management and orchestration in their architectural reference [13], which includes SDN, cloud computing, virtualization, and so on. Therefore, cloud computing is being employed to enhance the scalability and computational efficiency of 5G SLSs by offloading the computational load [14].

Software Defined Network
The tasks of the SDN are described as follows.
Channel Information Update: The SLS running in a MATLAB server with Open Database Connector (ODBC) driver updates the topology and channel modeling parameters to a centralized database.
SDN Emulation: The topology information is reactively fetched and emulates using Topo-Build in an SDN data plane through GNS3 and Mininet-wifi. The control plane starts communicating with Open-Flow.
Route Calculation: The application plane fetches topology from the control plane, translates it into a graph (using TopoSense), and finds the shortest paths between node pairs (using TopoRoute). These paths are fed back to the database and the controllers.
The work-flow and communication sequence taking place between the various elements of the architecture are as given in Fig. 2.
Update Communication Parameters: The SLS performs channel modeling, radio resource allocation, and system performance evaluation. After optimizing the model for a given scenario, it generates several channel parameters (bandwidth, distance, path loss, latency, and delay) and BS parameters (position, range, received signal strength indication [RSSI], transmission power, etc.). Since parameters are calculated per BS, each BS updates their local dataset to the centralized database.
Fetch Node and Link Parameters: Topobuild is a bespoke program to fetch SLS parameters from the central database and relay them to the SDN data plane for replicating the topology. BSs are placed as wireless access points, running Open-Flow protocol. Links, depending on types (i.e., fronthaul or backhaul), are assigned to BSs. A fronthaul link is wireless, therefore carrying several radio parameters including path loss, frequency band, RSSI, and so on.
Topology Replication: Topobuild translates parameters obtained from the central database into a series of commands. In the proposed architecture, there are two possible infrastructures (GNS3 and Mininet-wifi) with three possible deployment options (full GNS3, full Mininet-wifi, and hybrid). Based on the deployment type and connection specifications, Topobuild.py generates a script to replicate the topology and injects it to the specific engines (GNS3 and/or Mininet-wifi). A feature comparison between GNS3 and Mininet-WiFi is given in Table 1 and briefly described as follows.
The 5G public-private partnership project (5GPPP) has described several aspects in softwarization, service management and orchestration in their architectural reference which includs, SDN, cloud computing, virtualization, etc. Therefore, cloud computing is being employed to enhance the scalability and computational efficiency of 5G SLSs by offloading the computational load. GNS3 Infrastructure: GNS3 is an open source network emulation software. It comprises a controller unit that interfaces through RESTful application programming interfaces (APIs), manages topology and so on, and a cluster of para-virtualized compute nodes that host virtual network functions (VNFs) such as routers, switches, firewalls as containers, or virtual machines (VMs). In SDN-Sim, Topobuild communicates with GNS3-Controller using a RESTful API, and the Open-V-switches (OVSs) run as Docker containers within the GNS3-Compute cluster. Therefore, GNS3 off ers realistic test cases with a testbed and produces more authentic results than those of a simulation. A downside is that it lacks support for wireless networking emulation; however, it allows access to a physical wireless card.
Mininet-WiFi Infrastructure: With a Wi-Fi extension, inheriting all the features of Mininet, it can now emulate wireless SDNs. With support of its python API, programming and configuring Mininet-Wifi is more user-friendly. Mininet-Wifi runs in a single sandbox with its own interactive command line. Thus, Topobuild uses raw sockets to inject Mininet-wifi commands for deployment and parameters updates. It also off ers four mobility and five propagation models. Since Mininet-Wifi uses HWSIM drivers to simulate wireless networking, emulating a very large network is constrained. Also, it does not support clustering,.
Accessing Control Plane with OpenFlow: Both GNS3 and Mininet-wifi host OVSs; thus, they communicate with the controller using OpenFlow v1.3 (OF1.3) as southbound protocol. OF1.3 uses bidirectional messaging to communicate with the switches. A switch requests a controller with a Packet-In message, and the controller replies back with a Packet-Out. SDN-Sim uses OpenDaylight -Beryllium SR4 (ODL) as controller, which runs as a VM. Both GNS3 (4.1) and Mininet-wifi (4.2) use TCP port 6633 and/or 6635 for communication. The control plane supports binding of several controller nodes with clustering to maintain scalability, high availability, and persistence in the data plane. ODL supports "Akka" clustering for this purpose.
Interfacing with the Application Plane with RESTConf: SDN-Controller interacts with the application plane using Northbound APIs. ODL uses RESTConf (RFC 8040). It is a RESTful version of NETCONF (RFC 6241) protocol, and uses JSON (RFC-7159) format to transfer data among REST enabled devices. In SDN-Sim, we use two RESTConf resources (Inventory and Topology). "Operational" resources are to read, and "Confi g" are to write. The inventory resource provides node-wise OpenFlow tables, and the topology of the network is provided by the Topology resource. The App TopoSense makes use of the resources to model a graph by fusing topology and flow-table information. TopoSense invokes TopoRoute to calculate the route for a given topology.
Route Calculation: TopoRoute uses the stochastic temporal edge normalization (STEN) [15] technique to fi nd routes. It receives the topology and fl ow tables from TopoSense. The link parameters are already present in the database. By fusing them, a single source shortest path algorithm is run over every pair of vertices. and a set of all With the size of the network, due to propagation delay, the routing becomes significantly slow. However, in an SDN paradigm, discovery is done by the controller hence the topology graph is mapped proactively. Therefore, the shortest paths between every pair can be calculated in parallel. This speeds up the routing process and allows scalability in the network.
possible routes are generated. In a traditional network, local routes are shared among neighbors to form routing tables. With the size of the network, due to propagation delay, the routing becomes significantly slow. However, in an SDN paradigm, discovery is done by the controller; hence, the topology graph is mapped proactively. Therefore, the shortest paths between every pair can be calculated in parallel. This speeds up the routing process and allows scalability in the network.
OpenFlow Tables Update: TopoRoute does event-driven update of the flow tables in the Central DB (7.1). The calculated routes are fed back to TopoSense, which eventually sends the routes back to the controller using the RESTConf Inventory-Config API.
Once the central DB is updated, SLS picks the information, and an inter-cellular route is discovered to leverage the lower (physical and data link) layer operations. The complete operational sequence diagram is given in Fig. 2.

Data Modeling and Design of the Central Database
The central database plays the role of middleware between the SLS and the SDN. Table 2 shows the related entities and their corresponding attributes for the data model. There are three main entities: node, flow table, and channel. A node can be either an access point (AP) or a host. One host is associated with one AP, and one AP can associate with many hosts. Each AP is an OVS; thus, it contains flow table(s) and a unique data path identifier (DPID). A pair of nodes makes a channel, with a unique channel ID (mapping is recorded at Node_Channel_Map  table). A channel between a host and an AP is called fronthaul (wireless), and between a pair of APs a backhaul (wired). No channel exists between two hosts. Since TopoSense reads information from the APs, the route calculation by TopoRoute takes place over the backhaul network. sites, respectively, with 3 sectors per site; d) depicts total time consumption and is predominated by the SLS channel scheduling, SDN tasks are comparatively lightweight, and routing time is bounded by a sub-second interval; e) shows that the total time consumption has a constant convergence time (it is too small to be visible on the staked bar chart).

Design of Application Plane
Three major apps in the application plane run three python apps: TopoBuild.py, TopoSense.py, and TopoRoute.py. In the previous sections, their usage has been mentioned. Figure 3 depicts their working principles using flowcharts.

Experimental Setup
During experiment, the following compute-node setup was followed. Latency and Time Complexity Figure 4 depicts the latency of several stages of integration; SLS shares most of it. The setup phase consumes a significant amount of time depending on the network size and topology, which includes channel allocation and scheduling, SDN setup, flow table population, proactive route calculation, and so on. However, the runtime is reactive in nature, and response is in millisecond scale (Fig.  4e). All the routes are pre-calculated and the network runs on a centralized virtual platform, which eliminates control packet exchange between devices to learn network topology. The time complexity of the proactive phase is of a high degree polynomial class, and the reactive phase is constant; thus, once the SDN is deployed, response time comes down to millisecond scale.

Conclusion and Future Work
In this article, we propose an architecture and methodology to integrate a system-level simulator with a software-defined networking infrastructure using a relational database as middleware. A comparative analysis on using a GNS3 and Mininetwifi-based data plane is given. A sequence model shows the data and control flow among several percolating entities. The data model of the middleware describes the information structuring, and a set of algorithms to fetch and replicate will extend this architecture by appending an analytics plane on top of it. Data analytics empowered by deep learning algorithms will learn the runtime behavior of the network and help to improve the network automation, self-organization, and state prediction ability. The proposed simulator facilitates the end-to-end performance evaluation of 5G and beyond 5G use cases such as 5G-V2X, where latency is a critical performance metric.