Multi-Agent Collaborative Learning for UAV Enabled Wireless Networks

Journal article


Xia, Wenchao, Zhu, Y., De Simone, L., Dagiuklas, A., Wong, Kai-Kit and Zhen, Gan (2022). Multi-Agent Collaborative Learning for UAV Enabled Wireless Networks. IEEE Journal on Selected Areas in Communications.
AuthorsXia, Wenchao, Zhu, Y., De Simone, L., Dagiuklas, A., Wong, Kai-Kit and Zhen, Gan
Abstract

The unmanned aerial vehicle (UAV) technique pro- vides a potential solution to scalable wireless edge networks. This paper uses two UAVs, with accelerated motions and fixed altitudes, to realize a wireless edge network, where one UAV forwards downlink signals to user terminals (UTs) distributed over an area while the other one collects uplink data. The conditional average achievable rates, as well as their lower bounds, of both the uplink and downlink transmission are derived considering the active probability of UTs and the service queues of two UAVs. In addition, a problem aiming to maximize the energy efficiency of the whole system is formulated, which takes into account communication related energy and propulsion energy consumption. Then, we develop a novel multi-agent Q- learning (MA-QL) algorithm to maximize the energy efficiency, through optimizing the trajectory and transmit power of the UAVs. Finally, simulation results are conducted to verify our analysis and examine the impact of different parameters on the downlink and uplink achievable rates, UAV energy consumption, and system energy efficiency. It is demonstrated that the proposed algorithm achieves much higher energy efficiency than other benchmark schemes.

KeywordsUAV swarm, energy efficiency, trajectory opti- mization, multi-agent reinforcement learning, queue theory.
Year2022
JournalIEEE Journal on Selected Areas in Communications
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
ISSN0733-8716
Publication process dates
Accepted23 Apr 2022
Deposited13 May 2022
Accepted author manuscript
License
File Access Level
Open
Permalink -

https://openresearch.lsbu.ac.uk/item/8zxz4

Restricted files

Accepted author manuscript

  • 64
    total views
  • 1
    total downloads
  • 5
    views this month
  • 0
    downloads this month

Export as

Related outputs

The role of NTN in 6G
Dagiuklas, A. and Zhu, Y. (2022). The role of NTN in 6G. https://futurenetworks.ieee.org. Virtual 19 - 21 Jul 2022 IEEE.
A Comparative Analysis of Deadlock Avoidance and Prevention Algorithms for Resource Provisioning in Intelligent Autonomous Transport Systems over 6G Infrastructure
Ugwuanyi, E., Iqbal, M. and Dagiuklas, A. (2022). A Comparative Analysis of Deadlock Avoidance and Prevention Algorithms for Resource Provisioning in Intelligent Autonomous Transport Systems over 6G Infrastructure. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2022.3169424
Forgery Image Dataset
Hossain, Faria, Gul, Asim, Raja, Rameez, Dagiuklas, A. and Galkandage, Chathura (2022). Forgery Image Dataset. Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.21227/9dmj-yn86
A Cognitive Routing Framework for Reliable Communication in IoT for Industry 5.0
Ghosh, S., Dagiuklas, A., Iqbal, M. and Wang, X. (2021). A Cognitive Routing Framework for Reliable Communication in IoT for Industry 5.0. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/TII.2022.3141403
SO-KDN: A Self-Organised Knowledge Defined Networks Architecture for Reliable Routing
Ghosh, S., El Boudani, B., Dagiuklas, A. and Iqbal, M. (2021). SO-KDN: A Self-Organised Knowledge Defined Networks Architecture for Reliable Routing. 2021 The 4th International Conference on Information Science and Systems. https://doi.org/10.1145/3459955.3460617
Robust, Resilient and Reliable Architecture for V2X Communication
Khan, M.A., Ghosh, S., Busari, S.A., Huq, K.M.S., Dagiuklas, A., Mumtaz, S., Iqbal, M. and Rodriguez, J. (2021). Robust, Resilient and Reliable Architecture for V2X Communication. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2021.3084519
Towards An SDN Assisted IDS
Sutton, R., Ludwiniak, R., Pitropakis, N., Chrysoulas, C. and Dagiuklas, A. (2021). Towards An SDN Assisted IDS. IFIP NTMS Workshop on CyberSecurity on Hardware 2021. Paris 19 - 21 Apr 2021 International Federation for Information Processing (IFIP).
Enabling Real-Time AI Edge Video Analytics
Tsakanikas, V. and Dagiuklas, A. (2021). Enabling Real-Time AI Edge Video Analytics. IEEE International Conference on Communications. Montreal 14 - 23 Jun 2021 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICC42927.2021.9500902
A Self-Organised Knowledge Defined Networks Architecture for Reliable Routing
Ghosh, S., El Boudani, B., Dagiuklas, A. and Iqbal, M. (2021). A Self-Organised Knowledge Defined Networks Architecture for Reliable Routing. 4th International Conference on Information Science and Systems ICISS 2021. Edinburugh 17 - 19 Mar 2021 ACM.
Positioning as Service for 5G IoT Networks
El Boudani, B., Kanaris, L., Kokkinis, A., Chrysoulas, C., Dagiuklas, A. and Stavrou, S. (2021). Positioning as Service for 5G IoT Networks. 2021 Telecoms Conference (ConfTELE). Leiria 10 - 12 Feb 2021 Institute of Electrical and Electronics Engineers (IEEE).
A Comparative Analysis of Deadlock Avoidance and Prevention Algorithms for Resource Provisioning in Distributed MEC
Ugwuanyi, E., Iqbal, M. and Dagiuklas, A. (2021). A Comparative Analysis of Deadlock Avoidance and Prevention Algorithms for Resource Provisioning in Distributed MEC. IEEE Transactions on Computational Social Systems.
A Novel Predictive-Collaborative-Replacement (PCR) Intelligent Caching Scheme for Multi-Access Edge Computing (MECs)
Ugwuanyi, E.E., Iqbal, M. and Dagiuklas, A. (2021). A Novel Predictive-Collaborative-Replacement (PCR) Intelligent Caching Scheme for Multi-Access Edge Computing (MECs). IEEE Access. 9, pp. 37103 - 37115. https://doi.org/10.1109/ACCESS.2021.3058769
User Selection in Reconfigurable Intelligent Surface Assisted Communication Systems
Gan, Xu, Zhong, C., Zhu, Y. and Zhong, Z. (2021). User Selection in Reconfigurable Intelligent Surface Assisted Communication Systems. IEEE Communications Letters. 25 (4), pp. 1353-1357. https://doi.org/10.1109/lcomm.2020.3048782
Survey for Smart Farming Technologies: Challenges and Issues
Idoje, G., Dagiuklas, A. and Iqbal, M. (2020). Survey for Smart Farming Technologies: Challenges and Issues. Computers and Electrical Engineering. 92, p. 107104. https://doi.org/10.1016/j.compeleceng.2021.107104
Multiobjective Optimization Algorithms for Wireless Sensor Networks
Kandris, D., Alexandridis, A., Dagiuklas, A., Panaousis, E. and Vergados, D. (2020). Multiobjective Optimization Algorithms for Wireless Sensor Networks. Wireless Communications and Mobile Computing. 2020, pp. 1-5. https://doi.org/10.1155/2020/4652801
Implementing Deep Learning Techniques in 5G IoT Networks for 3D Indoor Positioning: DELTA (DeEp Learning-Based Co-operaTive Architecture)
El Boudani, B., Kanaris, L., Kokkinis, A., Kyriacou, M., Chrysoulas, C., Stavrou, S. and Dagiuklas, A. (2020). Implementing Deep Learning Techniques in 5G IoT Networks for 3D Indoor Positioning: DELTA (DeEp Learning-Based Co-operaTive Architecture). Sensors. 20 (19), p. e5495. https://doi.org/10.3390/s20195495
A Centralised Hybrid Routing Model for Multi-Controller SD-WANs
Ghosh, S, Iqbal, M. and Dagiuklas, A. (2020). A Centralised Hybrid Routing Model for Multi-Controller SD-WANs. Transactions on Emerging Telecommunications Technologies. 32 (6), p. e4252. https://doi.org/10.1002/ett.4252
Performance Analysis of Hybrid UAV Networks for Probabilistic Content Caching
Zhu, Y. (2020). Performance Analysis of Hybrid UAV Networks for Probabilistic Content Caching. IEEE Systems Journal. https://doi.org/10.1109/JSYST.2020.3013786
Large System Analysis of Downlink MISO-NOMA System via Regularized Zero-Forcing Precoding with Imperfect CSIT
Zhang, J., Zhu, Y., Ma, S., Li, X. and Wong, K.K. (2020). Large System Analysis of Downlink MISO-NOMA System via Regularized Zero-Forcing Precoding with Imperfect CSIT. IEEE Communications Letters. 24 (11), pp. 2454-2458. https://doi.org/10.1109/lcomm.2020.3010422
Programmable Metasurface Based Multicast Systems: Design and Analysis
Zhu, Y (2020). Programmable Metasurface Based Multicast Systems: Design and Analysis. IEEE Journal on Selected Areas in Communications. https://doi.org/10.1109/JSAC.2020.3000809
Stochastic Geometry Analysis of Large Intelligent Surface-Assisted Millimeter Wave Networks
Zhu, Y (2020). Stochastic Geometry Analysis of Large Intelligent Surface-Assisted Millimeter Wave Networks. IEEE Journal on Selected Areas in Communications. https://doi.org/10.1109/JSAC.2020.3000806
Spectrum and Energy Efficiency in Dynamic UAV-Powered Millimeter Wave Networks
Zhu, Y and Tasos, D (2020). Spectrum and Energy Efficiency in Dynamic UAV-Powered Millimeter Wave Networks. IEEE Communications Letters. https://doi.org/10.1109/LCOMM.2020.3001357
Shared Hybrid ARQ with Incremental Redundancy (SHARQ-IR) in Overloaded MIMO Systems to support Energy-Efficient Transmissions
Kashif, M, Iqbal, M, Ullah, Z, Dagiuklas, A, Sarwar, S, Ul Qayyum, Z and Safyan, M (2020). Shared Hybrid ARQ with Incremental Redundancy (SHARQ-IR) in Overloaded MIMO Systems to support Energy-Efficient Transmissions. IEEE Access. https://doi.org/10.1109/ACCESS.2020.3001146
Incomplete Information based Collaborative Computing in Emergency Communication Networks
Wang, Q, Zhu, Y and Wang, X (2020). Incomplete Information based Collaborative Computing in Emergency Communication Networks. IEEE Communications Letters. https://doi.org/10.1109/LCOMM.2020.2995501
DocsChain: Blockchain based IoT Solution for Verficiation of Degree Documents
Rasool, S., Saleem, A., Iqbal, M., Dagiuklas, A., Mumtaz, S. and Ul Qayyum, Z. (2020). DocsChain: Blockchain based IoT Solution for Verficiation of Degree Documents. IEEE Transactions on Computational Social Systems. pp. 827-837. https://doi.org/10.1109/TCSS.2020.2973710
Blockchain-Empowered Decentralized Storage in Air-to-Ground Industrial Networks
Zhu, Y., Zheng, G. and Wong, K-K. (2019). Blockchain-Empowered Decentralized Storage in Air-to-Ground Industrial Networks. IEEE Transactions on Industrial Informatics. 15 (6), pp. 3593-3601. https://doi.org/10.1109/tii.2019.2903559
Achievable Rate and Capacity Analysis for Ambient Backscatter Communications
Qian, J, Zhu, Y, He, C, Gao, F and Jin, S (2019). Achievable Rate and Capacity Analysis for Ambient Backscatter Communications. IEEE Transactions on Communications. 67 (9), pp. 6299-6310. https://doi.org/10.1109/tcomm.2019.2918525
On the Uplink Achievable Rate of Massive MIMO System with Low-Resolution ADC and RF Impairments
Xu, L, Lu, X, Jin, S, Gao, F and Zhu, Y (2019). On the Uplink Achievable Rate of Massive MIMO System with Low-Resolution ADC and RF Impairments. IEEE Communications Letters. 23 (3), pp. 502-505. https://doi.org/10.1109/lcomm.2019.2895823
Deep Learning Empowered Task Offloading for Mobile Edge Computing in Urban Informatics
Zhang, K, Zhu, Y, Leng, S, He, Y, Maharjan, S and Zhang, Y (2019). Deep Learning Empowered Task Offloading for Mobile Edge Computing in Urban Informatics. IEEE Internet of Things Journal. 6 (5), pp. 7635-7647. https://doi.org/10.1109/jiot.2019.2903191
A Deep Learning Framework for Optimization of MISO Downlink Beamforming
Xia, W, Zheng, G, Zhu, Y, Zhang, J, Wang, J and Petropulu, AP (2019). A Deep Learning Framework for Optimization of MISO Downlink Beamforming. IEEE Transactions on Communications. 68 (3), pp. 1866 - 1880. https://doi.org/10.1109/TCOMM.2019.2960361
Modelling quality of experience for online video advertisement insertion
Bulkan, U., Dagiuklas, A. and Iqbal, M. (2019). Modelling quality of experience for online video advertisement insertion. IEEE Transactions on Broadcasting. https://doi.org/10.1109/TBC.2020.2965064
Lightweight Computation to Robust Cloud Infrastructure for Future Technologies (Workshop Paper)
Shahzadi, S., Iqbal, M., Wang, X., Ubakanma, G., Dagiuklas, A. and Tchernykh, A. (2019). Lightweight Computation to Robust Cloud Infrastructure for Future Technologies (Workshop Paper). Wang, Xinheng, Gao, Honghao, Iqbal, Muddesar and Min, Geyong (ed.) CollaborateCom 2019 - 15th EAI International Conference on Collaborative Computing: Networking, Applications and Worksharing. London 19 - 22 Nov 2019 Springer International Publishing. pp. 3-11
Co-Operative and Hybrid Replacement Caching for Multi-Access Mobile Edge Computing
Ugwuaynyi, E.E., Ghosh, S., Iqbal, M., Dagiuklas, A., Mumtaz, S. and Al-Dulaimi, A. (2019). Co-Operative and Hybrid Replacement Caching for Multi-Access Mobile Edge Computing. 2019 European Conference on Networks and Communications (EuCNC). Valencia, Spain 18 - 21 Jun 2019 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/EuCNC.2019.8801991
Predicting quality of experience for online video service provisioning
Bulkan, U and Dagiuklas, A. (2019). Predicting quality of experience for online video service provisioning. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-019-7164-9
Evolution of Orchestration Towards 5G
Mohades Kassai, S K, Ghosh, S and Dagiuklas, A (2019). Evolution of Orchestration Towards 5G. Journal of Communication. 14 (12). https://doi.org/10.12720/jcm.14.12.1134-1140
BlueArch – An implementation of 5G Testbed
Ghosh, S, Ugwuanyi, E, Dagiuklas, A and Iqbal, M (2019). BlueArch – An implementation of 5G Testbed. Journal of Communication. https://doi.org/10.12720/jcm.14.12.1110-1118
Secrecy Rate Analysis of UAV-Enabled mmWave Networks Using Matérn Hardcore Point Processes
Zhu, Y, Zheng, G and Fitch, M (2018). Secrecy Rate Analysis of UAV-Enabled mmWave Networks Using Matérn Hardcore Point Processes. IEEE Journal on Selected Areas in Communications. 36 (7), pp. 1397-1409. https://doi.org/10.1109/jsac.2018.2825158
Content Placement in Cache-Enabled Sub-6 GHz and Millimeter-Wave Multi-Antenna Dense Small Cell Networks
Zhu, Y., Zheng, G., Wang, L., Wong, K-K. and Zhao, L. (2018). Content Placement in Cache-Enabled Sub-6 GHz and Millimeter-Wave Multi-Antenna Dense Small Cell Networks. IEEE Transactions on Wireless Communications. 17 (5), pp. 2843-2856. https://doi.org/10.1109/twc.2018.2794368
A Novel Optimal Mapping Algorithm With Less Computational Complexity for Virtual Network Embedding
Cao, H, Zhu, Y, Zheng, G and Yang, L (2018). A Novel Optimal Mapping Algorithm With Less Computational Complexity for Virtual Network Embedding. IEEE Transactions on Network and Service Management. 15 (1), pp. 356-371. https://doi.org/10.1109/tnsm.2017.2778106
Performance Analysis of Cache-Enabled Millimeter Wave Small Cell Networks
Zhu, Y., Zheng, G., Wong, K-K., Jin, S. and Lambotharan, S. (2018). Performance Analysis of Cache-Enabled Millimeter Wave Small Cell Networks. IEEE Transactions on Vehicular Technology. 67 (7), pp. 6695-6699. https://doi.org/10.1109/tvt.2018.2797047
A Efficient Mapping Algorithm With Novel Node-Ranking Approach for Embedding Virtual Networks
Cao, H, Zhu, Y, Yang, L and Zheng, G (2017). A Efficient Mapping Algorithm With Novel Node-Ranking Approach for Embedding Virtual Networks. IEEE Access. 5, pp. 22054-22066. https://doi.org/10.1109/access.2017.2761840
Secure Communications in Millimeter Wave Ad Hoc Networks
Zhu, Y., Wang, L., Wong, K-K. and Heath, R.W. (2017). Secure Communications in Millimeter Wave Ad Hoc Networks. IEEE Transactions on Wireless Communications. 16 (5), pp. 3205-3217. https://doi.org/10.1109/twc.2017.2676087
A Novel Mobility-Aware Playout Algorithm for VoIP Services
Lykourgiotis, A, Kotsopoulos, S and Dagiuklas, A. (2017). A Novel Mobility-Aware Playout Algorithm for VoIP Services. Wireless Personal Communications. 96 (2), pp. 2427-2446. https://doi.org/10.1007/s11277-017-4305-1
Predicting Quality Of Experience For Online Video Systems Using Machine Learning
Dagiuklas, T and Bulkan, U (2017). Predicting Quality Of Experience For Online Video Systems Using Machine Learning. 19th IEEE International Workshop on Multimedia Signal Processing. Luton 16 - 18 Oct 2017 IEEE.
IoTA: IoT Automated SIP-based Emergency Call Triggering System for general eHealth purposes
Andriopoulou, F, Orphanoudakis, T and Dagiuklas, A. (2017). IoTA: IoT Automated SIP-based Emergency Call Triggering System for general eHealth purposes. 13th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications. Rome 09 - 11 Oct 2017 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/WiMOB.2017.8115830
Wireless Power Transfer in Massive MIMO-Aided HetNets With User Association
Zhu, Y., Wang, L., Wong, K-K., Jin, S. and Zheng, Z. (2016). Wireless Power Transfer in Massive MIMO-Aided HetNets With User Association. IEEE Transactions on Communications. 64 (10), pp. 4181-4195. https://doi.org/10.1109/tcomm.2016.2594794
Geometric Power Control for Time-Switching Energy-Harvesting Two-User Interference Channel
Zhu, Yongxu, Wong, Kai-Kit, Zhang, Yangyang and Masouros, Christos (2016). Geometric Power Control for Time-Switching Energy-Harvesting Two-User Interference Channel. IEEE Transactions on Vehicular Technology. 65 (12), pp. 9759-9772. https://doi.org/10.1109/tvt.2016.2520565
End-to-end quality aware optimization for multimedia clouds
Athanasopoulos, D, Politis, I, Lykourgiotis, A, Tselios, C and Dagiuklas, A. (2016). End-to-end quality aware optimization for multimedia clouds. IEEE International Conference on Telecommunications and Multimedia. Heraklion, Greece 25 - 27 Jul 2016 Institute of Electrical and Electronics Engineers (IEEE).
V2I Applications in Highways: How RSU Dimensioning Can Improve Service Delivery
Charalampopoulos, G, Dagiuklas, T and Chrysikos, T (2016). V2I Applications in Highways: How RSU Dimensioning Can Improve Service Delivery. 2nd International Conference on Telecommunications (ICT). Thessaloniki, Greece 16 - 18 May 2016 London South Bank University.
Non-Conforming Behavior Detection for VoIP-Based Network Systems
Galiotos, P, Anagnostopoulos, C, Dagiuklas, T and Kotsopoulos, S (2016). Non-Conforming Behavior Detection for VoIP-Based Network Systems. IEEE International Conference on Communications. Kuala Lumpur, Malaysia 23 - 27 May 2016 IEEE. https://doi.org/10.1109/ICC.2016.7511103
Techno-economic analysis for the deployment of PPDR services over 4G/4G+ Networks
Chrysikos, T, Galiotos, P, Dagiuklas, A. and Kotsopoulos, S (2016). Techno-economic analysis for the deployment of PPDR services over 4G/4G+ Networks. IEEE International Conference on Telecommunications and Multimedia. Heraklion, Greece 25 - 27 Jul 2016 Institute of Electrical and Electronics Engineers (IEEE).
EMYNOS: Next Generation Emergency Communication
Markakis, E, Lykourgiotis, A, Politis, I, Dagiuklas, T. and Rebahi, Y (2016). EMYNOS: Next Generation Emergency Communication. IEEE Communications Magazine. 55 (1), pp. 139-145. https://doi.org/10.1109/MCOM.2017.1600284CM
Integrating IoT and Fog Computing for Healthcare Service Delivery
Andriopoulou, F, Dagiuklas, T. and Orphanoudakis, T (2016). Integrating IoT and Fog Computing for Healthcare Service Delivery. in: Keramidas, G, Voros, N and Hübner, M (ed.) Components and Services for IoT Platforms: Paving the Way for IoT Standards Springer. pp. 213-232
Wireless Information-Theoretic Security: Theoretical analysis & experimental measurements with multiple eavesdroppers in an outdoor obstacle-dense MANET
Chrysikos, T, Birkos, K, Dagiuklas, A. and Kotsopoulos, S (2016). Wireless Information-Theoretic Security: Theoretical analysis & experimental measurements with multiple eavesdroppers in an outdoor obstacle-dense MANET. Physical Communication. https://doi.org/10.1016/j.phycom.2016.11.003