Loading...
28 results
Search Results
Now showing 1 - 10 of 28
- Reinforcement Learning for Scheduling Wireless Powered Sensor CommunicationsPublication . Li, Kai; Ni, Wei; Abolhasan, Mehran; Tovar, EduardoIn a wireless powered sensor network, a base station transfers power to sensors by using wireless power transfer (WPT). Inadequately scheduling WPT and data transmission causes fast battery drainage and data queue overflow of some sensors who could have potentially gained high data reception. In this paper, scheduling WPT and data transmission is formulated as a Markov decision process (MDP) by jointly considering sensors’ energy consumption and data queue. In practical scenarios, the prior knowledge about battery level and data queue length in MDP is not available at the base station. We study reinforcement learning at the sensors to find a transmission scheduling strategy, minimizing data packet loss. An optimal scheduling strategy with full-state information is also investigated, assuming that the complete battery level and data queue information are well known by the base station. This presents the lower bound of the data packet loss in wireless powered sensor networks. Numerical results demonstrate that the proposed reinforcement learning scheduling algorithm significantly reduces network packet loss rate by 60%, and increases network goodput by 67%, compared to existing non-MDP greedy approaches. Moreover, comparing the optimal solutions, the performance loss due to the lack of sensors’ full-state information is less than 4.6%.
- Proactive Eavesdropping via Jamming for Trajectory Tracking of UAVsPublication . Li, Kai; Kanhere, Salil S.; Ni, Wei; Tovar, Eduardo; Guizani, MohsenThis paper considers that a legitimate UAV tracks suspicious UAVs’ flight for preventing intended crimes and terror attacks. To enhance tracking accuracy, the legitimate UAV proactively eavesdrops suspicious UAVs’ communication via sending jamming signals. A tracking algorithm is developed for the legitimate UAV to track the suspicious flight by comprehensively utilizing eavesdropped packets, angle-of-arrival and received signal strength of the suspicious transmitter’s signal. A new co-simulation framework is implemented to combine the complementary features of optimization toolbox with channel modeling (in Matlab) and discrete event-driven mobility tracking (in NS3). Moreover, numerical results validate the proposed algorithms in terms of tracking accuracy of the suspicious UAVs’ trajectory
- LSTM-characterized Deep Reinforcement Learning for Continuous Flight Control and Resource Allocation in UAV-assisted Sensor NetworkPublication . Li, Kai; Ni, Wei; Dressler, FalkoUnmanned aerial vehicles (UAVs) can be employed to collect sensory data in remote wireless sensor networks (WSN). Due to UAV's maneuvering, scheduling a sensor device to transmit data can overflow data buffers of the unscheduled ground devices. Moreover, lossy airborne channels can result in packet reception errors at the scheduled sensor. This paper proposes a new deep reinforcement learning based flight resource allocation framework (DeFRA) to minimize the overall data packet loss in a continuous action space. DeFRA is based on Deep Deterministic Policy Gradient (DDPG), optimally controls instantaneous headings and speeds of the UAV, and selects the ground device for data collection. Furthermore, a state characterization layer, leveraging long short-term memory (LSTM), is developed to predict network dynamics, resulting from time-varying airborne channels and energy arrivals at the ground devices. To validate the effectiveness of DeFRA, experimental data collected from a real-world UAV testbed and energy harvesting WSN are utilized to train the actions of the UAV. Numerical results demonstrate that the proposed DeFRA achieves a fast convergence while reducing the packet loss by over 15%, as compared to existing deep reinforcement learning solutions.
- Deep Reinforcement Learning for Persistent Cruise Control in UAV-aided Data CollectionPublication . Kurunathan, John Harrison; Li, Kai; Ni, Wei; Tovar, Eduardo; Dressler, FalkoAutonomous UAV cruising is gaining attention dueto its flexible deployment in remote sensing, surveillance, andreconnaissance. A critical challenge in data collection with theautonomous UAV is the buffer overflows at the ground sensorsand packet loss due to lossy airborne channels. Trajectoryplanning of the UAV is vital to alleviate buffer overflows as wellas channel fading. In this work, we propose a Deep DeterministicPolicy Gradient based Cruise Control (DDPG-CC) to reducethe overall packet loss through online training of headings andcruise velocity of the UAV, as well as the selection of the groundsensors for data collection. Preliminary performance evaluationdemonstrates that DDPG-CC reduces the packet loss rate byunder 5% when sufficient training is provided to the UAV.
- Onboard Deep Deterministic Policy Gradients for Online Flight Resource Allocation of UAVsPublication . Li, Kai; Emami, Yousef; Ni, Wei; Tovar, Eduardo; Han, ZhuIn Unmanned Aerial Vehicle (UAV) enabled data collection, scheduling data transmissions of the ground nodes while controlling flight of the UAV, e.g., heading and velocity, is critical to reduce the data packet loss resulting from buffer overflows and channel fading. In this letter, a new online flight resource allocation scheme based on deep deterministic policy gradients (DDPG-FRAS) is studied to jointly optimize the flight control of the UAV and data collection scheduling along the trajectory in real time, thereby asymptotically minimizing the packet loss of the ground sensor networks. Numerical results confirm that the proposed DDPG-FRAS can gradually converge, while enlarging the buffer size can reduce the packet loss by 47.9%.
- Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary SurveyPublication . Kurunathan, John Harrison; Li, Kai; Ni, WeiOver the past decade, Unmanned Aerial Vehicles (UAVs) have provided pervasive, efficient, and cost-effective solutions for data collection and communications. Their excellent mobility, flexibility, and fast deployment enable UAVs to be extensively utilized in agriculture, medical, rescue missions, smart cities, and intelligent transportation systems. Machine learning (ML) has been increasingly demonstrating its capability of improving the automation and operation precision of UAVs and many UAV-assisted applications, such as communications, sensing, and data collection. The ongoing amalgamation of UAV and ML techniques is creating a significant synergy and empowering UAVs with unprecedented intelligence and autonomy. This survey aims to provide a timely and comprehensive overview of ML techniques used in UAV operations and communications and identify the potential growth areas and research gaps. We emphasize the four key components of UAV operations and communications to which ML can significantly contribute, namely, perception and feature extraction, feature interpretation and regeneration, trajectory and mission planning, and aerodynamic control and operation. We classify the latest popular ML tools based on their applications to the four components and conduct gap analyses. This survey also takes a step forward by pointing out significant challenges in the upcoming realm of ML-aided automated UAV operations and communications. It is revealed that different ML techniques dominate the applications to the four key modules of UAV operations and communications. While there is an increasing trend of cross-module designs, little effort has been devoted to an end-to-end ML framework, from perception and feature extraction to aerodynamic control and operation. It is also unveiled that the reliability and trust of ML in UAV operations and applications require significant attention before the full automation of UAVs and potential cooperation between UAVs and humans come to fruition.
- Employing Intelligent Aerial Data Aggregators for Internet of Things: Challenges and SolutionsPublication . Li, Kai; Ni, W.; Noor, Alam; Guizani, MohsenInternet-of-Things (IoT) devices equipped with temperature and humidity sensors, and cameras are increasingly deployed to monitor remote and human-unfriendly areas, e.g., farmlands, forests, rural highways or electricity infrastructures. Aerial data aggregators, e.g., autonomous drones, provide a promising solution for collecting sensory data of the IoT devices in human-unfriendly environments, enhancing network scalability and connectivity. The flexibility of a drone and favourable line-of-sight connection between the drone and IoT devices can be exploited to improve data reception at the drone. This article first discusses challenges of the drone-assisted data aggregation in IoT networks, such as incomplete network knowledge at the drone, limited buffers of the IoT devices, and lossy wireless channels. Next, we investigate the feasibility of onboard deep reinforcement learning-based solutions to allow a drone to learn its cruise control and data collection schedule online. For deep reinforcement learning in a continuous operation domain, deep deterministic policy gradient (DDPG) is suitable to deliver effective joint cruise control and communication decision, using its outdated knowledge of the IoT devices and network states. A case study shows that the DDPG-based framework can take advantage of the continuous actions to substantially outperform existing non-learning-based alternatives.
- When Internet of Things meets Metaverse: Convergence of Physical and Cyber WorldsPublication . Li, Kai; Cui, Yingping; Li, Weicai; Lv, Tiejun; Yuan, Xin; Li, Shenghong; Ni, Wei; Simsek, Meryem; Dressler, FalkoIn recent years, the Internet of Things (IoT) is studied in the context of the Metaverse to provide users immersive cyber-virtual experiences in mixed reality environments. This survey introduces six typical IoT applications in the Metaverse, including collaborative healthcare, education, smart city, entertainment, real estate, and socialization. In the IoT-inspired Metaverse, we also comprehensively survey four pillar technologies that enable augmented reality (AR) and virtual reality (VR), namely, responsible artificial intelligence (AI), high-speed data communications, cost-effective mobile edge computing (MEC), and digital twins. According to the physical-world demands, we outline the current industrial efforts and seven key requirements for building the IoT-inspired Metaverse: immersion, variety, economy, civility, interactivity, authenticity, and independence. In addition, this survey describes the open issues in the IoT-inspired Metaverse, which need to be addressed to eventually achieve the convergence of physical and cyber worlds.
- An Experimental Localization Testbed based on UWB Channel Impulse Response MeasurementsPublication . Li, Kai; Ni, Wei; Zhang, PeiIn this paper, we demonstrate a new ultra-wideband (UWB) localization testbed, which tracks a UWB tag and estimates locations of obstacles based on channel impulse response measurements. Anchor nodes that are developed with off-the-shelf Decawave DW1000 UWB transceivers are deployed to cover the area of interest. The testbed is implemented and preliminary experiments are carried out to estimate the location of the object by analyzing channel impulse response strength of the UWB tag.
- Federated Learning for Energy-balanced Client Selection in Mobile Edge ComputingPublication . Zheng, Jingjing; Li, Kai; Tovar, Eduardo; Guizani, MohsenMobile edge computing (MEC) has been considered as a promising technology to provide seamless integration of multiple application services. Federated learning (FL) is carried out at edge clients in MEC for privacy-preserving training of data processing models. Despite that the edge clients with small data payloads consume less energy on FL training, the small data payload gives rise to a low learning accuracy due to insufficient input to the FL training. Inadequate selection of the edge clients can result in a large energy consumption at the edge clients, or a low learning accuracy of the FL training. In this paper, a new FL-based client selection optimization is proposed to balance the trade-off between energy consumption of the edge clients and the learning accuracy of FL. We first show that this optimization problem is NP-complete. Next, we propose a FL-based energy-accuracy balancing heuristic algorithm to approximate the optimal client selection in polynomial time. The numerical results show the advantage of our proposed algorithm.
- «
- 1 (current)
- 2
- 3
- »