Loading...
3 results
Search Results
Now showing 1 - 3 of 3
- Reinforcement Learning for Scheduling Wireless Powered Sensor CommunicationsPublication . Li, Kai; Ni, Wei; Abolhasan, Mehran; Tovar, EduardoIn a wireless powered sensor network, a base station transfers power to sensors by using wireless power transfer (WPT). Inadequately scheduling WPT and data transmission causes fast battery drainage and data queue overflow of some sensors who could have potentially gained high data reception. In this paper, scheduling WPT and data transmission is formulated as a Markov decision process (MDP) by jointly considering sensors’ energy consumption and data queue. In practical scenarios, the prior knowledge about battery level and data queue length in MDP is not available at the base station. We study reinforcement learning at the sensors to find a transmission scheduling strategy, minimizing data packet loss. An optimal scheduling strategy with full-state information is also investigated, assuming that the complete battery level and data queue information are well known by the base station. This presents the lower bound of the data packet loss in wireless powered sensor networks. Numerical results demonstrate that the proposed reinforcement learning scheduling algorithm significantly reduces network packet loss rate by 60%, and increases network goodput by 67%, compared to existing non-MDP greedy approaches. Moreover, comparing the optimal solutions, the performance loss due to the lack of sensors’ full-state information is less than 4.6%.
- Deep Reinforcement Learning for Real-Time Trajectory Planning in UAV NetworkPublication . Li, Kai; Ni, Wei; Tovar, Eduardo; Guizani, MohsenIn Unmanned Aerial Vehicle (UAV)-enabled wireless powered sensor networks, a UAV can be employed to charge the ground sensors remotely via Wireless Power Transfer (WPT) and collect the sensory data. This paper focuses on trajectory planning of the UAV for aerial data collection and WPT to minimize buffer overflow at the ground sensors and unsuccessful transmission due to lossy airborne channels. Consider network states of battery levels and buffer lengths of the ground sensors, channel conditions, and location of the UAV. A flight trajectory planning optimization is formulated as a Partial Observable Markov Decision Process (POMDP), where the UAV has partial observation of the network states. In practice, the UAV-enabled sensor network contains a large number of network states and actions in POMDP while the up-to-date knowledge of the network states is not available at the UAV. To address these issues, we propose an onboard deep reinforcement learning algorithm to optimize the realtime trajectory planning of the UAV given outdated knowledge on the network states.
- Deep Q-Learning based Resource Management in UAV-assisted Wireless Powered IoT NetworksPublication . Li, Kai; Ni, Wei; Tovar, Eduardo; Jamalipour, AbbasIn Unmanned Aerial Vehicle (UAV)-assisted Wireless Powered Internet of Things (IoT), the UAV is employed to charge the IoT nodes remotely via Wireless Power Transfer (WPT) and collect their data. A key challenge of resource management for WPT and data collection is preventing battery drainage and butter overflow of the ground IoT nodes in the presence of highly dynamic airborne channels. In this paper, we consider the resource management problem in practical scenarios, where the UAV has no a-prior information on battery levels and data queue lengths of the nodes. We formulate the resource management of UAV-assisted WPT and data collection as Markov Decision Process (MDP), where the states consist of battery levels and data queue lengths of the IoT nodes, channel qualities, and positions of the UAV. A deep Q-learning based resource management is proposed to minimize the overall data packet loss of the IoT nodes, by optimally deciding the IoT node for data collection and power transfer, and the associated modulation scheme of the IoT node.