Loading...
15 results
Search Results
Now showing 1 - 10 of 15
- Buffer-Aware Scheduling for UAV Relay Networks with Energy FairnessPublication . Emami, Yousef; Li, Kai; Tovar, EduardoFor assisting data communications in human-unfriendly environments, Unmanned Aerial Vehicles (UAVs) are employed to relay data for ground sensors thanks to UAVs' flexible deployment, high mobility, and line-of-sight communications. In UAV relay networks, energy efficient data relay is critical due to limited battery of the ground sensing devices. In this paper, we propose a butter-aware transmission scheduling optimization to minimize the energy consumption of the ground devices under constraints of butter overflows and energy cost fairness on the ground devices. Moreover, we show that the problem is NP-complete and propose a heuristic algorithm to approximate the optimal scheduling solution in polynomial time. The performance of the proposed algorithm is evaluated in terms of network sizes, packet arrival rates, and fairness of the energy consumption. Numerical results confirm that the proposed scheduling algorithm reduces the energy consumption of the ground devices in a fair fashion, while the butter overflow constraint holds.
- Deep Reinforcement Learning for Persistent Cruise Control in UAV-aided Data CollectionPublication . Kurunathan, John Harrison; Li, Kai; Ni, Wei; Tovar, Eduardo; Dressler, FalkoAutonomous UAV cruising is gaining attention dueto its flexible deployment in remote sensing, surveillance, andreconnaissance. A critical challenge in data collection with theautonomous UAV is the buffer overflows at the ground sensorsand packet loss due to lossy airborne channels. Trajectoryplanning of the UAV is vital to alleviate buffer overflows as wellas channel fading. In this work, we propose a Deep DeterministicPolicy Gradient based Cruise Control (DDPG-CC) to reducethe overall packet loss through online training of headings andcruise velocity of the UAV, as well as the selection of the groundsensors for data collection. Preliminary performance evaluationdemonstrates that DDPG-CC reduces the packet loss rate byunder 5% when sufficient training is provided to the UAV.
- Proactive Eavesdropping via Jamming for Trajectory Tracking of UAVsPublication . Li, Kai; Kanhere, Salil S.; Ni, Wei; Tovar, Eduardo; Guizani, MohsenThis paper considers that a legitimate UAV tracks suspicious UAVs’ flight for preventing intended crimes and terror attacks. To enhance tracking accuracy, the legitimate UAV proactively eavesdrops suspicious UAVs’ communication via sending jamming signals. A tracking algorithm is developed for the legitimate UAV to track the suspicious flight by comprehensively utilizing eavesdropped packets, angle-of-arrival and received signal strength of the suspicious transmitter’s signal. A new co-simulation framework is implemented to combine the complementary features of optimization toolbox with channel modeling (in Matlab) and discrete event-driven mobility tracking (in NS3). Moreover, numerical results validate the proposed algorithms in terms of tracking accuracy of the suspicious UAVs’ trajectory
- Online Velocity Control and Data Capture of Drones for the Internet of Things: An Onboard Deep Reinforcement Learning ApproachPublication . Li, Kai; Ni, Wei; Tovar, Eduardo; Jamalipour, AbbasApplications of unmanned aerial vehicles (UAVs) for data collection are a promising means to extend Internet of Things (IoT) networks to remote and hostile areas and to locations where there is no access to power supplies. The adequate design of UAV velocity control and communication decision making is critical to minimize the data packet losses at ground IoT nodes that result from overflowing buffers and transmission failures. However, online velocity control and communication decision making are challenging in UAV-enabled IoT networks, due to a UAV?s lack of up-to-date knowledge about the state of the nodes, e.g., the battery energy, buffer length, and channel conditions.
- Reinforcement Learning for Scheduling Wireless Powered Sensor CommunicationsPublication . Li, Kai; Ni, Wei; Abolhasan, Mehran; Tovar, EduardoIn a wireless powered sensor network, a base station transfers power to sensors by using wireless power transfer (WPT). Inadequately scheduling WPT and data transmission causes fast battery drainage and data queue overflow of some sensors who could have potentially gained high data reception. In this paper, scheduling WPT and data transmission is formulated as a Markov decision process (MDP) by jointly considering sensors’ energy consumption and data queue. In practical scenarios, the prior knowledge about battery level and data queue length in MDP is not available at the base station. We study reinforcement learning at the sensors to find a transmission scheduling strategy, minimizing data packet loss. An optimal scheduling strategy with full-state information is also investigated, assuming that the complete battery level and data queue information are well known by the base station. This presents the lower bound of the data packet loss in wireless powered sensor networks. Numerical results demonstrate that the proposed reinforcement learning scheduling algorithm significantly reduces network packet loss rate by 60%, and increases network goodput by 67%, compared to existing non-MDP greedy approaches. Moreover, comparing the optimal solutions, the performance loss due to the lack of sensors’ full-state information is less than 4.6%.
- Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor NetworksPublication . Emami, Yousef; Wei, Bo; Li, Kai; Ni, Wei; Tovar, EduardoUnmanned Aerial Vehicles (UAVs) can collaborate to collect and relay data for ground sensors in remote and hostile areas. In multi-UAV-assisted wireless sensor networks (MA-WSN), the UAVs' movements impact on channel condition and can fail data transmission, this situation along with newly arrived data give rise to buffer overflows at the ground sensors. Thus, scheduling data transmission is of utmost importance in MA-WSN to reduce data packet losses resulting from buffer overflows and channel fading. In this paper, we investigate the optimal ground sensor selection at the UAVs to minimize data packet losses . The optimization problem is formulated as a multiagent Markov decision process, where network states consist of battery levels and data buffer lengths of the ground sensor, channel conditions, and waypoints of the UAV along the trajectory. In practice, an MA-WSN contains a large number of network states, while the up-to-date knowledge of the network states and other UAVs' sensor selection decisions is not available at each agent. We propose a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MUAIS) to minimize the data packet loss, where the UAVs learn the underlying patterns of the data and energy arrivals at all the ground sensors. Numerical results show that the proposed MUAIS achieves at least 46\% and 35\% lower packet loss than an optimal solution with single-UAV and an existing non-learning greedy algorithm, respectively.
- Deep Reinforcement Learning for Real-Time Trajectory Planning in UAV NetworkPublication . Li, Kai; Ni, Wei; Tovar, Eduardo; Guizani, MohsenIn Unmanned Aerial Vehicle (UAV)-enabled wireless powered sensor networks, a UAV can be employed to charge the ground sensors remotely via Wireless Power Transfer (WPT) and collect the sensory data. This paper focuses on trajectory planning of the UAV for aerial data collection and WPT to minimize buffer overflow at the ground sensors and unsuccessful transmission due to lossy airborne channels. Consider network states of battery levels and buffer lengths of the ground sensors, channel conditions, and location of the UAV. A flight trajectory planning optimization is formulated as a Partial Observable Markov Decision Process (POMDP), where the UAV has partial observation of the network states. In practice, the UAV-enabled sensor network contains a large number of network states and actions in POMDP while the up-to-date knowledge of the network states is not available at the UAV. To address these issues, we propose an onboard deep reinforcement learning algorithm to optimize the realtime trajectory planning of the UAV given outdated knowledge on the network states.
- On-Board Deep Q-Network for UAV-Assisted Online Power Transfer and Data CollectionPublication . Li, Kai; Ni, Wei; Tovar, Eduardo; Jamalipour, AbbasUnmanned Aerial Vehicles (UAVs) with Microwave Power Transfer (MPT) capability provide a practical means to deploy a large number of wireless powered sensing devices into areas with no access to persistent power supplies. The UAV can charge the sensing devices remotely and harvest their data. A key challenge is online MPT and data collection in the presence of on-board control of a UAV (e.g., patrolling velocity) for preventing battery drainage and data queue overflow of the devices, while up-to-date knowledge on battery level and data queue of the devices is not available at the UAV. In this paper, an on-board deep Q-network is developed to minimize the overall data packet loss of the sensing devices, by optimally deciding the device to be charged and interrogated for data collection, and the instantaneous patrolling velocity of the UAV. Specifically, we formulate a Markov Decision Process (MDP) with the states of battery level and data queue length of devices, channel conditions, and waypoints given the trajectory of the UAV; and solve it optimally with Q-learning. Furthermore, we propose the on-board deep Q-network that enlarges the state space of the MDP, and a deep reinforcement learning based scheduling algorithm that asymptotically derives the optimal solution online, even when the UAV has only outdated knowledge on the MDP states. Numerical results demonstrate that our deep reinforcement learning algorithm reduces the packet loss by at least 69.2%, as compared to existing non-learning greedy algorithms.
- Cooperative Secret Key Generation for Platoon-Based Vehicular CommunicationsPublication . Li, Kai; Lu, Lingyun; Ni, Wei; Tovar, Eduardo; Guizani, MohsenIn a vehicular platoon, the lead vehicle that is responsible for managing the platoon's moving directions and velocity periodically disseminates messages to the following automated vehicles in a multi-hop vehicular network. However, due to the broadcast nature of wireless channels, vehicle-to-vehicle (V2V) communications are vulnerable to eavesdropping and message modification. Generating secret keys by extracting the shared randomness in a wireless fading channel is a promising way for V2V communication security. We study a security scheme for platoon-based V2V communications, where the platooning vehicles generate a shared secret key based on the quantized fading channel randomness. To improve conformity of the generated key, the probability of secret key agreement is formulated, and a novel secret key agreement algorithm is proposed to recursively optimize the channel quantization intervals, maximizing the key agreement probability. Numerical evaluations demonstrate that the key agreement probability achieved by our security protocol given different platoon size, channel quality, and number of quantization intervals. Furthermore, by applying our security protocol, it is shown that the probability that the encrypted data being cracked by an eavesdropper is less than 5%.
- Onboard Deep Deterministic Policy Gradients for Online Flight Resource Allocation of UAVsPublication . Li, Kai; Emami, Yousef; Ni, Wei; Tovar, Eduardo; Han, ZhuIn Unmanned Aerial Vehicle (UAV) enabled data collection, scheduling data transmissions of the ground nodes while controlling flight of the UAV, e.g., heading and velocity, is critical to reduce the data packet loss resulting from buffer overflows and channel fading. In this letter, a new online flight resource allocation scheme based on deep deterministic policy gradients (DDPG-FRAS) is studied to jointly optimize the flight control of the UAV and data collection scheduling along the trajectory in real time, thereby asymptotically minimizing the packet loss of the ground sensor networks. Numerical results confirm that the proposed DDPG-FRAS can gradually converge, while enlarging the buffer size can reduce the packet loss by 47.9%.