| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 2.76 MB | Adobe PDF |
Advisor(s)
Abstract(s)
Quality of Experience (QoE) in multi-view streaming systems is known to be severely affected
by the latency associated with view-switching procedures. Anticipating the navigation intentions of the
viewer on the multi-view scene could provide the means to greatly reduce such latency. The research work
presented in this article builds on this premise by proposing a new predictive view-selection mechanism.
A VGG16-inspired Convolutional Neural Network (CNN) is used to identify the viewer’s focus of attention
and determine which views would be most suited to be presented in the brief term, i.e., the near-term viewing
intentions. This way, those views can be locally buffered before they are actually needed. To this aim, two
datasets were used to evaluate the prediction performance and impact on latency, in particular when compared
to the solution implemented in the previous version of our multi-view streaming system. Results obtained
with this work translate into a generalized improvement in perceived QoE. A significant reduction in latency
during view-switching procedures was effectively achieved. Moreover, results also demonstrated that the
prediction of the user’s visual interest was achieved with a high level of accuracy. An experimental platform
was also established on which future predictive models can be integrated and compared with previously
implemented models.
Description
Keywords
Multimedia; streaming; multi-view; focus-of-attention; deep learning
Citation
3. T. S. Costa, P. Viana and M. T. Andrade (2023). Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications, in IEEE Access, vol. 11, pp. 93883-93897, 2023, doi: 10.1109/ACCESS.2023.3310822