Séminaire du 12 février 2024

Beyond Pixels and 3D point Clouds: Exploring Deep Learning for Semantic Image Segmentation and 3D Point Cloud Quality Assessment

Ayoub Karine, ISEN Yncréa Ouest, Nantes

Résumé :In this seminar, I will discuss some of our contributions centered around deep learning, specifically addressing two computer vision tasks. The first task will be on semantic image segmentation, a task aiming to assign a category label to each pixel in input images. While deep learning methods have demonstrated impressive performance in semantic segmentation, their computationally intensive nature poses challenges for deployment on resource-limited devices. To overcome this weakness, various methods, including Knowledge Distillation (KD), have been proposed. KD involves training a compacted neural network called student under the supervision of a heavy one called teacher. The challenge is to find, for a student network, a better tradeoff between the segmentation quality and its efficiency. In this context, I will present some of our proposed KD methods that leverage self-attention principles to capture contextual dependencies in both spatial and channel dimensions. For the second task, it concerns 3D point cloud quality assessment (PCQA). Recently, PC is considered one of the most widely used data for digital representation to model 3D realistic content in various applications. During the processing pipeline (acquisition, representation, compression, and rendering), various degradation may appear in PCs which affect its perceived visual quality. Consequently, it is essential to develop effective methods that accurately assess the quality of PCs and preserve the quality of the user experience. Unlike 2D images, PCs are unordered and unstructured, posing a challenge for direct application of convolutional neural networks using discrete convolutions. To circumvent this limitation, I will discuss some of our proposed PCQA metrics categorized into projection-based and point-based methods.