Workshop: 3D Point Cloud Processing, Analysis, and Communication (PC-PAC)
Intro: One emerging representation of 3D world objects is point clouds where in addition to the spatial coordinates of points on the surface of the objects, attributes such as color, reflectance, transparency, normal direction, motion direction, and so forth are captured. Point clouds are receiving increased attention due to their potential to improve immersive experience in virtual reality, augmented reality, mixed reality, and telepresence applications. The aim of the workshop is to bring together researchers interested in recent developments in this field, stimulate discussions, and promote new approaches and solutions to current challenges. This workshop is supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 836192.
Organizer: Hui Yuan, Shandong University
Agenda:
15:40-16:10
High-Efficiency Point Cloud Compression: Learnt Approach & Distortion Measurement
Zhan Ma
16:10-16:40
Technique and standardization about point cloud
Yiling Xu
16:40-17:10
Video Encoder optimization for video-based point cloud compression
Li Li
17:10-17:40
Deep Regular Geometry Representations for 3D Point Clouds
Junhui Hou
Title: High-Efficiency Point Cloud Compression: Learnt Approach & Distortion Measurement
Speaker: Zhan Ma，Nanjing University
Abstract: This talk will cover two topics in recently-emerged point cloud processing: one is the learning driven point cloud geometry (PCG) compression and the other is graph based point cloud distortion measurement. For PCG compression, we start with the theoretical motivation and discuss the explorations along this avenue. Currently, our learnt PCG compressions offers state-of-the-art efficiency in comparision to the standard compliant solutions. As reported in many studies that existing point cloud distortion measurements are of great limitation due to its unstructured geometry appearance in 3D space, e.g., point-to-point, or point-to-plane, we introduce graph based similarity (GraphSIM) to address this problem. Extensive simulations have shown that our GraphSIM have demonstrated encouraging efficiency in different point cloud applications.
Biography: Zhan Mais with the school of Electronic Science and Engineering, Nanjing University, Jiangsu, 210093, China. He received the B.S. and M.S. from the Huazhong University of Science and Technology, Wuhan, China, in 2004 and 2006 respectively, and his Ph.D. degree from the New York University, New York, in 2011. From 2011 to 2014, he has been with Samsung Research America, Dallas, TX, and Futurewei Technologies, Inc., Santa Clara, CA, respectively. His research focuses include the learning-based image/video coding, and computational imaging. He is a co-winner of the 2018 PCM Best Paper Finalist, 2019 IEEE Broadcast Technology Society Best Paper Award, and 2020 IEEE MMSP Grand Challenge Best Image Coding Solution.
Title: Technique and standardization about point cloud
Speaker: Yiling Xu，Shanghai Jiaotong University
Abstract: As a typical presentation format for visual media, 3D point clouds have drawn wide attention from both industry and academia. Different from image or video pixels that are distributed evenly over the 2D grid, the structures of the point clouds exhibit high irregularity, which puts forward new requirements for related processing techniques. For point cloud compression, the well-known Moving Picture Experts Group (MPEG) establishes specific working groups, such as the Video-based point cloud compression (V-PCC) and Geometry-based point cloud compression (G-PCC), to develop efficient point cloud compression methods using data projection and octree-based data partition. As one of the earliest participants of MPEG-PCC, Shanghai Jiao Tong University (SJTU) carried out systematic study on point cloud compression and achieved SOTA performance. Meanwhile, the development of lossy compression techniques induces urgent demands for robust point cloud quality assessment. In this perspective, the SJTU group proposed projection-based, graph signal processing (GSP)-based and point potential energy-based models to realize effective and robust subjective perception prediction. These quality assessment models can also be used as loss functions to assist the generation of high-quality samples, in an unsupervised learning manner, in tasks such as point cloud reconstruction, shape completion, and upsampling.
Yiling Xu received the B.S., M.S. and Ph.D. from the University of Electronic Science and Technology of China, in 1999, 2001 and 2004 respectively. From 2004 to 2013, she has been with Samsung Electronics as a senior engineer in Multimedia Communication Research Institute of Samsung Electronics Inc, Korea. She joined the Shanghai Jiaotong University (SJTU) and is now a professor in the areas of multimedia communication, 3D point cloud compression and assessment, system design and network optimization. She is also an active member in standard organizations including MPEG, 3GPP, AVS.
Title: Video Encoder optimization for video-based point cloud compression
Speaker: Li Li，University of Science and Technology of China
Abstract: Moving pictures experts group (MPEG) video-based point cloud compression (V-PCC) achieves state-of-the-art performance for dynamic point cloud compression. The main idea of V-PCC is to use mature video coding technologies to compress point clouds. However, the current video coding technologies are mainly designed for natural videos, which limits the performance of the V-PCC. In this talk, I would like to introduce some video encoder optimization methods to further improve the performance of the V-PCC. First, I will give a very brief introduction about V-PCC. Second, to handle the non-corresponding patches between neighboring point cloud frames, I will introduce a 3D motion prediction method to find an accurate motion vector predictor for each patch. Third, to handle the unoccupied pixels in point cloud frames, I will introduce an occupancy-map-based rate-distortion optimization method to ignore distortions of the unoccupied pixels in V-PCC. Finally, to handle the boundaries between the occupied and unoccupied pixels, I will introduce an adaptive partition method to give a proper partition to each boundary block. The experimental results show that the proposed algorithm leads to significant BD-rate savings compared with the V-PCC anchor.
Li Li received the B.S. and Ph.D. degrees in electronic engineering from the University of Science and Technology of China (USTC), Hefei, Anhui, China, in 2011 and 2016, respectively. He is currently a research fellow in the University of Science and Technology of China. He was a visiting assistant professor in University of Missouri-Kansas City from 2016 to 2020. His research interests include image/video/point cloud compression and processing. He has 60+ publications in book chapters, journals, and conferences in these areas. He received the Top 10% Paper Award at the 2016 IEEE Visual Communications and Image Processing (VCIP) Conference and 2019 IEEE International Conference on Image Processing (ICIP) Conference. He was the winner of the 2016 IEEE International Conference on Multimedia and Expo Grand Challenge on Light Field Image Compression.
Title: Deep Regular Geometry Representations for 3D Point Clouds
Speaker: Junhui Hou，City University of Hong Kong
Abstract: Although convolutional neural networks have achieved remarkable success in analyzing 2D images/videos, it is still non-trivial to apply the well-developed 2D techniques in regular domains to the irregular 3D point cloud data. To bridge this gap, we propose ParaNet, a novel end-to-end deep learning framework, for representing 3D point clouds in a completely regular and nearly lossless manner. To be specific, ParaNet converts an irregular 3D point cloud into a regular 2D color image, named point geometry image (PGI), where each pixel encodes the spatial coordinates of a point. In contrast to conventional regular representation modalities based on multi-view projection and voxelization, the proposed representation is differentiable and reversible. The PGIs can be seamlessly coupled with a task network established upon standard and mature techniques for 2D images/videos to realize a specific task for 3D point clouds. We evaluate ParaNet over shape classification and point cloud upsampling, in which our solutions perform favorably against the existing state-of-the-art methods. We believe such a paradigm will open up many possibilities to advance the progress of deep learning-based point cloud processing and understanding.
Junhui Hou received the B.Eng. degree in information engineering (Talented Students Program) from the South China University of Technology, Guangzhou, China, in 2009, the M.Eng. degree in signal and information processing from Northwestern Polytechnical University, Xian, China, in 2012, and the Ph.D. degree in electrical and electronic engineering from the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, in 2016. He has been an Assistant Professor with the Department of Computer Science, City University of Hong Kong, since 2017. His research interests fall into the general areas of visual computing, such as image/video/3D geometry data representation, processing and analysis, semi/un-supervised data modeling, and data compression and adaptive transmission Dr. Hou was the recipient of several prestigious awards, including the Chinese Government Award for Outstanding Students Study Abroad from China Scholarship Council in 2015, and the Early Career Award (3/381) from the Hong Kong Research Grants Council in 2018. He is currently serving as an Associate Editor for IEEE Transactions on Circuits and Systems for Video Technology, The Visual Computer, an Area Editor for Signal Processing: Image Communication, the Guest Editor for the IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. He also served an Area Chair of ACM International Conference on Multimedia (ACM MM) 2019 and 2020, and IEEE International Conference on Multimedia & Expo (IEEE ICME) 2020. He is a senior member of IEEE.