The primary goal of our research is to develop algorithms and techniques that can help create and enhance visual contents such as images and videos. To this end, we study various research problems that include image/video restoration and enhancement, human-computer interaction, 3D reconstruction, and various other graphics applications such as 2D/3D caricatures. We also actively study deep learning algorithms to create more intelligent solutions to these problems. We value the transfer of academic results to industry. Our technologies on image deblurring and photo upright adjustment have been transferred to Adobe Creative Cloud and Adobe Photoshop Lightroom.


3D scene reconstruction

  • RGB-D image processing

With the improvement of computer graphics, humans have tried to construct 3D geometry of an actual object from photos. These reconstructed objects can be used in film or commercial industry for immersive and realistic computer graphics effects. We study 3D reconstruction to producing accurate 3D geometry and texture of real objects. Furthermore, with the advent of Microsoft Kinect, the public attention of RGB-D image, which includes color and depth information, increases, we also try to extract useful information from RGB-D image.

Related area: RGB-D images processing, Texture optimization, Non-rigid 3D reconstruction


Image and video processing

  • Upright adjustment and video stabilization

Image and video are the most intuitive information medium for humans. Humans use millions of images and videos every day and are familiar with them. However, we sometimes are not satisfied with the raw image and video. We want to recognize people or cars in the image of a surveillance camera or remove the shakiness of the video. That is, we want to extract useful information from images or videos, and modify them for better recognition. In this area, we study the essence of images and videos to get useful information.

Related area: Upright adjustment, Video stabilization, Instance segmentation, defocus estimation, Deblurring


Computational photography with big visual data 

  • Super resolution

Making a beautiful and moving image is a humans desire. People from ancient to modern age studied what makes touching images. With the big visual data and deep learning technology, we decided to enhance the beauty of images. Specifically, we analyze the essence of visual beauty from big visual data and train the deep neural network which produces the pleasing images

Related area: Super-resolution, Color enhancement, Photo composition enhancement, Photo aesthetic analysis


New media application 

  • 360 image and video upright

There are many specialized cameras that have a different goal and physical components such as a lens or sensor. One of the specialized cameras is a 360 camera. The 360 camera has 360° horizontal FoV and 180° vertical FoV so that it can take a picture including every object around the camera. A picture taken by 360 camera has different visual property from an ordinary image, so we research the 360 camera to make pleasing 360 images and videos

Related area: 360 image and video upright, 360 video navigation for virtual reality