1. summary image

    Depth Completion with TWIN Surface Extrapolation at Occlusion Boundaries

    Saif Imran, Xiaoming Liu, Daniel Morris

    Depth completion starts from a sparse set of known depth values and estimates the unknown depths for the remaining image pixels. Most methods model this as depth interpolation and erroneously interpolate depth pixels into the empty space between spatially distinct objects, resulting in depth-smearing across occlusion boundaries. Here we propose ...

    Continue reading

    Keywords: Depth Completion, Sensor Fusion

  2. summary image

    Towards High Fidelity Face Relighting with Realistic Shadows

    Andrew Hou, Ze Zhang, Michel Sarkis, Ning Bi, Yiying Tong, Xiaoming Liu

    Existing face relighting methods often struggle with two problems: maintaining the local facial details of the subject and accurately removing and synthesizing shadows in the relit image, especially hard shadows. We propose a novel deep face relighting method that addresses both problems. Our method learns to predict the ratio (quotient ...

    Continue reading

    Keywords: Face Relighting

  3. summary image

    GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection

    Abhinav Kumar, Garrick Brazil, Xiaoming Liu

    Modern 3D object detectors have immensely benefited from the end-to-end learning idea. However, most of them use a post-processing algorithm called Non-Maximal Suppression (NMS) only during inference. While there were attempts to include NMS in the training pipeline for tasks such as 2D object detection, they have been less widely ...

    Continue reading

    Keywords: 3D Object Detection

  4. summary image

    Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction

    Feng Liu, Luan Tran, Xiaoming Liu

    Inferring 3D structure of a generic object from a 2D image is a long-standing objective of computer vision. Conventional approaches either learn completely from CAD-generated synthetic data, which have difficulty in inference from real images, or generate 2.5D depth image via intrinsic decomposition, which is limited compared to the ...

    Continue reading

    Keywords: 3D Shape Reconstruction

  5. summary image

    Face Anti-spoofing, Face Presentation Attack Detection

    Yaojie Liu, Jeol Stehouwer, Amin Jourabloo, Yousef Atoum, Xiaoming Liu

    Biometrics utilize physiological, such as fingerprint, face, and iris, or behavioral characteristics, such as typing rhythm and gait, to uniquely identify or authenticate an individual. As biometric systems are widely used in real-world applications including mobile phone authentication and access control, biometric spoof, or Presentation Attack (PA) are becoming a ...

    Continue reading

    Keywords: Face antispoofing

  6. summary image

    Kinematic 3D Object Detection in Monocular Video

    Garrick Brazil, Gerard Pons-Moll, Xiaoming Liu, Bernt Schiele

    Perceiving the physical world in 3D is fundamental for selfdriving applications. Although temporal motion is an invaluable resource to human vision for detection, tracking, and depth perception, such features have not been thoroughly utilized in modern 3D object detectors. In this work, we propose a novel method for monocular video-based ...

    Continue reading

    Keywords: 3D Object Detection, Video

  7. summary image

    Facial Forgery Detection

    Hao Dang*, Feng Liu*, Joel Stehouwer*, Xiaoming Liu, Anil Jain

    The prevalence of facial recognition, biometric unlock, and social media presents a significant opportunity for bad actors to introduce forged or manipulated images to spread false information or damage reputations. This is aided by the continuing improvement in realistic image synthesis and manipulation by generative adversarial network, GAN, based methods ...

    Continue reading

    Keywords: Deepfake, Image forgery

  8. summary image

    Generic Object Sensor and Spoof Noise Classification, Modeling, and Synthesis

    Joel Stehouwer, Amin Jourabloo, Yaojie Liu, Xiaoming Liu

    Biometric recognition is increasingly used in commercial and high-security settings. Because of this, the threat of spoofing techniques, the act of presenting a fake biometric object to a sensor, is a large concern. Recent research has focused on face, fingerprint, and iris anti-spoofing. However, no other research attempts to use ...

    Continue reading

    Keywords: Anti-Spoofing, Noise Synthesis

  9. summary image

    The Edge of Depth: Explicit Constraints between Segmentation and Depth

    Shengjie Zhu, Garrick Brazil, Xiaoming Liu

    In this work we study the mutual benefits of two common computer vision tasks, self-supervised depth estimation and semantic segmentation from images. For example, to help unsupervised monocular depth estimation, constraints from semantic segmentation has been explored implicitly such as sharing and transforming features. In contrast, we propose to explicitly ...

    Continue reading

    Keywords: Depth Prediction

  10. summary image

    M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

    Garrick Brazil, Xiaoming Liu

    Understanding the world in 3D is a critical component of urban autonomous driving. Generally, the combination of expensive LiDAR sensors and stereo RGB imaging has been paramount for successful 3D object detection algorithms, whereas monocular image-only methods experience drastically reduced performance. We propose to reduce the gap by reformulating the ...

    Continue reading

    Keywords: 3D Object Detection

  11. summary image

    Pedestrian Detection with Autoregressive Network Phases

    Garrick Brazil, Xiaoming Liu

    We present an autoregressive pedestrian detection framework with cascaded phases designed to progressively improve precision. The proposed framework utilizes a novel lightweight stackable decoder-encoder module which uses convolutional re-sampling layers to improve features while maintaining efficient memory and runtime cost. Unlike previous cascaded detection systems, our proposed framework is designed ...

    Continue reading

    Keywords: Pedestrian Detection

  12. summary image

    Depth Coefficients for Depth Completion

    Saif Imran, Yunfei Long, Xiaoming Liu, Daniel Morris

    Depth completion involves estimating a dense depth image from sparse depth measurements, often guided by a color image. While linear upsampling is straight forward, it results in artifacts including depth pixels being interpolated in empty space across discontinuities between objects. Current methods use deep networks to upsample and "complete" the ...

    Continue reading

    Keywords: Depth Completion, 3D Object Detection

  13. summary image

    Gait Recognition via Disentangled Representation Learning

    Ziyuan Zhang, Luan Tran, Xi Yin, Yousef Atoum, Xiaoming Liu, Jian Wan, Nanxin Wang

    Gait, the walking pattern of individuals, is one of the most important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as the gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and view angle. To ...

    Continue reading

    Keywords: Gait Recognition

  14. summary image

    Towards Interpretable Face Recognition

    Bangjie Yin, Luan Tran, Haoxiang Li, Xiaohui Shen, Xiaoming Liu

    Deep CNNs have been pushing the frontier of visual recognition over past years. Besides recognition accuracy, strong demands in understanding deep CNNs in the research community motivate developments of tools to dissect pre-trained models to visualize how they make predictions. Recent works further push the interpretability in the network learning ...

    Continue reading

    Keywords: Face recognition

  15. summary image

    MSU-AVIS dataset: Fusing Face and Voice Modalities for Biometric Recognition in Indoor Surveillance Videos

    Anurag Chowdhury, Yousef Atoum, Luan Tran, Xiaoming Liu, Arun Ross

    Indoor video surveillance systems often use the face modality to establish the identity of a person of interest. However, the face image may not offer sufficient discriminatory information in many scenarios due to substantial variations in pose, illumination, expression, resolution and distance between the subject and the camera.

    In such ...

    Continue reading

    Keywords: Application

  16. summary image

    Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild

    Luan Tran, Kihyuk Sohn, Xiang Yu, Xiaoming Liu, Manmohan Chandraker

    Recent developments in deep domain adaptation have allowed knowledge transfer from a labeled source domain to an unlabeled target domain at the level of intermediate features and input pixels. We propose that advantages may be derived by jointly investigating the two, in the form of different level insights that lead ...

    Continue reading

    Keywords: Domain Adaptation

  17. summary image

    Feature Transfer Learning for Deep Face Recognition with Long-Tail Data

    Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

    Real-world face recognition datasets exhibit long-tail characteristics, which results in biased classifiers in conventionally-trained deep neural networks, or insufficient data when long-tail classes are ignored. In this paper, we propose to handle long-tail classes in the training of a face recognition engine by augmenting their feature space under a center-based ...

    Continue reading

    Keywords: Face Recognition

  18. summary image

    Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition

    Xi Yin, Xiaoming Liu

    This work explores Multi-Task Learning (MTL) for face recognition. First, we propose a multi-task Convolutional Neural Network (CNN) for face recognition where identity classification is the main task and Pose, Illumination, and Expression (PIE) estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss ...

    Continue reading

    Keywords: Face Recognition

  19. summary image

    Monocular Video-Based Trailer Coupler Detection using Multiplexer Convolutional Neural Network

    Yousef Atoum, Joseph Roth, Michael Bliss, Wende Zhang, Xiaoming Liu

    This paper presents a monocular camera-based computer vision system for autonomous selfbacking-up a vehicle towards a trailer, by continuously estimating the 3D trailer coupler position and feeding it to the vehicle control system, until the alignment of the tow hitch with the trailers coupler. This system is made possible through ...

    Continue reading

    Keywords: Application

  20. summary image

    Disentangled Representation Learning GAN for Pose-Invariant Face Recognition

    Luan Tran, Xi Yin, Xiaoming Liu

    The large pose discrepancy between two face images is one of the fundamental challenges in automatic face recognition. Conventional approaches to pose-invariant face recognition either perform face frontalization on, or learn a pose-invariant representation from, a non-frontal face image. We argue that it is more desirable to perform both tasks ...

    Continue reading

    Keywords: Face Recognition, Face Reconstruction

  21. summary image

    Towards Large-Pose Face Frontalization in the Wild

    Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

    Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments. Learning pose-invariant features is one solution, but needs expensively labeled large scale data and carefully designed feature learning algorithms. In this work, we focus on frontalizing faces in the ...

    Continue reading

    Keywords: Face Recognition, Face Reconstruction

  22. summary image

    Spatio-Temporal Alignment of Non-Overlapping Sequences from Independently Panning Cameras

    Seyed Morteza Safdarnejad, Xiaoming Liu

    We tackle a novel scenario of spatio-temporal alignment of seuqences referred to as Nonoverlapping Sequences (NOS). NOS are captured by multiple freely panning handheld cameras whose field of views (FOV) might have no direct spatial overlap. With the popularity of mobile sensors, NOS rise when multiple cooperative users capture a ...

    Continue reading

    Keywords: Sequence Alignment, Motion Compensation

  23. summary image

    Illuminating Pedestrians via Simultaneous Detection & Segmentation

    Garrick Brazil, Xi Yin, Xiaoming Liu

    Pedestrian detection is a critical problem in computer vision with significant impact on safety in urban autonomous driving. In this work, we explore how semantic segmentation can be used to boost pedestrian detection accuracy while having little to no impact on network efficiency. We propose a segmentation infusion network to ...

    Continue reading

    Keywords: Pedestrian Detection, Semantic Segmentation

  24. summary image

    Temporally Robust Global Motion Compensation by Keypoint-based Congealing

    Seyed Morteza Safdarnejad, Yousef Atoum, Xiaoming Liu

    Global motion compensation (GMC) removes the impact of camera motion and creates a video in which the background appears static over the progression of time. Various vision problems, such as human activity recognition, background reconstruction, and multi-object tracking can benefit from GMC. Existing GMC algorithms rely on sequentially processing consecutive ...

    Continue reading

    Keywords: Motion Compensation, Activity Recognition

  25. summary image

    Robust Global Motion Compensation in Presence of Predominant Foreground

    Seyed Morteza Safdarnejad, Xiaoming Liu, Lalita Udpa

    Global motion compensation (GMC) removes intentional and unwanted camera motion. GMC is widely applicable for video stitching and, as a pre-processing module, for motion-based video analysis. While state-of-the-art GMC algorithms generally estimate homography satisfactorily between consecutive frames, their performances deteriorate on real-world unconstrained videos, for instance, videos with predominant foreground ...

    Continue reading

    Keywords: Motion Compensation, Activity Recognition

  26. summary image

    Sports Videos in the Wild (SVW): A Video Dataset for Sports Analysis

    Seyed Morteza Safdarnejad, Xiaoming Liu, Lalita Udpa, Brooks Andrus, John Wood, Dean Craven

    The amount of digital videos being created is increasing exponentially, e.g., YouTube has reached the upload rate of 100 hours of video per minute. A great deal of this growth is due to the tremendous popularity of smartphones and ubiquitous Internet access. This means that amateur-user generated videos form ...

    Continue reading

    Keywords: Database, Activity Recognition

  27. summary image

    Multi-Modality Imagery Database for Plant Phenotyping

    Jeffrey A. Cruz, Xi Yin, Xiaoming Liu, Saif M. Imran, Daniel D. Morris, David M. Kramer, Jin Chen

    We have collected a multi-modality plant imagery database named “MSU-PID” including two types of plants: Arabidopsis and bean. It is captured using four types of imaging sensors:fluorescence, infrared(IR), RGB color, and depth. The imaging setup and the variety of manual labels allow MSU-PID to be used for a ...

    Continue reading

    Keywords: Plant Vision, Database, Application, Multi-Modality

  28. summary image

    Plant Vision

    Xi Yin, Xiaoming Liu, Jin Chen, David M. Kramer

    Plants are the major organisms that can absorb the light energy from the sun to produce biomass and oxygen. One key problem in studying plant growth is to understand the photosynthetic activities of plants under various external stimuli or genetic variations. Because leaves at different developmental ages may response to ...

    Continue reading

    Keywords: Plant vision

  29. summary image

    On developing and enhancing plant-level disease rating systems in real fields

    Yousef Atoum, Muhammad Jamal Afridi, Xiaoming Liu, J. Mitchell McGrath, Linda E. Hanson

    Cercospora leaf spot (CLS) is one of the most serious diseases of sugar beet worldwide, and if uncontrolled, causes nearly complete defoliation and loss of revenue for beet growers. The beet sugar industry continuously seeks CLS-resistant sugar beet cultivars as one strategy to combat this disease. Normally human experts manually ...

    Continue reading

    Keywords: Application

  30. summary image

    Image Segmentation of Mesenchymal Stem Cells in Diverse Culturing Conditions

    Muhammad Jamal Afridi, Chun Liu, Christina Chan, Seungik Baek, Xiaoming Liu

    Researchers in the areas of regenerative medicine and tissue engineering have an enormous interest in understanding the relationship of different sets of culturing conditions and applied mechanical stimuli on the behavior of Mesenchymal Stem Cells (MSCs). However, it remains a challenge to design a general tool to perform automatic cell ...

    Continue reading