1. summary image

    SeaBird: Segmentation in Bird’s View with Dice Loss Improves Monocular 3D Detection of Large Objects

    Abhinav Kumar, Yuliang Guo, Xinyu Huang, Liu Ren and Xiaoming Liu

    Monocular 3D detectors achieve remarkable performance on cars and smaller objects. However, their performance drops on larger objects, leading to fatal accidents. Some attribute the failures to training data scarcity or receptive field requirements of large objects. In this paper, we highlight this understudied problem of generalization to large objects ...

    Continue reading

    Keywords: 3D Object Detection, Image Segmentation

  2. summary image

    Learning Clothing and Pose Invariant 3D Shape Representation for Long-Term Person Re-Identification

    Feng Liu, Minchul Kim, Ziang Gu, Anil Jain, Xiaoming Liu

    Long-Term Person Re-Identification (LT-ReID) has become increasingly crucial in computer vision and biometrics. In this work, we aim to extend LT-ReID beyond pedestrian recognition to include a wider range of real-world human activities while still accounting for cloth-changing scenarios over large time gaps. This setting poses additional challenges due to ...

    Continue reading

    Keywords: Body Matching, 3D Shape Reconstruction, Biometrics, 3D Human Reconstruction, Person Re-identification

  3. summary image

    DEVIANT: Depth EquiVarIAnt NeTwork for Monocular 3D Object Detection

    Abhinav Kumar, Garrick Brazil, Enrique Corona, Armin Parchami, Xiaoming Liu

    Modern neural networks use building blocks such as convolutions that are equivariant to arbitrary 2D translations. However, these vanilla blocks are not equivariant to arbitrary 3D translations in the projective manifold. Even then, all monocular 3D detectors use vanilla blocks to obtain the 3D coordinates, a task for which the ...

    Continue reading

    Keywords: 3D Object Detection

  4. summary image

    Controllable and Guided Face Synthesis for Unconstrained Face Recognition

    Feng Liu, Minchul Kim, Anil Jain, Xiaoming Liu

    Although significant advances have been made in face recognition (FR), FR in unconstrained environments remains challenging due to the domain gap between the semi-constrained training datasets and unconstrained testing scenarios. To address this problem, we propose a controllable face synthesis model (CFSM) that can mimic the distribution of target datasets ...

    Continue reading

    Keywords: Face Recognition, Face Synthesis, Biometrics

  5. summary image

    AdaFace: Quality Adaptive Margin for Face Recognition

    Minchul Kim, Anil K. Jain, Xiaoming Liu

    Recognition in low quality face datasets is challenging because facial attributes are obscured and degraded. Advances in margin-based loss functions have resulted in enhanced discriminability of faces in the embedding space. Further, previous studies have studied the effect of adaptive losses to assign more importance to misclassified (hard) examples. In ...

    Continue reading

    Keywords: Face recognition, Biometrics

  6. summary image

    Face Relighting with Geometrically Consistent Shadows

    Andrew Hou, Michel Sarkis, Ning Bi, Yiying Tong, Xiaoming Liu

    Most face relighting methods are able to handle diffuse shadows, but struggle to handle hard shadows, such as those cast by the nose. Methods that propose techniques for handling hard shadows often do not produce geometrically consistent shadows since they do not directly leverage the estimated face geometry while synthesizing ...

    Continue reading

    Keywords: Face Relighting, Low-level Vision

  7. summary image

    Proactive Image Manipulation Detection

    Vishal Asnani, Xi Yin, Tal Hassner, Sijia Liu, Xiaoming Liu

    Image manipulation detection algorithms are often trained to discriminate between images manipulated with particular Generative Models (GMs) and genuine/real images, yet generalize poorly to images manipulated with GMs unseen in the training. Conventional detection algorithms receive an input image passively. By contrast, we propose a proactive scheme to image ...

    Continue reading

    Keywords: Image Manipulation, Low-level Vision

  8. summary image

    Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image

    Feng Liu, Xiaoming Liu

    Inferring 3D locations and shapes of multiple objects from a single 2D image is a long-standing objective of computer vision. Most of the existing works either predict one of these 3D properties or focus on solving both for a single object. One fundamental challenge lies in how to learn an ...

    Continue reading

    Keywords: 3D Object Detection, 3D Shape Reconstruction, Generic Object 3D Reconstruction

  9. summary image

    Depth Completion with TWIN Surface Extrapolation at Occlusion Boundaries

    Saif Imran, Xiaoming Liu, Daniel Morris

    Depth completion starts from a sparse set of known depth values and estimates the unknown depths for the remaining image pixels. Most methods model this as depth interpolation and erroneously interpolate depth pixels into the empty space between spatially distinct objects, resulting in depth-smearing across occlusion boundaries. Here we propose ...

    Continue reading

    Keywords: Depth Completion, Camera+LiDAR+Radar, Depth Prediction

  10. summary image

    Towards High Fidelity Face Relighting with Realistic Shadows

    Andrew Hou, Ze Zhang, Michel Sarkis, Ning Bi, Yiying Tong, Xiaoming Liu

    Existing face relighting methods often struggle with two problems: maintaining the local facial details of the subject and accurately removing and synthesizing shadows in the relit image, especially hard shadows. We propose a novel deep face relighting method that addresses both problems. Our method learns to predict the ratio (quotient ...

    Continue reading

    Keywords: Face Relighting, Low-level Vision

  11. summary image

    GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection

    Abhinav Kumar, Garrick Brazil, Xiaoming Liu

    Modern 3D object detectors have immensely benefited from the end-to-end learning idea. However, most of them use a post-processing algorithm called Non-Maximal Suppression (NMS) only during inference. While there were attempts to include NMS in the training pipeline for tasks such as 2D object detection, they have been less widely ...

    Continue reading

    Keywords: 3D Object Detection

  12. summary image

    Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction

    Feng Liu, Luan Tran, Xiaoming Liu

    Inferring 3D structure of a generic object from a 2D image is a long-standing objective of computer vision. Conventional approaches either learn completely from CAD-generated synthetic data, which have difficulty in inference from real images, or generate 2.5D depth image via intrinsic decomposition, which is limited compared to the ...

    Continue reading

    Keywords: Generic Object 3D Reconstruction, 3D Shape Reconstruction, Semantic Segmentation

  13. summary image

    Face Anti-spoofing, Face Presentation Attack Detection

    Yaojie Liu, Joel Stehouwer, Amin Jourabloo, Yousef Atoum, Xiaoming Liu

    Biometrics utilize physiological, such as fingerprint, face, and iris, or behavioral characteristics, such as typing rhythm and gait, to uniquely identify or authenticate an individual. As biometric systems are widely used in real-world applications including mobile phone authentication and access control, biometric spoof, or Presentation Attack (PA) are becoming a ...

    Continue reading

    Keywords: Face Antispoofing, Biometrics, Low-level Vision, Database

  14. summary image

    Kinematic 3D Object Detection in Monocular Video

    Garrick Brazil, Gerard Pons-Moll, Xiaoming Liu, Bernt Schiele

    Perceiving the physical world in 3D is fundamental for selfdriving applications. Although temporal motion is an invaluable resource to human vision for detection, tracking, and depth perception, such features have not been thoroughly utilized in modern 3D object detectors. In this work, we propose a novel method for monocular video-based ...

    Continue reading

    Keywords: 3D Object Detection

  15. summary image

    The Edge of Depth: Explicit Constraints between Segmentation and Depth

    Shengjie Zhu, Garrick Brazil, Xiaoming Liu

    In this work we study the mutual benefits of two common computer vision tasks, self-supervised depth estimation and semantic segmentation from images. For example, to help unsupervised monocular depth estimation, constraints from semantic segmentation has been explored implicitly such as sharing and transforming features. In contrast, we propose to explicitly ...

    Continue reading

    Keywords: Depth Prediction, Image Segmentation, Semantic Segmentation

  16. summary image

    Facial Forgery Detection

    Hao Dang*, Feng Liu*, Joel Stehouwer*, Xiaoming Liu, Anil Jain

    The prevalence of facial recognition, biometric unlock, and social media presents a significant opportunity for bad actors to introduce forged or manipulated images to spread false information or damage reputations. This is aided by the continuing improvement in realistic image synthesis and manipulation by generative adversarial network, GAN, based methods ...

    Continue reading

    Keywords: Image Manipulation, Low-level Vision, Database

  17. summary image

    Generic Object Sensor and Spoof Noise Classification, Modeling, and Synthesis

    Joel Stehouwer, Amin Jourabloo, Yaojie Liu, Xiaoming Liu

    Biometric recognition is increasingly used in commercial and high-security settings. Because of this, the threat of spoofing techniques, the act of presenting a fake biometric object to a sensor, is a large concern. Recent research has focused on face, fingerprint, and iris anti-spoofing. However, no other research attempts to use ...

    Continue reading

    Keywords: Low-level Vision, Database

  18. summary image

    M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

    Garrick Brazil, Xiaoming Liu

    Understanding the world in 3D is a critical component of urban autonomous driving. Generally, the combination of expensive LiDAR sensors and stereo RGB imaging has been paramount for successful 3D object detection algorithms, whereas monocular image-only methods experience drastically reduced performance. We propose to reduce the gap by reformulating the ...

    Continue reading

    Keywords: 3D Object Detection

  19. summary image

    Pedestrian Detection with Autoregressive Network Phases

    Garrick Brazil, Xiaoming Liu

    We present an autoregressive pedestrian detection framework with cascaded phases designed to progressively improve precision. The proposed framework utilizes a novel lightweight stackable decoder-encoder module which uses convolutional re-sampling layers to improve features while maintaining efficient memory and runtime cost. Unlike previous cascaded detection systems, our proposed framework is designed ...

    Continue reading

    Keywords: Pedestrian Detection, Object Detection

  20. summary image

    Depth Coefficients for Depth Completion

    Saif Imran, Yunfei Long, Xiaoming Liu, Daniel Morris

    Depth completion involves estimating a dense depth image from sparse depth measurements, often guided by a color image. While linear upsampling is straight forward, it results in artifacts including depth pixels being interpolated in empty space across discontinuities between objects. Current methods use deep networks to upsample and "complete" the ...

    Continue reading

    Keywords: Depth Completion, Camera+LiDAR+Radar, Multi-modality, Depth Prediction

  21. summary image

    Gait Recognition via Disentangled Representation Learning

    Ziyuan Zhang, Luan Tran, Xi Yin, Yousef Atoum, Xiaoming Liu, Jian Wan, Nanxin Wang

    Gait, the walking pattern of individuals, is one of the most important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as the gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and view angle. To ...

    Continue reading

    Keywords: Biometrics, Gait Recognition, Database

  22. summary image

    Towards Interpretable Face Recognition

    Bangjie Yin, Luan Tran, Haoxiang Li, Xiaohui Shen, Xiaoming Liu

    Deep CNNs have been pushing the frontier of visual recognition over past years. Besides recognition accuracy, strong demands in understanding deep CNNs in the research community motivate developments of tools to dissect pre-trained models to visualize how they make predictions. Recent works further push the interpretability in the network learning ...

    Continue reading

    Keywords: Face Recognition, Biometrics

  23. summary image

    Recurrent Flow-Guided Semantic Forecasting

    Adam M. Terwilliger, Garrick Brazil, Xiaoming Liu

    Understanding the world around us and making decisions about the future is a critical component to human intelligence. As autonomous systems continue to develop, their ability to reason about the future will be the key to their success. Semantic anticipation is a relatively under-explored area for which autonomous vehicles could ...

    Continue reading

    Keywords: Forecasting, Semantic Segmentation, Image Segmentation

  24. summary image

    MSU-AVIS dataset: Fusing Face and Voice Modalities for Biometric Recognition in Indoor Surveillance Videos

    Anurag Chowdhury, Yousef Atoum, Luan Tran, Xiaoming Liu, Arun Ross

    Indoor video surveillance systems often use the face modality to establish the identity of a person of interest. However, the face image may not offer sufficient discriminatory information in many scenarios due to substantial variations in pose, illumination, expression, resolution and distance between the subject and the camera.

    In such ...

    Continue reading

    Keywords: Application, Biometrics, Face Recognition, Surveillance, Multi-modality

  25. summary image

    Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild

    Luan Tran, Kihyuk Sohn, Xiang Yu, Xiaoming Liu, Manmohan Chandraker

    Recent developments in deep domain adaptation have allowed knowledge transfer from a labeled source domain to an unlabeled target domain at the level of intermediate features and input pixels. We propose that advantages may be derived by jointly investigating the two, in the form of different level insights that lead ...

    Continue reading

    Keywords: Domain Adaptation

  26. summary image

    Feature Transfer Learning for Deep Face Recognition with Long-Tail Data

    Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

    Real-world face recognition datasets exhibit long-tail characteristics, which results in biased classifiers in conventionally-trained deep neural networks, or insufficient data when long-tail classes are ignored. In this paper, we propose to handle long-tail classes in the training of a face recognition engine by augmenting their feature space under a center-based ...

    Continue reading

    Keywords: Face Recognition, Biometrics

  27. summary image

    Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition

    Xi Yin, Xiaoming Liu

    This work explores Multi-Task Learning (MTL) for face recognition. First, we propose a multi-task Convolutional Neural Network (CNN) for face recognition where identity classification is the main task and Pose, Illumination, and Expression (PIE) estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss ...

    Continue reading

    Keywords: Face Recognition, Biometrics

  28. summary image

    Monocular Video-Based Trailer Coupler Detection using Multiplexer Convolutional Neural Network

    Yousef Atoum, Joseph Roth, Michael Bliss, Wende Zhang, Xiaoming Liu

    This paper presents a monocular camera-based computer vision system for autonomous selfbacking-up a vehicle towards a trailer, by continuously estimating the 3D trailer coupler position and feeding it to the vehicle control system, until the alignment of the tow hitch with the trailers coupler. This system is made possible through ...

    Continue reading

    Keywords: Application, Object Detection

  29. summary image

    Disentangled Representation Learning GAN for Pose-Invariant Face Recognition

    Luan Tran, Xi Yin, Xiaoming Liu

    The large pose discrepancy between two face images is one of the fundamental challenges in automatic face recognition. Conventional approaches to pose-invariant face recognition either perform face frontalization on, or learn a pose-invariant representation from, a non-frontal face image. We argue that it is more desirable to perform both tasks ...

    Continue reading

    Keywords: Face Recognition, Surveillance, Biometrics, Face Synthesis

  30. summary image

    Towards Large-Pose Face Frontalization in the Wild

    Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

    Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments. Learning pose-invariant features is one solution, but needs expensively labeled large scale data and carefully designed feature learning algorithms. In this work, we focus on frontalizing faces in the ...

    Continue reading

    Keywords: Face Recognition, Face Reconstruction, 3D Shape Reconstruction, Face Synthesis, Biometrics

  31. summary image

    Spatio-Temporal Alignment of Non-Overlapping Sequences from Independently Panning Cameras

    Seyed Morteza Safdarnejad, Xiaoming Liu

    We tackle a novel scenario of spatio-temporal alignment of seuqences referred to as Nonoverlapping Sequences (NOS). NOS are captured by multiple freely panning handheld cameras whose field of views (FOV) might have no direct spatial overlap. With the popularity of mobile sensors, NOS rise when multiple cooperative users capture a ...

    Continue reading

    Keywords: Motion Compensation, activity recognition

  32. summary image

    Illuminating Pedestrians via Simultaneous Detection & Segmentation

    Garrick Brazil, Xi Yin, Xiaoming Liu

    Pedestrian detection is a critical problem in computer vision with significant impact on safety in urban autonomous driving. In this work, we explore how semantic segmentation can be used to boost pedestrian detection accuracy while having little to no impact on network efficiency. We propose a segmentation infusion network to ...

    Continue reading

    Keywords: Pedestrian Detection, Semantic Segmentation, Object Detection, Image Segmentation

  33. summary image

    Temporally Robust Global Motion Compensation by Keypoint-based Congealing

    Seyed Morteza Safdarnejad, Yousef Atoum, Xiaoming Liu

    Global motion compensation (GMC) removes the impact of camera motion and creates a video in which the background appears static over the progression of time. Various vision problems, such as human activity recognition, background reconstruction, and multi-object tracking can benefit from GMC. Existing GMC algorithms rely on sequentially processing consecutive ...

    Continue reading

    Keywords: Motion Compensation, Activity Recognition

  34. summary image

    Robust Global Motion Compensation in Presence of Predominant Foreground

    Seyed Morteza Safdarnejad, Xiaoming Liu, Lalita Udpa

    Global motion compensation (GMC) removes intentional and unwanted camera motion. GMC is widely applicable for video stitching and, as a pre-processing module, for motion-based video analysis. While state-of-the-art GMC algorithms generally estimate homography satisfactorily between consecutive frames, their performances deteriorate on real-world unconstrained videos, for instance, videos with predominant foreground ...

    Continue reading

    Keywords: Motion Compensation, Activity Recognition

  35. summary image

    Sports Videos in the Wild (SVW): A Video Dataset for Sports Analysis

    Seyed Morteza Safdarnejad, Xiaoming Liu, Lalita Udpa, Brooks Andrus, John Wood, Dean Craven

    The amount of digital videos being created is increasing exponentially, e.g., YouTube has reached the upload rate of 100 hours of video per minute. A great deal of this growth is due to the tremendous popularity of smartphones and ubiquitous Internet access. This means that amateur-user generated videos form ...

    Continue reading

    Keywords: Activity Recognition, Database, Application

  36. summary image

    Multi-Modality Imagery Database for Plant Phenotyping

    Jeffrey A. Cruz, Xi Yin, Xiaoming Liu, Saif M. Imran, Daniel D. Morris, David M. Kramer, Jin Chen

    We have collected a multi-modality plant imagery database named “MSU-PID” including two types of plants: Arabidopsis and bean. It is captured using four types of imaging sensors:fluorescence, infrared(IR), RGB color, and depth. The imaging setup and the variety of manual labels allow MSU-PID to be used for a ...

    Continue reading

    Keywords: Plant Vision, Database, Application, Multi-Modality

  37. summary image

    On developing and enhancing plant-level disease rating systems in real fields

    Yousef Atoum, Muhammad Jamal Afridi, Xiaoming Liu, J. Mitchell McGrath, Linda E. Hanson

    Cercospora leaf spot (CLS) is one of the most serious diseases of sugar beet worldwide, and if uncontrolled, causes nearly complete defoliation and loss of revenue for beet growers. The beet sugar industry continuously seeks CLS-resistant sugar beet cultivars as one strategy to combat this disease. Normally human experts manually ...

    Continue reading

    Keywords: Application, Plant Vision

  38. summary image

    Image Segmentation of Mesenchymal Stem Cells in Diverse Culturing Conditions

    Muhammad Jamal Afridi, Chun Liu, Christina Chan, Seungik Baek, Xiaoming Liu

    Researchers in the areas of regenerative medicine and tissue engineering have an enormous interest in understanding the relationship of different sets of culturing conditions and applied mechanical stimuli on the behavior of Mesenchymal Stem Cells (MSCs). However, it remains a challenge to design a general tool to perform automatic cell ...

    Continue reading

    Keywords: Medical Imaging, Image Segmentation, Application, Semantic Segmentation