Long-Term Person Re-Identification (LT-ReID) has become increasingly crucial in computer vision and biometrics. In this work, we aim to extend LT-ReID beyond pedestrian recognition to include a wider range of real-world human activities while still accounting for cloth-changing scenarios over large time gaps. This setting poses additional challenges due to ...

Continue reading

Keywords: Body Matching, 3D Shape Reconstruction, Biometrics, 3D Human Reconstruction, Person Re-identification

Modern neural networks use building blocks such as convolutions that are equivariant to arbitrary 2D translations. However, these vanilla blocks are not equivariant to arbitrary 3D translations in the projective manifold. Even then, all monocular 3D detectors use vanilla blocks to obtain the 3D coordinates, a task for which the ...

Continue reading

Keywords: 3D Object Detection

Although significant advances have been made in face recognition (FR), FR in unconstrained environments remains challenging due to the domain gap between the semi-constrained training datasets and unconstrained testing scenarios. To address this problem, we propose a controllable face synthesis model (CFSM) that can mimic the distribution of target datasets ...

Continue reading

Keywords: Face Recognition, Face Synthesis, Biometrics

Recent research has shown that controllable image generation based on pre-trained GANs can benefit a wide range of computer vision tasks. However, less attention has been devoted to 3D vision tasks. In light of this, we propose a novel image-conditioned neural implicit field, which can leverage 2D supervisions from GAN-generated ...

Continue reading

Keywords: 3D Shape Reconstruction, Generic Object 3D Reconstruction

Feature fusion plays a crucial role in unconstrained face recognition where inputs (probes) comprise of a set of N low quality images whose individual qualities vary. Advances in attention and recurrent modules have led to feature fusion that can model the relationship among the images in the input set. However ...

Continue reading

Keywords: Face recognition, Biometrics

Recognition in low quality face datasets is challenging because facial attributes are obscured and degraded. Advances in margin-based loss functions have resulted in enhanced discriminability of faces in the embedding space. Further, previous studies have studied the effect of adaptive losses to assign more importance to misclassified (hard) examples. In ...

Continue reading

Keywords: Face recognition, Biometrics

Most face relighting methods are able to handle diffuse shadows, but struggle to handle hard shadows, such as those cast by the nose. Methods that propose techniques for handling hard shadows often do not produce geometrically consistent shadows since they do not directly leverage the estimated face geometry while synthesizing ...

Continue reading

Keywords: Face Relighting, Low-level Vision

Image manipulation detection algorithms are often trained to discriminate between images manipulated with particular Generative Models (GMs) and genuine/real images, yet generalize poorly to images manipulated with GMs unseen in the training. Conventional detection algorithms receive an input image passively. By contrast, we propose a proactive scheme to image ...

Continue reading

Keywords: Image Manipulation, Low-level Vision

Inferring 3D locations and shapes of multiple objects from a single 2D image is a long-standing objective of computer vision. Most of the existing works either predict one of these 3D properties or focus on solving both for a single object. One fundamental challenge lies in how to learn an ...

Continue reading

Keywords: 3D Object Detection, 3D Shape Reconstruction, Generic Object 3D Reconstruction

Depth completion starts from a sparse set of known depth values and estimates the unknown depths for the remaining image pixels. Most methods model this as depth interpolation and erroneously interpolate depth pixels into the empty space between spatially distinct objects, resulting in depth-smearing across occlusion boundaries. Here we propose ...

Continue reading

Keywords: Depth Completion, Camera+LiDAR+Radar, Depth Prediction

Existing face relighting methods often struggle with two problems: maintaining the local facial details of the subject and accurately removing and synthesizing shadows in the relit image, especially hard shadows. We propose a novel deep face relighting method that addresses both problems. Our method learns to predict the ratio (quotient ...

Continue reading

Keywords: Face Relighting, Low-level Vision

Modern 3D object detectors have immensely benefited from the end-to-end learning idea. However, most of them use a post-processing algorithm called Non-Maximal Suppression (NMS) only during inference. While there were attempts to include NMS in the training pipeline for tasks such as 2D object detection, they have been less widely ...

Continue reading

Keywords: 3D Object Detection

Inferring 3D structure of a generic object from a 2D image is a long-standing objective of computer vision. Conventional approaches either learn completely from CAD-generated synthetic data, which have difficulty in inference from real images, or generate 2.5D depth image via intrinsic decomposition, which is limited compared to the ...

Continue reading

Keywords: Generic Object 3D Reconstruction, 3D Shape Reconstruction, Semantic Segmentation

The goal of this paper is to learn dense 3D shape correspondence for topology-varying objects in an unsupervised manner. Conventional implicit functions estimate the occupancy of a 3D point given a shape latent code. Instead, our novel implicit function produces a part embedding vector for each 3D point, which is ...

Continue reading

Keywords: 3D Shape Correspondence, Semantic Segmentation

Biometrics utilize physiological, such as fingerprint, face, and iris, or behavioral characteristics, such as typing rhythm and gait, to uniquely identify or authenticate an individual. As biometric systems are widely used in real-world applications including mobile phone authentication and access control, biometric spoof, or Presentation Attack (PA) are becoming a ...

Continue reading

Keywords: Face Antispoofing, Biometrics, Low-level Vision, Database

Perceiving the physical world in 3D is fundamental for selfdriving applications. Although temporal motion is an invaluable resource to human vision for detection, tracking, and depth perception, such features have not been thoroughly utilized in modern 3D object detectors. In this work, we propose a novel method for monocular video-based ...

Continue reading

Keywords: 3D Object Detection

The prevalence of facial recognition, biometric unlock, and social media presents a significant opportunity for bad actors to introduce forged or manipulated images to spread false information or damage reputations. This is aided by the continuing improvement in realistic image synthesis and manipulation by generative adversarial network, GAN, based methods ...

Continue reading

Keywords: Image Manipulation, Low-level Vision, Database

Biometric recognition is increasingly used in commercial and high-security settings. Because of this, the threat of spoofing techniques, the act of presenting a fake biometric object to a sensor, is a large concern. Recent research has focused on face, fingerprint, and iris anti-spoofing. However, no other research attempts to use ...

Continue reading

Keywords: Low-level Vision, Database

In this work we study the mutual benefits of two common computer vision tasks, self-supervised depth estimation and semantic segmentation from images. For example, to help unsupervised monocular depth estimation, constraints from semantic segmentation has been explored implicitly such as sharing and transforming features. In contrast, we propose to explicitly ...

Continue reading

Keywords: Depth Prediction, Image Segmentation, Semantic Segmentation

Understanding the world in 3D is a critical component of urban autonomous driving. Generally, the combination of expensive LiDAR sensors and stereo RGB imaging has been paramount for successful 3D object detection algorithms, whereas monocular image-only methods experience drastically reduced performance. We propose to reduce the gap by reformulating the ...

Continue reading

Keywords: 3D Object Detection

We present an autoregressive pedestrian detection framework with cascaded phases designed to progressively improve precision. The proposed framework utilizes a novel lightweight stackable decoder-encoder module which uses convolutional re-sampling layers to improve features while maintaining efficient memory and runtime cost. Unlike previous cascaded detection systems, our proposed framework is designed ...

Continue reading

Keywords: Pedestrian Detection, Object Detection

Depth completion involves estimating a dense depth image from sparse depth measurements, often guided by a color image. While linear upsampling is straight forward, it results in artifacts including depth pixels being interpolated in empty space across discontinuities between objects. Current methods use deep networks to upsample and "complete" the ...

Continue reading

Keywords: Depth Completion, Camera+LiDAR+Radar, Multi-modality, Depth Prediction

Traditional 3D face models learn a latent representation of faces using linear subspaces from no more than 300 training scans of a single database. The main roadblock of building a large-scale face model from diverse 3D databases lies in the lack of dense correspondence among raw scans. To address these ...

Continue reading

Keywords: 3D Face Modeling, Face Reconstruction, 3D Shape Reconstruction

Gait, the walking pattern of individuals, is one of the most important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as the gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and view angle. To ...

Continue reading

Keywords: Biometrics, Gait Recognition, Database

Deep CNNs have been pushing the frontier of visual recognition over past years. Besides recognition accuracy, strong demands in understanding deep CNNs in the research community motivate developments of tools to dissect pre-trained models to visualize how they make predictions. Recent works further push the interpretability in the network learning ...

Continue reading

Keywords: Face Recognition, Biometrics

Understanding the world around us and making decisions about the future is a critical component to human intelligence. As autonomous systems continue to develop, their ability to reason about the future will be the key to their success. Semantic anticipation is a relatively under-explored area for which autonomous vehicles could ...

Continue reading

Keywords: Forecasting, Semantic Segmentation, Image Segmentation

Indoor video surveillance systems often use the face modality to establish the identity of a person of interest. However, the face image may not offer sufficient discriminatory information in many scenarios due to substantial variations in pose, illumination, expression, resolution and distance between the subject and the camera.

In such ...

Continue reading

Keywords: Application, Biometrics, Face Recognition, Surveillance, Multi-modality

As a classic statistical model of 3D facial shape and texture, 3D Morphable Model (3DMM) is widely used in facial analysis, including model fitting, image synthesis, etc. Conventional 3DMM is learned from a collection of wellcontrolled 2D face images with associated 3D face scans, and represented by two sets of ...

Continue reading

Keywords: Face Alignment, Face Reconstruction, 3D Face Modeling, 3D Shape Reconstruction

Recent developments in deep domain adaptation have allowed knowledge transfer from a labeled source domain to an unlabeled target domain at the level of intermediate features and input pixels. We propose that advantages may be derived by jointly investigating the two, in the form of different level insights that lead ...

Continue reading

Keywords: Domain Adaptation

Real-world face recognition datasets exhibit long-tail characteristics, which results in biased classifiers in conventionally-trained deep neural networks, or insufficient data when long-tail classes are ignored. In this paper, we propose to handle long-tail classes in the training of a face recognition engine by augmenting their feature space under a center-based ...

Continue reading

Keywords: Face Recognition, Biometrics

This work explores Multi-Task Learning (MTL) for face recognition. First, we propose a multi-task Convolutional Neural Network (CNN) for face recognition where identity classification is the main task and Pose, Illumination, and Expression (PIE) estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss ...

Continue reading

Keywords: Face Recognition, Biometrics

This paper presents a monocular camera-based computer vision system for autonomous selfbacking-up a vehicle towards a trailer, by continuously estimating the 3D trailer coupler position and feeding it to the vehicle control system, until the alignment of the tow hitch with the trailers coupler. This system is made possible through ...

Continue reading

Keywords: Application, Object Detection

The large pose discrepancy between two face images is one of the fundamental challenges in automatic face recognition. Conventional approaches to pose-invariant face recognition either perform face frontalization on, or learn a pose-invariant representation from, a non-frontal face image. We argue that it is more desirable to perform both tasks ...

Continue reading

Keywords: Face Recognition, Surveillance, Biometrics, Face Synthesis

Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments. Learning pose-invariant features is one solution, but needs expensively labeled large scale data and carefully designed feature learning algorithms. In this work, we focus on frontalizing faces in the ...

Continue reading

Keywords: Face Recognition, Face Reconstruction, 3D Shape Reconstruction, Face Synthesis, Biometrics

Recently, Convolutional Neural Network (CNN) based models have achieved great success in Single Image SuperResolution (SISR). Owing to the strength of deep networks, these CNN models learn an effective nonlinear mapping from the low-resolution input image to the high-resolution target image, at the cost of requiring enormous parameters. This paper ...

Continue reading

Keywords: Low-level Vision

We tackle a novel scenario of spatio-temporal alignment of seuqences referred to as Nonoverlapping Sequences (NOS). NOS are captured by multiple freely panning handheld cameras whose field of views (FOV) might have no direct spatial overlap. With the popularity of mobile sensors, NOS rise when multiple cooperative users capture a ...

Continue reading

Keywords: Motion Compensation, activity recognition

We present an algorithm for "unconstrained" 3D face reconstruction from a 2D photo collection of face images of a subject captured under a diverse variation of poses, expressions, and illuminations, without meta data about the cameras, timing, or light conditions. The output of our algorithm is a true 3D face ...

Continue reading

Keywords: Face reconstruction, 3D shape reconstruction

Pedestrian detection is a critical problem in computer vision with significant impact on safety in urban autonomous driving. In this work, we explore how semantic segmentation can be used to boost pedestrian detection accuracy while having little to no impact on network efficiency. We propose a segmentation infusion network to ...

Continue reading

Keywords: Pedestrian Detection, Semantic Segmentation, Object Detection, Image Segmentation

Face alignment is a process of applying a supervised learned model to a face image and estimating the locations of a set of facial landmarks, such as eye corners, mouth corners, etc. Face alignment is a key module in the pipeline of most facial analysis algorithms, normally after face detection ...

Continue reading

Keywords: Face alignment, Image Alignment, Face Reconstruction, 3D Shape Reconstruction

Global motion compensation (GMC) removes the impact of camera motion and creates a video in which the background appears static over the progression of time. Various vision problems, such as human activity recognition, background reconstruction, and multi-object tracking can benefit from GMC. Existing GMC algorithms rely on sequentially processing consecutive ...

Continue reading

Keywords: Motion Compensation, Activity Recognition

One of the key GEOINT capabilities is to be able to automatically recognize a large array of objects from visual data. Depending on the resolution of imagery, objects may range from specific locations or scenes, road, building, forest to vehicle, human, etc. This is clearly a technically challenging problem for ...

Continue reading

Keywords: Data Imputation, Multi-modality

The human is the dominant object of interest in the huge network of surveillance cameras deployed in buildings, airports, streets, and so on. The captured images and videos enable various applications such as face recognition, person re-identification, crowd and behavior analysis, etc. Certain applications, such as face recognition, rely on ...

Continue reading

Keywords: Biometrics, Face Deidentification

Global motion compensation (GMC) removes intentional and unwanted camera motion. GMC is widely applicable for video stitching and, as a pre-processing module, for motion-based video analysis. While state-of-the-art GMC algorithms generally estimate homography satisfactorily between consecutive frames, their performances deteriorate on real-world unconstrained videos, for instance, videos with predominant foreground ...

Continue reading

Keywords: Motion Compensation, Activity Recognition

The amount of digital videos being created is increasing exponentially, e.g., YouTube has reached the upload rate of 100 hours of video per minute. A great deal of this growth is due to the tremendous popularity of smartphones and ubiquitous Internet access. This means that amateur-user generated videos form ...

Continue reading

Keywords: Activity Recognition, Database, Application

In this paper, we introduce a multimedia analytics system to perform automatic and continuous online exam proctoring (OEP) . The overall goal of this system is to maintain academic integrity of exams, by providing real-time proctoring to detect the majority of cheating behaviors of the test taker. To achieve such goals ...

Continue reading

Keywords: Application

We hypothesize that an individual computer user has a unique and consistent habitual pattern of Typing Behavior, independent of the text, while typing on a keyboard. This habit manifests itself visually through the shapes and motions of the hands, as well as audibly through the sounds produced when pressing and ...

Continue reading

Keywords: Typing behavior, biometrics

We have collected a multi-modality plant imagery database named “MSU-PID” including two types of plants: Arabidopsis and bean. It is captured using four types of imaging sensors:fluorescence, infrared(IR), RGB color, and depth. The imaging setup and the variety of manual labels allow MSU-PID to be used for a ...

Continue reading

Keywords: Plant Vision, Database, Application, Multi-Modality

Plants are the major organisms that can absorb the light energy from the sun to produce biomass and oxygen. One key problem in studying plant growth is to understand the photosynthetic activities of plants under various external stimuli or genetic variations. Because leaves at different developmental ages may response to ...

Continue reading

Keywords: Plant vision, Application

Cercospora leaf spot (CLS) is one of the most serious diseases of sugar beet worldwide, and if uncontrolled, causes nearly complete defoliation and loss of revenue for beet growers. The beet sugar industry continuously seeks CLS-resistant sugar beet cultivars as one strategy to combat this disease. Normally human experts manually ...

Continue reading

Keywords: Application, Plant Vision

Researchers in the areas of regenerative medicine and tissue engineering have an enormous interest in understanding the relationship of different sets of culturing conditions and applied mechanical stimuli on the behavior of Mesenchymal Stem Cells (MSCs). However, it remains a challenge to design a general tool to perform automatic cell ...

Continue reading

Keywords: Medical Imaging, Image Segmentation, Application, Semantic Segmentation