This paper presents a monocular camera-based computer vision system for autonomous selfbacking-up a vehicle towards a trailer, by continuously estimating the 3D trailer coupler position and feeding it to the vehicle control system, until the alignment of the tow hitch with the trailers coupler. This system is made possible through our proposed distance-driven Multiplexer-CNN method, which selects the most suitable CNN using the estimated coupler-to-vehicle distance. The input of the multiplexer is a group made of a CNN detector, trackers, and 3D localizer. In the CNN detector, we propose a novel algorithm to provide a confidence score with each detection. The score reflects the existence of the target object in a region, as well as how accurate is the 2D target detection. We demonstrate the accuracy and efficiency of the system on a large trailer database. Our system achieves an estimation error of 1.4 cm when the ball reaches the coupler, while running at 18.9 FPS on a regular PC.

Plant-Disease Introduction

Our Multiplexer-CNN system has five CNN inputs: DCNN, TCNN1, TCNN2, TCNN3, CCNN. As shown in the image, our system consists of three main stages: (1) 2D coupler detection, (2) 2D coupler tracking, and (3) 3D coupler localization for vehicle automation. Stage 1 initializes the 2D coordinate of the coupler for Stage 2. Stage 2 and 3 collaborate in estimating both 2D and 3D coupler positions.

Plant-Disease Introduction

Publications

  • Monocular Video-Based Trailer Coupler Detection using Multiplexer Convolutional Neural Network
    Yousef Atoum, Joseph Roth, Michael Bliss, Wende Zhang, Xiaoming Liu
    In Proceeding of International Conference on Computer Vision (ICCV 2017), Venice, Italy, Oct. 2017
    Bibtex | PDF
  • @inproceedings{ monocular-video-based-trailer-coupler-detection-using-multiplexer-convolutional-neural-network,
      author = { Yousef Atoum and Joseph Roth and Michael Bliss and Wende Zhang and Xiaoming Liu },
      title = { Monocular Video-Based Trailer Coupler Detection using Multiplexer Convolutional Neural Network },
      booktitle = { In Proceeding of International Conference on Computer Vision },
      address = { Venice, Italy },
      month = { October },
      year = { 2017 },
    }