Probabilistic Framework for the Characterization of Surfaces and Edges in Range Images,
with Application to Edge Detection

Download the paper
[Supplemental material]

Summary

In [1],we develop a powerful probabilistic framework for the local characterization of surfaces and edges in range images. We use the geometrical nature of the data to derive an analytic expression of the joint probability density function (pdf) for the random variables used to model the ranges of a set of pixels in a local neighborhood of an image. We decompose this joint pdf by considering independently the cases where two real world points corresponding to two neighboring pixels are locally on the same real world surface or not. In particular, we show that this joint pdf is linked to the Voigt pdf and not to the Gaussian pdf as it is assumed in some applications. We apply our framework to edge detection and develop a locally adaptive algorithm that is based on a probabilistic decision rule. We show in an objective evaluation that this new edge detector performs better than prior art detectors. This proves the benefits of the probabilistic characterization of the local neighborhood as a tool to improve applications that involve range images.

Database + source code in C++

1 Download the source code

The source code can be downloaded here. It is licensed under a basic 3-close BSD license (see 3↓).

1.1 Instructions

The program has been developed in standard C++. In order to compile it, you need a C++ compiler, a copy of the OpenCV library, and the CMake build automation tool. An optional example code uses the Point Cloud Library (PCL). On UNIX-like environments, the program can be compiled as follows, considering that your terminal is in the source code directory:

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ..
$ make

1.2 Extract edges from a depth image

1.2.1 From our database (YAML file)

You can extract the edge map from an image of our database2↓ as follows, considering that your terminal is in the build directory of the source code:

$ ./bin/detect_edges <path_to_an_yml_image> [threshold=0.5] [prior=0.1] [spatial_support=8]

with <path_to_an_yml_image> substituted by the path to the image and where [threshold], [prior] and [spatial_support] are the parameters of the edge detector and are optional.

1.2.2 From a 16 bits PNG file

You can also extract the edges from a 16 bits PNG image where each 16 bits unsigned integer represents the depth in millimeters as follows:

$ ./bin/detect_edges <path_to_a_png_image> <fx> <fy> <px> <py> [threshold=0.5] [prior=0.1] [spatial_support=8]

with <fx> <fy> <px> <py> substituted by the intrinsic parameters (focal length and principal point along the x and y axis) of the camera used to capture the image. In this case, the noise standard deviation is assumed to be the one of the Microsoft Kinect 1 (0.002 × z²).

This permits to extract edges from datasets such as the RGB-D SLAM Dataset.

1.2.3 Outputs

The program will output four different images in the working directory:

[edges.png] Detected edges as given by our method. The result for each pixel is a series of flags encoded on a single byte. The flags are described lower.
[binary_edges.png] Detected edges as a binary mask (pixel value at 255 if an edge was detected on that pixel)
[depth_gray.png] Gray-level visualization of the depth map (normalized between the minimum and maximum depth of the image)
[depth_rgb.png] RGB visualization of the depth map (normalized between the minimum and maximum depth of the image)

Our detector encodes more information that a simple binary “edge/surface” information. Eight flags can be activated to indicate the neighbors with which the pixel forms an edge and wether the pixel is occluding the other one or occluded by it. A surface occludes another one if it is closer to the camera. It means that occluding edge pixels form an actual boundary of a surface while it is not necesseraly the case for occluded edge pixels. The flags are defined in include/ped.h:

enum EdgeFlags { 
    WEST  = 1,        //!< Indicates wether there is an edge to the left/west of this pixel 
    NORTH = 1 << 1, 
    EAST  = 1 << 2,
    SOUTH = 1 << 3,
    OCCLUDING_WEST  = 1 << 4, //!< Indicates wether the pixel is on a surface border 
                              //!< that is occluding the surface to the west/on its left, 
                              //!< i.e. the current pixel has a smaller depth than the one
                              //!< to its left. 
    OCCLUDING_NORTH = 1 << 5, 
    OCCLUDING_EAST  = 1 << 6, 
    OCCLUDING_SOUTH = 1 << 7,
};

Note that the program has been successfully tested on Debian-like GNU/Linux operating systems (compiled with g++).

1.3 Live stream edge detection

A program performing an edge detection on a live stream of range image is provided. The program requires the Point Cloud Library (PCL) and the OpenNI library for range image acquisition. It can be used as follows, considering that your terminal is in the build directory of the source code:

$ ./bin/pcl_example

The ID of the camera can be specified by using the option -d <camera_id>.

2 JUMP Evaluation database

The evaluation database can be downloaded here. The archive contains a folder per camera:

kinect/ for the Microsoft Kinect 1
kinect2/ for the Microsoft Kinect2
camcube-2000/ for the PMD[vision] CamCube 2 with an integration time of 2000µs.

Each folder contains the six range image recorded as YAML file using OpenCV with their corresponding groundtruth edge map recorded as PNG binary mask.

3 License

Copyright (c) 2015, University of Liège
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Acknowledgments

A. Lejeune is under a contract funded by the European Regional Development Fund (ERDF) program of the Walloon Region, Belgium.

References

[1] A. Lejeune, J. Verly, M. Van Droogenbroeck. Probabilistic Framework for the Characterization of Surfaces and Edges in Range Images, with Application to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence:1-14, 2017. URL http://ieeexplore.ieee.org/document/8019874/.