Aashish Nagpal (2012CS50271)

Shivam Verma (2012CS10253)

Goals

Input Gathering:- Taking input of an arrangement of primitive objects

placed on a planar surface from a 3D scanning device .

Segmentation:- Segment the image captured into different regions such

that all the objects are in the different region.

Point Cloud Generation:- Generate point clouds for the different segments
based on the additional factor of depth captured from scanner.

Identification:- Identify the point clouds using training data.

Parameter Calculation:- Based on the object identified calculate the

dimensions/parameters for the pointcloud.

Input Gathering

We used two devices for scanning the arrangement:

Kinect v2.0

CIMCORE Portable Laser Scanner

Input Gathering From Kinect

Two types of images were captured using Kinect v2.0:

RGB

Depth

Input was taken for different arrangements and from different distance
and angles.

Gathered image data was then preprocessed for segmentation which

involved scaling the image

Segmentation

Input image was then segmented using the previous work based on
Learning Rich Features from RGB-D Images for Object Detection
and Segmentation [1]

This technique uses geocentric embedding for depth images that encodes
height above ground and angle with gravity for each pixel in addition to
the horizontal disparity to calculate features.

Decision forest approach that classifies pixels in the detection window as

foreground or background

Point Cloud Generation

PLY and PCD files were generated using segmented image and the mesh
data collected by Kinect.

Significant Point Clouds of different segments were generated.

Only the points with significant depth information were considered.

Point with zero depth in the depth image were rejected.

Final output was point clouds of primitive objects and platform.

Identification

PCL library was used to match the obtained point clouds with the
training point clouds in the database.

VFH(viewpoint feature histogram) signature were calculated for each of

the pointcloud (.pcd) .

These features were then used to identify the object from the training
data.

Search algorithm used for nearest neighbor search was KD-tree.

Detecting Parameters

PCL library functions were used to detect the parameters for the object
identified in the previous step.

Point Cloud is represented as pcl::PointCloud<PointT>

For normal calculation pcl::NormalEstimation<PointT, pcl::Normal>

was used

Point cloud was normalized using Moving Least Square method.

Detecting Parameters

Segmentation object was created using

pcl::SACSegmentationFromNormals<pcl::PointNormal, pcl::Normal>.
Currently the library supports the following models for fitting:

Cylinder

Cone

Sphere

Plane

Appropriate model was defined based on the object identified in the

Identification step.

Detecting Parameters

Random sample consensus (RANSAC)was used to estimate parameters

of the segmented object.

Following output parameters were obtained for the objects:

Cylinder: Radius
Height
Orientation of the cylinder in terms of angle the axis make.
Position of centroid

Detecting Parameters

Cone: Base Radius and Top Radius

Height
Orientation of the cone in terms of angle the axis make.
Position of centroid

Sphere: Radius
Position of the center

Quality of Input(RGB-D)

Good Input

Enough depth information available

Quality of Input(RGB-D)

Bad Input

Very less depth information

Results
Cylinder 1(Kinect)

Cylinder 1(Actual)

Radius

1.57 cm

1.6 cm

Height

6.25 cm

6.5 cm

Identified as cylinder

Cylinder 2 (Kinect)

Cone(Actual)

Radius

2.89 cm

3.0 cm

Height

4.6 cm

Results

Results

Results
Cylinder1 (Kinect)

Cylinder1 (Actual)

Radius

1.57 cm

1.6 cm

Height

6.2 cm

6.5 cm

Cylinder2 (Kinect)

Cylinder2 (Actual)

Radius

2.85 cm

3.0 cm

Height

4.6 cm
Cone (Kinect)

Cone (Actual)

Base Radius

Not detected

2.5 cm

Top Radius

Not detected

2.0 cm

Height

Not detected

6.1 cm

Results

Results

Results
Cylinder (Kinect)

Cylinder (Actual)

Radius

1.53 cm

1.6 cm

Height

6.23 cm

6.5 cm

Sphere (Kinect)

Sphere (Actual)

2.9 cm

3.0 cm

Cone (Kinect)

Cone (Actual)

Base Radius

2.21 cm

2.5 cm

Top Radius

2.033 cm

2.0 cm

Height

5.94 cm

6.1 cm

Radius