Vision System for Industrial Robots

10. Software: Background_Area_Configuration module
Update #10195  |  28 Apr 2015

Few words about markers and background

At the beginning I would like to explain what I mean by a marker in the later sections. Markers are simply 4 little squares placed in corners of background working area. User measure distances between them and enters this data in configuration process. Application calculate data from user and data from camera to get mm/pixel ratio for future position calculations. Markers also will indicate local base for robot and will help in robot frame calibration process.

I decided to use this idea of ratio calculation because it greatly simplify calibration process. Thanks to this approach I can easily take my system to another environment and in few minutes make it working very well.

Other way to solve problem of mm/pixel ratio calculation would be placing camera and whole system in stable fixed position, measure all data required for calculation, and hope nobody will ever move anything. But this is unacceptable because I want Robot-Vision to be easily useable in various environments.

Also I decided to use black background because it solve problem of shadows which occurs in the case of white background.


Background_Area_Configuration application

Main tasks of this module are:

  •  Extract markers from background 
  •  Determine markers image position 
  •  Determine markers image and real distances 
  •  Capture static background image 


How to use this module for proper background calibration?

First step is to manipulate color filter scroll bars to extract markers from background. If there are some noises change settings of size filter to remove them. Calibration of markers recognition is correct when on the screen you will see only 4 white shapes of markers. Then head in to marker distance settings and enter measured distances between markers. Final step is to hit “Calibrate” button save data and that is it.

Image readings of marker distances will tell you if camera is placed in right way. For example if real distance between left bottom and right bottom, and left upper and right upper markers are the same in mm so they should be almost the same in pixels. If they are not that simply tell user that camera is not directed straight to the background and it has to be changed.

Background Area Configuration user interface - not calibrated


Background Area Configuration user interface - proper settings


Background Area Configuration user interface - calibration done


Methodology - how does Background_Area_Configuration works?

All of my classes that has user interface use my standard functions to save settings, load settings, enable, disable and refresh ui, to solve basic problems connected with dialog actions. In this section I will focus on more specific functions that are connected with methodology of image processing and core functionality of class, rather than explain mention before basic methods. Also I assume that reader has some basic knowledge of OpenCV.

Module Camera_Image [1] delivers information about current image. Then first thing that has to be done in order to get marker position is to extract them from background. To do this I use cv::inRange [2] function with variables connected to scrollbars. That’s basically RGB color filter that leave on the result matrix only pixels that pass through it. Thanks to this I receive binary image with extracted markers and nothing else except potential noises. Next step is to get image position of markers, to do this I use cv::findContours [3] method. All objects contours from binary image are placed in vector, and then I deal with noises by contour size filter that is simply for loop with size conditions. After that I got 4 markers contours in array and here I determine their mass centers using cv::Moments [3] and mathematical operations. Array of mass centers cv::Point [4] of markers is sorted to assign values to proper variables (left-upper marker to left-upper marker variable and so on). On the next move I can finally calculate distances between markers on image and save this, and user information of real distances for future mm/pixel ratio calculations. All data relevant for future calculations are placed in structure to be able to share with other classes. Background working zone is displayed using cv::line [4] and cv::putText [4] functions. Also if calibration process will end with success background image is taken for future image subtractions. Of course if something go wrong during calibration process (more than 4 contours are found etc.) error will display information and calibration will stop.

Whole configuration process can be done in few secounds if camera is placed straight to background.