Locations

Vision System for Industrial Robots

 
Mateusz Dziedzic
Poznan University of Technology
Wrocław, Poland
See full Bio Contact
 
Entry date: 06-Aug-2014
Final Submission: 01-Jul-2015
Robot-Vision is a system which calculate objects real position based on readings from camera. Also system allow for color and size object identification and classification. Data about objects can be delivered to a robot to create random placed objects managing applications.
 

Project details

Project goal and motivation


The aim of this project is to create a system that will provide information about the location of objects to an industrial robot based on readings from the camera. Information received by a robot can be used in many industrial applications where position of object is unknown at the stage of creation robot program, or when there is a need for objects identification for example randomly placed object sorting application. Creation of Robot-Vision was motivated by possibilities of embedded computers and benefits of using industrial robots along with computer vision systems. Using computer vision can highly decrease cost of industrial applications and give additional possibilities that will improve whole process. Industrial computer vision systems are very expensive that can cost up to 9000 euro. By making this project I show that Colibri T20 computer on module can handle main processing unit task of that kind of system, and it can become affordable.


Key aspects of work


Main part of project - software application for Colibri T20 included implementation of:

  • Function that periodically read image from camera
  • User interface dialog that allow for camera configuration
  • User interface dialog that allow for background settings calibration
  • Image processing algorithms that extract objects from background
  • Functions that calculate object mass center position on image
  • Functions that calculate object orientation on image
  • User interface dialog that allow for calibration of image processing algorithms in order to calibrate system for different environments to increase quality of objects position calculation
  • Image processing algorithms that identifies objects color and size
  • User interface dialog that allow for auto model object calibration
  • Functions that classify objects based on model object information
  • Position calculation algorithms that calculate real position of objects based on their image position and background data
  • Functions that prepare communication frame with information about objects
  • Functions that allow for RS232 communication
  • User interface dialog that allow for communication settings configuration
  • User main main window that allow user to operate the system, and deliver object list, processed image and live preview

 

Colibri T20 software configuration included preparation of:

  • Linux operating system
  • OpenCV and Qt libraries
  • Qt Creator environment

 

Hardware development included:

  • Selection and integration of system components
  • Creation of power supply and communication PCB module
  • Creation of device case and custom camera mounting

 

Industrial robot Stäubli TX60L sorting randomly placed objects software project included:

  • Calibration of robot tool
  • Calibration of local base
  • Implementation of frame decoding agorithm
  • Implementation of robot moves algorithm

 

System overview



Final look of Robot-Vision case

 


Main window of user interface

 


Robot-Vision system with Stäubli TX60L

 


System hardware connections

 

Contact and information


 
  For any questions feel free to write on robot.vision.system@gmail.com

  Also if you want, you can visit google+ profile and facebook page:

 Google+             Facebook 

Updates Hide

Update #10119  |  21 Oct 2014
0
The first update describes all elements of the system except the robot.
Update #10120  |  23 Oct 2014
0
Update include information about my first steps with Colibri T20: flashing linux, first boot, and first program.
Update #10125  |  10 Nov 2014
0
Update include information about Qt configuration and first test application.
Update #10169  |  18 Jan 2015
0
Update include information about OpenCV cross compilation and test application.
Update #10180  |  27 Feb 2015
0
Update include information about hardware and software architecture.
Update #10187  |  28 Mar 2015
0
Update include information about board project, schematic and application.
Update #10188  |  01 Apr 2015
0
Update include information about custom made device case for core elements of system. Also I will tell why I decided to replace old USB hub.
Update #10189  |  01 Apr 2015
0
Update include information about custom made mounting for camera.
Update #10191  |  13 Apr 2015
0
Update include information about software modules responsible for cyclic taking image from camera and configuration camera settings.
Update #10195  |  28 Apr 2015
0
Update include information about software module used for background area configuration.
Update #10196  |  30 Apr 2015
0
Update include information about software module responsible for find and classify objects.
Update #10198  |  23 May 2015
0
Update include information about software module responsible for calculation object local base position and orientation including conversion from pixels to millimeters.
Update #10204  |  12 Jun 2015
0
Update include information about last two software modules. User interface main window and module responsible for RS232 communication.
Update #10205  |  12 Jun 2015
0
Update include information about accuracy measurements presented in tables and 3D graphs, system total cost information, conclusions and plans for next updates.
Update #10207  |  23 Jun 2015
0
Update include information about robot, controller and objects used in sorting application that show potential usage of created vision system.
Update #10210  |  28 Jun 2015
0
Update include information about software for robot application development and robot algorithm.
Update #10211  |  30 Jun 2015
0
Update include my conclusions, thanks and final video in which you will find quick system overview and result of final application where industrial robot use data from Robot-Vision in order to sort randomly placed objects.

Comments Hide

Vinicius Bazan Pinto Fernandes
 

Dear Mateuszm, Robot-Vision is a great project. Reading through it, I've noticed that what you are creating could be also useful for my project, WITRA (you can find it on Toradex Challenge projects). I've created a wearable interface for the user to teleoperate a industrial robot. Putting these ideas together could be really useful, since we could use your computer vision for obstacle avoidance along with teleoperation and some kind of feedback to provide the user with a unique experience.

Mateusz Dziedzic
 

Hello, I am glad to hear that you like my project. After I develop application of object inspection for sorting I plan to use my system for medical purpose as well so after this we can think how to combine our ideas.