Building applications with OpenCV inside Ubuntu with Code::Blocks and CMake

www.gunjangupta.net

For building applications with OpenCV we need to set our Ubuntu as per earlier chapters where we discussed about how to set Ubuntu for PandaBoard and How to install OpenCV in Ubuntu. After these steps we can check in terminal whether successful installation of OpenCV took place or not.

pkg-config --cflags OpenCV

pkg-config --libs OpenCV

Fig 1: Ensuring OpenCV Installation

1    OpenCV with CMake

CMake is a cross-platform free software program for managing the build process of software using a compiler-independent method. It is designed to support directory hierarchies and applications that depend on multiple libraries, and for use in conjunction with native build environments such as make, Apple's Xcode, and Microsoft Visual Studio. It also has minimal dependencies, requiring only a C++ compiler on its own build system.

CMake can handle in-place and out-of-place builds, enabling several builds from the same source tree, and cross-compilation. The ability to build a directory tree outside the source tree is a key feature, ensuring that if a build directory is removed, the source file remains unaffected.

Another feature of CMake is the ability to generate a cache to be used with a graphical editor, which, when CMake is run, can locate executables, files and libraries. This information goes into the cache, which can then be tailored before generating the native build files.

Complicated directory hierarchies and applications that rely on several libraries are well supported by CMake. For instance, CMake is able to accommodate a project that has multiple toolkits, or libraries that each have multiple directories. In addition, CMake can work with projects that require executables to be created before generating code to be compiled for the final application. Its open-source, extensible design allows CMake to be adapted as necessary for specific projects.

CMake can generate makefiles for many platforms and IDEs including UNIX, Windows, Mac OS X, OS/2, MSVC, Cygwin, MinGW and Xcode.

The build process with CMake takes place in two stages. First, standard build files are created from configuration files. Then the platform's native build tools are used for the actual building.

Each build project contains a CMakeLists.txt file in every directory that controls the build process. The CMakeLists.txt file has one or more commands in the form COMMAND (args...), with COMMAND representing the name of each command and args the list of arguments, each separated by white space. While there are many built-in rules for compiling the software libraries (static and dynamic) and executables, there are also provisions for custom build rules. Some build dependencies can be determined automatically. Advanced users can also create and incorporate additional makefile generators to support their specific compiler and OS needs.

Example CMake file

cmake_minimum_required (VERSION 2.6)

PROJECT(WebcamFaceRec)

# Requires OpenCV v2.4.1 or later

FIND_PACKAGE( OpenCV REQUIRED )

IF (${OpenCV_VERSION} VERSION_LESS 2.4.1)

    MESSAGE(FATAL_ERROR "OpenCV version is not compatible : ${OpenCV_VERSION}. FaceRec requires atleast OpenCV v2.4.1")

ENDIF()

SET(SRC

    main.cpp

    detectObject.cpp

    preprocessFace.cpp

    recognition.cpp

    ImageUtils_0.7.cpp

)

ADD_EXECUTABLE( ${PROJECT_NAME} ${SRC} )

TARGET_LINK_LIBRARIES( ${PROJECT_NAME}  ${OpenCV_LIBS} )

Now go to the folder containing CMakeLists.txt file in terminal and write command for configuring cmake. The command is

gunjan@ubuntu:~/Desktop/Projects/Chapter8_FaceRecognition$ ./cmake .

Fig 2: CMake

gunjan@ubuntu:~/Desktop/Projects/Chapter8_FaceRecognition$ ./make

Fig 3: Make

After make of the project is successful you can execute the generated application with its name

For example here it is

 gunjan@ubuntu:~/Desktop/Projects/Chapter8_FaceRecognition$ ./WebcamFaceRec

Fig 4: Execution of Program

2    OpenCV with Code::Blocks

Code::Blocks is a free and open source, cross-platform IDE which supports multiple compilers including GCC and Visual C++. It is developed in C++ using wxWidgets as the GUI toolkit. Using a plugin architecture, its capabilities and features are defined by the provided plugins. Currently, Code::Blocks is oriented towards C, C++, and Fortran. It can also be used for creating ARM, AVR, D, DirectX, GLUT, GTK+,MATLAB, OpenGL, Qt and wx programs and applications, although in some cases installing third-party SDKs or frameworks is necessary.

Code::Blocks is being developed for Windows, Linux, and Mac OS X and has been ported to FreeBSD, OpenBSD,and Solaris.

2.1    Features

Compilers

Code::Blocks supports multiple compilers, including MinGW / GCC, Digital Mars, Microsoft Visual C++, Borland C++, LLVM Clang, Watcom, LCC and the Intel C++ compiler. Although the IDE was designed for the C++ language, there is some support for compilers of other languages, including GNU Fortran, Digital Mars D and GNU GDC. A plug-in system is included to support other programming languages.

Code editor

The IDE features syntax highlighting and code folding through the use of the Scintilla editor component, C++ code completion and class browser and an integrated to-do list. All of the open files are organized into tabs, which can be closed and opened at the user's will with the navigation pane or the close (X) button on the tabs. The code editor supports font and font size selection, this allows the user to choose the font and font size with which he is comfortable. In addition, it can provide specific syntax highlighting color theme that can be personalized.

Debugger

The Code::Blocks debugger has full breakpoint support. It also allows the user to debug their program by having access and using the local function symbol and argument display, user-defined watches, call stack, disassembly, custom memory dump, thread switching, CPU registers, GNU GDB Interface and MS CDB (not fully supported yet).

GUI designer

As of Code::Blocks 13.12 comes with a GUI designer called wxSmith. It is a derivative port of wxWidgets based on the 2.9.4 version.To make a complete wxWidgets application, the appropriate wx SDK must be installed with set environment variables.

User migration

Some of Code::Blocks' features are targeted at users migrating from other IDEs - these include Dev-C++ and Microsoft Visual C++ project import (MSVC 7 & 10), and Dev-C++ Devpak support.

Project files and build system

Code::Blocks uses a custom build system, which stores its information in XML-based project files, but it can optionally use external makefiles, which simplifies interfacing with projects using the GNU and Qt Software's qmake build systems.

2.2    Installing Code::Blocks on Ubuntu

This is a quick guide to get Code::Blocks up and running on your Ubuntu based Linux distribution. It is also going to make sure you can develop wxWidgets applications on your box as well. Look at the bottom of this guide for a complete command line that will install all the packages in one operation.

First be sure you have the necessary software to compile and debug programs.

1. Install the compiler.

sudo apt-get install build-essential

2. Install the debugger.

sudo apt-get install gdb

You'll need to install wxWidgets to use Codeblocks. Revisions from 4051 and after use wxWidgets 2.8.4. If you want to install them, make sure you have your universe and multiverse repositories enabled and install the following packages.

3. Install wxWidgets library. (This package is all that is needed to run any application that uses wxWidgets. ie. Code::Blocks)

sudo apt-get install libwxgtk2.8-0

4. Install the wxWidgets developement packages. (This is used to develop wxWidgets applications of your own.)

sudo apt-get install libwxgtk2.8-dev

5. Install Code::Blocks.

sudo apt-get install codeblocks

2.3    Configuring CodeBlocks IDE for use with OpenCV

·         Create a new C++ console project in CodeBlocks.

·         With the new project, right click on the project in the left bar and choose "Build Options..".

·         Click on the "Search Directories" tab, then Compiler tab, and add these locations:                    /usr/local/include/opencv

/usr/local/include/opencv2

·         Still on the "Search Directories" tab, then click on the the Linker tab and add these locations:

/usr/local/lib

·         Then click on the "Linker Settings" tab and be sure to add all of the OpenCV libraries. This is a list of all the libraries that I had installed on my system:

§  /usr/local/lib/libopencv_calib3d.so

§  /usr/local/lib/libopencv_contrib.so

§  /usr/local/lib/libopencv_core.so

§  /usr/local/lib/libopencv_features2d.so

§  /usr/local/lib/libopencv_flann.so  

§  /usr/local/lib/libopencv_gpu.so

§  /usr/local/lib/libopencv_highgui.so

§  /usr/local/lib/libopencv_imgproc.so

§  /usr/local/lib/libopencv_legacy.so  

§  /usr/local/lib/libopencv_ml.so

§  /usr/local/lib/libopencv_nonfree.so

§  /usr/local/lib/libopencv_objdetect.so

§  /usr/local/lib/libopencv_photo.so

§  /usr/local/lib/libopencv_stitching.so

§  /usr/local/lib/libopencv_ts.so

§  /usr/local/lib/libopencv_video.so

§  /usr/local/lib/libopencv_videostab.so

·         Once you have all that taken care of you can now go ahead and edit the main.cpp source code file. Copy and paste this sample code to test out your setup:

#include

#include

#include

#include

int main(int argc, char** argv)

{

cv::Mat frame;

int c;

CvCapture *capture = cvCaptureFromCAM(0);

while(1)

{

frame = cvQueryFrame(capture);

cv::imshow("result",frame);

c = cv::waitKey(10);

if(c==27)

break;

}

return 0;

}

·         Done! You can now build and run the sample application to make sure it's working. You should get a new window displaying the image from your webcam. You should now be able to copy the project and use it as a template for future work.

Fig 5: Result

Sixth Sense Image Capturing Technology

www.gunjangupta.net

1 Introduction

The Sixth Sense image capturing technology is founded by Pranav Mistry at MIT Media Lab. The demo of sixth sense device also presented at TED Talks. The technology of capturing image is based on gesture detection. There are four different strips bent on the fingers of two hands as shown in figure 5.1. The camera records the video and processing of video can be performed frame by frame to detect the gesture shape based on color detection of the strips. When all the four strips is detected for certain amount of time, the image will be captured and cropped with the Region of Interest (ROI) evaluated from four points.

Figure 1: Gesture detection using 4 points

The improved design of this technology is getting ROI from two points, which are placed diagonally in rectangle. It has an advantage of less computational power and increased processing speed. Hence, the device can be designed on low cost embedded system. For that, the requirement of image processing is color detection and finding the position of color. Another thing is to use circles to detect colors as circle is the most compressed shape with given area. Detection of circle can be achieved by “Circular Hough Transform”.

2 Hough’s Transform 

The Hough transform is a feature extraction technique used in image analysis, computer vision, and digital image processing. The purpose of the technique is to find imperfect instances of objects within a certain class of shapes by a voting procedure. This voting procedure is carried out in a parameter space, from which object candidates are obtained as local maxima in a so-called accumulator space that is explicitly constructed by the algorithm for computing the Hough transform.

In automated analysis of digital images, a sub problem often arises of detecting simple shapes, such as straight lines, circles or ellipses. In many cases an edge detector can be used as a pre-processing stage to obtain image points or image pixels that are on the desired curve in the image space. Due to imperfections in either the image data or the edge detector, however, there may be missing points or pixels on the desired curves as well as spatial deviations between the ideal line/circle/ellipse and the noisy edge points as they are obtained from the edge detector. For these reasons, it is often non-trivial to group the extracted edge features to an appropriate set of lines, circles or ellipses. The purpose of the Hough transform is to address this problem by making it possible to perform groupings of edge points into object candidates by performing an explicit voting procedure over a set of parameterized image objects.

The simplest case of Hough transform is the linear transform for detecting straight lines. In the image space, the straight line can be described as,

y = mx + b

where, the parameter m is the slope of the line, and b is the intercept (y-intercept).

This is called the slope-intercept model of a straight line. In the Hough transform, a main idea is to consider the characteristics of the straight line not as discrete image points (x1, y1), (x2, y2), etc., but instead, in terms of its parameters according to the slope-intercept model, i.e., the slope parameter m and the intercept parameter b. In general, the straight line y = mx + b can be represented as a point (b, m) in the parameter space. However, vertical lines pose a problem. They are more naturally described as x = a, and would give rise to unbounded values of the slope parameter m. Thus, for computational reasons, Duda and Hart proposed the use of a different pair of parameters, denoted r and θ, for the lines in the Hough transform. These two values, taken in conjunction, define a polar coordinate. It is shown in figure 5.2. The points on line can be shown in (x,y) plane with (r, θ) coordinates in Hough’s space.

It is therefore possible to associate with each line of the image a pair (r, θ) which is unique if θ ∈ [0, π] and r ∈ R, or if θ ∈ [0, 2π] and r > 0.

Figure 2: Hough’s space denotation as (r, θ) for line.

The linear Hough transform algorithm uses a two-dimensional array, called an accumulator, to detect the existence of a line described by r = x cosθ + y sinθ. The dimension of the accumulator equals the number of unknown parameters, i.e., two, considering quantized values of r and θ in the pair (r,θ). For each pixel at (x,y) and its neighborhood, the Hough transform algorithm determines if there is enough evidence of a straight line at that pixel. If so, it will calculate the parameters (r,θ) of that line, and then look for the accumulator's bin that the parameters fall into, and increment the value of that bin. By finding the bins with the highest values, typically by looking for local maxima in the accumulator space, the most likely lines can be extracted, and their (approximate) geometric definitions read off. The simplest way of finding these peaks is by applying some form of threshold, but other techniques may yield better results in different circumstances - determining which lines are found as well as how many. Since the lines returned do not contain any length information, it is often necessary, in the next step, to find which parts of the image match up with which lines. Moreover, due to imperfection errors in the edge detection step, there will usually be errors in the accumulator space, which may make it non-trivial to find the appropriate peaks, and thus the appropriate lines.

The final result of the linear Hough transform is a two-dimensional array (matrix) similar to the accumulator -- one dimension of this matrix is the quantized angle θ and the other dimension is the quantized distance r. Each element of the matrix has a value equal to the number of points or pixels that are positioned on the line represented by quantized parameters (r, θ). So the element with the highest value indicates the straight line that is most represented in the input image.

The line can be found in image using Hough’s transform as explained following example.

Figure 3: Accumulator tables for (r, θ) for three points

Following steps are used to find the actual (r, θ) point representing the line passing by three points given in the figure 3:

• For each data point, a number of lines are plotted going through it, all at different angles. These are shown here as solid lines.
• For each solid line a line is plotted which is perpendicular to it and which intersects the origin. These are shown as dashed lines.
• The length (i.e. perpendicular distance to the origin) and angle of each dashed line is measured. In the diagram above, the results are shown in tables.
• This is repeated for each data point.
• A graph of the line lengths for each angle, known as a Hough space graph, is then created. It is shown in figure 4. The intersection point of all 3 curves is the actual value of (r, θ) representing the line passing through 3 points.

Figure 4: (r, θ) graph for 3 points

3 Circular Hough Transform

Unlike the linear HT, the CHT relies on equations for circles. The equation of the a circle is,

 r² = (x – a)² + (y – b)²

Here, a and b represent the coordinates for the center, and r is the radius of the circle. The parametric representation of this circle is

  x = a + r*cosθ

  y = b + r*sinθ

In contrast to a linear HT, a CHT relies on 3 parameters, which requires a larger computation time and memory for storage, increasing the complexity of extracting information from our image. For simplicity, most CHT programs set the radius to a constant value (hard coded) or provide the user with the option of setting a range (maximum and minimum) prior to running the application. 

For each edge point, a circle is drawn with that point as origin and radius r. The CHT also uses an array (3D) with the first two dimensions representing the coordinates of the circle and the last third specifying the radii. The values in the accumulator (array) are increased every time a circle is drawn with the desired radii over every edge point. The accumulator, which kept counts of how many circles pass through coordinates of each edge point, proceeds to a vote to find the highest count. The coordinates of the center of the circles in the images are the coordinates with the highest count.

4 Block diagram for circle detection

The block diagram of detection of circle is shown in figure 5.5. The input image is preprocessed to remove noise. The image is enhanced by adjusting the contrast and brightness. Also, the image is converted into grayscale. The next is to segment the image by edge detector to detect the edges in image. If the solid circle is present, the boundary of the circle will be extracted in white color in the output image from segmentation process. Finally, the image is given into Hough’s transform block to detect the lines or circle according to the transformation parameters.

Figure 5: Block diagram for Circle detection 

Result

The detection of two different circles is necessary to detect the two diagonal points of rectangle. For that purpose, application of Hough’s circle detection algorithm is repeated twice with different preprocessing parameters. The preprocessing block involves filtering and thresholding with the RGB value of pixels. So, red color can be detected by keeping the (R, G, B) filtering value in (150, 0, 0) to (255, 30, 30). The first set of RGB value shows the minimum value which is allowed in image to have white pixel value or 255 grayscale value. The second set of RGB value shows the maximum value, which are allowed to have white pixel value. If the image pixel value is in between the minimum and maximum values, the output image will contain white color or 255 grayscale value at the corresponding pixel position. If the image pixel value doesn’t satisfy the criteria and falls beneath the minimum or maximum value, the output image will contain black color or 0 grayscale value at the corresponding pixel position.

Same way the detection of blue color can be done by setting RGB minima at (0, 0, 150) and maxima at (30, 30, 255).

Hence, two different preprocessed images are available, which are filtered with red and blue color criteria. Applying Hough’s circle detection on these images produce two points, which are center of the red and blue color circle. Assuming that, (x₁, y₁) and (x₂, y₂) are the center of red and blue circles, respectively.

From two points (x₁, y₁) and (x₂, y₂), which are center of two circles, the rectangle can be drawn joining these points in diagonal direction. This rectangle can be considered as ROI for cropping an image. It is shown in figure 6.

Figure 6: Deciding the Region of Interest (ROI) from two points

Finally, the cropped image is stored. The above image processing can be done by OpenCV libraries. For designing the device pandaboard can be used. Raspberry Pi, which is low cost and lower specification than pandaboard, can also be used as this is the optimized sixth sense capturing using two color detection.

IMAGE IS CAPTURED USING GESTURE DETECTION & IT WILL BE STORED ON GOOGLE DRIVE THROUGH CLOUD COMPUTING.

Guide for Ubuntu Terminal Commands

www.gunjangupta.net

1    FILE COMMANDS

#ls –al =>Display all information about files/ directories

#pwd =>Show the path of current directory

#mkdir directory-name =>Create a directory

#rm file-name =>Delete file

#rm -r directory-nam =>Delete directory recursively

#rm -f file-name =>Forcefully remove file

#rm -rf directory-name =>Forcefully remove directory recursively

#cp file1 file2 =>Copy file1 to file2

#cp -r dir1 dir2 =>Copy dir1 to dir2, create dir2 if it doesn’t exist

#mv file1 file2 =>Rename source to dest / move source to directory

#ln –s /path/to/file-name link-name #Create symbolic link to file-name

#touch file =>Create or update file

#cat > file =>Place standard input into file

#more file =>Output contents of file

#head file =>Output first 10 lines of file

#tail file =>Output last 10 lines of file

#tail -f file =>Output contents of file as it grows starting with the last 10 lines

#gpg -c file =>Encrypt file

#gpg file.gpg =>Decrypt file

#wc =>print the number of bytes, words, and lines in files

#xargs =>Execute command lines from standard input

2    FILE PERMISSION RELATED

#chmod octal file-name =>Change the permissions of file to octal

Example

#chmod 777 /data/test.c =>Set rwx permission for owner,group,world

#chmod 755 /data/test.c =>Set rwx permission for owner,rw for group and world

#chown owner-user file =>Change owner of the file

#chown owner-user:owner-group file-name =>Change owner and group owner of the file

#chown owner-user:owner-group directory =>Change owner and group owner of the directory

3    COMPRESSION / ARCHIVES

#tar cf home.tar home =>Create tar named home.tar containing home/

#tar xf file.tar =>Extract the files from file.tar

#tar czf file.tar.gz files =>Create a tar with gzip compression

#gzip file =>Compress file and renames it to file.gz

4    INSTALL PACKAGE

#rpm -i pkgname.rpm =>Install rpm based package

#rpm -e pkgname =>Remove package

5    INSTALL FROM SOURCE

#./configure

#make

#make install

6    DIRECTORY TRAVERSE

#cd .. =>To go up one level of the directory tree

#cd =>Go to $HOME directory

#cd /test =>Change to /test directory

7    NETWORK

#ifconfig –a =>Display all network ports and ip address

#ifconfig eth0 =>Display specific ethernet port

#ethtool eth0 =>Linux tool to show ethernet status

#mii-tool eth0 =>Linux tool to show ethernet status

#ping host =>Send echo request to test connection

#whois domain =>Get who is information for domain

#dig domain =>Get DNS information for domain

#dig -x host =>Reverse lookup host

#host google.com =>Lookup DNS ip address for the name

#hostname –i =>Lookup local ip address

#wget file =>Download file

#netstat -tupl =>List active connections to / from system

8    DISK USAGE

#df –h =>Show free space on mounted filesystems

#df -i =>Show free inodes on mounted filesystems

#fdisk -l =>Show disks partitions sizes and types

#du -ah =>Display disk usage in human readable form

#du -sh =>Display total disk usage on the current directory

9    SEARCH

#grep pattern files =>Search for pattern in files

#grep -r pattern dir =>Search recursively for pattern in dir

#locate file =>Find all instances of file

#find /home/tom -name 'index*' =>Find files names that start with "index"

#find /home -size +10000k =>Find files larger than 10000k in /home

10  LOGIN (SSH AND TELNET)

#ssh user@host =>Connect to host as user

#ssh -p port user@host =>Connect to host using specific port

#telnet host =>Connect to the system using telnet port

11  FILE TRANSFER

scp

#scp file.txt server2:/tmp =>Secure copy file.txt to remote host /tmp folder

rsync

#rsync -a /home/apps /backup/ =>Synchronize source to destination

12  SYSTEM

#uname -a =>Displaylinux system information

#uname -r =>isplay kernel release information

#uptime =>Show how long the system has been running + load

#hostname =>Show system host name

#hostname -i =>Display the IP address of the host

#last reboot =>Show system reboot history

#date =>Show the current date and time

#cal =>Show this month calendar

#w =>Display who is online

#whoami =>Who you are logged in as

#finger user =>Display information about user

13  HARDWARE

#dmesg =>Detected hardware and boot messages

#cat /proc/cpuinfo =>CPU model

#cat /proc/meminfo =>Hardware memory

#cat /proc/interrupts =>Lists the number of interrupts per CPU per I/O device

#lshw =>Displays information on hardware configuration of the system

#lsblk =>Displays block device related information in Linux

#free -m =>Used and free memory (-m for MB)

#lspci -tv =>Show PCI devices

#lsusb -tv =>Show USB devices

#dmidecode =>Show hardware info from the BIOS

#hdparm -i /dev/sda =>Show info about disk sda

#hdparm -tT /dev/sda =>Do a read speed test on disk sda

#badblocks -s /dev/sda =>Test for unreadable blocks on disk sda

14  USERS

#id =>Show the active user id with login and group

#last =>Show last logins on the system

#who =>Show who is logged on the system

#groupadd admin =>Add group "admin"

#useradd -c "Sam Tomshi" =>g admin -m sam #Create user "sam"

#userdel sam =>Delete user sam

#adduser sam =>Add user "sam"

#usermod =>Modify user information

15  PROCESS RELATED

#ps =>Display your currently active processes

#ps aux | grep 'telnet' =>Find all process id related to telnet process

#pmap =>Memory map of process

#top =>Display all running processes

#killpid =>Kill process with mentioned pid id

#killall proc =>Kill all processes named proc

#pkill process-name =>Send signal to a process with its name

#bg =>Lists stopped or background jobs

#fg =>Brings the most recent job to foreground

#fg n =>Brings job n to the foreground