Using OpenCV in GTK+ applications

Although OpenCV comes with it's own windowing system for displaying any OpenCV images, for fully functional applications with user interface elements such as buttons, menus, and radio buttons, maybe you would find it useful to use widget libraries such as GTK+ or QT. Drawing OpenCV image surfaces in GTK+ applications is not much of a difficult task than adding a couple of extra lines in your existing GTK+ applications. And again there are plenty of ways to accomplish the same results, but here I will show how I have achieved it. The sample program here is written using GTK+ C++ binding GTKmm, but the technique and functions should be same across all the GTK+ bindings.

As usual we create a GTK+ top level window by creating a MainWindow object. MainWindow, which is itself a GtkWindow widget contains one GtkFrame and one GtkDrawingArea widgets. The class definition of MainWindow is shown below:

class MainWindow : public Gtk::Window
{
    protected:
        Gtk::Frame video_frame;
        VideoArea video_area;
       
    public:
        MainWindow ();
        virtual ~MainWindow();
};

In the above code, VideoArea is the GtkDrawingArea widget object defined as follows:

class VideoArea : public Gtk::DrawingArea
{
    protected:
        cv::VideoCapture cv_cap;
        bool cv_opened;
        virtual bool on_draw (const Cairo::RefPtr<Cairo::Context> &cr);
        bool on_timeout ();
    public:
        VideoArea ();
        virtual ~VideoArea();  
};

We are going to use GtkDrawingArea widget for the purpose of displaying OpenCV image surface. To do that, we have to put OpenCV related functions in the GtkDrawingArea on_draw function. This on_draw function will be called every time draw signal is emitted from GtkDrawingArea widget. As can be seen from the VideoArea object definition, we also have a timer function on_timeout to call on_draw function in a regular manner by invalidating GtkDrawingArea widget. So when a VideoArea object is created in the MainWindow object OpenCV is initialized like shown in the code below:

VideoArea::VideoArea() : cv_opened(false)
{
    cv_cap.open(0);
   
    if (cv_cap.isOpened() == true) {
        cv_opened = true;
        Glib::signal_timeout().connect(sigc::mem_fun(*this, &VideoArea::on_timeout), 50);
    }
   
}

Here, we connect a glib timeout signal, which will call on_timeout function every 50 milliseconds interval. The functions for on_timeout and on_draw are shown below:

bool VideoArea::on_timeout()
{
    Glib::RefPtr<Gdk::Window> win = get_window();
    if (win)
    {
        Gdk::Rectangle r(0, 0, get_allocation().get_width(), get_allocation().get_height());
        win->invalidate_rect(r, false);
    }
    return true;
}

bool VideoArea::on_draw(const Cairo::RefPtr<Cairo::Context> &cr)
{
    if (!cv_opened) return false;
   
    cv::Mat cv_frame, cv_frame1;

    cv_cap.read(cv_frame);
   
    if (cv_frame.empty()) return false;
   
    cv::cvtColor (cv_frame, cv_frame1, CV_BGR2RGB);
       
    Gdk::Cairo::set_source_pixbuf (cr, Gdk::Pixbuf::create_from_data(cv_frame1.data, Gdk::COLORSPACE_RGB, false, 8, cv_frame1.cols, cv_frame1.rows, cv_frame1.step));
   
    cr->paint();
       
    return true;
}

In the on_draw function, we convert OpenCV surface to RGB channel by using cvtColor, since GTK widgets uses RGB format. Finally, we use Gdk::Pixbuf::create_from_data to fill Cairo surface with OpenCV surface pixel buffers.

Download the complete program source code and test it by yourself!

gtkcv.zip

To compile the files in gtkcv.zip, you need to have OpenCV and gtkmm development files installed on your operating system. On Ubuntu just follow the instructions below:

Installing OpenCV:

Downloading OpenCV and creating compile directory,

From http://opencv.org/ grab opencv-3.0.0 and extract the opencv-3.0 archive file. Then, create a folder named build inside the opencv-3.0.0 folder. Then, open a terminal into the build folder. Then, in the terminal put the following commands:

# setup compile environment for OpenCV
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr -D WITH_OPENMP=ON -D WITH_CUDA=OFF .. 

# compile sources in 4 threads
make -j 4 

# install OpenCV

sudo make install

Installing gtkmm:

sudo apt-get install libgtkmm-3.0-dev

that's it!, now extract the gtkcv.zip file and open a terminal in the gtkcv folder and execute the following command:

g++ -o gtkcv main.cpp MainWindow.cpp VideoArea.cpp `pkg-config --cflags --libs gtkmm-3.0 opencv`

this will produce gtkcv executable file.