Fun Computer Vision opencv tutorials and ..


My favourite

  • Opencv tutorial people detection
  • Head people cascade download
  • Opencv tutorial optical flow
  • Opencv Video stabilization
  • Opencv car dataset download
  • Opencv tutorial Transparent mask
  • Opencv videowriter
  • Opencv FFMPEG
  • Opencv Canny edge and hough lines
  • Opencv in Docker, Build app in docker tutorial

    How to run and build application based on Opencv in Docker container is purpose of this tutorial. The process described bellow is how to install Docker engine in windows machine, by command line download base Ubuntu image, install Opencv inside the container and commit changes to create own image to save your installation to add what ever opencv app you want to scale in Docker.. Cool ? No magic. Really
    docker opencv app

    Install Docker in windows

    The important think is that windows running docker machine in Hyper-V virtual machine available under Windows 10. If you select this installation you can have troubles with Virtual Box no longer work after Hyper-V is set to enable.. There is option use also installation based on Oracle Virtual Box but i select easy install using the first options which is nicely integrated with Windows 10 and brings me advantage of docker development under ubuntu on windows machines.. 

    For the installation just follow the step by step tutorial on first link for Hyper-V based installation

    Download and install Links !Read text before!
    To download Docker for Windows Hyper-V use this link
    To download Docker for Windows using Oracle VirtualBox link

    Play with docker

    After the easy installation you should have available icon in Windows like in following image. 

    Docker windows opencv
    This allows you to control your docker engine like start stop and update, all the work is done by easy commands in CMD command line.. Do not worry of command line.. It is easy like to write email.. 
    Docker opencv

    The installed docker is now available through cmd line.. 
    Docker help tutorial
    You can simply verify by issue command as me docker help and if you have succesfully installed docker you should see the same output of possible command options. 

    Docker basics 

    Just briefly about docker.. I try to use simplest word to describe this. By issue command docker ps you can see that i do not have anything set up. The main components are docker container and image. 


    Docker image

    It is easy, Image is something like CD, all you need to install is there ready to be used by specific application. The most often the application you want to develop is included in this image. For example, basic ubuntu installation. This image could be extend to satisfy specific needs of developer. 

    Docker contained

    The container is running image instance. I have one image with ubuntu and http server inside. I could run from one image several containers with exactly same settings. This allows me easy scalable  deployment of my application from 10 users  on one container to 10 milion in hundred container of same app and setting from this base image. CD that could be installed several times is just container..

    Install Opencv in ubuntu docker image

    You can use different kind of images to start. I would like to recommend ubuntu, where we install our opencv library. The process is that you take some base image from already exist repository. This is just set up that you run container and image is basically downloaded in background. After you install opencv and prerequisites and commit changes inside container. This commit create your unique image with your saved changes (installed) opencv and save locally.. If you skipp this step your changes inside container are lost..  

    C:\docker run -it ubuntu bash

    By issue this command you run ubuntu basic container and start ubuntu bash command line.. You can exit bash command line inside ubuntu containe by type EXIT.
    I just issue different command docker run -it ID_of_image bash because i have already image installed. But after you issue C:\docker run -it ubuntu bash you will be inside linux like on picture bellow. 
    opencv docker

    Now install opencv follow the documentation.

    All is done in bash ubuntu command line. Again no mystery and who is afraid of command line is just because he used to work in different environment. But it is necessary and behind is nothing really hard.  Now we follow exactly same installation for Linux distribution, I got somethink for Debian here just a short version but you will be find with follow approach.. 

    1. install Required dependencies 
    We are root that mean sudo is not necessary, just copy this 3 lines one by one and follow the several instructions. Basically sometimes is write Y to command line. Nothink else. 

    apt-get install build-essential

    apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev

    apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

    Now you have everything you need to build opencv. Now issue this command to download source code from git. This will create you the opencv folder, and after by cd jump into opencv folder. By issue mkdir build you create new directory. Name as build! Jump into by cd build.. You can start install right now.

    git clone

    cd opencv 

    mkdir build

    cd build

    Now you are in build folder. This one is empty but is under the opecv where the all source code are located. Here just run following command for basic configuration.. Now take some time and after you will see some overview what is available to be installed. For example avlibs to use video captures and other opencv vide options.. Do not forgot the dots on next line :)

    cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..

    Now is time to issue another command which build your opencv from source based on cmake configuration. By type simple make in linux bash.

    make                      OR    make -j4 to run in parallel 4 build jobs.

    But just make is quite fine. Now this steps take several minutes. 10 to 40 depends how much power and -j jobs you run to make opencv. The last is to install opencv into the system by make install. 

    make install

    Done, your library are ready and you can develop your app under Linux (But this will be in next tutorial) in docker (/usr/local/lib) check the screen shot below for location cd /usr/local/lib , and list them by ls to display this.
    docker ubuntu opencv

     NOW we need to save what is in our container. When we are saving new container we create new image from we are starting containers.  

    !!Do not issue exit yet.. save first!! 

    Save changes mean - commit and create new image

    This create new image with all the changes and allow you start container from this image with all the configuration from previous step. Maybe run 1000 containers to serve the video application for millions of customers.  Open new commad line windows like on image. Run command that display running container docker -ps.

    Docker image

    Now from new cmd line save the container by this command in c:\> docker commit your_container_id name/description:version ,BE sure that c3f279d17e0a is the same container id  obtain by docker -ps command.. Like in example. 

    docker commit c3f279d17e0a ubuntu/image:version3

    This take some time and create your local image. Than could be again started by same command.
    C:\docker run -it ubuntu/image:version3 bash 

    Next steps for docker/opencv tutorials 

    Build the app inside the environment which is just prepare and use visual studio with remote build in ubuntu container.. The next tutorial already in progress.

    Reference for docker

    The best easy manual for docker is here

    As a big fan of OPENCV 3.3.0 There is what is new!

    Some my notes about new released. Based on changelog and released notes. 

    Deep neural network module is now accelerated with improved performance also moved into the main repository branch under opencv/modules/dnnThis module is also no more available in contrib branch. There was improved loading models from Troch and TensorFlow and many performance improvements. 

    Support of
    • Caffe 1
    • TensorFlow
    • Torch/PyTorch
    Interesting is available layer types list 
    • AbsVal
    • AveragePooling
    • BatchNormalization
    • Concatenation
    • Convolution (including dilated convolution)
    • Crop
    • Deconvolution, a.k.a. transposed convolution or full convolution
    • DetectionOutput (SSD-specific layer)
    • Dropout
    • Eltwise (+, *, max)
    • Flatten
    • FullyConnected
    • LRN
    • LSTM
    • MaxPooling
    • MaxUnpooling
    • MVN
    • NormalizeBBox (SSD-specific layer)
    • Padding
    • Permute
    • Power
    • PReLU (including ChannelPReLU with channel-specific slopes)
    • PriorBox (SSD-specific layer)
    • ReLU
    • RNN
    • Scale
    • Shift
    • Sigmoid
    • Slice
    • Softmax
    • Split
    • TanH

    And even more

    Advanced core performance AVX, AVX2, SSE and NEON
    New Python and C++ samples DNN C++ and Python api

    Another mainly performance improvements is 15% speed according to IPPICV from 2015.12 to 2017.2 version upgrade

    Opencv C++ 11 ready 

    I am interesting about news C++ 11 support. This should speed up development a bit. Tested on fedora distribution and Opencv should be build with -DENABLE_CXX11=ON. I will try on windows. I do not expect any problem. Build Opencv 3.3 with Visual Studio 2017 and CXX11 support.
    Examples like

    And dummy auto containers :)
    auto A = Mat_<double>({0, -1, 0, -1, 5, -1, 0, -1, 0}).reshape(1, 3);

    Opencv hardware-accelerated video encoding/decoding

    This is big in my eyes. Continue doing this.. Encoding and decoding of raw H.264 and MPEG1/2 video streams is supported, media containers are not supported yet.

    Big thanks to all contributors of OPENCV 3.3.0 to doing this great lib even better. 



    Machine learning by Andrew Ng

    machine learning

    I would like to summarize couple of thoughts about this famous coursera course. I just guess what you want to know. There is couple of facts important for me.

    Facts about

    • Link to course here
    • Price 75 dollars, not sure 
    • 11 weeks, 25 assignment from that 8 programming exercises in Matlab Octave.
    • You need to pass all assignment to get Certificate
    • Not a problem to finish in 5 weeks.
    • Video lectures, PDF, discussion, Matlab Octave background materials, data sets, and much more, teachers to help you any time
    • As a student, you focus on critical elements to really understand the machine learning. Not how to use and build model in TensorFlow like in some other course.. 
    • In Matlab, Octave you design and programming critical parts to be able understand how  machine learning works. Not python call of some black box. Real Math behind..
    Again and again constructing different Cost function for different type of problems to be able to evaluate how good your model is. 
    Gradient descent in different application and debugging if you are on right track. Advance method of optimization and different types of learning batch, stochastic gradient descent and many other for large scale machine learning.  

    • Some course about deep learning focus for example to using tensorflow and design model in tensorFlow. Not to essentials of ML implementations and    
    What is included

    • Linear regression to predict or maybe better to understand evaluation of some metrics based on some observation. 
    • Logistic regression to be able assigned something to class or not 0, 1
    • Neural networks, Cost function, Learning by back propagation and handling of over fitting.
    • SVM one of the best machine learning result of all time. 
    • K mean to be able observe several cluster center in unsupervised learning  
    • Evaluation, error according to recognize the problem before your learning ends nowhere. This practical part to recognize over fitting and high bias is everywhere.  It is right. The concept is quite known. The experience in ML is hard to learn almost harder than the theory itself. 
    • Unsupervised learning, K mean and PCA 
    • Quite lots of application example, Building and design of recommendation system and anomaly detection

    Basically like any others ML bachelor course in little bit extended focus in neural networks application and implementation details. Which is plus and very good starting point. 

    What is missing ? There is no topics like. This topics you should study after you fully understand the previous topics quite well, anyway.

    • Recurrent Neural networks 
    • Deep learning 
    • Convolution neural network 
    Thanks for this course.. I got my certificate on Linkedin. My knowledge are far behind but still this is great extension and recapitulation of already known thinks towards better understanding.. 

    Go and take it.. 

    VideoCapture IP camera stream, Web camera, File, images and VideoWriter

    Opencv reading video files, reading video stream, Images, IP and Web cameras. I would like to cover this all in one post. Yes, video writer is also important to store your results and achievements in video. There is couple of simple trick and if you follow them, you will never have a problem with the reading and writing video, stream, files in future.
    opencv web camera

    Basic opencv web camera reading

    There is couple think you need to take care. My favorite installation on windows platform is trough NUGET package system. It is easy in few steps. I describe this many times for example VS 2017 here. Nuget set up your project without any linking settings, library path selection, global environmental variables and you can directly start coding in few seconds. Just select and install nuget and compile code below. Nothing else.  You need to take care if you have included several thinks. highgui.hpp core.hpp, imgproc.hpp, videoio, imgcodecs. All of them are not necessary to read the web camera but for example for video stream from IP camera is possible that you really need them all.

    VideoCapture web camera code

    VideoCapture cap(0); is mean open the default camera web camera. Most of the time this mean web camera on your laptop or plugged in any USB camera. The video is read in 'never ending for(;;) loop' which is break when the video from camera is not available by condition if (!cap.isOpened()). Finally the Mat img;   cap >> img;  copy image from default camera devices into your MAT container. The rest is just display. 

    #include "opencv2\highgui.hpp"
    #include "opencv2\imgproc.hpp"
    #include "opencv2\objdetect\objdetect.hpp"
    #include "opencv2\videoio\videoio.hpp"
    #include "opencv2\imgcodecs\imgcodecs.hpp"
    #include "opencv2\core\core.hpp"
    #include <vector>
    #include <stdio.h>
    #include <windows.h>
    #include <iostream>
    #include <time.h>
    using namespace cv;
    using namespace std;
    int main(int argc, const char** argv)
     VideoCapture cap(0);
     for (;;)
      if (!cap.isOpened()) {
       cout << "Video Capture Fail" << endl;
      else {
       Mat img;
       cap >> img;
       namedWindow("Video", WINDOW_AUTOSIZE);
       imshow("Video", img);
       int key2 = waitKey(20);
     return 0;

    Opencv video file reading

    Look at the example above for reading the camera. There is almost no difference. Just one, small and straightforward. As a parameter of cap put instead of default devices cap(0) the file name or path you want to open. There is almost always trouble with path.  In this example you just read the files that are located under your project. You can also read the file from different location or on one place by using the full path into some video folder as you can see in following examples. 

    VideoCapture cap("movie.vmw");
    VideoCapture cap("movie.mp4");
    VideoCapture cap("");
    VideoCapture cap("");

    VideoCapture cap("C:/cm/");
    VideoCapture cap("C:/cm/movie.mp4");

    Opencv Image read from file and writing 

    This is super easy task. Into the our Mat container  image load the image 6.jpg on this C:/adress/ path. There is something different what is great to have in case you are reading lots of images inside the folder. 

    Mat image;
    image = imread("C:/adress/6.jpg", CV_LOAD_IMAGE_COLOR);

    CV_LOAD_IMAGE_COLOR is defined parameter to tell reader that i want MAT with 3 colors. Basically 3 MAT array container of image size. One for blue, red and green color channel. CV_LOAD_IMAGE_GRAYSCALE is defined to tall reader that i want gray scale. Basically only one Mat of the weight(cols) and high(rows) of the image.. 

    To write results into file just use imwrite where the first string is just name of your result and image. Image is MAT containing what you want to save..

    imwrite("image.jpg", image);

    Opencv video stream verification

    I am using good practice. instead of try stream directly in opencv. I prefer to verify my stream in VLC player. It is faster than modify code and compile again of passing the camera URL as parameter. Also the VLC ask for potential user name and password if its necessary. What is annoying is that all the cameras own stream URL format.  The best approach is to find your IP camera model on and apply to verify inside the VLC. After verification put this directly to VideoCapture cap("http://IP:PORT/mjpeg/video.mjpg?counter"); 

    rtsp://IP:PORT/various url
    Remember that VLC ask for password Opencv NOT.. Just add rtsp://username:password@IP:PORT
    Important FFMPEG is needed in Linux. In case of Nuget packages depends but the stream sometimes needs special installation. 

    Opencv tutorial code IP camera pseudo code

    There is 3 function.. 
    First of all, the main function at the end, where are established 2 threads to read the camera stream..

    In Main
    • Thread call the stream function for both camera with different IP camera URL                       thread cam1(stream, "http://xxxxxxxR");
    • To run the function stream inside the thread with url as parametr use.                       cam1.join();
    void stream

    • Capture video from url strCamera VideoCapture cap(strCamera) 
    • Fill the frame from cap  cap >> frame;
    • Detect people in camera detect(frame, strCamera);
    void detect

    Opencv C++ IP camera code

    #include <iostream>
    #include <thread>
    #include "opencv2/opencv.hpp"
    #include <vector>
    using namespace std;
    using namespace cv;
    void detect(Mat img, String strCamera) {
      string cascadeName1 = "haar_cascade_for_people_detection.xml";
      CascadeClassifier detectorBody;
      bool loaded1 = detectorBody.load(cascadeName1);
      Mat original;
      vector human;
      cvtColor(img, img, CV_BGR2GRAY);
      equalizeHist(img, img);
      detectorBody.detectMultiScale(img, human, 1.1, 2, 0 | 1, Size(40, 80), Size(400,480 ));
      if (human.size() > 0) 
          for (int gg = 0; gg < human.size(); gg++) 
          rectangle(original, human[gg].tl(), human[gg].br(), Scalar(0, 0, 255), 2, 8, 0);
      imshow("Detect " + strCamera, original);
      int key6 = waitKey(40);
    //End of the detect
    void stream(String strCamera) {
    VideoCapture cap(strCamera);
     if (cap.isOpened()) { 
          while (true) {
            Mat frame;
            cap >> frame; 
            resize(frame, frame, Size(640, 480));  
            detect(frame, strCamera);
    int main() {
        thread cam1(stream, "http://xxxxxxxR");
        thread cam2(stream, "http://xxxxxxxR");
        return 0;

    write video into file

    On windows machine i usually works with simple wmv format. Works perfectly. Remember the golden rule of video writer in opencv. image Mat have to match same size as VideoWriter. The image is mat that i want to write as frame into the video.. Before I put them into the VideoWriter, I always resize to target size.. This causing the lots of trouble. You cannot see the video result only for that reason.  
    Size SizeOfVideo = cv::Size(1024, 740);  VideoWriter video("Result.wmv", CV_FOURCC('W', 'M', 'V', '2'), CAP_PROP_FPS,SizeOfVideo, true);
    resize(image, image, Size(800, 600)); 
     video << image; 

    Microsoft and cognitive service computer vision is one of the most visible on build 2017 

    Looks pretty cool, Microsoft machine learning for safety in work environment. Recognizes people identity, real time tracking, evaluate their role in respect with position  against to safety. We will see in near future this technology more often. Lets have a look to strategy little bit closer. All who play with computer vision just know, that the demonstration is one think and the general service available to the Employers is something totally different. Cool, interesting as amazon GO. Lets compare with Amazon Go much closer.
    Microsoft Build conference 2017

    Microsoft cognitive vs Amazon Go

    Both are huge co-leaders on the market with cloud computing. Obviously, Who have this computational power should entered the future of everything, starting from small IOT devices to large scale distributed intelligent platforms driven by machine learning. There is maybe question, why microsoft do not follow the Amazon Go concept in general form, open to everybody. Analysis of video stream in retail statistics and marketing. I know it is conference demonstration, mentioned everywhere. Maybe I can add some original idea, why to slightly change the focus. 

    Employees monitoring system 

    This is related to mentality, human resources and people comfort in working environment. Sure as a employer you can control how effectively your money are spend covered by all by safety of the employees buzzwords. In some environment with strict rules question of life or serious health problems is necessary to go this way. Hard stop and sensors are already part of this kind of environments. Can we expect that this environments send video to be analysed somewhere as a service? Is this really the best market? Is this what everybody want. Is this dangerous or provide benefit with safety or making just people leaving the companies that push privacy questions so hard behind the borders.

    Do you know details about the architecture ? Let me comment.. As a service to process video streaming in cloud, There is several problem and much more critical one, when we are talking about heath monitoring of anyone. 

    Amazon go do right think 

    Advantage of the Amazon go concept is more features. They are count not only to computer vision but also to sensor fusion from different sources to provide better features processed by deep learning. Main advantage against Microsoft is that Amazon focus on their own environment but Microsoft to general one. Problem is general one not in retail but in environment, where the human health is concerned. This should be critical problem. Where Amazon has same situation in same environment and lights conditions handled hundred times. Microsoft should adapt on every possible light, environment and other variables related to deployment in different places. What is worse in much more critical applications and situations.

    This could be hard and slow down to go with this on market..

    Microsoft go to harder segment for computer vision

    Again, Just a comparison Microsoft concept and situation of and Amazon go. Why the microsoft position is just a little bit uncomfortable. 

    Amazon go can easily solve customers problem, refund the money for customers complains and the application is  basically not so critical. In the other hand. Microsoft need in such a cases like hospital and security critical environment count whit certification problems, law issues and more. In Microsoft segment is much more responsibility. This is why amazon can speed up the development based on experience in real application instead of segment when is necessary be perfect.
    On the other hands, AI is able control cars itself without human. This segment is also little bit risky from that point of view.

    I really think that they should start competition with amazon Go as general service for all retails. Somehow bound the requirements for the stores environment and use sensor fusion. For example in medical application like hospital is maybe good idea to use thermo cameras combined which normal one..

     Provide benefits for customer and high valuable information about the customers behavior to boost the retail and advertisement impact. This is just a save. No problem issue. 

    Video stream delivery 

    I think that microsoft mostly provide all AI in form of service. You deliver stream or image request and We give you a results back. The limitation here is bandwidth, Video stream quality and time delays. What if the network go down, resolution or frame rate and something happened. Cars has brain in their body.. This service has brain somewhere else. I expect, do not know for sure. 
    Microsoft can guaranty availability of the service and accuracy, but who will be responsible in case that the system is not able to connect and something happened. Just a case. You always need to count with the worse one and hope that never happened. 

    Delay, Video stream delays of real time broad casting. Delay is here. Delay in video will be here also in future. When some situation occurred the second play roles in microsoft case. If the alert response come back with delay. This service will be replace by something else.   Maybe is better to deploy something like in form of IoT devices than services. Maybe this kind of service should provide pretrained parameters of deep neural network for IoT devices, compute only forward pass without learning. But not transfer, analyze the whole content and response somewhere..  Who know what will be final solution. Power of cloud, power of machine learning in this case needs to be response available on time. That mean calculation directly in camera devices or on local network.

    Good luck to Microsoft. Here is a fan. Let me test your solution ! :) . Be careful

    Do you like this post ?? Feel free to share. This keep me doing this kind of content. Hopefully I have got time to also some new tutorial post.. Hopefully