******************
Camera Calibration
******************

Introduction
============

This tutorial explains how to calibrate the camera. A good calibration
is necessary for image processing. The output of calibration process is
a calibration matrix. A calibration contains rectification coefficients
for lens distortion as well as intrinsic and extrinsic coefficients,
which are needed for reconstruction of 3D objects from 2D images.

-  Lines beginning with $ are terminal commands.

   -  To open a new terminal use the shortcut :kbd:`Ctrl + Alt + T`.

   -  To open a new tab inside an existing terminal use the shortcut
      :kbd:`Ctrl + Shift + T`.

   -  To kill a process in a terminal use the shortcut :kbd:`Ctrl + C`.

-  Lines beginning with # indicate the syntax of the commands.

-  Code is separated in boxes.

-  Code is case sensitive.


Using a USB Camera
==================

.. attention:: 
   If you don't have an usb camera or webcam, you have to use the gazebo camera in the turtlebot3 simulation.
   Gazebo already publishes the camera_info topic with calibration data, but you should still learn the procedure.

   #. Download the the calibration target gazebo model.
      :download:`Calibration Target Gazebo Model </_resources/code/tutorials/misc/calibration_target.zip>` 

   #. Extract it in ``~/.gazebo/models``.

   #. ``$ ros2 launch turtlebot3_gazebo turtlebot3_house.launch.py``

   #. Place it in your world.

   #. Skip this chapter.



To run the webcam, use the ROS2 package **usb_camera_driver**. 
It provides an interface to all standard USB cameras.

.. code-block:: bash

   $ cd ~/robot_ws/src
   $ git clone https://github.com/klintan/ros2_usb_camera
   $ cd ~/robot_ws
   $ colcon build
   $ source ~/.bashrc
   $ ros2 launch usb_camera_driver usb_camera_node.launch.py

.. admonition:: Task
   :class: attention

   Start the camera with the new launch file and visualize the image stream
   in RViz.

   #. ``$ rviz2``
   #. :guilabel:`Add` > :guilabel:`Image`

As you maybe already noticed, you got a ROS warning during the starting
process of the camera node: Using default calibration URL. This happens
because you are not providing any calibration information to the camera
node. For many image processing applications in ROS you **have to**
calibrate the camera.

Calibrating a Camera
====================

For calibration, the ROS package **camera_calibration** can be used.

.. code-block:: bash

   $ cd ~/robot_ws/src``
   $ git clone -b ros2 https://github.com/ros-perception/image_pipeline``
      # - Delete all folders in "image_pipeline" except "camera_calibration"
      #   or install all package dependencies
      # - On cv2 import error: ``sudo apt install python3-opencv``
   $ cd ~/robot_ws``
   $ colcon build``
   $ source ~/.bashrc``


Calibration Process
-------------------

#. **Read** the whole process once before starting calibration.

#. Print Checkerboard_ (not for simulation).

   .. _Checkerboard: http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration?action=AttachFile&do=get&target=check-108.pdf

#. Start the camera with 
   ``$ ros2 launch usb_camera_driver usb_camera_node.launch.py`` 
   or the simulation with
   ``$ ros2 launch turtlebot3_gazebo turtlebot3_house.launch.py``
   and place the calibration target in front of the turtlebot3

#. Check your *camera info* topic. All intrinsic camera matrices
   should be empty or zero (not in simulation).

#. Launch the calibration with correct parameters. 
   ``$ ros2 run camera_calibration cameracalibrator -c camera --size=8x6 --square=0.05 image:=/camera/image_raw``
   It will pop up a new window, like shown in :numref:`fig_checkerboard_pattern`.

   .. image:: /_resources/image/parameters.png
      :scale: 100%
      :align: center

   .. figure:: /_resources/image/checkerboard_pattern.png
      :scale: 100%
      :align: center
      :name: fig_checkerboard_pattern

      Calibration Tool

#. In order to get a good calibration result, you will need to move the
   checkerboard around in the camera view such that the checkerboard:

   -  is on the camera's left, right, top and bottom of field of view

   -  is filling the whole field of view

   -  is tilted to the left, right, top and bottom (skew)

   At each step (s. :numref:`fig_calibration_steps`), hold the checkerboard
   still until the pattern is highlighted in the calibration window.

   .. figure:: /_resources/image/steps.png
      :scale: 100%
      :align: center
      :name: fig_calibration_steps

      Calibration Steps

#. When all 4 bars (X, Y, Size and Skew) are in green click on calibrate
   button.

   .. note:: It may take some time due to large amount of data to be processed.

#. After the calibration is done, click on the :guilabel:`Save` button.

#. Now click on :guilabel:`Commit`.

#. Copy the generated calibration data (:file:`calibrationdata.tar.gz`) from
   :file:`/tmp` directory into the :file:`config` folder of **usb_camera_driver**

#. Extract the :file:`calibrationdata.tar.gz` file:

   .. code-block:: bash

      $ cd ~/robot_ws/src/ros2_usb_camera/config
      $ mkdir calib
      $ tar –xvzf calibrationdata.tar.gz -C calib

#. Rename the :file:`ost.txt` file to :file:`calibration.ini`.

#. Convert the :file:`calibration.ini` file to a yml format.

   .. code-block:: bash

      $ cd calib
      $ ros2 run camera_calibration_parsers convert calibration.ini webcam.yml

#. Move the :file:`webcam.yml` file to the :file:`config` folder in your
   **usb_camera_driver** package.

#. Open the :file:`usb_camera_node.launch.py` file and edit the ``camera_info_url`` parameter
   with the new calibration file.

   .. code-block:: python

      usb_camera = launch_ros.actions.Node(
         package='usb_camera_driver', node_executable='usb_camera_driver_node', output='screen', node_namespace=ns, 
         parameters=[{"camera_calibration_file": "file:///config/webcam.yml"}])

#. Launch :file:`usb_camera_node.launch.py`, take a look at the ``camera_info`` topic and
   check, if the correct calibration data is published.

For more information visit http://wiki.ros.org/camera_calibration
and http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration.