DLC+¶

This module belongs to the manuscript "Burchardt, L., Van de Sande, Y., Kehy, M., Gamba, M., Ravignani, A., Pouw, W."

This contains a module for tracking Siamang head and air sac postures. The following keypoints will be tracked by a trained resnet 101 model:

  • UpperLip
  • LowerLip
  • Nose
  • EyeBridge
  • Start_outline_outer_left
  • Start_outline_outer_right
  • LowestPoint_outline
  • MidLowleft_outline
  • MidLowright_outline

Note that Deeplabcut needs to be installed (in command prompt "pip install -r requirements.txt"). By default the CPU version is installed. Please see the original documentation of DeepLabCut to ensure GPU compatibility if you want to speed up the tracking process. We do recommend to use a GPU supported deeplabcut.

The trained resnet101 model needs to be downloaded first from google drive; so please go to the following folder and follow the download link and download to that folder: "./AirSacTracker/Toolkit/module_process_video_DLC_model/trained_model_and_metainfo/dlc-models/iteration-0/Deep_AirSacTrackingV1Jan1-trainset95shuffle1/train/".

Example Output¶

In [1]:
from IPython.display import Video
video_url = 'https://tsg-131-174-75-200.hosting.ru.nl/samples_airsactoolkit/June16_02_circle_rec.mp4'  # Replace this with your video URL
Video(video_url, width=600, height=00)
Out[1]:
Your browser does not support the video element.

Loading packages¶

In [2]:
# load in all the packages needed:

# general packages:
from tqdm import tqdm

# processing the videos with deeplabcut:
import deeplabcut
import os
import shutil
from os.path import isfile, join
from IPython.display import Video

# performing a circle estimation with Landau algorithm:
import glob
import pandas as pd
import numpy as np
from pprint import pprint

# plot landau circles on processed video:
from os import listdir
from os.path import isfile, join
import cv2
import math

Loading DLC 2.3.0...

D:\Programs\Anaconda\Lib\site-packages\qtpy\__init__.py:267: RuntimeWarning: Selected binding "pyside6" could not be found, using "pyqt5"
  warnings.warn('Selected binding "{}" could not be found, '
D:\Programs\Anaconda\Lib\site-packages\paramiko\transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,

DLC loaded in light mode; you cannot use any GUI (labeling, relabeling and standalone GUI)

Part 1: processing your video with DLC¶

In [3]:
# load the pre-trained model settings
config_path = r"./DLC/trained_model_and_metainfo/config.yaml"

    # does config exist?
if not os.path.exists(config_path):
    print('the config file is not correctly pointed to: ' + config_path)

# where are we going to save our DLC tracked results to?
intermediate_output = "./intermediate_output/"
if not os.path.isdir(intermediate_output):
    print('the output folder is not correctly pointed to: ' + intermediate_output)
    
# where are we going to save our DLC+ final results to?
output_dir = "./output/"
if not os.path.isdir(output_dir):
    print('the output folder is not correctly pointed to: ' + output_dir)
    
# set videofolder from which we are going to process
videofolder = "../input/"
if not os.path.isdir(videofolder):
    print('the input folder is not correctly pointed to: ' + videofolder)

# loading in the videos, make sure you only have video files in your folder
vids = [f for f in os.listdir(videofolder) if isfile(join(videofolder, f))]
for i in vids:
    print('to process: ' + i)

to process: example8.mp4
In [4]:
# display the first video in the set
Video(videofolder + vids[0], width=300, height=200, embed = True)
Out[4]: