An interactive software tool for tracking and visualizing animal motion patterns using computer vision
Table of Contents
- Introduction
- Section 1: Video demo for using AnimalMotionViz software
- Section 2: Set up the environment for running AnimalMotionViz locally
- Section 3 (optional): Creating a custom mask for tracking motion patterns in a specific region
- Section 4: How AnimalMotionViz works and its application
- 4.1: How AnimalMotionViz Works
- 4.2: Guidelines on setting up the video processing parameters
- 4.2.1: Uploading the input video
- 4.2.2: Uploading the mask image if available
- 4.2.3: Select a background subtraction algorithm
- 4.2.4: Specify the interval for frame processing (e.g., every nth frame)
- 4.2.5: Select a kernel size for the morphological operation
- 4.2.6: Choose the overlay parameters
- 4.2.7: Select a colormap
- 4.2.8: Start the video processing
- Section 5: Results
- Contact information and help
- References
- License
To provide novel insight into the movement and space use of dairy cattle, we developed AnimalMotionViz, an open-source software tool that processes video data to monitor animal movement patterns using computer vision. The software generates a motion heatmap image and video, along with motion metrics that highlight and quantify movement intensity and patterns. This software tool aims to support the broader adoption of computer vision systems, thereby further enabling precision livestock farming.
We provide a video demo below to showcase the usage of our AnimalMotionViz software.
1.1 AnimalMotionViz demo video [Video demo]
2.1 Install Conda on your local computer
# cloning "AnimalMotionViz/" folder from GitHub
git clone https://github.com/uf-aiaos/AnimalMotionViz.git
# change to the cloned/downloaded folder
cd AnimalMotionViz/# "environment.yml" is available under "AnimalMotionViz/"
conda env create -f environment.ymlconda activate animalmotionviz# change to the source code directory
cd AnimalMotionViz_sourcecode/
# run the app
python app.pyAfter that, open the follow link http://127.0.0.1:8050/ in your web brower and now you can use AnimalMotionViz locally!
For users interested in tracking animal motion patterns within a specific region, a mask image can be uploaded that is created using annotation tools, allowing a region of interest to be specified in the image. While this step is optional, it is recommended as it enables the definition of specific areas to be considered during video processing, thereby increasing the focus and relevance of the analysis. Below, we have provided a tutorial on creating a mask image using the open-source graphical annotation tool LabelMe.
3.1.1 Create a conda environment with Python installed
# create a conda env named `labelme`
conda create --name labelme
# activate the created `labelme`
conda activate labelme#installing LabelMe
pip install labelmelabelme --version# opening the app
labelme 
Get a frame from your video, and use it for creating a mask. Make sure the frame size is the same as your video's resolution. If you're using VLC/Quicktime Player, there's a function there that captures a frame from the video.
# go to directory where you save the .json file
cd Pictures\New Folder
#convert the .json files
labelme_json_to_dataset annotation.json -o annotation_json
Note that only the mask image, label.png, will need to be uploaded in step 4.2.2 to specify a region of interest.
After setting up the necessary dependencies, we are ready to use the AnimalMotionViz software. This section provides detailed instructions on how to run and operate the AnimalMotionViz app. The source code of AnimalMotionViz is available at AnimalMotionViz_sourcecode AnimalMotionViz_sourcecode.
AnimalMotionViz facilitates the uploading of video files via the dash-uploader component, with an option to include an optional mask defining the Region of Interest (ROI). The software offers various background subtraction algorithms like MOG2, KNN, GMG, CNT, GSOC, and LSBP, implemented using OpenCV. Then users can specify the interval for frame processing (e.g., every nth frame). Users can also select a kernel size for the morphological operation, where it can mitigate small noises (birds, leaves, et.), as well as the weight (alpha and beta) of the original frame and motion heatmap overlay, respectively. Several colormaps like Bone, Ocean, Pink, and Hot are available for enhancing the visualization of the motion heatmap. The motion heatmap is generated by applying a colormap to the accumulated image obtained from background subtraction and filtering, which is then overlaid on the original frame.
This section provides clear and concise guidelines for configuring parameters in AnimalMotionViz, including uploading video and mask files, selecting background subtraction algorithms, adjusting overlay settings, and applying colormaps. By following these instructions, users can fully utilize AnimalMotionViz to track and visualize animal movement patterns.
Upload the video file for tracking animal motion patterns. The software supports various video formats, including mp4, avi, mov, wmv, etc. We have also provided an example video to demonstrate the usage of the software.
You may upload a mask (e.g., label.png file created above using LabelMe) to define the region or area of interest, which can be applied during video processing.
Choose from a selection of background subtraction algorithms, including MOG2, KNN, GMG, CNT, GSOC, and LSBP, implemented using OpenCV.
In this step, the frame processing interval allows users to select a subset of frames from the video for analysis by choosing to process every nth frame (e.g., every 5th frame), which reduces computational load. This approach is particularly useful for long videos or when changes between consecutive frames are trivial, allowing users to capture key movements efficiently.
Here, users can specify the kernel size for the morphological operation. A smaller kernel focuses on removing minor noise while maintaining the integrity of key structures in the image. On the other hand, a larger kernel is more effective at eliminating substantial noise but may also risk removing important details or features, depending on the complexity of the image.
Customize the weight (alpha and beta) of the original frame and the motion heatmap overlay to achieve the desired visual effect. Alpha is the weight of the first array elements (frame image), while Beta is the weight of the second array elements (overlay).
Select from various colormaps such as Bone, Ocean, Pink, and Hot to enhance the visualization of the returned motion heatmap.
Run the video analysis to track and visualize animal motion patterns based on the provided parameters.
-
Peak Intensity Location: This metric refers to the specific location within the region where the most movement is detected. The table shows three peak intensity locations, each with an X and Y coordinate.
-
Overall Percentage of Used Region: This metric represents the portion of the entire region that’s being used for movement.
-
Quadrant X Percentage of Used Region: This metric breaks down how much space is being used for movement in each quadrant of the region. There are four quadrants, labelled 1 to 4. For each quadrant, the metric shows the percentage of the quadrant’s space that is being used.
- Angelo De Castro (decastro.a@ufl.edu)
- Haipeng Yu (haipengyu@ufl.edu)
If you use the materials provided in this repository, we kindly ask you to cite our papers:
-
The software paper: De Castro, A. L., Wang, J., Bonnie-King, J. G., Morota, G., Miller-Cushon, E. K., & Yu, H. (2024). AnimalMotionViz: An interactive software tool for tracking and visualizing animal motion patterns using computer vision. bioRxiv. https://doi.org/10.1101/2024.10.22.619671
-
The application paper: Marin, M.U., Gingerich, K.N., Wang, J., Yu, H. and Miller-Cushon, E.K., 2024. Effects of space allowance on patterns of activity in group-housed dairy calves. JDS Communications.
This project is primarily licensed under the GNU General Public License version 3 (GPLv3).