Body Tracking Project

1. Introduction

The project was carried out in two main ways

The first method is using Microsoft's Azure Kinect DK, and the second method is to process video stream data received through a webcam on a web page using the body tracking model disclosed as an open source

2. Body of Text

2.1. Using Azure Kinect DK

The way Azure Kinect DK is used is very simple, but you just need to call the equipment provided by Microsoft and the corresponding API

2.1.1. Example

Example of a simple application using Azure Kinect DK

2.2. Using Open Source ML Model and Webcam Video Stream Data

In this case, it was implemented by combining codes through web search, and consisted of simple html and javascript codes

<!-- main.html -->
<h2> align=center>Auto Video Stream to Still Image</h2>
                                
<video id="myVideo" width="400" height="300" style="border: 1px solid #ddd;"></video>
<canvas id="myCanvas" width="160" height="140" style="border: 1px solid #ddd;"></canvas><br>
                                
<input type=button value="get Video" onclick="{getVideo()}">
<input type=button value="get Pic" onclick="{takeSnapshot()}"><br>Take snapshot every
<input type=number id="myInterval" value="3000"> milliseconds
<input type=button value="Auto" onclick="{takeAuto()}">
                                
<script src="processing.js"></script>
                                
<script src="tf.min.js"></script>
<script src="posenet.min.js"></script>

// processing.js
var myVideoStream = document.getElementById('myVideo')     // make it a global variable
var myStoredInterval = 0

function getVideo() {
    navigator.getMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
    navigator.getMedia({ video: true, audio: false },

    function (stream) {
        myVideoStream.srcObject = stream
        myVideoStream.play();
    },

    function (error) {
        alert('webcam not working');
    });
}

function takeSnapshot() {
    var myCanvasElement = document.getElementById('myCanvas');
    var myCTX = myCanvasElement.getContext('2d');
    myCTX.drawImage(myVideoStream, 0, 0, myCanvasElement.width, myCanvasElement.height);

    StartTracking()
}

function takeAuto() {
    takeSnapshot() // get snapshot right away then wait and repeat
    clearInterval(myStoredInterval)
    myStoredInterval = setInterval(function () {
        takeSnapshot()
    }, document.getElementById('myInterval').value);
}

function StartTracking() {
    var imageScaleFactor = 0.5;
    var outputStride = 16;
    var flipHorizontal = false;

    var imageElement = document.getElementById('myVideo');

    posenet.load().then(function (net) {
        return net.estimateSinglePose(imageElement, imageScaleFactor, flipHorizontal, outputStride)
    }).then(function (pose) {
        console.log(pose);
    })
}

The files required for ML are as follows:

posenet.min.js
tf.min.js

2.2.1. Related Post

My Original Blog Post Related to

2.2.2. Example

Body Tracking Demo Page

3. Result

After testing the two methods, the pros and cons of each were clear

First of all, the difficulty of development becomes easier when Azure Kinect DK is used, but the disadvantages can be pointed out as the fact that the equipment is not cheap and that the available platforms can be limited to some extent because the API provided is based on C#

Next, in the case of a combination of ML models and webcams, the ML model currently tested was written in JavaScript, confirming the possibility of widespread use in the web environment and native, and by using webcams together, it could be easily utilized without spending money for additional equipment. However, since ML models are open-source based, it is unclear whether they are updated or not, and the quality of body tracking results can depend on various issues such as resolution of webcams