Skip to main content

Counting how many people walks by using Python, tensorflow and height estimation

In this article, I'd like to share my project to count the number of people walking past a store/shop etc. The motivation was when I was part of a family oriented digital theme park company, we would want to know if daily sales (visitors) are good enough. 

First, some background. Our theme park (store) is part of a department store/shopping complex. So we did not have a large piece of land like Disneyland, where all visitors are definitely there to visit the park. In our case, visitors to the shopping complex will consider taking their children to our entertainment theme park, so we would get a certain percentage of the total visitors of the shopping complex. So for example if we have 100 visitors on one single day, how do we know if this number is good or bad? If we know the number of people passing through our store we would have a better context. If the total number of people is say around 500 then we would be getting 20%, which doesn't seem so bad. But if the shopping complex was packed and we have say around 5000 people walking past our store, that would mean that we could only generate 2% conversion, and we would then decide to do further research on why we did so badly.

Another challenge we had was to detect if there are children present. Because we are targeting families, it would not make sense without getting the number of children present.

System overview

  • People detection with tensorflow
  • Children detection with height estimation
  • Sensor used was one webcam
  • Windows 10
  • Data stored to Google Sheets

Preliminary testing of height estimation algorithm

People Detection

Let's go through the modules one by one. The first component is the people detection module. This is achieved by using the model provided by Google and Tensorflow. This is easy enough since people detection models and sample code are available on the internet.

Height estimation

Height is estimated using single camera using trigonometry. The assumptions for the camera are
  • Focal length is known, so the camera intrinsics and extrinsics parameters are known.
  • Height of camera is known
  • Angle/orientation of camera is known

At the time of this writing, I just measured the height manually and the angle was also measured visually using a protractor. Of course, this is not accurate and for future work, an automatic calibration process is needed to recover the height and angle.

The height estimation process is simply calculating the bounding boxes of people detected by the AI model and converting their height in pixels into meters.

Future work

As mentioned above, this is just a simple installation of people counter and height estimation and more work is needed to increase accuracy especially with an automatic calibration method to recover height and angle of camera placement.


Comments

Post a Comment

Popular posts from this blog

Using FCM with the new HTTP v1 API and NodeJS

When trying to send FCM notifications I found out that Google has changed their API specifications. The legacy API still works but if you want to use the latest v1 API you need to make several changes. The list of changes is listed on their site so I won't be repeating them again but I'll just mention some of the things that caused some trial and error on my project. The official guide from Google is here : Official Migration Guide to v1 . The request must have a Body with the JSON containing the message data. Most importantly it needs to have "message" field which must contain the target of the notification. Usually this is a Topic, or Device IDs. Since my previous project was using GAS, my request had a field called "payload" instead of "body". Using the request from my previous project, my request in Node JS was as follows: request ({ url: 'https://fcm.googleapis.com/v1/projects/safe-door-278108/messages:send' , method: ...
Post a Comment
Read more

Building a native plugin for Intel Realsense D415 for Unity

Based on a previous post , I decided to write a plugin for the Intel Realsense SDK methods so we can use these methods from within Unity. FYI Intel also has their own Unity wrapper in their Github repository , but for our projects, I needed to perform image processing with OpenCV and passing the results to Unity instead of just the raw image/depth data. There is a plugin called OpenCVForUnity to use OpenCV functions from Unity but previous experiments indicate the image processing inside Unity can take a long time. I hope this post can help someone else who wants to use Intel's cameras or any other devices natively in Unity. Test Environment Windows 10 64bit Unity 2017.2.0f3 x64 bit Realsense SDK from Intel CMake 3.0 or higher Steps Checkout the native plugin code here . Don't worry about the other projects in the same repository. The relevant code is in the link above. Checkout the Unity sample project here . However, instead of master you need to go to the br...
5 comments
Read more

Microsoft Azure Face API and Unity

During one of my projects, I came across Microsoft's face recognition API (Azure Face API) and it looked good enough to recognize people's faces and detect if a person is a newcomer or a repeating customer to our store. As our installations mainly use the game engine Unity, I wanted to be able to use the Face API from Unity. Face API does not have an SDK for Unity but their requests are just HTTP requests so the Networking classes in Unity can be wrapped into methods to make it easy to call these APIs. First of all, to those who just want to see the code, here it is . My tests focus on the identification of a face in an input image. The full tutorial I followed can be found here . The Main scene goes through the steps in the tutorial such as creating a PersonGroup and adding Persons to the group if it is not created yet. Just make sure you: Change the API key. I used a free trial key which is no longer valid. Use whatever images you want. I don't mind you us...
Post a Comment
Read more