Skip to main content

Image Processing with Intel Realsense for Digital Interactive Installations (1/3)

This is a three part post on using Intel's Realsense depth cameras in a digital theme park, particularly for detecting people from a top-view position. The detected positions are used as input to provide visual effects projected back onto the floor, or in some cases, onto a ball pool.

The following image will probably better explain the setup used.


If you look closely, there are three Realsense cameras attached to the ceiling close to the projectors.

The targets of the project are:
  • We want to project wave ripple effects whenever someone is present.
  • The effect center is where the person is standing and ripples out just like a ripple in the water.
In this first part, I will share how we decided to detect a person using Realsense D415 cameras.

Person Detection Process

There are several ways to detect people, but since we do not need a very accurate position information I found that one of the easiest ways is to utilize Realsense's depth sensor to detect people's heads. This is done by thresholding the depth image at a certain value as shown in the following figure.


In the actual implementation, because I need to detect kids as well as adults, I ended up using two range values as thresholds.

After thresholding depth, we get an image similar to the image below.

In reality, the image is noisy and the blobs are not clean circles. However, the above illustration should convey the general idea of the algorithm. Therefore the output of the sensor is an array of blob positions that correspond to people's positions.

[(x1, y1), (x2, y2), ... , (xn, yn)]

Keep in mind that the above positions are seen from one Realsense device, with a resolution of 1280x720. In the next part, I will share on the data merging between several Realsense cameras.

Appendix : Trial and Experiments

When considering how to easily detect people from top view, and still maintain a certain accuracy, I tried to implement the detection method using paper cups as experiments. It took several tuning and experiments before I could be sure the method would work relatively well.

Paper cups taped to the wall and detected by the depth camera

This setup also served as the basis of multi camera calibration for my system later on.

The paper cups height from the wall

As seen in the image, the depth camera was able to determine the height (depth) of the paper cups. This eventually became the base for the people detection system mentioned in the previous section.

See Part 3 here: http://www.dailytinkerer.com/2020/04/image-processing-with-intel-realsense_18.html


Comments

Post a Comment

Popular posts from this blog

Object detection with Google Colab and Tensorflow

This is just a memo of the challenges I faced when running a model training on Google Colab, while following a great tutorial here . Mind the versions Tensorflow is currently at version 2.2.0 but most tutorials are still using the contrib package, and there is no known easy way to update the code to remove dependency on contrib. So my best bet is to downgrade the tensorflow version to 1.x. Since Google Colab only gives the options of either 1.x or 2.x and we cannot specify the exact version, I ended up with version 1.15.2. Even with the command :  %tensorflow_version  1.15.0 I ended up with : 1.15.2 Another pitfall was the version of numpy. Installing numpy gives us the version 1.18.3 but for some reason this generates the error : TypeError: 'numpy.float64' object cannot be interpreted as an integer Downgrading numpy to version 1.17.4 solved this for me. It seems we don't need ngrok for tensorboard With the command :  %load_ext tensorboard W...
Post a Comment
Read more

Installing a custom ROM on Android (on the GT-N8013)

It's been a while since my last entry and since it is a new start in 2019, I thought I'd write something about "gone with the old and in with the new". I've had my Samsung Galaxy Note 10.1 (pnotewifi) since 2014, and it's one of the early Galaxy Note tablet series. It has served me well all this years but now it just sits there collecting dust. My old Samsung GT-N8013 I've known a long time about custom Android ROMs like CyanogenMod but has never had the motivation to try them out, until now ! Overview of the process For beginners like me, I didn't have an understanding of the installation process and so it looked complicated and it was one of the reasons I was put off in trying the custom ROM. I just want to say, it's not complicated at all!   Basically you will need to Prepare an SD card and install Android SDK (you need adb ). Install a custom boot loader ( TWRP is the de facto tool at the moment). Use adb to copy custom...
15 comments
Read more

Using FCM with the new HTTP v1 API and NodeJS

When trying to send FCM notifications I found out that Google has changed their API specifications. The legacy API still works but if you want to use the latest v1 API you need to make several changes. The list of changes is listed on their site so I won't be repeating them again but I'll just mention some of the things that caused some trial and error on my project. The official guide from Google is here : Official Migration Guide to v1 . The request must have a Body with the JSON containing the message data. Most importantly it needs to have "message" field which must contain the target of the notification. Usually this is a Topic, or Device IDs. Since my previous project was using GAS, my request had a field called "payload" instead of "body". Using the request from my previous project, my request in Node JS was as follows: request ({ url: 'https://fcm.googleapis.com/v1/projects/safe-door-278108/messages:send' , method: ...
Post a Comment
Read more