Word Adventure, gather letters to complete the word!

This is an interactive installation that aims to combine learning words and fun in devising strategies to gather letters.

Collect letters to complete the word

The letters are dangling at various locations, and only the yellow colored characters can collect them. But these characters cannot jump that high, so players need to put blocks as stepping stones to help these cute yellow fellas to reach the letters. The blocks are generated by touching the screen (wall) and can be removed by touching it again.

System overview

The system is simply a projector and a laser range finder to detect touches on the wall. You may realize the black object in the photo above at the top of the wall. This is the laser range finder and for this project I used Hokuyo's UST10-LX. Content is developed with Unity.

How the touch detection works

Readings from the sensor

This image is the visualization of the data points detected. With the sensor placed on top of the wall looking down, if there are no hands touching the wall the laser beams will reach the floor. This will result in a rectangular shape with the floor points aligning as a straight line. However, if someone touches the wall, its position will be detected as shown in the above image. So to detect the touches we just have to first define the rectangular touch area, and all points that are within this area are touch (hand) points.

Limitations

There are some limitations in accuracy and detection rate. If there are two hands (touch points) that overlap each other from the sensor's point of view, the one on the outer side will not be detected.

Undetected touch point

As shown in the image, the bottom touch point will not be detected because it is hidden by the other one above it. The black line represents the laser beam that sweeps the wall for obstacles (hand/touch points) and we can see it cannot reach the bottom point.

Comments

Installing a custom ROM on Android (on the GT-N8013)

It's been a while since my last entry and since it is a new start in 2019, I thought I'd write something about "gone with the old and in with the new". I've had my Samsung Galaxy Note 10.1 (pnotewifi) since 2014, and it's one of the early Galaxy Note tablet series. It has served me well all this years but now it just sits there collecting dust. My old Samsung GT-N8013 I've known a long time about custom Android ROMs like CyanogenMod but has never had the motivation to try them out, until now ! Overview of the process For beginners like me, I didn't have an understanding of the installation process and so it looked complicated and it was one of the reasons I was put off in trying the custom ROM. I just want to say, it's not complicated at all! Basically you will need to Prepare an SD card and install Android SDK (you need adb ). Install a custom boot loader ( TWRP is the de facto tool at the moment). Use adb to copy custom...

Building a native plugin for Intel Realsense D415 for Unity

Based on a previous post , I decided to write a plugin for the Intel Realsense SDK methods so we can use these methods from within Unity. FYI Intel also has their own Unity wrapper in their Github repository , but for our projects, I needed to perform image processing with OpenCV and passing the results to Unity instead of just the raw image/depth data. There is a plugin called OpenCVForUnity to use OpenCV functions from Unity but previous experiments indicate the image processing inside Unity can take a long time. I hope this post can help someone else who wants to use Intel's cameras or any other devices natively in Unity. Test Environment Windows 10 64bit Unity 2017.2.0f3 x64 bit Realsense SDK from Intel CMake 3.0 or higher Steps Checkout the native plugin code here . Don't worry about the other projects in the same repository. The relevant code is in the link above. Checkout the Unity sample project here . However, instead of master you need to go to the br...

OpenCV native plugin for Unity in IOS

In one of my recent projects, I needed to use OpenCV from within Unity, in IOS. The asset called OpenCVForUnity is overkill because I didn't need the whole OpenCV library, just a few functions. In addition, this asset does not implement the whole OpenCV library so unless you know that what you need is included you may find it lacking when you discover it does not support some functions you need. As my project involves some trial and error and mixing algorithms together I decided to go with a native plugin. Overview In IOS, a native library is built as a bundle . We need to put this bundle inside Unity's Plugins/OSX folder to use it. Therefore, we need to create two projects. An XCode project to build the native plugin. A Unity project to use the plugin. Dependencies Of course, since we need to use OpenCV we will have to install it first. Tutorials on installing OpenCV on IOS are abundant and I will not include them here. Assuming you have installed OpenCV go to t...

Time is a limited resource

Search This Blog