Time is a limited resource

When trying to send FCM notifications I found out that Google has changed their API specifications. The legacy API still works but if you want to use the latest v1 API you need to make several changes. The list of changes is listed on their site so I won't be repeating them again but I'll just mention some of the things that caused some trial and error on my project. The official guide from Google is here : Official Migration Guide to v1 . The request must have a Body with the JSON containing the message data. Most importantly it needs to have "message" field which must contain the target of the notification. Usually this is a Topic, or Device IDs. Since my previous project was using GAS, my request had a field called "payload" instead of "body". Using the request from my previous project, my request in Node JS was as follows: request ({ url: 'https://fcm.googleapis.com/v1/projects/safe-door-278108/messages:send' , method: ...

In this post I'd like to share about one of my projects involving doodles and bringing them to live with Unity. We prepare doodle papers and some crayons and let children color them. After they're done, we scan the images and they appear on the screen. The screen is projected on walls using projectors. Doodles come alive on the screen Project flow I utilized document scanners readily available such as the following. A document scanner The scanner has many helpful features, such as cropping and rotating the scanned images so they are nice and straight even if the paper is slightly rotated or not aligned properly. The scanned images are stored in a server and a Unity application polls the server for new images every few seconds. For the server I initially used AWS S3 for image storage and later on we switched to a local image server with Node JS. Attaching 2D Texture to a 3D Model I no longer have access to the actual doodle papers but they look l...

This is just a memo of the challenges I faced when running a model training on Google Colab, while following a great tutorial here . Mind the versions Tensorflow is currently at version 2.2.0 but most tutorials are still using the contrib package, and there is no known easy way to update the code to remove dependency on contrib. So my best bet is to downgrade the tensorflow version to 1.x. Since Google Colab only gives the options of either 1.x or 2.x and we cannot specify the exact version, I ended up with version 1.15.2. Even with the command :  %tensorflow_version  1.15.0 I ended up with : 1.15.2 Another pitfall was the version of numpy. Installing numpy gives us the version 1.18.3 but for some reason this generates the error : TypeError: 'numpy.float64' object cannot be interpreted as an integer Downgrading numpy to version 1.17.4 solved this for me. It seems we don't need ngrok for tensorboard With the command :  %load_ext tensorboard W...

In one of my recent projects, I needed to use OpenCV from within Unity, in IOS. The asset called OpenCVForUnity is overkill because I didn't need the whole OpenCV library, just a few functions. In addition, this asset does not implement the whole OpenCV library so unless you know that what you need is included you may find it lacking when you discover it does not support some functions you need. As my project involves some trial and error and mixing algorithms together I decided to go with a native plugin. Overview In IOS, a native library is built as a bundle . We need to put this bundle inside Unity's Plugins/OSX folder to use it. Therefore, we need to create two projects. An XCode project to build the native plugin. A Unity project to use the plugin. Dependencies Of course, since we need to use OpenCV we will have to install it first. Tutorials on installing OpenCV on IOS are abundant and I will not include them here. Assuming you have installed OpenCV go to t...

This is an interactive installation that aims to combine learning words and fun in devising strategies to gather letters. Collect letters to complete the word The letters are dangling at various locations, and only the yellow colored characters can collect them. But these characters cannot jump that high, so players need to put blocks as stepping stones to help these cute yellow fellas to reach the letters. The blocks are generated by touching the screen (wall) and can be removed by touching it again. System overview The system is simply a projector and a laser range finder to detect touches on the wall. You may realize the black object in the photo above at the top of the wall. This is the laser range finder and for this project I used Hokuyo's UST10-LX. Content is developed with Unity. How the touch detection works Readings from the sensor This image is the visualization of the data points detected. With the sensor placed on top of the wall looking down, ...

This is one of the installations I developed for a digital entertainment theme park. It is an interactive digital sandbox with normal white sand and I used a projector and depth sensor to project topological colors and virtual objects depending on the height and shape of the sand. I also used QR markers on 3D printed objects to interact with the virtual content. Overview of installation A sandbox of 2 x 1.5 meters with projected content As shown in the image, depending on the height of the sand, the projected colors change from deep water, shallow water, beach, green grass, dark green forest, rocky mountains and finally snowy mountains. These are the 7 different height levels defined by the system. In addition to the topological colors, it also has fishes and boats in the waters, lava coming out of a crater and other interactive elements. The sand depth is detected real time and projection updated Fish cannot go on land Hardware setup Sandbox made of w...

This post is a side note for anyone trying to setup a Jenkins job to build an Android application. Error because google-services.json is not found google-services.json is used to access Google services, APIs etc. and is probably the most used file in almost all applications. However this is automatically ignored if you use gitignore and it won't be available in the build machine where Jenkins is run. The solution is obviously to un-ignore this file, push and try again. Cannot run nohup.exe error This was solved by adding git command to Windows env path. Merge debug resources failed error The full error is as follows: java.util.concurrent.ExecutionException:com.android.builder.internal.aapt.v2.Aapt2Exception: AAPT2 error: check logs for details :app:mergeDebugResources FAILED This can be solved by setting the  GRADLE_USER_HOME  Global properties in  Jenkins > Manage jenkins > Configure System JDK error Kotlin could not find the required JDK tools ...

In this third and final entry, I will share how we use Google's Protocol Buffer to package the data structure when sharing and transmitting between the PCs. Basic information on Protocol Buffer can be found on Google's official page . But to put it briefly it is a serialization library that manages your structured data to and from a variety of data streams. In my case, since the detection result is a person's position on a 2D image, my data (proto file) looks like this. message people_position { repeated float x = 1 ; repeated float y = 2 ; repeated string device_number = 3 ; } x, y of course is the detected person's position on the image, and device_number is serial number of the device that detected it. The field rule "repeated" means that there will be multiple instances of this field, which in my case applies to when there are multiple people detected. Protocol Buffer Compilation In my project, the detection algorithm is written...

In this second part, I will share how the data between several Realsense devices are merged. First, an overview on how the devices are connected together. The data flow is as follows PC 1 and PC 2 gathers detection data from the devices attached to them. PC 2 has another process that merges the data and sends them to PC 3. All PCs have knowledge of the device layout and transformation Matrix to transform coordinates into Unity coordinates.* A little more on this later. PC 2 transfers the transformed coordinates to PC 3 through TCP/IP connection. As mentioned above, the devices need to know its position in the layout and transforms it detection results into the larger area axes as shown in the figure below. Each dotted rectangle denotes the view of one Realsense depth camera. Each camera is assigned an identifier (a, b) that reflects its position in the layout. Both PC 1 and PC 2 holds a settings file that tells them which devices are attached to each PC, ma...

This is a three part post on using Intel's Realsense depth cameras in a digital theme park, particularly for detecting people from a top-view position. The detected positions are used as input to provide visual effects projected back onto the floor, or in some cases, onto a ball pool. The following image will probably better explain the setup used. If you look closely, there are three Realsense cameras attached to the ceiling close to the projectors. The targets of the project are: We want to project wave ripple effects whenever someone is present. The effect center is where the person is standing and ripples out just like a ripple in the water. In this first part, I will share how we decided to detect a person using Realsense D415 cameras. Person Detection Process There are several ways to detect people, but since we do not need a very accurate position information I found that one of the easiest ways is to utilize Realsense's depth sensor to detect ...

Normally apps use database and a backend to operate. Here I'd like to share another way of using Google Sheets as a database to replace SQL, Mongo and the likes. It is not very customizable or extendable but it is a very good way to prototype or for small scale services or internal tools. Whenever a change happens on the sheets, the app gets a notification so it can re-poll the sheets to get new data. Using Google Sheets as a Database Overview What is needed for this to work. Google account, Google Sheets file. Created Firebase project. Created android app and registered to use FCM. The app has access rights to the Google Sheets to read its contents. Reading a Google Sheets file has been covered many times before so I will just refer you to one of them here . Flow of the application Someone in charge of the data will make changes to the Google Sheets file and finally when everything is done, he/she will modify a single cell that acts as the "Apply button...