Have you seen these game controllers and wished you could make one yourself and customize exactly as you wish? If so, keep reading! ClipWidgets is a technique that helps create super custom external controls for smartphones. “Smartphones are pretty cool, why would I need external controls?”, I hear you say. Well, yes, while smartphones and other touchscreen-based devices are incredibly versatile and are capable of presenting content and capture input at the same time, they lack the physicality of controls (e.g., buttons, dials). Our hands are these incredible things that have evolved for millions and millions of years to give us rich dexterity and sensing, however touchscreens do not leverage them at all. This is why plane cockpits are so full of physical dials and switches, and why masters of their craft (e.g., musicians, gamers, software developers) chose for physical interfaces to express their craft.
ClipWidgets is not the first technique to tackle this problem, but it has some cool tricks up its sleeve. The most impressive thing is that in order for you to create your own ClipWidget controls you need no specialized equipment, just a 3D-printer, your phone, and some color markers. Another cool thing about ClipWidgets is its flexibility. With ClipWidgets you can attach three different types of widgets to your smartphone: buttons, sliders, or dials. These widgets then clip (hence the name) to a custom printed smartphone case and you’re ready to roll!
How does it work?
ClipWidgets is super clever! It makes use of your phone’s camera and flash to detect how the widgets are being interacted with. Once the phone case and widgets you proceed to click them together in the position that you need these widgets to be. Each widget has two markers (the blue and yellow cone objects you can see in the picture), and move as you interact with the widgets. We can see these movements using the phone’s camera and flash (because it’s very dark once the phone is in the case) and using our custom software, we can identify what is being done with the widgets.
“But wait a minute, how can we make our camera monitor the entirety of the case?” That’s a very good question! We created a conical mirror and placed it in front of where the camera would sit on the case. After some post-processing of the image to fix the distortions caused by this mirror and done!
ClipWidgets can add different modalities of input
ClipWidgets can be used to add many new interaction modalities to your smartphone, like dials, sliders and buttons. We translate the movements in the widget to circular movements of our markers, we made a super cleaver belt system for our slider widget—check our paper!
An example application for ClipWidgets is a DJ app. You can change the tempo, volume, and playback of two different tracks using our widgets.
Widgets can be stacked
A cool thing about ClipWidgets is that we are not limited to the space on the size of the phone to add our widgets. For example, we can clip widgets to the back of other widgets, like in the image below. We can do this simply by changing where the cone markers, the ones we described above, are in the case.
You can download this paper from the ACM Digital Library, or directly from this website.