Whirled Worlds: Pointing and Spinning Smartphones and Tablets to Control Multimodal Augmented Reality Displays


  • Whirled Worlds: Pointing and Spinning Smartphones and Tablets to Control Multimodal Augmented Reality Displays, 

Modern smartphones and tablets have magnetometers that can be used to detect yaw, which data can be distributed to modulate ambient media. We have implemented such functionality for both a Google Android smartphone and Apple iOS iPhone & iPad. A client-server architecture synchronizes distributed displays across shared channels, including image-based renderings of panoramic photos and object movies, spatial sound (periphonic) speaker arrays, rotary motion platforms [4], and the position of avatars or other objects in virtual environments such as Alice1 [5] and Open Wonderland.2 Embedding such devices into a spinnable affordance allows a “spinning plate”-style interface, a novel interaction technique [1] [2]. Broad configurability allows flexible deployment. Transmission may be one-shot or continuous, including thresholded filtering for choked bandwidth, azimuthal (rotation) and/or circumferential (revolution), and azimuthal transmissions may be wrapped or unwrapped. Device vertical orientation may be upright or inverted. and a modal loop disables multitouch control for duration of play, preventing accidental change of the parameters. A soft transmission can scale the control:display ratio, allowing fast whirling to be shared as more leisurely turning (or even “overdriven” to exaggerate such torque). Such interfaces suggest the potential for embodied interaction exploiting the happy alignment of a gravityoriented horizontal spinning gesture, the latitudinal magnetic field, and the horizontally favored visual field and auditory directionalization acuity. It’s a “come as you are” interface, requiring no special markers or clothing. A video of our new system is posted to http://sonic. u-aizu.ac.jp/spatial-media/mixedreality/ VideoClips/Whirled_Worlds.mov. Even though we can’t bring the rotary motion platform, the speaker array, or the large display to San Diego, for the proposed demonstration at HotMobile, we would show the head-tracking controlling spatial sound and avatars in Alice or Open Wonderland, the double-headed configuration and split control of avatars, and the whirling of panoramic and/or turoramic imagery. For infrastructure, we would need just a desk near an ordinary power strip (to drive a laptop with stereo speakers). (We’ll bring our own wi-fi router.)

Please fill the following information to view the document.