Gentle Gestures of Control: On the Somatic Sensibilities of an IoT Remote App

This is an outdated version published on 2022-01-31. Read the most recent version.

PDF PDF (Español (España))

Published: Jan 31, 2022

Versions:

2022-03-10 (2)

2022-01-31 (1)

DOI: https://doi.org/10.7764/disena.20.Article.1

Keywords:

domestic robots , soma design , embodied interaction , gestures , IoT experiences

Jūra Miniotaitė

KTH Royal Institute of Technology, Department of Media Technology and Interaction Design

Vaida Pakulytė

Electrolux CX, Software Team

Ylva Fernaeus

KTH Royal Institute of Technology, Department of Media Technology and Interaction Design

Abstract

The design of user experiences for physical appliances increasingly involves connection, monitoring, and control via smartphone applications. Despite the rich possibilities for interaction provided by smartphones, the current standard mode of engagement with such apps is through graphical user interface manipulations. To explore new felt experiences for this use context, a remote-control app for a robotic vacuum cleaner was designed, enabling participants to have their gaze focused on the robot, while steering it by gently tilting the phone. This particular interaction is used as a case to emphasize the role of somatic sensibilities when designing smartphone applications in the context of IoT. Through a phenomenologically-inspired analysis, we describe the user experience in terms of physical manipulation, perception, effort, and utility, and through social and emotional engagement. An important attribute was how the interaction, through its subtleness, created a somatically connected experience.

Downloads

Download data is not yet available.

How to Cite

Miniotaitė, J., Pakulytė, V., & Fernaeus, Y. (2022). Gentle Gestures of Control: On the Somatic Sensibilities of an IoT Remote App. Diseña, (20), Article.1. https://doi.org/10.7764/disena.20.Article.1

Issue

No. 20 (2022): Design and Somatic Sensibilities

Section

Original Articles (part 1)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International license.

All contents of this electronic edition are distributed under the Creative Commons license of "Attribution-ShareAlike 4.0 Internacional" (CC-BY-SA). Any total or partial reproduction of the material must mention its origin.

The rights of the published images belong to their authors, who grant to Diseña the license for its use. The management of the permits and the authorization of the publication of the images (or of any material) that contains copyright and its consequent rights of reproduction in this publication is the sole responsibility of the authors of the articles.

Author Biographies

Jūra Miniotaitė, KTH Royal Institute of Technology, Department of Media Technology and Interaction Design

BA in Media Technology, KTH Royal Institute of Technology. M.Sc. in Interactive Media Technology, KTH Royal Institute of Technology. She is a Doctoral student in Multimodal Machine Learning at KTH. As a researcher, she explores Human-robot Interaction, social robotics, and embodied interaction. She is the co-author of ‘A Case Study of Deep Enculturation and Sensorimotor Synchronization to Real Music’ (with O. Misgeld, T. Gulz, and A. Holzapfel; in International Society for Music Information Retrieval Conference 2021).

Vaida Pakulytė, Electrolux CX, Software Team

BA in International Communication and Multimedia, Aarhus Universitet. MA in Digital Marketing & Social Media, London School of Business and Finance. She works on user experience and interaction design as part of the Consumer Experience Software team at Electrolux, specializing on applications related to robotic vacuum cleaners and air purifiers.

Ylva Fernaeus, KTH Royal Institute of Technology, Department of Media Technology and Interaction Design

BA in Computer and Systems Sciences, Stockholm University. M.Sc. in Interactive Multimedia, University of Westminster. Ph.D. in Human-Machine Interaction, Stockholm University. Associate Professor in the Department of Media Technology and Interaction Design at KTH and guest researcher at Umeå Institute of Design. Her research is focuses on physical interaction design, embodied programming, preciousness in interaction, and hybrid crafting. Her latest publications include ‘Envisioning Arduino Action: A Collaborative Tool for Physical Computing in Educational Settings’ (with G. J. Roumen, International Journal of Child-Computer Interaction, Vol. 29, Issue 100277); ‘Co-watching 360-Films in Nursing Homes’ (with A. Lundström and S. Ghebremikael; in Proceedings of 13th IFIP Conference on Human-Computer Interaction); and ‘Troubling Care: Four Orientations for Wickedness in Design’ (with K. Helms; in Designing Interactive Systems Conference 2021).

References

Aloi, G., Caliciuri, G., Fortino, G., Gravina, R., Pace, P., Russo, W., & Savaglio, C. (2017). Enabling IoT Interoperability through Opportunistic Smartphone-based Mobile Gateways. Journal of Network and Computer Applications, 81(C), 74–84. https://doi.org/10.1016/j.jnca.2016.10.013

Ando, T., Isomoto, T., Shizuki, B., & Takahashi, S. (2018). Press & Tilt: One-handed Text Selection and Command Execution on Smartphone. Proceedings of the 30th Australian Conference on Computer-Human Interaction, 401–405. https://doi.org/10.1145/3292147.3292178

Benford, S., Magerkurth, C., & Ljungstrand, P. (2005). Bridging the Physical and Digital in Pervasive Gaming. Communications of the ACM, 48(3), 54–57. https://doi.org/10.1145/1047671.1047704

Bentley, F., & Barrett, E. (2012). Building Mobile Experiences. MIT Press.

Buur, J., Jensen, M. V., & Djajadiningrat, T. (2004). Hands-only Scenarios and Video Action Walls: Novel Methods for Tangible User Interaction Design. Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, 185–192. https://doi.org/10.1145/1013115.1013141

Candau, Y., Françoise, J., Alaoui, S. F., & Schiphorst, T. (2017). Cultivating Kinaesthetic Awareness through Interaction: Perspectives from Somatic Practices and Embodied Cognition. Proceedings of the 4th International Conference on Movement Computing, 1–8. https://doi.org/10.1145/3077981.3078042

Coronado, E., Villalobos, J., Bruno, B., & Mastrogiovanni, F. (2017). Gesture-based Robot Control: Design Challenges and Evaluation with Humans. 2017 IEEE International Conference on Robotics and Automation, 2761–2767. https://doi.org/10.1109/ICRA.2017.7989321

Dourish, P. (2001). Where the Action Is: The Foundations of Embodied Interaction. MIT Press. https://doi.org/10.7551/mitpress/7221.001.0001

Eriksson, S., Höök, K., Shusterman, R., Svanes, D., Unander-Scharin, C., & Unander-Scharin, Å. (2020). Ethics in Movement: Shaping and Being Shaped in Human-Drone Interaction. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–14. https://doi.org/10.1145/3313831.3376678

Fernaeus, Y., & Jacobsson, M. (2009). Comics, Robots, Fashion and Programming: Outlining the Concept of ActDresses. Proceedings of the 3rd International Conference on Tangible and Embedded Interaction, 3–8. https://doi.org/10.1145/1517664.1517669

Fernaeus, Y., & Sundström, P. (2012). The Material Move How Materials Matter in Interaction Design Research. Proceedings of the Designing Interactive Systems Conference, 486–495. https://doi.org/10.1145/2317956.2318029

Fernaeus, Y., Tholander, J., & Jonsson, M. (2008). Beyond Representations: Towards an Action-centric Perspective on Tangible Interaction. International Journal of Arts and Technology, 1(3/4), 249–267.

Forlizzi, J. (2007). How Robotic Products Become Social Products: An Ethnographic Study of Cleaning in the Home. Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, 129–136. https://doi.org/10.1145/1228716.1228734

Frischen, A., Bayliss, A. P., & Tipper, S. P. (2007). Gaze Cueing of Attention: Visual Attention, Social Cognition, and Individual Differences. Psychological Bulletin, 133(4), 694–724. https://doi.org/10.1037/0033-2909.133.4.694

Hallnäs, L., & Redström, J. (2001). Slow Technology – Designing for Reflection. Personal and Ubiquitous Computing, 5(3), 201–212. https://doi.org/10.1007/PL00000019

Hardy, R., & Rukzio, E. (2008). Touch & Interact: Touch-based Interaction of Mobile Phones with Displays. Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services, 245–254. https://doi.org/10.1145/1409240.1409267

Hendriks, S., Mare, S., Gamboa, M., & Baytaþ, M. A. (2021). Azalea: Co-experience in Remote Dialog through Diminished Reality and Somaesthetic Interaction Design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Article 261). Association for Computing Machinery. https://doi.org/10.1145/3411764.3445052

Höök, K. (2018). Designing with the Body: Somaesthetic Interaction Design. MIT Press.

Hung, C.-H., Bai, Y.-W., & Wu, H.-Y. (2016). Home Outlet and LED Array Lamp Controlled by a Smartphone with a Hand Gesture Recognition. 2016 IEEE International Conference on Consumer Electronics, 5–6. https://doi.org/10.1109/ICCE.2016.7430502

La Delfa, J., Baytas, M. A., Patibanda, R., Ngari, H., Khot, R. A., & Mueller, F. F. (2020). Drone Chi: Somaesthetic Human-Drone Interaction. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–13. https://doi.org/10.1145/3313831.3376786

Loke, L., & Núñez-Pacheco, C. (2018). Developing Somatic Sensibilities for Practices of Discernment in Interaction Design. The Senses and Society, 13(2), 219–231. https://doi.org/10.1080/17458927.2018.1468690

Miniotaité, J. (2021). JoyTilt: Beyond GUI App Design for Embodied Experience of Controlling a Robot Vacuum Cleaner [Master´s Thesis]. KTH, School of Electrical Engineering and Computer Science. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-294338

Nintendo. (n.d.). Nintendo History. Nintendo of Europe GmbH. Retrieved March 24, 2020, from https://www.nintendo.co.uk/Hardware/Nintendo-History/Nintendo-History-625945.html

Norman, D. A. (1993). Cognition in the Head and in the World: An Introduction to the Special Issue on Situated Action. Cognitive Science, 17(1), 1–6. https://doi.org/10.1207/s15516709cog1701_1

Soma, R., Dønnem Søyseth, V., Søyland, M., & Schulz, T. (2018). Facilitating Robots at Home: A Framework for Understanding Robot Facilitation. The Eleventh International Conference on Advances in Computer-Human Interactions, 1–6.

Sung, J., Grinter, R. E., & Christensen, H. I. (2010). Domestic Robot Ecology. International Journal of Social Robotics, 2(4), 417–429. https://doi.org/10.1007/s12369-010-0065-8

Székely, M., & Michael, J. (2020). The Sense of Effort: A Cost-Benefit Theory of the Phenomenology of Mental Effort. Review of Philosophy and Psychology. Advance Online Publication, 1–16. https://doi.org/10.1007/s13164-020-00512-7

Vertesi, J. (2008). “Seeing Like a Rover”: Embodied Experience on the Mars Exploration Rover Mission. CHI ’08 Extended Abstracts on Human Factors in Computing Systems, 2523–2532. https://doi.org/10.1145/1358628.1358709

Wu, L., Alqasemi, R., & Dubey, R. (2020). Development of Smartphone-Based Human-Robot Interfaces for Individuals with Disabilities. IEEE Robotics and Automation Letters, 5(4), 5835–5841. https://doi.org/10.1109/LRA.2020.3010453

Article Sidebar

Main Article Content