Our laboratory conducts research on intelligent information systems that support human communication and the high-performance networks that enable them. We study a wide range of media and their underlying theories, core technologies, and applications, including media that watch over people, teach them, connect people with each other, simplify information, and help them notice things. As foundational technologies, we explore the processing and automatic recognition of images, speech, and natural language, along with human behavior understanding based on such analysis, as well as next-generation Internet technologies. For applications, we consider the realization of media that can significantly transform the future of education, daily life, and remote communication (e.g., lectures, meetings, and experience sharing).

• Estimating internal states from sensing of facial expressions and gestures

We observe outward expressions such as human gestures and facial expressions to infer internal states such as intent and emotion, and to estimate indicators such as QOL (Quality of Life).

• Fine-grained evaluation of facial expression based on expression comparison

People often display subtle intermediate facial expressions in daily life that are not obvious at a glance. To estimate psychological state and emotion from such expressions, we proposed a method that recognizes changes in facial expression by comparing two facial images and demonstrated that this enables fine-grained evaluation of expression intensity according to an individual’s expression scale.



• K. Kondo, T. Nakamura, Y. Nakamura, and S. Satoh, ``Siamese-structure Deep Neural Network Recognizing Changes in Facial Expression According to the Degree of Smiling'', In proc. of 25th International Conference on Pattern Recognition, 2021.
• K. Shimonishi, J. Zhang, K. Kondo, H. Ueda, and Y. Nakamura, ``Assessment of smile intensity using an ordinal scale toward QOL monitoring'', Intelligent Systems with Applications, Vol. 30, 2026.
• Estimation of attention state through measurement and modeling of gestures

We represent and estimate how people allocate attention across multiple targets using a dynamic model based on head orientation, hand orientation, trunk orientation, and the relationships among them. We analyzed the relationship between patterns of attention distribution among two objects and features of motion in gestures from the perspectives of attention and embodiment.



• Takaki Kurisu, Kazuaki Kondo, Yuichi Nakamura, ``A Proposal of a Method for Analyzing Attention Distribution Based on Coordination of Multiple Body Parts'', HCG Symposium 2016 (in Japanese).
• Takaki Kurisu, Kazuaki Kondo, Yuichi Nakamura, ``Analysis of Coordinated Motion of the Head, Trunk, and Hands Depending on Attention Allocation'', HCG Symposium 2017 (in Japanese).
• Assistive motion and behavior support based on biosensing

By observing muscle activity, motion, and posture, we estimate intent and predict movement to explore ways of assisting users through power assistance and joint constraints. We also design and implement assistive devices for this purpose.

• Exoskeleton power assistance that adapts to motion intent

To provide natural support that does not interfere with human activity or make the user feel it too strongly, we propose a mechanism that provides power assistance only when needed and in the required amount. Based on analysis of muscle coordination across multiple muscles, we predict motion intent before actual movement begins, and we are also developing passive assistive devices that do not inhibit the user’s voluntary actions.



• J. Furukawa, S. Okajima, Qi An, Y. Nakamura, and J. Morimoto, ``Selective Assist Strategy by Using Lightweight Carbon Frame Exoskeleton Robo'', IEEE Robotics and Automation Letters, Vol.7, No.7, pp.3890-3897, 2022.
• J. Ilham, Y. Nakamura, K. Kondo, T. Ito, Qi An, J. Akita, and M. Toda, ``Stability of muscle synergy patterns for sit-to-stand motions under different condition'', IEICE Tech. Report, MBE2021-88, Vol.121, No.389, 2022.
• Subtle guidance of attention and movement through tactile presentation

As a gentle method to provide awareness and guidance without obstructing human activity, we propose tactile presentation that lightly pulls or stretches the skin. We designed a system that transmits the movement of the neck measured by electromyography to a person in a remote location by pulling the skin of their neck, and we verified transmission performance while investigating the potential for communication through the neck.



• T. Ito, K. Kondo, Y. Nakamura, J. Rossiter, J. Akita, and M. Toda, ``Skin-pulling device for conveying and monitoring activity and motion state'', IEICE Technical Report, vol. 118, no. 35, PRMU2018-5, 2018.
• T. Ito, Y. Nakamura, K. Kondo, J. Rossiter, J. Akita, and M. Toda, ``Motion Information Transmission for On-Neck Communication'', The 3rd Int. Conf. on Computer-Human Interaction Research and Applications, 2019.
• Modeling and supporting human communication

We analyze communication between humans in various settings, including conversation, presentations, and collaborative work, and design information systems and robots to support such interactions.

• Communication analysis in remote work support

In “remote dialogue-based behavior support,” where the work situation is monitored from a distance and necessary information or instructions are provided, we analyze how communication is structured. As an indicator of communication quality, we proposed the “consistency of speech and actions between two people,” and analyzed its relationship with temporal characteristics in reciprocal interaction patterns.



• Y. Koizumi, K. Obata, Y. Watanabe, K. Kondo, and Y. Nakamura, ``Analysis of the attitudes of workers and supporters in video-dialogue behavioral support'', HCG Symposium 2013.
• Y. Koizumi, K. Obata, Y. Watanabe, K. Kondo, and Y. Nakamura, ``Analysis of communication based on frequent patterns in video-dialogue behavioral support'', IPSJ Journal, Vol.56, No.3, pp.1068-1079, 2015.
• Pointing interface based on human motion characteristics

We design interfaces and measurement/presentation systems that enable smooth communication in presentation styles where users point to materials shown on a large display. To address the problem that pointing poses do not always accurately indicate the intended location, we proposed display methods and intention-estimation techniques based on human visual perception and motion characteristics.



• K. Kondo, G. Mizuno, and Y. Nakamura, ``Feedback Control Model of a Gesture-based Pointing Interface for a Large Display'', IEICE TRANSACTIONS on Information and Systems, Vol. E101-D, No.7, pp. 1894-1905, Jul. 2018.
• T. Fujiwara, Y. Nakamura, and K. Kondo, ``Design of a collaborative pointing interface based on user intent estimation and prediction'', IEICE Technical Report, vol. 120, no. 185, HIP2020-39, 2020.
• K. Kondo, T. Fujiwara, and Y. Nakamura, ``Non-linear distance filter for modeling effect of a large pointer used in a gesture-based pointing interface'', IEICE TRANSACTIONS on Information and Systems, Vol.E103-D, No.11, pp.2302-2313, 2020.
• Sharing and utilizing memories and experiences

By recording human behavior and group activities using video, location data, physiological signals, and other sensing data, we share memories and experiences and utilize them for education and training.



• Analysis of group work experience recordings

We are developing methods for recording and analyzing group work so that participants and organizers can review collaborative activities in a meaningful way. We proposed the amount of hands appearing in participants’ field of view as a feature of group work involving both consultation and task execution, and clarified how scene changes over time and cooperation styles differ across groups.



• D. Deguchi, K. Kondo, and A. Shimada, ``Visualization of real world activity on group work'', In Proc. of 6th Int. Conf. on Distributed, Ambient and Pervasive Interactions, 2018.
• K. Kondo, K. Shimada, D. Deguchi, ``Proposal of a feature for group collaborative work based on the appearance of hands in first-person video'', HCG Symposium 2018 (in Japanese).