RESERACH
Current Research Projects
RP3: Conversational Human-Robot Interaction Vision-Language-Action (cHRI-VLA) Models for Mobile Manipulators
The research objective of this project is to design, develop, and evaluate a Cyber-Physical System called the WheelArm that integrates and supports (i) multi-modal user intent identification
and adaptive planning (ii) autonomous and safe task execution for both the wheelchair and the arm, and (iii) an edge computing architecture that enables real-time processing of sensory inputs to enhance system's responsiveness and reliability.
RP2: Multi-Modal Human Intent Recognition and Understanding for Human-Robot Interaction
The research objective of this project is to apply fundamental theories at the intersection of artificial intelligence, deep learning, human-robot interaction, and multi-modal signal processing to develop new solutions for human intent recognition and understanding for assistive devices.
to bridge the gap between user intent and device action to enhance the interaction between individuals with mobility impairments and smart technologies, such as wheelchairs equipped with robotic arms.
RP1: From Verbal Intent to Robotics Action Through Large Language Models (LLMs) and Natural Language Processing (NLP)
The research objective is to explore the feasibility of deploying LLMs and NLP for intuitive human-robot interaction, focusing on enabling mobile robots and assistive robotic arms to understand and execute complex commands in real-time, even in edge computing environments with limited or no cloud connectivity.
In this research, we used LLMs and NLP to explore intuitive human-robot interactions for controlling mobile robots and robotic arms. Firstly, we investigated deploying GPT-4-Turbo and a quantized version of LLaMA 2 on edge devices to control mobile robots using natural language commands. The study highlighted the superior performance of GPT-4-Turbo compared to LLaMA 2 in interpreting and executing tasks.
