Mecharithm Lab

Menu
cropped-IMG_4474.png
Menu
Search
Layer_1

RP1: From Verbal Intent to Robotics Action Through Large Language Models (LLMs) and Natural Language Processing (NLP)

back to research

Project Lead

  • Madi Dian, Faculty Member, Aerospace and Mechanical Engeneering, SLU

Collaborators

    • Flavio Esposito, Faculty Member, Computer Science, SLU
    • Hadi AliAkbarpour, Faculty Member, Computer Science, SLU

Sponsors

Researchers

    • Paskal Sikorski, Undergraduate Research Assistant, Computer Science, SLU
    • Kaleb Yu, Undergraduate Research Assistant, Computer Science, SLU
    • Lucy Billadeau, Undergraduate Research Assistant, Aerospace & Mechanical Engineering, SLU
    • Leo Schrader, Undergraduate Research Assistant, Aerospace & Mechanical Engineering, SLU
    • Naveena Mutharasan, Undergraduate Research Assistant, Aerospace & Mechanical Engineering, SLU
The research objective is to explore the feasibility of deploying LLMs and NLP for intuitive human-robot interaction, focusing on enabling mobile robots and assistive robotic arms to understand and execute complex commands in real-time, even in edge computing environments with limited or no cloud connectivity.

In this research, we used LLMs and NLP to explore intuitive human-robot interactions for controlling mobile robots and robotic arms. Firstly, we investigated deploying GPT-4-Turbo and a quantized version of LLaMA 2 on edge devices to control mobile robots using natural language commands. The study highlighted the superior performance of GPT-4-Turbo compared to LLaMA 2 in interpreting and executing tasks.

Secondly, we focused on integrating NLP, LLMs, and computer vision in an edge-computing framework to enhance robotic arms, improving user experience and command interpretation through natural language.

videos


Products

RP1: From Verbal Intent to Robotics Action Through Large Language Models (LLMs) and Natural Language Processing (NLP)
Scroll to Top
Menu