Research

This project develops a unified supervisory framework for mobile manipulators operating in human-centered environments such as homes, clinics, and assisted-living facilities. Combining autonomous navigation (TIAGo Pro, Open-RMF fleet management) with dexterous manipulation (UR12e + Robotiq Hand-E), the research investigates how human operators can efficiently oversee and intervene in multi-robot workflows — bridging full autonomy and direct teleoperation through adjustable autonomy interfaces.

Designing and evaluating a suite of teleoperation interfaces for the UR12e + Robotiq Hand-E platform, spanning five sensing modalities: webcam-based hand tracking (MediaPipe), wearable exoskeleton mapping (EduExo), leader-arm joint mirroring (PincherX-100), gamepad control (DualShock 4 / Nintendo Switch Pro), and immersive VR (Meta Quest 2). The research asks: how should interface modality be matched to user capability, task structure, and deployment context — and what design principles generalize across assistive robotics applications?

Large Vision-Language-Action (VLA) models offer a promising path toward generalizable robot manipulation — but deploying them reliably in assistive contexts requires understanding how humans can efficiently supervise, correct, and trust autonomous policies. This project fine-tunes the π₀ framework (PaliGemma 3B backbone, cross-embodiment pre-training on UR5e and other platforms) on a UR12e arm for language-conditioned pick-and-place tasks. Three interconnected research questions guide the work: how many teleoperation demonstrations are needed to achieve reliable task performance (sample efficiency); how demonstration quality and collection modality affect fine-tuning outcomes (data strategy); and how to design meaningful human-oversight mechanisms that balance autonomy with appropriate intervention. Graduate student collaborators develop the data collection pipeline, fine-tuning workflow, and evaluation protocols.