NSF-REU Program 2025 at UMBC
Professional Development Activities
COEIT Professional Development Sessions
Summer Undergraduate Research Fest (SURF)
Weekly Individual Research Meeting and Training
Longitudinal Oral Health Monitoring Using IMU and Audio from Wearable Sensors
REU student: Kodilinye Mkpasi
Mentors: Dong Li, Dr. Anuradha Ravi, Dr. Nirmalya Roy
Abstract—Maintaining proper oral hygiene remains critical across all age groups. However, individuals frequently miss spots, neglect areas, apply too much pressure, or overemphasize specific areas during brushing. Incorrect brushing techniques can lead to plaque buildup, enamel abrasion, gingivitis, and other oral health issues. With increasing popularity of smartwatches, there exists significant potential to leverage these devices for monitoring brushing behavior and techniques. This research investigates application of consumer-grade smartwatches for detecting and analyzing toothbrushing activities, with a specific focus on recognizing distinct brushing techniques, including horizontal scrub, vertical scrub, and circular motion patterns, using heterogeneous sensors across various smartwatch platforms. The research addresses multiple challenges in smartwatch-based toothbrushing activity recognition, including gaps in sampling data with unequal data stream lengths, gyroscope drift during rapid motions, establishing appropriate coordinate systems to determine tooth surface locations, synchronizing video annotations with multiple IMU and audio data streams, and managing inherent variability in sensor configurations across Samsung and Apple smartwatch ecosystems. Linear interpolation was employed to resample and synchronize disparate data streams, ensuring alignment across all sensor inputs. Complementary filters were developed to extract gravity vectors from accelerometer data, thereby mitigating gyroscope drift during dynamic brushing motions.
Signal Quality and ROI Analysis for Video-Based Contactless Respiratory Rate Estimation
REU student: Haley Patel
Mentors: Zahid Hasan, Dr. Nirmalya Roy
Abstract—Contactless respiratory rate (RR) estimation refers to a class of noninvasive techniques, including video-based methods, for measuring breathing in clinical, home, and research settings. However, accurately capturing respiratory motion is challenging due to its variability across individuals and conditions. Factors such as distance, posture, occlusion, camera angle, clothing, and background significantly affect which body region—chest, shoulder, or abdomen—provides the most reliable motion cues. This study investigates how variations in subject-to-camera distance affect the visibility of respiratory motion and aims to identify the most dominant region of interest (ROI) for robust and efficient RR estimation. We conducted a series of controlled video recordings of five subjects at varying distances, capturing normal, no-breathing, and heavy-breathing phases. To analyze how different factors affect respiratory motion visibility, we tested the impact of subject-to-camera distance, ROI placement, and point selection across the torso. We extracted pixel intensity signals and applied both small and large RAFT optical flow models to manually selected regions on a sample of frame pairs. Pearson correlation analysis revealed that the RAFT-large model consistently aligned more closely with pixel intensity trends, capturing respiratory motion more robustly across conditions.
Collaborative Robot Localization via Network Channel Characteristics and Visual Sensing in GPS-Denied Environments
REU student: Henry Gardiner
Mentors: Gaurav Shinde, Dr. Anuradha Ravi, Dr. Nirmalya Roy
Abstract—Ground Robots require precise localization to create a scene knowledge graph for accomplishing various missions, such as situational awareness. A scene knowledge graph is essential for scene understanding, as it creates a structured representation of the objects, agents, locations, and relationships between entities present in a visual scene. While robots are equipped with GPS, they are not effective in localizing themselves in GPS-denied environments. Hence, robots need to communicate with each other to estimate their relative location in the visual scene, which is essential for applications such as collaborative scene perception and navigation, as well as distributed task execution that requires multi-robot coordination. Received Signal Strength Indicator (RSSI) is widely used for localization and inter-robot distance estimation due to its low computational overhead. However, RSSI alone is highly susceptible to interference and environmental variability, leading to significant localization errors when used in isolation. Our preliminary studies on estimating distance from RSSI values, using a dataset curated in both home and lab environments, have resulted in median errors of 3.65 m using linear regression, 3.27 m using KNN Classification, and 1.47 m with tree-based ML models. Thus, to improve the estimation accuracy, we introduce a novel fusion framework that integrates RSSI with signal-to-noise ratio (SNR) measurements and selective RGB vision cues, yielding more robust distance estimates. Our unified model continuously fuses RSSI and SNR data, invoking visual observations selectively to account for measurement noise. We plan to test our approach on a heterogeneous multi-agent system comprising two TurtleBot3 Burger robots and a ROSbot 2 PRO. For long-range communication in GPS-denied settings, we employ Doodle Radios, which enable an extended operational range. We plan to conduct comparative evaluations against conventional path-loss models and RSSI-based regression techniques to assess how our fusion method reduces localization error and improves distance estimation across varying environments.
Deterring Wild Animals From Agricultural and Residential Settings Using Acoustics
REU student: Elijah Polyakov
Mentors: Snehalraj Chugh, Dr. Anuradha Ravi, Dr. Nirmalya Roy, Dr. Nirupam Roy, Bipendra Basnyat
Abstract—Human-wildlife conflicts have increased over the past decade, with wildlife causing an estimated $600 million in crop losses (corn, soybeans, wheat, cotton) in the United States between 2015 and 2019. Beyond agriculture, animals such as deer and foxes also damage gardens in urban and peri-urban areas, and most control methods remain harmful to them. This project seeks to develop effective and humane deterrent solutions to reduce wildlife intrusion in both agricultural and residential areas. Given the varying coverage requirements of these environments, different technologies must be employed to deliver sound-based deterrents tailored to each context. For the agricultural setting, we create two 'sound pod' devices: Sound pod A, which only produces tones, and Sound pod B, which can play sound files. For the residential setting, we create a phased array of speakers that can only generate tones. This will enable us to control the direction of sound propagation, ensuring minimal disruption to pets and neighboring areas. 'Sound Pod' A and the phased speaker array employ the Arduino Nano ESP32 microcontroller and the MCP4151 digital potentiometer to control the volume. 'Sound Pod' A uses the AD9833 function generator to create sound output, while the phased array uses direct GPIO pin output. 'Sound Pod' B employs an ESP32-WROOM module development board and plays sound through a Bluetooth speaker. We observed that the phased array was able to steer the sound with an accuracy of 15° and had a 10 dB (SPL) decrease in volume outside of the beam (in both the left and right directions). The pattern of sound propagation created by our phased array matched simulations using MATLAB. We also observed, using sound pod A, that a 2 kHz square wave tone was able to deter deer within approximately 2 meters of the device. Some directions of future research include increasing the precision of the phased array and determining whether tones, predator recordings, or a mix of both are more effective in deterring animals.
Cross-Modal Key Point Transfer from Land RGB-Depth to Underwater RGB-Sonar for Occluded Human Recognition
REU student: David Paz Menendez
Mentors: Milind Rampure, Dr. Anuradha Ravi
Abstract - This research will achieve human identification in aquatic environments under occlusion scenarios. This requires non-RGB features such as Anthropometric Data generated from key points. The research aims at collecting anthropometric data of humans on land and use the same for detection underwater. We analyzed the performance of keypoint based person re-identification using state of the art models. We identified that severe occlusions (75%) drops performance accuracy and evaluated which body parts are essential for improved person re-identification. The challenges are multifold in underwater scenario. Future work will involve: Transfer of land-based key point generation methods to aquatic scenarios; allowing re-identification through key points underwater. Pairing RGB with Sonar sensors to perform key point reconstruction given occlusions.Expanding upon datasets for occluded aquatic contexts with real divers in a more expansive underwater space. Implementing autonomous way point navigation through PyMAVlink to approach identified divers.