Anik Sarker

Real Time Hand Tracking Algorithm

Paper under review.

• Novel hand tracking algorithim fusing ML prediction with anaytical model based prediction

This project was completed during my internship at Meta.

Spherical Point Pattern Marching and Applications

1 paper accepted for ICCV 2025. 2 Papers under review.

Developed novel algorithms for correspondence-free spherical point pattern matching. Example applications include point cloud registration and spherical image registration. Our methods are rigorously evaluated against benchmark algorithms such as TEASER, ICP, and deep learning approaches like PointNetLK and Deep Closest Point. We are also extending our work to spherical image registration using our fast spherical correlation algorithm. Applications span pose estimation, 3D reconstruction, and image processing.

Accurate On-Body Distance Estimation Using BLE RSSI and IMU Sensor Fusion

In this work we develop a novel sensor fusion algorithm that combines IMU and BLE RSSI data to accurately estimate on-body distances. By leveraging the complementary strengths of these sensors, our approach significantly improves distance estimation accuracy compared to using RSSI alone. We validate our method through experiments involving multiple subjects performing various upper-body movements while wearing IMUs and RSSI transmitters/receivers. Our results demonstrate the effectiveness of sensor fusion in enhancing pose estimation capabilities.

The project is being funded by NSF.

Meta (Reality Labs Research) - Research Scientist

Working as a Research Scientist at Meta Reality Labs Research in Redmond, Washington.

Foundation Models for Multi-Agent Movement

Worked with the Foundation Model team at Amazon Robotics to develop and scale multimodal foundation models for multi-agent movement and robotics applications. Enhanched existing models to incorporate diverse modalities for robust multi-agent coordination and prediction. Applied scaling laws, Mixture of Experts, multi-task learning, and optimizing with multiple loss functions for efficient inference in both encoder-only and decoder-only architectures. Provided recommendations for model configuration and evaluation to improve performance across various robotics benchmarks.

Real Time Hand Tracking Algorithm

• Gained hands-on experience in multimodal perception for real-time detection and tracking across multiple sensors, including IMUs, tactile sensors, cameras, EMG, and other time-series data sources.
• Leveraged state-of-the-art machine learning models, analytical filters, and signal processing techniques to develop robust solutions.
• Built large-scale ML models from the ground up, including data extraction and preprocessing pipelines for efficient and accurate data handling.
• Designed a novel approach that led to a 40% increase in accuracy for a real-time tracking problem.
• Successfully deployed these solutions on one of Meta’s flagship tracking devices, significantly enhancing overall performance compared to previous versions.

Object Detection: Fine-tune custom dataset with YOLO V8, NAS, Facebook Detr

The goal of this project was to explore the performance of different object detection and tracking algorithms on a custom dataset. Fine-tuned "SODA10M" dataset (the largest 2D object detection dataset in autonomous driving) using state of the art "YOLO-NAS", "YOLO-V8" and "Facebook Detr".

Applied several unique visualization techniques to compare performances of different models both quantitatively and subjectively.

Dataset Website

Input and Output (detect + track) video of unseen test data.

Human pose reconstruction and ergonomic pose classification

Human pose reconstruction and ergonomics evaluation have many applications, such as stroke rehabilitation, underground mining, safety and rescue, bio-mechanical monitoring, sports and fitness training, assistive devices, and virtual reality interaction.

This project presents an approach for Human Pose Reconstruction and Human Posture (Ergonomics) Evaluation based on five low-cost IMU sensors (XSens DOT). We use information from 5 IMU sensors (orientation, acceleration) to predict full-body kinematics (23 human body segments). For ergonomics evaluation, we classify walking and sitting postures into normal or poor posture using auto-encoders (sequence-to-sequence, transformers) to map the latent space (five segments) to the full pose-space (23 segments). Ergonomics parameters include Flexion, Lateral Bending, and Extension for ‘Pelvis’ and ‘T8’ segments.

We compare reconstruction and classification error with respect to ground truth.
Sequence-to-sequence, Transformers, PyTorch, XSens   (PDF)

B-SPLINE Global Surface Interpolation and Approximation

Developed a GUI app for demonstrating both B-Spline global surface interpolation and approximation.

The user interface allows input through mouse and keyboard to control several parameters related to surface interpolation and approximation.

C++, OpenGL   (PDF)