Anik Sarker

Foundation Models for Multi-Agent Movement

Working in the Foundation Model team at Amazon Robotics to develop and scale multimodal foundation models for multi-agent movement and robotics applications. Enhancing existing models to incorporate diverse modalities for robust multi-agent coordination and prediction. Applying scaling laws, Mixture of Experts, multi-task learning, and optimizing with multiple loss functions for efficient inference in both encoder-only and decoder-only architectures. Providing recommendations for model configuration and evaluation to improve performance across various robotics benchmarks.

This project is ongoing as part of my internship at Amazon Robotics.

Real Time Hand Tracking Algorithm

Paper under review.

• Gained hands-on experience in multimodal perception for real-time detection and tracking across multiple sensors, including IMUs, tactile sensors, cameras, EMG, and other time-series data sources.
• Leveraged state-of-the-art machine learning models, analytical filters, and signal processing techniques to develop robust solutions.
• Built large-scale ML models from the ground up, including data extraction and preprocessing pipelines for efficient and accurate data handling.
• Designed a novel approach that led to a 40% increase in accuracy for a real-time tracking problem.
• Successfully deployed these solutions on one of Meta’s flagship tracking devices, significantly enhancing overall performance compared to previous versions.

This project was completed during my internship at Meta.

Spherical Point Pattern Marching and Applications

2 Papers under review.

Developed novel algorithms for correspondence-free spherical point pattern matching. Example applications include point cloud registration and spherical image registration. Our methods are rigorously evaluated against benchmark algorithms such as TEASER, ICP, and deep learning approaches like PointNetLK and Deep Closest Point. We are also extending our work to spherical image registration using our fast spherical correlation algorithm. Applications span pose estimation, 3D reconstruction, and image processing.

Human Upper-Body Pose Estimation Using IMU and RSSI

This is an extension of previous work. Previously, we used 3 IMUs to estimate upper-body kinematics. Now, we experiment with both IMU and RSSI (signal transmitters and receivers) to evaluate the feasibility of human pose estimation.

The project is being funded by NSF.

Object Detection: Fine-tune custom dataset with YOLO V8, NAS, Facebook Detr

The goal of this project was to explore the performance of different object detection and tracking algorithms on a custom dataset. Fine-tuned "SODA10M" dataset (the largest 2D object detection dataset in autonomous driving) using state of the art "YOLO-NAS", "YOLO-V8" and "Facebook Detr".

Applied several unique visualization techniques to compare performances of different models both quantitatively and subjectively.

Dataset Website

Input and Output (detect + track) video of unseen test data.

Human pose reconstruction and ergonomic pose classification

Human pose reconstruction and ergonomics evaluation have many applications, such as stroke rehabilitation, underground mining, safety and rescue, bio-mechanical monitoring, sports and fitness training, assistive devices, and virtual reality interaction.

This project presents an approach for Human Pose Reconstruction and Human Posture (Ergonomics) Evaluation based on five low-cost IMU sensors (XSens DOT). We use information from 5 IMU sensors (orientation, acceleration) to predict full-body kinematics (23 human body segments). For ergonomics evaluation, we classify walking and sitting postures into normal or poor posture using auto-encoders (sequence-to-sequence, transformers) to map the latent space (five segments) to the full pose-space (23 segments). Ergonomics parameters include Flexion, Lateral Bending, and Extension for ‘Pelvis’ and ‘T8’ segments.

We compare reconstruction and classification error with respect to ground truth.
Sequence-to-sequence, Transformers, PyTorch, XSens   (PDF)

B-SPLINE Global Surface Interpolation and Approximation

Developed a GUI app for demonstrating both B-Spline global surface interpolation and approximation.

The user interface allows input through mouse and keyboard to control several parameters related to surface interpolation and approximation.

C++, OpenGL   (PDF)