Innovations

AmazonGo - Retail Shopping

AmazonGo is a re-imagining of what a retail shopping experience should look like. My team and I invented new solutions and capabilities. Computer vision, sensors, hardware and artificially intelligent algorithms blend into the background to create a magical shopping experience with AmazonGo.

Patents:

US9996818B1 - Counting inventory items using image analysis and depth information
US10882692B1 - Item replacement assistance
US10169677B1 - Counting stacked inventory using image analysis
US10789483B1 - Image based inventory item counting
US10949801B1 - Inventory item release apparatus and method
US10671856B1 - Detecting item actions and inventory changes at an inventory location

Product Recommendations

"Once in a decade leap", was what Jeff Wilke (CEO of Amazon's Consumer Division) had called it at his re:MARS 2019 keynote.

I invented a new formulation for product recommendation which was 2x better than Amazon's state of art in recommendation.

Video - Jeff Wilkie's Talk at re:MARS 2019
Paper - The Effectiveness of Two-layer Neural Network for Recommendations
Article - The History of Amazon’s Recommendation Algorithm
Patent - US10290040B1 - Discovering cross-category latent features

Ad Targeting using ML

I worked with the early team that built the machine learning based display ad targeting platform at Amazon. We saw on average 3x more relevant advertisements. Over the years Amazon's Ad business has grown. In Q3, 2023, Amazon Advertisement generated 12 billion in revenue for the quarter.

Patent - US11138631B1 - Predictive user segmentation modeling and browsing interaction analysis for digital advertising

Precision Agriculture

Combining multi-spectral images (from planes, drones or satellite) with other sensor data, can help us track health of an individual tree in an orchard. I invented a method to align multiple images, so that we could use distributed compute on big data to analyse 100,000 acres in 15 minutes.

Patent - US10089712B2 - System and method providing automatic alignment of aerial/satellite imagery to known ground features

IoT Sensor Data Synthesis for ML

IoT sensors can produce valuable data, but can also expose user privacy. We invented an algorithm to synthesis IoT sensor data, which protects the original source data privacy, while allowing teams to use it with machine learning algorithms to find patterns and make predictions.

Patent - US11392850B2 - Synthesizing high-fidelity time-series sensor signals to facilitate machine-learning innovations

Medical Image Analysis

High Resolution CT scans are a non-invasive way to help radiologists diagnose diseases. Yet going through 100s of images per patient can be time-consuming. LMIK project was looking at ways to automate the analysis of lung anatomy and diseases in scans, to help radiologists.

Papers:

Misra et al - Automatic lung segmentation: a comparison of anatomical and machine learning approaches
Misra et al - Incremental learning for segmentation in medical images
Massoptier et al - Combining Graph-cut Technique and Anatomical Knowledge for Automatic Segmentation of Lungs Affected by Diffuse Parenchymal Diseases in HRCT Images
Massoptier et al - Automatic lung segmentation in HRCT images with diffuse parenchymal lung disease using graph-cut
Rudrapatna et al - LMIK - learning medical image knowledge: an Internet-based medical image knowledge acquisition framework

Incremental Engineering

Software engineering of vision systems is challenging when data trickles in and the algorithms need to evolve over time. I used incremental knowledge acquisition techniques to provide ways to deal with the changes rapidly and in doing so speed up the development process, and accuracy over time.

Papers:

Misra et al - Incremental learning for segmentation in medical images
Misra et al - Incremental learning of control knowledge for lung boundary extraction
Misra et al - Incremental system engineering using process networks
Misra et al - Incremental engineering of computer vision systems
Misra et al - Impact of quasi-expertise on knowledge acquisition in computer vision