Nitin Nilesh

👋 Hey, I am Nitin Nilesh, currently employed as the Generative AI Lead at Harman, where I collaborate with IQVIA to develop healthcare solutions. My educational background includes a Master of Science by Research degree from the International Institute of Information Technology, Hyderabad (IIIT Hyderabad). During my time at IIIT Hyderabad, I conducted research at the intersection of Computer Vision, Machine Learning, Deep Learning, and IoT, under the guidance of Prof. Sachin Chaudhari and Prof. C.V. Jawahar.

Before joining Harman, I worked as a Camera Modeling Engineer at Qualcomm, where I focused on developing and enhancing Image Signal Processor pipelines. Prior to that, I interned with the Amazon India Machine Learning team, collaborating with exceptional individuals on projects involving Graph Neural Networks.

Research

My research interests encompass various domains, including Computer Vision, Computational Photography, Generative AI, Environmental Sensing (Air Pollution Monitoring), Machine Learning, and the Internet of Things (IoT).

Teaching

In addition to my research pursuits, I am passionate about education and mentoring. I actively mentor at TalentSprint, where I contribute to the AI and MLOps Programme in collaboration with IISc Bangalore and TalentSprint, as well as the AI/ML Programme in collaboration with IIIT Hyderabad and TalentSprint. In this role, I deliver lectures on fundamental concepts in Machine Learning and Deep Learning to industry professionals, guiding them through their project work.

Apart from this, I am an enthusiast of sports, particularly Badminton and Chess. Additionally, I enjoy engaging in cycling and running as part of my recreational pursuits.

Updates

Mar 7, 2024	Our patent named System and Method for Implementing an Experiment Remotely and Determining an Output Using a Computer Vision Model accepted for publication at United States Patent and Trademark Office 💡
Feb 20, 2024	Joined Harman as Generative AI Tech Lead! :🥳:
Jul 8, 2023	Defended my Master’s thesis titled “Learning Methods for IoT: Use Cases of Air Pollution Monitoring”.
Nov 1, 2022	Presented two Papers at WF-IoT 2022 titled IoT-based AQI Estimation using Image Processing and Learning Methods and IoT and ML-based AQI Estimation using Real-time Traffic Data 🗞️
Sep 27, 2021	Joined Qualcomm as Camera Modeling Engineer!

Selected Publications

2023

Towards Real-Time Analysis of Broadcast Badminton Videos

Nitin Nilesh, Tushar Sharma, Anurag Ghosh, and C.V. Jawahar

Arxiv Preprint, Oct 2023

An end-to-end framework for real-time badminton analysis, tested live in Premier Badminton League (PBL). Find the live broadcast by Hotstar showing the analysis here.

Abs arXiv HTML Blog Code

Analysis of player movements is a crucial subset of sports analysis. Existing player movement analysis methods use recorded videos after the match is over. In this work, we propose an end-to-end framework for player movement analysis for badminton matches on live broadcast match videos. We only use the visual inputs from the match and, unlike other approaches which use multi-modal sensor data, our approach uses only visual cues. We propose a method to calculate the on-court distance covered by both the players from the video feed of a live broadcast badminton match. To perform this analysis, we focus on the gameplay by removing replays and other redundant parts of the broadcast match. We then perform player tracking to identify and track the movements of both players in each frame. Finally, we calculate the distance covered by each player and the average speed with which they move on the court. We further show a heatmap of the areas covered by the player on the court which is useful for analyzing the gameplay of the player. Our proposed framework was successfully used to analyze live broadcast matches in real-time during the Premier Badminton League 2019 (PBL 2019), with commentators and broadcasters appreciating the utility.

2022

IoT-based AQI Estimation using Image Processing and Learning Methods

Nitin Nilesh, Ishan Patwardhan, Jayati Narang, and Sachin Chaudhari

IEEE 8th World Forum on Internet of Things (WF-IoT), Oct 2022

Winner of the Environmental Sensing Project Competition 2022.

Abs PDF Blog Code Website

Air pollution is a concern to the health of all living beings. It is essential to check on the quality of air in the surroundings. This article presents an IoT-based real-time air quality index (AQI) estimation technique using images and weather sensors on Indian roads. A mixture of image features, i.e., traffic density, visibility, and sensor features, i.e., temperature and humidity, were used to predict the AQI. Object detection and localization-based Deep Learning (DL) method along with image processing techniques were used to extract image features while an Machine Learning (ML) model was trained on those features to estimate the AQI. In order to conduct this experiment, a dataset containing 5048 images along with co-located AQI values across different seasons was collected by driving on the roads of Hyderabad city in India. The experimental results report an overall accuracy of 82% for AQI prediction.
IoT and ML-based AQI Estimation using Real-time Traffic Data

Nitin Nilesh, Jayati Narang, Ayu Parmar, and Sachin Chaudhari

IEEE 8th World Forum on Internet of Things (WF-IoT), Oct 2022

Abs PDF Website

This paper proposes an IoT and machine learning (ML)-based novel method to estimate the air quality index (AQI) using traffic data in real-time. With the help of particulate matter (PM) monitoring nodes deployed in fifteen locations with diverse traffic scenarios of Indian roads, and using digital map service providers, a rich traffic dataset with approximately 210,000 samples has been collected. Three different ML models, namely random forest (RF), support vector machine (SVM), and multi-layer perceptron (MLP), are trained on this dataset to predict the AQI category into five levels. The experimental results show an accuracy of 82.60% with the F1-score of 83.67% on the complete dataset. Apart from this, ML models were also trained on individual node datasets, and the behavior of AQI levels was observed.

2024

System and Method for Implementing an Experiment Remotely and Determining an Output Using a Computer Vision Model

Sachin Chaudhari, Venkatesh Choppella, Om Rajendra Kathalkar, Viswanadh Savita Kandala, and Nitin Nilesh

United States Patent and Trademark Office (USPTO), Mar 2024

Abs PDF Website

A system and method for implementing an experiment remotely and determining an output using a computer-vision model is provided. The system includes an image capturing device, an experiment setup, a microcontroller, a user device, and a relay unit. The microcontroller (i) receives the input of the experiment from the image capturing device, (ii) extracts one or more frames from the input data, (iii) pre-process the one or more frames to obtain a binary image, (iv) obtain a closed curve around the binary image to locate the experiment, (v) determine the coordinates of the experiment to track the experiment in each frame, (vi) determine an output of the experiment from every two consecutive frames of the one or more frames, and (vii) optimize the determined output of the experiment using a linear regression model.