Research Assistant

Carnegie Mellon University


I am Ayush Jain, a final year student pusrsuing B.E. in computer science from BITS Pilani, Pilani Campus. Currently, I am also a research assistant at Carnegie Mellon University advised by Prof. Katerina Fragkiadaki. My research interests include computer vision and its intersection with language and control. I want to enable self-supervised robots that can learn about the world in a never-ending learning fashion without requiring explicit human supervision.


  • Computer Vision
  • Natural Language Processing
  • Reinforcement Learning


  • B.E. in Computer Science, 2021

    BITS PILANI, Pilani Campus


[Aug 2020] I’ll be attending Google Research India’s AI Summer School (Computer Vision track) from August 20-22, 2020!

[May 2020] Joined Prof. Katerina Fragkiadaki’s Lab as Research Associate at Carnegie Mellon University.

Recent Publications

Move to See Better: Towards Self-Supervised Amodal Object Detection

Humans learn to better understand the world by moving around their environment to get more …
Under Review (CVPR 2021).

AI-enabled Object Detection in Unmanned Aerial Vehicles for Edge Computing Applications

Unmanned Aerial Vehicles (UAVs) are emerging as a powerful tool for various industrial and smart …
Under Review (IEEE comsoc).

VisDrone-DET2020: The Vision Meets Drone Object Detection in Image Challenge Results

The Vision Meets Drone Object Detection in Image Challenge (VisDrone-DET 2020) is the third annual …
ECCV 2020 Workshop Paper (In Press).

NukeBERT: A Pre-trained language model for Low Resource Nuclear Domain

Significant advances have been made in recent years on Natural Language Processing with machines …



Research Associate

Carnegie Mellon University

May 2020 – Present Pittsburgh, Pennsylvania, USA
Under the supervision of Prof. Katerina Fragkiadaki, I worked on enabling an embodied agent to learn about objects without ground truth supervision in an unseen 3D environment just by moving around. I am currently working on embodied language grounding under weak supervision.

Research Assistant


Aug 2019 – Apr 2020 Rajasthan, India
As a research assistant, I worked under the guidance of Prof. Pratik Narang on Aerial Object Detection. My team and I proposed a novel architectural design that improved the performance of previous methods by a substantial margin.

Research Assistant

Indira Gandhi Center for Atomic Research

May 2019 – Jul 2019 Tamil Nadu, India
I worked on building and releasing two new Nuclear Domain Textual Datasets, and proposing a data efficient BERT based baseline.


Towards Learning Spatial Common Sense Through Weak Supervision : A Comparitive Study

Reviewed, compared and analysed three recent papers proposed in generalising to novel views through the use of view-prediction …

ABB Robot Simulation

Built simulation of various tasks like writing a cursive and block alphabets on a paper and building complex curved designs using …

Named Entity Recognition from scratch

Built multilayer perceptron network for Named Entity Recognition on CONLL2003 dataset from scratch, implementing backpropagation and …

Compiler Construction

erplag-cc is a compiler for the custom language ERPLAG, written in C. The compiler generates assembly code that can be run on a Linux …

AI Minesweeper

An artificially agent created to play Minesweepers. external_link.

Earthquake Prediction and Management

Implemented Rundle et al.’s research paper for probabilistic prediction of earthquake based on nowcasting. Using the previous …

Teaching and Mentoring

I have served as a teaching assistant for the following courses at BITS Pilani:

  • CS F407 Artificial Intelligence, Fall 2020
  • CS F464 Machine Learning, Spring 2020
  • CS F111 Computer Programming, Spring 2019 and Fall 2019