Abhinav Gupta Homepage

Abhinav Gupta

Professor The Robotics Institute Carnegie Mellon University

Office: EDSH 121
Phone: 412-268-2067 (email is the best option to reach me)
Email : abhinavg [at] cs [dot] cmu [dot] edu

ME

News

Back full-time at CMU starting May 2022.
Best paper award at ICRA22 Workshop - Scaling Robot Learning.
Awarded 2020 JK Aggarwal Prize for contributions to self-supervised learning
Awarded ONR Young Investigator Award 2018
Awarded Okawa Research Grant
Seven papers accepted in ICCV 2017
Eight papers accepted in CVPR 2017
Eight papers accepted in ECCV 2016
Awarded PAMI Young Researcher Award
Awarded ICRA Best Student Paper Award 2016
IJCAI Early Career Spotlight 2016
Awarded Sloan Fellowship.
NEIL appears in a Discover Magazine article.
Invited as panelist at Council on Foreign Relations
NEIL selected as CNN Top 10 Ideas of 2013.
Invited Speaker at Aspen Ideas Festival

About Me [biography]

I am a professor at Carnegie Mellon University. Prior to this, I was a post-doctoral fellow here working with Alyosha Efros and Martial Hebert. Before coming to Pittsburgh, I was working with Larry Davis at UMD and Jianbo Shi at UPenn. My PhD thesis was on "Beyond Nouns and Verbs".

Postdocs and Students

Unnat Jain (Postdoc)
Aravind Rajeswaran
Sudeep Dasari(Robotics PhD)
Shikhar Bahl(Robotics PhD, co-advised with Deepak Pathak)
Yufei Ye(Robotics PhD, co-advised with Shubham Tulsiani)
Raunaq Bhirangi(Robotics PhD, co-advised with Carmel Majidi)
Homanga Bharadwaj(Robotics PhD, co-advised with Shubham Tulsiani)

Visitors: NONE

Former PhD Students

David Fouhey PhD 2016, NSF fellow, NDSEG Fellow (now Asst Professor at U Michigan)
Carl Doersch PhD 2016, NDSEG Fellow, Google Fellowship (now Research Scientist at Google DeepMind)
Abhinav Shrivastava PhD 2017, MSR Fellow (now Asst. Professor at U. Maryland)
Xinlei Chen PhD 2018, Yahoo Fellow (now Research Scientist at Facebook AI Research)
Jacob Walker PhD 2018 (now Research Scientist at Google DeepMind)
Ishan Misra PhD 2018 (now Research Scientist at Facebook AI Research)
Xiaolong Wang PhD 2019, Facebook Fellow (now Asst. Professor at UCSD)
Lerrel Pinto PhD 2019 (now Asst. Professor at NYU)
Gunnar Sigurdsson PhD 2020 (now Research Scientist at Amazon AI)
Kenneth Marino PhD 2020, NDSEG Fellow (now RS at Google Deepmind)
Adithya Murali PhD 2020, Uber Fellow (now Research Scientist at NVidia Research, Seattle)
Senthil Purushwalkam PhD 2021 (now Research Scientist at Salesforce)
Victoria Dean NSF and Hertz Fellow, PhD 2023 (now Asst Prof at Olin)
Samantha Powers PhD 2023, Presidential Fellowship

Former Research Fellows/Members

Olga Russakovsky (now Asst. Professor at Princeton)
Yin Li (now Asst. Professor at U Wisconsin)
Saurabh Gupta (now Asst. Professor at UIUC)
Shubham Tulsiani (now Asst. Professor at CMU)
Simone Parisi
Tess Hellebrekers (now Research Scientist at Meta AI)
Pedro Morgado (now Asst. Professor at UWisc Madison)

Former MS Students

Dhiraj Gandhi(now Research Engineer at Facebook AI Research)
Tian Ye (now Software Engineer at Google)
Nilesh Kulkarni(now PhD Student at U Michigan)
Tao Chen (now PhD Student at MIT)
Wenxuan Zhou(now PhD Student at CMU)
Nadine Chang (now PhD Student at CMU)
Rohit Girdhar (now Research Scientist at Facebook AI Research)
Aayush Bansal (now PhD Student at CMU)
Saurabh Singh (Robotics MS, now PhD Student at UIUC)
Aravindh Mahendran (Research Staff, now PhD student at Oxford)

Courses

Press Coverage

Self-Supervised Grasping and Curious Robot

Never Ending Image Learner (NEIL)

.........

What makes Paris look like Paris?

Data Driven Visual Similarity

Blocks World Revisited

Storylines

Selected Projects
(Please see Publications for a complete list)
(Please see Downloads for code and datasets)

curious

Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park and Abhinav Gupta. The Curious Robot: Learning Visual Representations via Physical Interactions. ECCV 2016. (Spotlight)

pdf project page

voxels

R. Girdhar, D. Fouhey, M. Rodriguez, A. Gupta. Learning a Predictable and Generative Vector Representation for Objects. ECCV 2016. (Spotlight)

pdf project page

SSGAN

Xiaolong Wang and Abhinav Gupta. Generative Image Modeling using Style and Structure Adversarial Networks. ECCV 2016.

pdf project page

priming

Abhinav Shrivastava, Abhinav Gupta. Contextual Priming and Feedback for Faster R-CNN. ECCV 2016.

pdf project page

HiH

Gunnar A. Sigurdsson, G�l Varol, Xiaolong Wang, Ivan Laptev, Ali Farhadi, Abhinav Gupta. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. ECCV 2016.

pdf project page

priming

J Walker, C Doersch, A Gupta, M Hebert. An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders. ECCV 2016.

pdf project page

priming

Roozbeh Mottaghi, Mohammad Rastegari, Abhinav Gupta, Ali Farhadi. "What happens if..." Learning to Predict the Effect of Forces in Images. ECCV 2016.

pdf project page

priming

Gunnar A. Sigurdsson, Xinlei Chen, Abhinav Gupta, "Learning Visual Storylines with Skipping Recurrent Neural Networks", ECCV 2016.

pdf project page

baxter_grip

Lerrel Pinto and Abhinav Gupta. Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours. ICRA 2016. (Best Student Paper Award)

pdf project page

voxels

D. Fouhey, A. Gupta, A. Zisserman. 3D Shape Attributes. CVPR 2016. (Oral)

pdf project page

ohem

Abhinav Shrivastava, Abhinav Gupta, Ross Girshick. Training Region-based Object Detectors with Online Hard Example Mining. CVPR 2016 (Oral)

pdf project page

actions

Xiaolong Wang, Ali Farhadi, and Abhinav Gupta. Actions ~ Transformations. CVPR 2016.

pdf project page

actions

Aayush Bansal, Bryan Russell, Abhinav Gupta. Marr Revisited: 2D-3D Alignment via Surface Normal Prediction. CVPR 2016.

pdf project page

ohem

Ishan Misra*, Abhinav Shrivastava*, Abhinav Gupta, Martial Hebert. Cross-stitch Networks for Multi-task Learning. CVPR 2016

pdf project page

esvm

Carl Doersch, Abhinav Gupta, Alexei Efros. Unsupervised Visual Representation Learning by Context Prediction. ICCV 2015. (Oral)

pdf project page

esvm

Xiaolong Wang, Abhinav Gupta. Unsupervised Learning of Visual Representations using Videos. ICCV 2015. (Models and Code available for download!)

pdf project page

esvm

Xinlei Chen, Abhinav Gupta. Webly Supervised Learning of Convolutional Networks. ICCV 2015. (Oral)

pdf project page

esvm

David Fouhey, Abhinav Gupta, Martial Hebert. Single Image 3D without a Single 3D Image. ICCV 2015.

pdf project page

esvm

Jacob Walker, Abhinav Gupta, Martial Hebert. Dense Optical Flow Prediction from a Static Image. ICCV 2015.

pdf project page

esvm

Xiaolong Wang, David Fouhey, Abhinav Gupta. Designing Deep Networks for Surface Normal Estimation. CVPR 2015. (Models and Code available on request!)

pdf project page

esvm

Xinlei Chen, Alan Ritter, Abhinav Gupta, Tom Mitchell. Sense Discovery via Co-Clustering on Images and Text. CVPR 2015.

pdf project page

esvm

David F. Fouhey, Abhinav Gupta, Martial Hebert. Unfolding an Indoor Origami World. ECCV 2014. (Oral)

pdf project page

esvm

Carl Doersch, Abhinav Gupta, and Alexei A. Efros. Context as Supervisory Signal: Discovering Objects with Predictable Context. ECCV 2014.

pdf project page

esvm

Jacob Walker, Abhinav Gupta, and Martial Hebert.Patch to the Future: Unsupervised Visual Prediction. CVPR 2014. (Oral)

pdf project page

esvm

Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta. Enriching Visual Knowledge Bases via Object Discovery and Segmentation. In CVPR 2014.

pdf project page

esvm

Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta. NEIL: Extracting Visual Knowledge from Web Data. In ICCV 2013. (Oral)

pdf project page

esvm

David Fouhey, Abhinav Gupta and Martial Hebert. Data-Driven 3D Primitives for Single Image Understanding. In ICCV 2013

pdf project page

esvm

Abhinav Shrivastava and Abhinav Gupta. Building Part-based Object Detectors via 3D Geometry. In ICCV 2013

pdf project page

esvm

Carl Doersch, Abhinav Gupta and Alexei Efros. Mid-level Visual Element Discovery as Discriminative Mode Seeking. In NIPS 2013

pdf project page

esvm

Arpit Jain, Abhinav Gupta, Mikel Rodriguez and Larry S. Davis. Representing Videos using Mid-level Discriminative Patches. In CVPR 2013.

pdf project page

esvm

David Fouhey, Vincent Delaitre, Abhinav Gupta, Alexei A. Efros, Ivan Laptev, Josef Sivic. People Watching: Human Actions as a Cue for Single View Geometry. In ECCV 2012.(Oral)

pdf project page

ssl

Abhinav Shrivastava, Saurabh Singh, Abhinav Gupta.Constrained Semi-Supervised Learning Using Attributes and Comparative Attributes. In ECCV 2012.(Oral)

pdf project page

esvm

Saurabh Singh, Abhinav Gupta, Alexei A. Efros. Unsupervised Discovery of Mid-Level Discriminative Patches. In ECCV 2012.

pdf project page

esvm

Vincent Delaitre, David Fouhey, Ivan Laptev, Josef Sivic Abhinav Gupta, Alexei A. Efros. Scene Semantics from Long-term Observation of People. In ECCV 2012.

pdf project page

esvm

Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, Alexei A. Efros. What makes Paris look like Paris? In SIGGRAPH 2012. (Oral)

pdf project page

esvm Abhinav Shrivastava, Tomasz Malisiewicz, Abhinav Gupta, Alexei A. Efros, Data-driven Visual Similarity for Cross-domain Image Matching, In SIGGRAPH Asia 2011

pdf project page

esvm
Tomasz Malisiewicz, Abhinav Gupta, Alexei A. Efros, Ensemble of Exemplar-SVMs for Object Detection and Beyond, In ICCV 2011.
pdf project page

Source code (beta version) available

affordances
Abhinav Gupta, Scott Satkin, Alexei A. Efros and M. Hebert, From 3D Scene Geometry to Human Workspace. In CVPR 2011. (Oral)
pdf project page ppt

Source code partly available

nips
David C. Lee, Abhinav Gupta, Martial Hebert, and Takeo Kanade, Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces, In NIPS 2010.
pdf

Source code available

blocksworld

Abhinav Gupta, Alexei A. Efros and M. Hebert, Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics. In ECCV 2010. (Oral) (Best Paper Runner Up Award)
pdf project page ppt
Featured in Science Daily and ZD Net.

Source code available

al
Behjat Siddiquie and Abhinav Gupta, Beyond Active Noun Tagging: Modeling Contextual Interactions for Multi-Class Active Learning, In CVPR 2010. (Oral)
pdf ppt

movie
Abhinav Gupta, Praveen Srinivasan, Jianbo Shi and Larry S. Davis, Understanding Videos, Constructing Plots: Learning a Visually Grounded Storyline Model from Annotated Videos, In CVPR 2009. (Oral)
pdf ppt

Featured in an IEEE Spectrum and Discovery article. Also covered in Ethiopian Review and SiliconIndia.

beyondnouns
Abhinav Gupta and Larry S. Davis, Beyond Nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers, In ECCV 2008 (Oral)
pdf ppt

dpdp`
Abhinav Gupta, Aniruddha Kembhavi and Larry S. Davis, Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition, In Trans. on PAMI (Special Issue on Probabilistic Graphical Models).
pdf
Downloads Available: Dataset

Abhinav Gupta and Larry S. Davis, Objects in Action: An Approach for Combining Action Understanding and Object Perception, In CVPR 2007
pdf
Downloads Available: Dataset

nips_shape Abhinav Gupta, Jianbo Shi and Larry S. Davis, A “Shape Aware” Model for semi-supervised Learning of Objects and its Context, In NIPS 2008 (Spotlight Poster)
pdf

Funding Sources

Office of Naval Research: High Level MURI, ONR Applied Research
NSF: IIS 1320083
IARPA: ALADDIN Video
Google: focused award (2011), faculty research awards (2012, 2014)
Bosch Research & Technology Center: Bosch Young Faculty Fellowship 2014, Gift Award 2013
YAHOO!: Cluster for NEIL, InMind
HighMark Grant
MITRE
DARPA Memex Program