Harman Singh

I am a researcher at Google Research, working on large scale multimodal and multilingual modelling with Dr. Partha Talukdar in the NLU team.

Previously, I was an AI Resident at Meta AI where I worked on reasoning abilities of Vision Language Models. Before this, I completed my undergrad from Indian Institute of Technology, Delhi, advised by Prof. Parag Singla. During this time, I have been a research intern at InkLab, USC, advised by Prof. Xiang Ren and a research intern at IBM Research AI.

Email  /  CV  /  Google Scholar  /  Semantic Scholar  /  Twitter  /  Github

profile photo
IndicGenBench IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages
Harman Singh, Nitish Gupta, Shikhar Bharadwaj, Dinesh Tewari, Partha Talukdar

Under Review, 2024
dataset link: https://github.com/google-research-datasets/indic-gen-bench

IndicGenBench is a multilingual, multi-way parallel benchmark for measuring language generation capabilities across diverse user-facing tasks in 29 Indic languages spanning 13 writing scripts and 4 language families.

MosaiCLIP Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality
Harman Singh, Pengchuan Zhang, Qifan Wang, Mengjiao Wang, Wenhan Xiong, Jingfei Du, Yu Chen

EMNLP 2023 (long paper, main conference)
Oral acceptance to CLVL Workshop at ICCV 2023

Improving compositional reasoning capabilities of SOTA Vision-Language Models through a new Coarse-to-Fine contrastive learning technique as well as effective hard negative mining.

NeuroSIM Image Manipulation via Multi-Hop Instructions -- A New Dataset and Weakly-Supervised Neuro-Symbolic Approach
Harman Singh, Poorva Garg, Mohit Gupta, Kevin Shah, Ashish Goswami, Satyam Modi, Arnab Kumar Mondal, Dinesh Khandelwal, Parag Singla, Dinesh Garg

EMNLP 2023 (long paper, main conference)

New datasets and a modular method for weakly-supervised instruction guided image manipulations.

fairr FaiRR: Faithful and Robust Deductive Reasoning over Natural Language
Soumya Sanyal, Harman Singh, Xiang Ren

ACL 2022 (long paper, main conference)

Proposed a modular method (FaiRR) for logical reasoning over natural language rule bases. Our methods ensure model faithfulness by assured causal relation from the proof step to the inference reasoning. FaiRR is more interpretable, efficient as compared to baselines, and generalizes better to OOD logical reasoning tasks.

Past Work (Bioinformatics Research)

covid-paper-nature-image Unlocking capacities of genomics for the COVID-19 response and future pandemics
Sergey Knyazev, Karishma Chhugan, Harman Singh*, Varuni Sarwal*, Ram Ayyala*, ..., Serghei Mangul
Nature Methods
e-MSN-isbra A Novel Network Representation of SARS-CoV-2 Sequencing Datas
Sergey Knyazev, Daniel Novikov, Mark Grinshpon, Harman Singh, ..., Serghei Mangul
International Symposium on Bioinformatics Research and Applications 2021
Reviewer for ICLR 2023, NeurIPS 2023, EMNLP 2023, MLRC 2022 (Outstanding Reviewer Award), COLM 2024, ACL Rolling Review (ARR) 2024
Teaching Assitant for Machine Learning (Dr. Sumeet Agarwal and Dr. Jayadeva, Fall 2021)
Teaching Assitant for Intro. to EE (Dr. Anuj Dhawan, Fall 2021)
Demo Leader, NeurIPS Education Outreach Program (for 240+ high school students), NeurIPS 2022
Website template by Jon Barron