Qingqing Cao

Research Scientist •  Apple AIML.

prof_pic.jpg

I am a research scientist at Apple AIML. My research interests include efficient NLP, mobile computing, and ML systems. I have focused on building efficient and practical NLP systems for both edge devices and the cloud, such as on-device (visual) question answering and faster Transformer models.

Previously, I was a postdoc in the UW NLP group at the University of Washington. I hold a Ph.D. degree in computer science at Stony Brook University. I was a recipient of the Catacosinos Fellowship at Stony Brook University and a Rising Star in Data Science at the University of Chicago.

News

May 01, 2024 APT paper got accepted to ICML 2024 🎉! Congrats to Bowen 👏!
Apr 22, 2024 Checkout OpenELM, a new efficient language model family that optimizes parameters for accuracy with fewer tokens using layer-wise scaling! Training code is on Github, models are also on HuggingFace.
Feb 15, 2024 Glad to be invited to serve as Action Editor / Area Chair for ACL 2024 !
Jan 16, 2024 BTR was accepted to ICLR as a spotlight paper! 🎊
Nov 10, 2023 Gave a talk at the Efficient ML workshop hosted by Google Research.

Recent publications

2024

  1. # OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
    Apr 2024
  2. # APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
    Bowen ZhaoHannaneh Hajishirzi, and Qingqing Cao
    Jan 2024

2023

  1. # BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
    Qingqing CaoSewon MinYizhong Wang, and Hannaneh Hajishirzi
    In , Oct 2023
  2. # PuMer: Pruning and Merging Tokens for Efficient Vision Language Models
    Qingqing CaoBhargavi Paranjape, and Hannaneh Hajishirzi
    In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023