Anna Kawakami

PhD Student in Human-Computer Interaction
School of Computer Science
Carnegie Mellon University

Passions

I explore whether and how we can design for human-AI complementarity: partnerships between humans and AI systems that build upon each others’ strengths and compensate for each others’ limitations. To poke at this problem space, I currently use mixed methods to study the use, design, and impacts of algorithmic decision support tools in high-stakes, complex domains such as social work. While addressing model- and interface-level design challenges, I aim to consider the impacts of organizational structures and pressures to contextualize our understanding of what effective human-AI partnerships might look like in the real world.

I care deeply about extending research to have non-negative real-world impact. My aspiration is to produce knowledge and approaches that can easily be transferred to support relevant community members, policymakers, and researchers. Findings from my recent research has contributed to policy efforts surrounding the use of algorithms in the public sector, and has prompted relevant public discourse through news coverage by AP News, NPR, PBS News, and other national and local media sources.

Positions

I am a second-year PhD student in the Human-Computer Interaction Institute at Carnegie Mellon University's School of Computer Science. I am fortunate to be co-advised by Ken Holstein (CoALA Lab) and Haiyi Zhu (Social AI Lab). Before starting my PhD, I was at Wellesley College, a historically women's liberal arts college, where I graduated with a BA in Computer Science (with honors) and minor in Economics, as a Durant Scholar and Trustee Scholar.

This past summer, I was a research intern at Microsoft Research FATE (Fairness, Accountability, Transparency, and Ethics in AI). During my undergraduate years, I also contributed to Microsoft's Responsible AI efforts as an undergraduate research intern at Microsoft Research Aether (AI Ethics and Effects in Engineering and Research). I also did research and studied abroad at Oxford University.

You can find more information in my CV.

Research

Recent Research Areas

Human-AI Complementarity.
E.g., how can we empower human decision-makers to identify and leverage their own and the AI's complementary strengths to make improved decisions?

Human-AI Value Alignment.
E.g., how might we improve evaluations and measures for AI-based decision support tools, to better align with stakeholders' notions of 'successful' decision-making?

Participatory & Mixed Methods.
E.g., what participatory design methods and tools can enable more meaningful, diverse stakeholder participation in model design?

Refereed Conference Publications (stringently peer-reviewed)

^{* denotes equal contribution}

Amanda Coston, Anna Kawakami, Haiyi Zhu, Kenneth Holstein, Hoda Heidari. “A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms" To appear in IEEE Conference on Secure and Trustworthy Machine Learning, IEEE SaTML 2023.

Anna Kawakami*, Venkat Sivaraman*, Logan Stapleton, Hao-Fei Cheng, Adam Perer, Steven Wu, Haiyi Zhu, Kenneth Holstein. “'Why Do I Care What's Similar?' Probing Challenges in AI-Assisted Child Welfare Decision-Making through Worker-AI Interface Design Concepts" In ACM Conference on Designing Interactive Systems, DIS 2022.
[paper] [video] [preview]

Anna Kawakami, Venkat Sivaraman, Hao-Fei Cheng, Logan Stapleton, Yang Cheng, Diana Qing, Adam Perer, Steven Wu, Haiyi Zhu, Kenneth Holstein. “Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support.” In ACM Conference on Human Factors in Computing Systems, CHI 2022.
[paper] [video] [preview]
Best Paper Honorable Mention Award (Top 5%)

Hao-Fei Cheng*, Logan Stapleton*, Anna Kawakami, Venkat Sivaraman, Yang Cheng, Diana Qing, Adam Perer, Steven Wu, Haiyi Zhu, Kenneth Holstein. “How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions.” In ACM Conference on Human Factors in Computing Systems, CHI 2022.
[paper] [video]

Anna Kawakami, Khonzoda Umarova, Jennifer Huang, Eni Mustafaraj. “The 'Fairness Doctrine' lives on? Theorizing about the Algorithmic News Curation of Google's Top Stories” In ACM Conference on Hypertext and Social Media, HT 2020.
[paper]

Anna Kawakami, Khonzoda Umarova, Eni Mustafaraj. “The Media Coverage of the 2020 US Presidential Election Candidates through the Lens of Google’s Top Stories.” In International AAAI Conference on Web and Social Media, ICWSM 2020.
[paper]

Ada Lerner, Helen He, Anna Kawakami, Silvia Zeamer, Roberto Hoyle. “Privacy and Activism in the Transgender Community.” In ACM Conference on Human Factors in Computing Systems, CHI 2020.
[paper]

Camille Cobb, Milijana Surbatovich, Anna Kawakami, Mahmood Sharif, Lujo Bauer, Limin Jia. “How Risky are Real Users’ IFTTT Applets?” In Sixteenth Symposium on Usable Security and Privacy, SOUPS 2020.
[paper]

Workshops Organized

Logan Stapleton, Devansh Saxena, Anna Kawakami, Tonya Nguyen, Asbjørn Ammitzbøll Flügge, Motahhare Eslami, Naja Holten Møller, Min Kyung Lee, Shion Guha, Kenneth Holstein, Haiyi Zhu. “Who Has an Interest in 'Public Interest Technology'? Critical Questions for Working with Local Governments & Impacted Communities.” Workshop at the ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2022.
[workshop website]

Workshop and Poster Publications (lightly peer-reviewed)

Amanda Coston, Anna Kawakami, Haiyi Zhu, Kenneth Holstein, Hoda Heidari. “A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms.” In ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO 2022.
[preprint]

Anna Kawakami, Luke Guerdan, Yanghuidi Cheng, Anita Sun, Alison Hu, Kate Glazko, Nikos Arechiga, Matthew Lee, Scott Carter, Haiyi Zhu and Kenneth Holstein. “Towards a Learner-Centered Explainable AI.” In Workshop on Human-Centered Explainable AI (HCXAI) at the ACM Conference on Human Factors in Computing Systems, CHI 2022.
[short paper]

Other Writing and Reports

Anna Kawakami, Mihaela Vorvoreanu, Ben Zorn, Nathan Liles, Ece Kamar. “AI Reliability & Safety: Practices and Challenges at Microsoft.” Microsoft-internal technical report, 2021.

Anna Kawakami. “The News We See When Searching: Exploring Users’ Mental Models of Google’s Top Stories.” Senior honors thesis, 2021.

Resources

Resources others have created that I found helpful when applying to grad school. All credits go to their respective owners.

For applying to grad school: Emma Lurie’s Aspirational Grad School Timeline.
I didn’t follow this exactly but it was a good frame of reference. Also many HCI / I-School and CS PhD programs removed the GRE requirement in the past year. It could make sense to make a list of programs you'd definitely be excited to attend before studying for the GRE (maybe you'll find you don't need to take it, like me!).
For navigating undergraduate research: Emma Lurie's Tips for Working with an Undergraduate Research Advisor.
Every lab is different but this guide is spot-on for Eni's Wellesley Cred Lab. I like to read through the common scenarios for validation.
For finding potential advisors: Eric Gilbert’s Syllabus for Eric's PhD students.
I especially like the sections on Ideas and Doing research. He also links an awesome PhD meeting agenda template that I ~aspire~ to follow.

For getting feedback on application materials: Jia-Bin Huang's Twitter thread on student-led pre-application programs for supporting PhD students.
If you're interested in Carnegie Mellon's HCI PhD program (or any other CS-related PhD program at CMU), you could get feedback on your application materials from a wonderful team of PhD student volunteers: Graduate Application Support Program.

... and several more resources, that I will add later.

If you think there’s anything I can help with, please don't hesitate to email me directly!

Celebratory news

Celebratory news about myself made possible through the support of several excellent peers, mentors, and advisors.

Oct 2022 Extremely excited to attend the 2022 INFORMS (The Institute for Operations Research and the Management Sciences) Annual Meeting for the first time! Giving a talk with Luke Guerdan on Understanding Human-AI Decision-making in the Real-world: From Observational Studies to Theoretical Models.
Oct 2022 One of 50 researchers invited to attend the International Symposium of Augmented Intelligence at Work in Hanover, Germany! Thank you to folks at University of Cambridge, the RWTH Aachen University, and the Volkswagen Stiftung for hosting.
May 2022 Ecstatic to join Microsoft Research FATE as a research intern! Really looking forward to examining harms and opportunities from data-driven worker wellbeing technologies with Koustuv Saha, Vera Liao, Alexandra Olteanu, Jina Suh, and Shamsi Iqbal.
May 2022 Presenting research and attending workshops at CHI 2022 in New Orleans, Louisiana - my first in-person conference!
April 2022 First workshop paper acceptances! Excited to be attending CHI 2022's Workshop on Human-Centered Explainable AI (HCXAI) and Workshop on Trust and Reliance in AI-Human Teams (TRAIT).
Jan 2022 Two papers accepted to CHI '22!
Sept 2021 Started my PhD at Carnegie Mellon University!
June 2021 Passed and submitted my senior honors thesis, with Profs. Eni Mustafaraj, Takis Metaxas, Orit Shaer, and Pinar Keskin on my thesis committee!
June 2021 Graduated from Wellesley College!
June 2021 Awarded the Academic Excellence Award, voted by the Wellesley Computer Science Department faculty!
May 2021 Nominated into Sigma Xi Scientific Honor Society by a Wellesley faculty!
April 2021 Awarded the Trustee Scholarship, voted by the Wellesley College Board of Trustees!
Sept 2020 Started my senior honors thesis at Wellesley College with Prof. Eni Mustafaraj as my advisor!
July 2020 Presented our research, in collaboration with Khonzoda Umarova Jennifer Huang, and Prof. Eni Mustafaraj, at HT 2020!
June 2020 Presented our research and dataset, in collaboration with Khonzoda Umarova and Prof. Eni Mustafaraj, at ICWSM 2020!
June 2020 Started my research internship virtually at Microsoft Research! I am excited to be working with Drs. Ece Kamar, Ben Zorn, and Mihaela Vorvoreanu in responsible AI (Aether)!
Dec 2020 Our paper, in collaboration with Prof. Ada Lerner, Silvia Zeamer, Helen He, and Prof. Roberto Hoyle, on transgender security and privacy is accepted to CHI 2020!
Sept 2019 Arrived in the UK to start my year abroad at Oxford University!
June 2020 Started Carnegie Mellon's REU in Software Engineering, with Drs. Lujo Bauer and Limin Jia as my co-advisors!