HITS Colloquium Victoria Stodden: Verifying Correctness in AI-enabled Scientific Research – a New Frontier

By Victoria Stodden, Daniel J. Epstein Department of Industrial & Systems Engineering, University of Southern California, USA

In the last 10 years colossal cloud infrastructure investments behind the rise of near-ubiquitous global mobile technologies have trickled down to scientific research through innovative infrastructure including cloud compute and storage, I/O tools, data analysis and modeling frameworks, which in turn have generated broad and expanding communities of users and supporters. Arguably, the recent success of Large Language Models were catalyzed by the resulting technological innovations of 1) open and accessible massive data, and 2) re-executable discovery pipelines for model estimation and prediction. These changes are deeply disruptive to the research community since they open new paths to knowledge creation that were previously inaccessible and largely culturally unknown.
The scientific community is faced with the challenge of responding to changes in research modalities due to these technological innovations. Research is now conducted as an “Olympics” of benchmarked competitions between machine learning models leveraged by the opaque results of Large Language Models, access to massive data, and redeployment of complex scientific discovery workflows. In this Colloquium talk I provide a roadmap of challenges and responses by various stakeholders in the research community to ensure that scientific results remain reliable and reproducible, and secure within a position of trust in the broader society.

A Short Biography

VICTORIA STODDEN is Associate Professor in the Department of Industrial and Systems Engineering at the University of Southern California.

She received a Ph.D. in Statistics from Stanford University and a Law Degree from Stanford Law School. She graduated magna cum laude with her Bachelor’s in Economics from the University of Ottawa and holds a master’s degree in Economics from the University of British Columbia. She held the Kauffman Innovation fellowship at Yale Law School and was a Berkman Klein fellow at Harvard Law School. She was a postdoctoral researcher at MIT and has held faculty positions at the University of California Berkeley, Columbia University, and a tenured position at the University of Illinois at Urbana Champaign.

Stodden is an internationally recognized leader in improving the reliability of scientific results in the face of increasingly sophisticated computational approaches to research: understanding when and how inferences from data are valid and reproducible, what it means to have replicated a result, the effect of big data and computation on scientific inference, the design and implementation of scientific validation systems, standards of openness and transparency for data and code sharing, and resolving legal and policy barriers to disseminating reproducible research.

She has published more than 50 papers in scientific journals and conference proceedings, and has co-edited two professional books, published in 2014, Privacy, Big Data, and the Public Good: Frameworks for Engagement, published by Cambridge University Press and Implementing Reproducible Research, published by Taylor & Francis.

In 2009 she won the Access to Knowledge Kaltura prize for her publication on legal issues in reproducible research and scientific innovation. She has served on the National Academies of Science, Engineering, and Medicine committees: “Reproducibility and Replication in Science” and “Fostering Research Integrity.” She co-chaired the National Science Foundation Advisory Committee for Cyberinfrastructure and was a member of the National Science Foundation Directorate for Computer and Information Science and Engineering (CISE) Advisory Committee. She has been quoted in The Economist (2013) and interviewed by publications such as Nature (2016) on reproducibility in science.

She also testified on scientific reproducibility before the Congressional House Committee on Science, Space and Technology for the March 5, 2013 hearing on Scientific Integrity & Transparency.

She is PI on NSF awards #1941443 EAGER: Reproducibility and Cyberinfrastructure for Computational and Data-Enabled Science, and #1839010: EAGER: Preserve/Destroy Decisions for Simulation Data in Computational Physics and Beyond; and she is co-PI on the NSF grant #1541450: CC*DNI DIBBS: Merging Science and Cyberinfrastructure Pathways: The Whole Tale.

Her ORCID ID is 0000-0003-2015-7825.

REGISTRATION: