The Center for Data Innovation spoke with Brian Nosek, co-founder and executive director of the Center for Open Science (COS), a nonprofit devoted to open science based in Charlottesville, Virginia. Nosek discussed how COS is working to evaluate the credibility of scientific research and the challenges and opportunities AI poses to open science.
Joshua New: Can you describe what open science is? How “open” would you say science is today?
Brian Nosek: Open science is showing and sharing the process, content, and outcomes of research. The outcomes of research are the findings and reports of research investigations. The content of research includes the materials used to generate the data, the data produced from the research, and the code to analyze the data. The process of research includes preregistration of studies and analysis plans, display of the analysis pipeline, and the discussion and critique of the process, content, and findings such as peer review.
It is hard to put a singular value on how open science is today, but I’ll do it anyway. I would guess that science is about four percent open, with a confidence interval of plus or minus three percent, based mostly on intuition. The outcomes of research, particularly papers, are the most open partly because of the emergence of open access and partly because of civil disobedience activities such as sci-hub making copyrighted papers publicly accessible. The process of research is the least open. Researchers do not yet make their research process discoverable because it isn’t yet normative, and incentives and infrastructure for doing so has emerged only in the last five years.
New: Can you describe the Open Science Framework (OSF) COS created? How does this differ from other research sharing platforms?
Nosek: OSF is a collaborative management platform that enables researchers to archive and open their research process, content, and outcomes. Researchers can use OSF privately for managing collaborations, preregistering projects, and storing data and materials during the lifecycle of the project. Also, researchers can make public any part of that research that they wish at any time during the project or after it is complete to archive the work publicly. OSF offers different ways to search and discover research such as by sharing preprints to which data and materials can be added, or registering study designs and then later adding data, preprints, or postprints to share what was found. Finally, different from other research sharing platforms, OSF connects to services that researchers use rather than trying to replace them. For example, if a research team has an analyst that uses GitHub, a data manager that uses Dropbox, and a writer that uses Zotero, they can link all of these services to a single OSF project so that all of the files are discoverable and managed in a single place. Those individual researchers can continue to use their preferred interfaces, and everyone has a single place that they can reference to find everything related to the project.
New: COS was recently selected to help DARPA build a “bullshit detector” for science. What does this mean, and why is it necessary?
Nosek: The purpose of this new program, called SCORE, is to investigate whether we can create automated tools to assess the credibility of research findings. In principle, it is well-understood that “published” and “true” are not synonyms, but there exists very little basis for quickly assessing the credibility of individual findings or claims. Even so, we have accumulated evidence that researchers do have insight on which claims are more credible than others. With prediction markets and surveys on social-behavioral science studies, we have found that researchers can anticipate which findings will successfully replicate or not with quite good accuracy. If machines can learn to do the same thing, then we can create early indicators of credibility that could be used to guide resources and attention to those findings that are important but do not yet have sufficient evidence to assign high credibility. This would be a boon to improving efficiency of discovery and confidence in transitioning findings to application, such as developing new therapeutics, advancing new technologies, or adopting new public policies.
New: Some are concerned that scientific discoveries made with the help of machine learning are untrustworthy because of the inscrutability of “black box” systems. Can you describe this concern?
Nosek: It is a very reasonable concern. Many machine learning approaches provide little insight on what the machine has learned to become accurate in its classification or prediction. And, sometimes, researchers think that the machine has learned something useful when actually the training and test tools were confounded with features that are not of interest or applicable to decision-making more generally. For example, if a machine were trained to detect criminality of faces, but the researchers used mugshots and school photos to train and test the algorithm, then it might actually be learning to detect variables that are not about criminality of faces but are confounded with that, such as the likelihood of smiling in a mugshot versus a school picture. Ultimately, in machine learning research, just like every other area of research, there is an important difference between prediction and explanation. We may be able to predict outcomes accurately without understanding why. Unpacking the black box is a big part of research that ultimately provides useful explanation.
New: Alternatively, how can data technologies like AI help promote more open and trustworthy science?
Nosek: At least in these early days, I think the biggest opportunity for AI to help promote better science is as providing rapid, scalable heuristics to help direct attention and resources. AI may not provide precise answers and insight for awhile. But, for example, if SCORE is successful, then the credibility scores generated by machines will be useful early indicators that human can use to decide where to invest their energy to assess, investigate, and improve confidence and precision of research findings and the explanations we provide for them.