Measuring Our Influence

Autism advocacy organizations have funded millions of dollars in pre- and post-doctoral fellowship programs that train emerging talents in autism science. Yet, there is little data on the short and long-term impact of these investments. How do we measure impact in an objective way – one that does not rely on soliciting information through personal contact, which is often infeasible on a larger scale and is influenced by bias in reporting?
Recently, we presented data from the first multi-year follow-up analysis of pre-doctoral and post-doctoral fellowship funding at INSAR in 2022. The goal was to identify scientific and career impact of ASF’s grant program and to better understand the contributions of the funding in terms of outputs produced by the recipients and their continued engagement in autism research. Autism Science Foundation tracked pre- and post-doctoral fellows and unfunded applicants’ outputs for a 5-year period, and measured the impact through qualitative and quantitative metrics, and bibliometric analyses.

WE ARE HAPPY TO REPORT THAT…

  • The fellows funded by ASF are more likely to stay in autism research compared to those who were not funded.
  • Funded fellows overall contribute substantially to scientific discoveries through high-quality publications and engagement.
  • Funding in autism research did not predict scientific success. In other words, non-funded researchers went on to publish and to be heavily cited as well.
  • Field of study, like genetics and neurobiology, was associated with variations in collaborations and impact of publications.
Poster presented at INSAR 2022

Funded applicants are significantly more likely to remain in ASD after five years, but this difference was not observed at ten years. The only applicants that could be tracked to 10 years were those who were funded in 2010 or 2011.

While the effect was significant when 5 year outcomes were analyzed, the retention for fellows funded in 2014 and 2015 was marginal.  We expected some variation across years.

No significant difference was found in publications, collaborations, or RCRs for funded and unfunded applicants.

Those who were in fields of genetics and neurobiology had more collaborators and a higher RCR ratio than those who were in other fields.  p<.0001 for both collaborations and RCR.  Collaborations were measured by the number of unique co-authors across granting years.  We noticed that those who were on genetics and neurobiology studies had far more collaborators than those in other fields, most likely because of the multi-author nature of large scale genetics projects. 

This is not to say that those who are in neurobiology and genetics are more productive or their work is more valuable. This analysis was done to explain the variability in the collaborators seen in the analysis of funded vs. non funded applicants

We are eager to hear your thoughts on what is a meaningful metric for collaborations, and how else to measure the impact of autism-specific research funding on career trajectories. Share your input by emailing ASF Chief Science Officer Alycia Halladay at ahalladay@autismsciencefoundation.org