The replication crisis has called attention to the fact that empirical studies are often not reproducible. While it is not always possible to reproduce a researcher’s data, it ought to be possible to reproduce a researcher’s analysis of that data. Unfortunately even this is not always possible.
Sometimes researchers simply did not carry out the analysis reported in a paper. Perhaps they wrote up the analysis they intended to carry out, but make mistakes along the way. Sometimes they carried out the reported analysis, though the report leaves out important details and reverse-engineering is necessary to determine exactly what analysis was done. (Some people call this “forensic statistics.”)
We can help replicate statistical analysis. In the simplest case, we verify that the stated analysis was carried out correctly and does indeed lead to the published results. If this is not the case, we may be able to reverse engineer what analysis was actually done and document the missing details and assumptions. If this is also not possible, we can attest that the analysis is incorrect and document what the analysis should have concluded. In short, we may do one of three things:
- Verify that the analysis is correct.
- Fill in the missing gaps in the analysis.
- Explain why the analysis is wrong.
If you would like someone to reproduce a statistical analysis, please let us know.
Trusted consultants to some of the world’s leading companies
