Median Watch

Eyes on statistics

Experimenting with peer review

RORI’s interesting report on the future of peer review noted a lack of cross-over of ideas between peer review in funding and peer review in journals. Having worked on both, I have ideas. Lotteries for journals Conditional lotteries are becoming popular with funders. After we were publicly made fun of for suggesting them with the Australian NHMRC in 2015, they are now all over the world. They are popular because they reduce biases, save time, and appear to increase diversity.

Posted by Adrian Barnett Sunday, July 12, 2026

Research integrity is locked into an arms race with agentic AI slop

Reproduced from the LSE impact blog.

Science prides itself on being self-correcting. While scientific fakery has always been a problem, cases of fraud have been isolated, and a combination of scepticism and scrutiny has up to now generally worked to highlight published papers that are unreliable. The 30 minute paper However, the world of research and publishing is changing. The introduction of agentic artificial intelligence (AI) allows an automated assembly line of research tasks without any human checkpoints.

Posted by Adrian Barnett, Matt Spick Sunday, March 22, 2026

A publication’s “what” should count more than it’s “where”: why we should waive journal titles

Reproduced from the DORA blog. The winter months can get cold in Belfast, the largest city in Northern Ireland where the Titanic was designed, built and launched in the early 1900s. Not seriously cold of course, never sub-zero depths of cold that were the undoing of the Titanic on its maiden voyage, but a ‘nippy’ cold… a cold that comes with biting winds and rain that means only the bravest of souls venture outside in the winter months without the insurance policy of a good coat.

Posted by Adrian Barnett Wednesday, November 13, 2024

Casual inference and pubic health – What a rise in common spelling errors says about the state of research culture

Reproduced from the LSE impact blog. Here’s their introduction: Based on an analysis of over 32 million abstracts published over the last fifty years, Adrian Barnett and Nicole White find a marked rise in common spelling errors. Evidence they suggest of a culture of quantity over quality in academic writing. Pressure Many academics feel pressure to publish lots of papers every year because demonstrating their productivity is key for securing jobs and promotions.

Posted by Adrian Barnett Wednesday, October 30, 2024

Randomisation can resolve the uncertainty at the heart of peer review

Reproduced from the LSE impact blog. Here’s their introduction: Peer review decisions are definitive, and depending on the style of peer review practiced at a journal, reviewers can usually make one of three recommendations: accept, reject, revise and resubmit. Discussing a new study into the levels of certainty reviewers have making these choices, Adrian Barnett suggests how embracing this doubt could improve peer review processes. Easy peer review Occasionally I find peer review easy.

Posted by Adrian Barnett Saturday, October 26, 2024