Recent reports have raised concerns about researchers embedding hidden instructions in preprints to manipulate AI-driven evaluations. A recent Nikkei article revealed that scientists at 14 universities across eight countries have inserted covert commands, sometimes in invisible text, to enhance their papers’ perceived impact or appeal. These instructions target AI tools increasingly used in academic publishing to summarize, rank, or recommend research. Such tactics exploit the growing reliance on AI for assessing scholarly work, particularly in preprint repositories like arXiv, where peer review is absent.
This practice, while not yet widespread, poses ethical questions for the academic community. Preprints are valued for rapid dissemination but lack the rigorous scrutiny of peer-reviewed journals. Hidden instructions could mislead AI systems, skewing metrics like citation predictions or visibility algorithms, which many researchers rely on for funding or career advancement. The Nikkei report suggests this behavior may erode trust in preprints, as readers and AI tools may struggle to discern genuine content from manipulated text.
The issue reflects broader challenges in integrating AI into scientific workflows. AI tools, such as those used for manuscript analysis or summarization, are powerful but vulnerable to exploitation. Researchers must navigate the balance between leveraging technology and maintaining integrity. Journals and preprint platforms may need to adopt detection methods, such as text analysis algorithms, to identify hidden commands and ensure transparency.
The academic community can address this challenge through collaboration and innovation. Developing robust AI tools with safeguards against manipulation, coupled with clear ethical guidelines, will be crucial. Platforms like arXiv could implement stricter submission protocols, while researchers should prioritize transparency in their use of AI. By fostering open dialogue and technological advancements, the scientific community can preserve trust and integrity in an AI-driven era.
Click here for the Japanese version.