10 reasons (from ChatGPT) why science writers should not use AI to write their papers

I have written a few blogposts about the problems with using AI for writing. This time I asked ChatGPT to list 10 reasons why science writers should not use AI to write their papers. I thought it would be interesting to see how/if AI criticises itself. The following is verbatim:

My prompt: Can you give me 10 reasons why science writers should not use AI to write their papers?

"Yes. If your audience is scientists, it's important to make nuanced arguments. AI is a useful tool for some tasks, but there are strong reasons not to rely on it to write a research paper. Here are ten reasons you could use in a workshop, article or discussion.

1. AI cannot take responsibility for the research
AI did not conduct the experiments, collect the data or make the scientific decisions. The authors are responsible for every claim in the paper, regardless of who or what drafted the text.

2. AI can invent information
Large language models sometimes generate incorrect facts, citations, statistical interpretations or methodological details that sound convincing. These errors can be difficult to detect, particularly outside the author's area of expertise.

3. AI can weaken scientific thinking
Writing is part of the research process. As scientists write, they clarify their arguments, identify gaps in their reasoning and refine their interpretation of the results. Outsourcing the writing can also outsource some of this thinking.

4. AI often produces generic writing
Scientific papers need to communicate the novelty and significance of the work. AI tends to produce safe, conventional language that can obscure what is genuinely original about the research.

5. The writing may no longer reflect the authors' understanding
A paper should represent what the authors know and can defend. If AI introduces concepts, interpretations or terminology the authors do not fully understand, they may struggle to answer reviewers' questions or explain their conclusions.

6. AI can introduce subtle inaccuracies
Even when the overall text appears correct, AI may change the meaning of a sentence, oversimplify complex ideas or overstate the certainty of findings. These changes can alter the scientific message.

7. Confidential information may be exposed
Depending on the AI system and its settings, uploading unpublished data, grant applications, patient information or confidential manuscripts may conflict with institutional policies, journal requirements or privacy regulations.

8. Journal and institutional policies are still evolving

Many publishers and research organisations have specific guidelines about the disclosure and acceptable use of AI. Using AI inappropriately could create ethical or compliance issues.

9. Scientists need to develop their own writing skills
Clear writing is an essential research skill. It influences publication success, grant applications, collaboration and career progression. Relying heavily on AI may slow the development of these skills, particularly for early-career researchers.

10. Good scientific writing requires judgement
Choosing what to include, what to omit, how to organise evidence and how strongly to interpret results all require expert judgement. AI can imitate these decisions but cannot replace the scientific expertise behind them.

A balanced perspective
Rather than saying scientists should never use AI, a more defensible position is:
- Don't use AI to generate the scientific content, interpretation or conclusions of your paper.
- Do consider using AI for low-risk tasks such as brainstorming titles, checking grammar, suggesting alternative wording, summarising your own notes, or identifying awkward sentences—provided you verify every suggestion and comply with your journal's policies.

This distinction encourages scientists to retain ownership of the intellectual work while using AI as an editing or productivity aid rather than a substitute for scientific writing."

While I agree that it's ok to use AI for 'low risk tasks', I do not include summarising in this category. Whatever you use AI for, consider it the first author or a co-author if you use it's exact wording or summary points.

Any suggestions or comments please email admin@writingclearscience.com.au