TL;DR: LLMs are just mimicking natural language and conversation. Fact checking and healthy skepticism is not part of their model. For example they can be easily tricked into advocating conspiracy theories, like a fake moon landing. Google Bard is even stating arithmetic falsehoods like 5*6 != 30
I tried to use ChatGPT help speed up my writing, like I tried using “give me a brief history of Queen Dido’s relationship with Caesar” when working on a character inspired by Queen Dido and then it gave me completely false information relating to some other historical figure.
Until it can get more trustworthy in that regard I’m not sure how effective it is for writing assistance let alone writing the entire paper by itself.
This might sound funny but try bing chat instead. Chat gpt uses 3.5 which is prone to hallucination as they call it. Bing uses gpt4 and also shows sources. I found it to be generally better at everything, especially since it has access to the Internet. You can look up stuff past 2021
Edit: bings response. I have no clue if it’s right or not you tell me.
Queen Dido was the founder and first queen of the Phoenician city-state of Carthage. She lived in the 8th century BC. Julius Caesar was a Roman general and statesman who lived in the 1st century BC. There is no historical evidence that suggests that Queen Dido had any relationship with Julius Caesar. However, there are many stories and legends about Queen Dido’s life and her relationship with Aeneas, a Trojan hero who founded Rome. Would you like me to look up more information about Queen Dido or Julius Caesar?
Source: Conversation with Bing, 6/13/2023 (1) The Truth About Princess Diana’s Relationship With Dodi Fayed. https://www.grunge.com/649740/the-truth-about-princess-dianas-relationship-with-dodi-fayed/. (2) Dido - Wikipedia. https://en.wikipedia.org/wiki/Dido. (3) Dido Character Analysis in The Aeneid | SparkNotes. https://www.sparknotes.com/lit/aeneid/character/dido/. (4) Dido - CliffsNotes. https://www.cliffsnotes.com/literature/a/aeneid/character-analysis/dido.