Chapter 19 Can an AI write my PhD?
The emergence of artificial intelligence (AI) platforms brings with it new issues around writing generally, including writing your PhD thesis. These bots are already widely available in early 2023, with both Microsoft (through their company OpenAI) and Google launching ChatGPT and Gemini, respectively. Note that Google originally launched as Bard in early 2023, but then later released as an update known as Gemini. There are already publications that list ChatGPT as an author (O’Connor & ChatGPT, 2023; Mijwil, Aljanabi & ChatGPT, 2023) in acknowledgement that a large amount of the text therein had been generated by the bot. More AI writing tools are going to be coming online in the near future. Microsoft’s plan to integrate the ChatGPT into Bing and the Bing platform directly into their office suite, means that as an author you may unwittingly use AI generated automated suggestions inside your word processor without realising it. Thus, you should be interested in knowing whether such tools can be used to help you in your own writing. If you haven’t already tried (Owens, 2023), once you do you will understand the tantalising possibility that they could be used to replace your needing to write.
In this chapter I set out some ideas about how you might use AI to help you write, as well as some things that you should consider if you do go that route. Bear in mind that this field is moving extremely quickly and it will not be possible to keep up with every change happening in AI all of the time. Instead, I am to discuss more generically about how to use these tools.
This field is moving very fast
Bear in mind when you read this that the original chapter was written in March 2023, with updates nearly every month since then. This current version was compiled on 13 September, 2024. If you think that the content here (or elsewhere in the book) is out of date, or needs additional text, please let me know (see here).
19.1 What is a Large Language Model (LLM)?
Back in February 2023, I decided to ask ChatGPT and Bard (as it was then) to contribute to this chapter by describing themselves. My input text: “I am writing a chapter for a book on academic writing and I want you to describe ChatGPT/Bard in your own words”. Here’s what they said:
ChatGPT (March 2023)
As a language model based on the GPT-3.5 architecture, ChatGPT is a sophisticated artificial intelligence program that has been trained on a massive amount of text data from various sources. It is designed to understand natural language and generate responses that are contextually relevant and coherent.
ChatGPT uses deep learning algorithms to analyze text inputs and predict the most probable next word or phrase, based on patterns and associations it has learned from its training data. This allows it to generate human-like responses to a wide range of questions and prompts, from simple queries to complex conversations.
One of the unique features of ChatGPT is its ability to understand and generate text in multiple languages, making it a valuable tool for communication and collaboration across different cultures and regions. Additionally, it has been used for a wide variety of applications, including language translation, content creation, and customer service.
Overall, ChatGPT represents a significant breakthrough in artificial intelligence research and has the potential to transform the way we communicate and interact with technology.
Bard (March 2023)
Bard is a large language model, also known as a conversational AI or chatbot trained to be informative and comprehensive. I am trained on a massive amount of text data, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions. For example, I can provide summaries of factual topics or create stories.
I am still under development, but I have learned to perform many kinds of tasks, including:
I will try my best to follow your instructions and complete your requests thoughtfully.
I will use my knowledge to answer your questions in a comprehensive and informative way, even if they are open ended, challenging, or strange.
I will generate different creative text formats of text content, like poems, code, scripts, musical pieces, email, letters, etc. I will try my best to fulfill all your requirements.
I am excited to be able to help you with your academic writing, and I hope that I can be a valuable tool for you.
Gemini (May 2024) - note the shift from using first person (Bard) to third person (Gemini). It’s a deliberate change made by Google.
Gemini is actually the new name for what was formerly known as Bard. It’s like a more powerful and upgraded version. Here’s the breakdown of the key differences:
Underlying Tech: Bard primarily relied on Recurrent Neural Networks (RNNs) for processing information. Gemini utilizes a transformer-based design, which is considered more powerful for various tasks. Training Data: Gemini is trained on a much larger and more diverse dataset compared to Bard. This includes text, code, and even images, allowing it to understand and respond to a wider range of prompts and questions. Capabilities: Bard was focused on text-based interactions. Gemini, however, has multimodal capabilities. This means it can not only process text but also understand and generate different formats of data, including images and potentially even video. In short, Gemini is designed to be more versatile and powerful than Bard, with the ability to handle complex tasks and integrate with other Google tools more seamlessly.
To summarise, LLMs are trained on an exceptionally large amount of data and use AI to first understand what you are asking it, and second pull out of their training set what they need to reply to your user input. The key then is that whereas previous chat bots had (relatively) small training sets and limited ability to understand user input, these new LLMs have vast amount of data.
19.2 Large language models have been around for a while
Although at the beginning of 2023 we are seeing a large number of popular articles about ChatGPT, the use of language platforms in writing in general, including writing in the Biological Sciences, has been around for a while. If English is not your first language, then you already know how incredibly useful it is to enter text into a translation app in your own language and get English back.
Some years ago, these translation platforms were rather literal, simply translating each word and providing text back that was barely usable. Today, with the help of AI, the translation that returns is more often than not a reasonable sounding (near) grammatically correct translation of what was entered. Hence, if you have used such translations in your own writing, then you should have already considered the a lot of contents of this chapter, and if you haven’t then you need to read on.
Interestingly, when ChatGPT was asked to provide text in French and Arabic, it failed to produce coherent text (Seghier, 2023), although it is possible that this will change over time as it becomes more sophisticated. It is interesting to note the difference between the claim that ChatGPT made (above) and this assertion from Seghier (2023) a few months ago. Whether or not ChatGPT has improved has yet to be tested.
19.3 Output from ChatGPT cannot be trusted
The first, and perhaps most important issue, about using ChatGPT outputs in your own writing is that you cannot trust it to generate text that is factually correct or unbiased. Indeed, OpenAI who own and administer ChatGPT readily admit that their bot can produce factually incorrect statements and is often biased in its output (Owens, 2023). Similarly, when you start to use Google’s Bard, it forces you to click on an button to acknowledge that text generated may not be correct.
Because these bots use data on the internet to generate the basis of the text that they generates, they are prone to providing the internet consensus about popular topics, and studies have already shown that there is a bias in what is produced (McGee, 2023a,b).
This means that you should be carefully checking any AI text that is generated before you add it to your chapter or manuscript. In particular, you should never trust a AI generated text with citations (see below).
19.3.1 Plagiarism by AI
There is the chance that your AI tool is going to plagiarise text that it finds on the internet, and so you may unwittingly be pasting the words of another author into your own text (see more on plagiarism here). Some authors consider that even if the AI is programmed to avoid plagiarism, using this text in any way should be considered as plagiarism (Salvagno, Taccone & Gerli, 2023). Certainly, if you use AI generated text and then claim that you have written this yourself, then this is clearly plagiarism. In this case, the plagiarism is due to the false claim that you wrote the text.
Perhaps a more fundamental consideration is whether AI will require us to redefine what we mean by plagiarism. Certainly, this is a philosophical consideration that right now (at the start of 2023) you will need to decide for yourself, although as more AI tools come online and are more commonly available, I think that many journal guidelines will be prescriptive on their use (for example see here).
Plagiarism can be thought of as taking someone else’s writing without attribution. Another definition of plagiarism is pretending that someone else’s work is your own. While these two definitions may appear interchangeable at first sight, when considering use of text from an AI, you might only infringe the attribution rule while the AI is programmed to infringe the pretense. In other words, if the AI is not a person, then following the first definition of AI you have not taken someone else’s writing. However, you might consider that as the AI itself was the product of someone else’s work, then they are the person (or group of people) that generated that text, and so the work should be attributed to them. This then brings you to the second definition of plagiarism. But what if that group of people don’t ask for any attribution or acknowledgement? Should you still give them as an author or put them into the acknowledgements? Moreover, could this team really be said to have generated the text when they have created software that searches text written by others on the internet and then collate and rewrite it? The huge number of content creators on the internet can never be acknowledged individually.
Another way of thinking about this might be to consider human interactions in writing text. If, for example, I help some colleagues with the English text of a manuscript and they offer to place my name in the acknowledgements, but I tell them that there is really no need. Should they still acknowledge me even though I have told them not to?
A further example might be the use of different layers in GIS that have been generated by different people. Many journals now insist that there is attribution to these layers in the legend of the figure or in the acknowledgements. We might legitimately ask whether this is necessary when those layers are freely available and have a non-attribution Creative Commons licence (see here)?
Clearly, the definition of plagiarism will need some work in the light of new abilities for AIs to write text. What I see happening is that both universities and publishers are asking students / authors to declare that they have not generated any text using AI. This is a catch-all and will mean that you simply cannot use AI generated text.
To me, the possibility that by using AI generated text verbatim you could be using someone else’s words means that you should avoid this. The same goes whether this is a translation from a large language platform or a new chat bot style AI. Using such platforms as tools is certainly acceptable, while using their text verbatim is probably an example of false attribution.
My own view is that any attempt to be overly restrictive on the use of AI generated text is bound to fail as AI itself becomes increasingly more sophisticated and widespread. Instead, I share the view of Lim (2023) that it is better to embrace and learn how to use AI as a tool. Learn about what it can do, as well as its limitations. There is an entire chapter on plagiarism (in Part II) where AI generated text is covered, and I’d advise you to read this to learn more.
19.3.2 Ethics
In addition to the attribution of authorship, and potential plagiarism issues, you should also be aware of the ethical component of using text from an AI tool.
The kind of ethics points that come up are similar to those that are raised by paper mills (Salvagno, Taccone & Gerli, 2023). This issue is covered comprehensively elsewhere: see Chapter on when you should be an author in Measey (2022).
19.3.3 Failure to learn
Lastly, a great reason not to use AI to help you write you thesis chapters is that you will learn next to nothing this way. Even if you aren’t going to continue in science, writing a PhD is a significant learning experience, and you will benefit in being able to write reports and other types of documents since you will have had the experience of going from having nothing, planning what to write and then executing this. Don’t underestimate the importance of this ability. The reality is that once you have accomplished writing your thesis you will have an important and marketable skill.
Using AI to generate text, even if it gets to the point in the future when it generates something useful, will impact your ability to generate text yourself. I can imagine a future where far fewer people are learning how to write because there is so much AI generated text. In this future, having the ability to write yourself will give you an advantage that should be valued.
19.4 Positive aspects of using AI
It will not come as any surprise that many authors often find it difficult to get started with their writing. The classic image is starting at a blank piece of paper, or these days a flashing cursor on an empty screen. In this book, I provide a lot of suggestions about how to get started with your writing task:
- using a phrasebank to generate text
- formula for writing a logical argument
- using a formula
- making an outline
but can ChatGPT provide another opportunity when you don’t know how to start? I think that it can.
19.4.1 Use AI as a tool
Essentially, if you want to try using AI to help you write, use it as a tool. Don’t try to use it to replace your own writing. That will likely end in a poor product, and could possibly end with factually incorrect or biased text that you won’t want to be associated with your name, and clearly not something that you should want your advisor to read.
Getting started is often a struggle and I regularly suggest that students start by writing something/anything to fill out their outline, even if all of that text gets replaced at a later date. This is a way that you could use AI as a tool. Produce your own outline, because you do still need to plan your chapter, but then try using an AI to flesh this out. Read through the output and get inspired. Hopefully, this will end by helping you to write your own text, replacing everything that the AI has produced.
But do not copy and paste it into your document.
Use the AI to help you get stated but don’t copy it. Just like you might read the text of an article and that helps you get some understanding, you can use AI generated text in the same way. But if you copy and paste it, you’ll start on a slippery slope towards plagiarism and potential disqualification.
Be aware that AI can get it wrong.
Back at the start, ChatGPT was dated to a portion of the internet from September 2021 (Azamfirei, Kudchadkar & Fackler, 2023). It cannot access the internet and so you will not get anything more recent. Whereas Bard is trained from Google search results, Wikipedia, The Common Crawl and other publicly available datasets. This has now changed, but it may well be that the AI you use has not accessed the most recent research. Currently, you are a much better researcher and judge of the written work than any AI. So, don’t believe what it tells you without checking everything!
19.4.1.1 Using ChatGPT to get started: an example
I have tried this approach below and you can judge the outcome for yourself. First, I write the text outline, and second I provide each point to ChatGPT. For the following outline, I provided the outline statement (each bullet point) in quotes and prefixed it with “Write a sentence with citations on:”…”
In this example, the outline below is similar to what I would produce when writing an outline for the first two paragraphs of an introduction to a manuscript on invasive fish:
- Invasive species are a growing global problem
- invasive species cost a huge amount of money to governments
- invasive species are a threat to biodiversity
- invasive species impact people’s quality of life
- Freshwater habitats have been particularly impacted by global change
- important changes have occurred to freshwater systems after the introduction of invasive fish
- invasive fish have been added to both rivers and lakes
- large numbers of introductions of invasive fish have already occurred globally, and are ongoing
The following text was generated by ChatGPT (on 19 March 2023) given the primer text above, and the stipulation that it should produce a sentence with citations. I have simply combined the output text into the paragraphs as might be expected in the introduction to a manuscript:
According to the International Union for Conservation of Nature (IUCN), invasive species are a growing global problem that pose a significant threat to biodiversity and ecosystem health. The National Wildlife Federation (NWF) reports that invasive species cost the United States government an estimated $120 billion annually in damages and control measures (Masterson, 2018). The Convention on Biological Diversity (CBD) states that invasive species are one of the main threats to global biodiversity, causing species extinctions, altering ecosystems, and reducing ecological resilience (CBD, 2021). A study conducted by the University of California, Davis, found that invasive species can have negative impacts on human health, recreation, and property values, affecting people’s quality of life in areas where they are present (Koch et al., 2011).
A global analysis of freshwater ecosystems found that they are among the most vulnerable to the impacts of climate change, pollution, and invasive species, with declines in biodiversity and ecosystem services being reported worldwide (Vörösmarty et al., 2010). Studies have shown that the introduction of invasive fish species can cause significant ecological and economic changes in freshwater systems, including alterations in food webs, declines in native fish populations, and reduced recreational and commercial fishing opportunities (Leprieur et al., 2008; Ricciardi & MacIsaac, 2011). Invasive fish species have been intentionally or unintentionally introduced into both rivers and lakes, causing significant ecological impacts and economic costs globally (Savini et al., 2010; Phelps et al., 2016). A study by Jeschke et al. (2014) found that over 12,000 introductions of invasive fish species have already occurred globally, with new introductions continuing to happen at an increasing rate, highlighting the urgent need for effective prevention and management strategies.
The following text was generated by Bard (on 27 March 2023) using identical input format:
Invasive species are a growing global problem, with the International Union for Conservation of Nature (IUCN) estimating that they cause $1.4 trillion in economic damages annually (IUCN, 2016; WWF, 2017). Invasive species cost governments a huge amount of money to control and eradicate, with the US National Invasive Species Council estimating that the cost in the US is $120 billion annually. (NISC, 2017). Invasive species are a major threat to biodiversity, as they can outcompete native species for resources, introduce new diseases, and alter habitats (IUCN, 2016). Invasive species impact people’s quality of life by causing economic losses, disrupting ecosystems, and harming human health. Economic losses can occur when invasive species damage crops, forests, and other natural resources. Disrupted ecosystems can lead to the decline of native species and the loss of ecosystem services, such as water purification and flood control. Harm to human health can occur when invasive species spread diseases or cause allergic reactions.
Freshwater habitats have been particularly impacted by global change, with climate change, invasive species, and habitat loss all playing a role. Climate change is causing changes in temperature, precipitation, and water flow that are altering freshwater habitats. Invasive species are non-native species that can outcompete native species for resources, introduce new diseases, and alter habitats. Habitat loss is occurring due to human activities such as development, agriculture, and mining. The impacts of these changes are already being felt, with many freshwater species in decline. If we do not take action to address these challenges, the consequences for freshwater ecosystems and the people who rely on them will be severe. “Invasive species can cause major changes to freshwater ecosystems, including the decline of native species, the alteration of food webs, and the spread of diseases. For example, the introduction of the invasive Asian carp to the Great Lakes has led to the decline of native fish populations and the alteration of the food web” (IUCN, 2016). “Invasive fish, such as Asian carp and largemouth bass, have been added to both rivers and lakes, often through illegal releases or escapes from aquaculture facilities.” (IUCN, 2016). “Large numbers of introductions of invasive fish have already occurred globally, and are ongoing. Between 1970 and 2000, an estimated 7,000 species of fish were introduced to new areas (IUCN, 2016).”
For full details of the input and output of each sentence, please see my blog here for ChatGPT and here for Bard.
19.4.1.2 The results of my experiment are that both bots failed
As an exercise, I have tried to find each of these citations in an attempt to determine whether ChatGPT was accurate. You can find the results in full here. So, it’s bad news for anyone who might want to try to rely on it for writing referenced statements for their PhD. Although my initial assessment was that the text looked good most of what was attributed to the citations given was fraudulent, when the paper could be found. The text sounds good, but the citations are bogus, and that’s no use for you, or anybody else. Potentially it’s even worse than bad sounding text as it sounds credible. Similar attempts to fact check ChatGPT generated text have also found it to be completely fabricated (Azamfirei, Kudchadkar & Fackler, 2023).
I have not attempted to verify the output for the Bard output as there were very few citations given and all are reports, not original works, and so would not be readily accepted by anyone. Interestingly, Bard repeated my input text as part of its output, even encasing it in quotes. Bard’s excessive use of quotes when I asked it to write a sentence is also noteworthy. It is unclear whether it had problems generating text, or whether using quotes was considered a viable alternative to generating text. Bard also had problems restricting itself to a single sentence output.
Getting the citations right, and making your text fit what they find is where the skill comes in. Most statements sounded good, and were certainly on message, but the citations were sometimes vague and when they could be pinned down turned out to be about other subjects or did not contain the information claimed.
For information on how to cite, see the chapter in Part II.
19.4.2 Were the bots a complete fail?
Yes, I would say that having checked the references for ChatGPT, most of what was said sounded good, but could not be backed up by the citations given, in the cases where they could be found. For this reason, I am confident in calling this experiment a complete fail, not because there was no truth, but because it was mostly misleading (incorrect citations and unhelpful misdirection to correct citations).
For most of you who are trying to write their PhD thesis, I would suggest that you are completely wasting your time to try this approach with ChatGPT. This doesn’t mean that you can’t use it as an to writing, but that you can’t rely on it to produce anything that is factually correct. Perhaps ChatGPT is getting all these citation uses wrong because it is looking to see how other people use citations? This is not unreasonable, as mis-citing is a real problem. But having this platform reinforcing false citation habits is the last thing you need.
Given that it’s taken me several hours to check two paragraphs, I feel no hesitation in suggesting that you avoid using ChatGPT in this way – and if you do, beware – it probably got it all wrong.
19.4.3 Can I use AI to create a first draft?
If you are all out of ideas and staring at a blank page, then you could use the AI bot to help you generate some first ideas for text. However, I would add a heavy caution not to rely on anything it generates, and to be very cautious with respect to plagiarism. Overall, I’d suggest that you only use it as a last resort, and then only do so with a plan to substantially change and rewrite the text afterwards.
19.4.4 Did the AI pass the plagiarism checker
Using a plagiarism checker (TurnItIn), the output from ChatGPT suggested that: No plagiarism was found. However, text generated by Bard was found to have plagiarism at 21%. Some of this included text that was already in quotes, but there was additional text that had been lifted from non-cited sources on the internet.
NB This was done before TurnItIn launched the trial of their “AI writing detection capability”. It will be interesting to repeat this exercise and test whether or not plagiarism detection is possible.
I suspect that if plagiarism checkers, already in use in many institutions and publishing platforms, release additional tools to detect AI writing generation, then this may set the policy at these same institutions. However, this also raises some concerns.
- What if I write my own original text and it is flagged as AI generated?
- Will plagiarism tools keep up worth AI improvement?
- Can I still use AI as a tool to get started?
- What is the aim of policing AI text generation?
- are we trying to improve science or improve an ability to write English?
19.5 Will others detect ChatGPT if I use it?
Some people claim that they can detect the output from ChatGPT as it lacks the depth and insight that original authors usually have. In other words, the aim of ChatGPT is to produce the words (in a grammatically correct manner), while the aim of an author is to transmit an idea to the reader. As you might expect and hope, AI is not at the point where it can generate the same intent to communicate.
In late 2023, an article in Nature (Conroy, 2023) highlighted research by Guillaume Cabanac that had found the phrase “regnerate response” in more than 30 published papers. It is suggested that the authors have been so sloppy with their use of AI that they have also included the text on the button that occurs with the output. The examples spotted include the most crude errors and likely represent a very small proportion of AI generated text now published. The real question remains why so few people are critically reading manuscripts before they are published. Let us not forget that this is not the first time computer generated text has been published by “reputable publishers”. An article from 2014 talks about how use of the crude text generation software SCIgen had been outed in dozens of published papers (Sample, 2014).
Now (in March 2024) that I’ve had more than a year to evaluate AI tools that write text, I am still of the opinion that AI is not a solution, but can be used as a crude tool. In essays that students have submitted for assessment, it has been easy to spot those who have used AI to generate their text. How? Because AI gets the facts wrong, but still makes it sound good. These mistakes are really unlikely for a human to make. For example, AI will use real species names to summarise the results from an experiment, but uses the wrong name. The AI interpretation of what the experiment is about and what the findings are is often faulty, and can sometimes be wildly inaccurate. I suspect that students who were more diligent in what they handed in, were unlikely to have gained much advantage to using AI. Maybe it helped them get started, or maybe it helped them find some useful phrases, but there is still little help from generating text and using it without considerable reworking.
There will likely be better AI chat bots in the future, and ChatGPT itself is constantly getting feedback from users that should improve its own output. Hence, we can expect that current problems will change.
19.6 Transparency
However you use an AI platform, you should be transparent to your advisor and any colleagues that you publish with about the exact level of use that was involved in generating your text. As a general rule with writing and academia, transparency is the best policy. Some authors have already called for regulations (Salvagno, Taccone & Gerli, 2023). Be aware that journals do update their instructions to authors, so you may need to look for statements on AI text generation. Similarly, familiarise yourself with your institutions requirements for thesis submission, and be careful you do not transgress any recently added rules.
19.6.1 Universities are asking you to declare that text in your thesis is not AI generated
It would be worth checking with your institution to find out what their latest guidelines are. I have already received PhD theses to examine where there are declarations made that no text is AI generated. It is very likely that most institutions will make policies on this soon, if they have not done so already.
19.6.2 COPE have already provided guidelines
The Committee on Publication Ethics (COPE) have already provided some guidelines to their members (mostly publishers) on the use of AI (COPE, 2023) which you can find here. In general, these rules emphasise the importance of oversight and transparency in the use of AI tools in decision making.
19.7 Publishers using AI bots
Publishers are very interested in using the text from AI bots to generate lay summaries of articles that are published on their platforms. Certainly, there is evidence that scientists are already using the bots to create their own summaries (Owens, 2023). This is certainly a possible creative use for AIs, but I would be concerned that without careful curation they may be prone to producing factually incorrect or misleading content. If you plan to use AI software to popularise your own science, then I suggest that you carefully read anything that is created and ensure that the text is correctly attributed when you use it.