Introduction
Good research demands a careful approach to source selection and usage. Different types of sources serve distinct purposes and possess varying levels of reliability and authority. In scholarly and even business discourse, primary, secondary, and peer-reviewed sources are three fundamental categories that play crucial roles in constructing and substantiating arguments. This blog article will explain the differences between these sources, highlighting their defining characteristics, uses, and importance in all kinds of research. I will conclude my article by providing a clear-eyed evaluation of chatbots and how they fit in.
Primary Sources
Primary sources are foundational documents or materials that provide direct evidence or firsthand information about a particular subject, event, or phenomenon. They are original records that have not been altered or interpreted by others. Primary sources include:
Historical Documents
Letters
Diaries
Photographs
Video
Eyewitness Accounts
Interviews
Research Data
Artifacts
These sources offer a direct window into the past or the specific subject of inquiry. They allow researchers to engage with the raw material of their field.
The significance of primary sources lies in their ability to offer insight into the past or the subject we are researching. Researchers can use primary sources to develop a deeper understanding of historical contexts. Primary sources give us access to the thoughts and perspectives of individuals from the past. They allow us to scrutinize original data. Primary sources are critical for history, anthropology, sociology, justice, and the sciences, where researcher depend on empirical data about their topic.
However, bias, subjectivity, and the constraints of the time during which the author created them often skew the information provided by primary sources. Researchers must assess primary sources' authenticity, credibility, and context to ensure reliability. As AI and authentic-looking fakes become increasingly convincing, researchers must carefully verify the authenticity of source digital materials.
Secondary Sources
Secondary sources are different than primary sources. Secondary sources explain or talk about the information from primary or other sources. Examples of secondary sources include:
Textbooks
Blog Posts
Magazine Articles
Video Documentaries
Encyclopedias and Wikis
Study Aids (e.g., Course Hero)
These sources help us understand complex topics by giving us insights and perspectives we wouldn't have otherwise. Secondary sources often make it easier to research a topic because they summarize lots of information for you. People use them in literature reviews to find out what others have said about a topic before writing their article or post.
As researchers, we need to be careful when using secondary sources. We depend on the author's knowledge when using a secondary source. My mother used to say that you must always cut open at least one sausage to see what's in it before serving it to your family. So it is with secondary sources! The author may add their own opinions or favor one idea.
For this reason, it is always important to check who wrote the article and critically examine whether it is good source by comparing it with other sources. Do they provide links or citations to other reliable sources? You need to understand how the author made 'the sausage.' When searching for trustworthy secondary sources, in the back of our minds, we as researchers should take into account the following list of considerations from a library website (Pilgrim Library: Finding Primary Sources: Evaluating Primary & Secondary Sources, 2023):
The author and what they do for a living (Are they professional journalists, researchers, bloggers, or other experts?)
Are they an expert in the field, and what other research activities can you find that show them as a credible source?
Did a scholarly publisher publish the book or journal?
How long has the publisher been active in the field, and what is its reputation?
What is the text's purpose or motivation?
Does the writer, publication, or website you found it on have a clear bias?
Does the book or article have a comprehensive bibliography?
What primary sources is the author referring to?
What secondary sources is the author using?
Does the text have verifiable citations so you can research further?
Peer-Reviewed Sources
Peer-reviewed sources are particular kinds of secondary sources. Experts look at the source's quality, truth, and importance before publication in a journal. They do this anonymously so they can be fair. These types of sources are usually excellent because they have high standards. Peer-reviewed sources include articles in journals, books about conferences, and academic books. They have a particular structure, like citing primary sources, discussing other literature about the topic, describing how the researchers conducted their research (usually in a methodology section), and are part of ongoing conversations in that field. They provide citations that researchers can use as a road map in their research of topics in the field.
Peer-reviewed sources provide the gold standard in academic research. While not perfect, peer-reviewed articles provide the highest probability that what we are reading is reliable and vetted knowledge and that research is grounded in the best available evidence. Scholars and researchers rely on peer-reviewed sources to stay current with field developments, support their arguments, and build upon existing research.
How should researchers regard the integrity of bot-generated essays and research?
I regard AI or Chat Bot-generated text as a turbo-charged, customizable, secondary source. As such, most AI applications suffer from the "sausage" problem plaguing secondary sources I articulated above. Where did the bot find its information? Is the bot developing its information from reliable, unbiased sources? You can only know the answer to these questions by researching yourself. (Perplexity.ai, which I came upon only recently, is a notable exception that lists its sources.)
AI-generated text comes from primary and secondary information sources and naturally includes interpretations or analyses. Chatbots generate text using natural language processing algorithms to interpret and analyze the source material and produce a new piece of writing.
How Chatbots Evaluate Information Sources
Like their human counterparts, chatbots also use source verification techniques to evaluate the authority of an article. This validation technique involves looking at who wrote the article and verifying whether they are experts in the subject matter. A chatbot may look at factors such as the author's credentials, previous publications, and any awards they have received related to their work. The chatbot may also check if the author has been quoted in other reputable sources or if they have links to reputable organizations or institutions (LibGuides: AI Chatbots (ChatGPT): Teaching And Learning: Introduction, 2023).
Citation analysis is another way chatbots evaluate the authority of an article. Citation analysis involves looking at how often a particular article has been cited by other authors and determining whether it is considered authoritative within its field. Chatbots can also examine which journals have published articles citing this particular article and their relative impact (Lee et al., 2023).
Chatbots likewise borrow a method from search ranking to determine which websites link back to a particular article and assess its credibility based on domain age, page rank, and number of incoming links from other sites. Chatbots may also look for links from trusted websites, such as government agencies or educational institutions, to assess an article's reputation and authority within its field of study (Kooli, 2023).
Finally, to parse the reliability of sources on an emotional level, programmers are teaching bots to analyze text for positive or negative sentiment toward a particular topic or idea to determine whether an author has expressed support for it. By considering sentiment when evaluating articles, chatbots can better identify those written by experts who are knowledgeable about their topics and more likely to provide accurate information (How to Develop Chatbots With Real-Time Sentiment Analysis, 2022).
What Are Chatbots Good At?
These techniques may ensure that output from bots provides valuable insights and a good summary of what is "known" about a topic. Chatbots are particularly useful for researchers seeking a quick understanding of a broad topic. They also provide an excellent road map for further investigation. For example, asking a chatbot to tell you five ways businesses apply AI to Marketing will necessarily lead an investigator to how AI affects keyword search and SEO. From there, the researcher can learn about specific examples and case studies -- all in the space of a few moments.
Chatbots And Learning
The pedagogical opportunities for a technology like this are breathtaking (Hensell & Cameron, 2023). Imagine a study buddy for a college student who can provide online tutoring about anything in their school curriculum. The chatbot can provide this information based on the student's needs and as the student requires, whenever and wherever the student needs it. Do you have a math problem you can't solve? You will find it hard to stump the chatbot. You can give Chat GPT a poorly written paragraph and tell it, "Please correct the grammar in the following paragraph," then cut and paste the paragraph you need to have edited. In my tests, it rewrote even the grammar-challenged prose into perfect, grammatically correct English. Finally, it can even help you learn how to code. An engineering master's student at a top university told me his professor allows students to use it to develop their coding assignments.
The technology can also help with college or graduate school planning by providing information about colleges and universities nationwide. It can help students compare schools based on academic ranking, student satisfaction, cost, location, size, etc. The chatbot can also answer questions and advise on applying for financial aid or scholarships. This buddy is free because the bot wants to learn from the students about their needs.
Talk about leveling the playing field!
Chatbot 'Imperfections'
Notwithstanding the above, it is essential to remember that chatbots are only sometimes accurate representations of the source material and reality (Fowler & Merrill, 2023). We are leaving it to a computer algorithm (and Open AI or Microsoft Bing Chat or Google Bard programmers) to determine whether the source it is accessing is reliable.
If you need to be 100 percent correct, you need to check. "Garbage in, garbage out" is the easiest way I can summarize the problem of using chatbots exclusively for researching or writing posts and essays (Alba, 2022). An entire cottage industry has been tracking and cataloging Chat Gpt and other bots' failings (Giuven, n.d.) Researchers, mathematicians, logicians, and political advocates of all stripes delight in tripping up Chatgpt and New Bing and gleefully documenting failures. Without verifying a few primary and secondary sources in your search to double-check what the bot found and wrote, you are flying blind. Generally, the broader and more codified the topic you are researching, the more likely the chatbot will return a good answer. Project Management's PIMBOK guide makes Bots exceptionally knowledgeable and accurate about Project Management best practices, and the FASB is excellent for guiding answers to accounting questions. If filet mignon goes in the meat grinder, you get good sausage.
Another thing chatbots are not good at is writing essays that reflect you as a person. Depending on the institution, colleges and universities' admission departments take different positions on using Chatbots to generate essays. Some, like the University of Michigan Law School, forbid it altogether (Singer, 2023), while others allow it but suggest that students adapt the output. At Georgia Tech, for example, according to a recent New York Times article, the College Admissions Department has guidelines "encouraging high school applicants to use AI tools as collaborators to "brainstorm, refine and edit" their ideas. At the same time, the site warned applicants that they should "not copy and paste content you did not create directly into your application (Singer, 2023)."
A Way Forward
My advice? Go forward, but verify. You will be mostly pleased -- if not amazed by what you get. We have yet to reach the place where chatbots are researchers' one-stop shop, but chatbots are getting better every day. Researchers should spend time with Google Search or Bing to cross—reference primary and secondary source materials with bot-generated essays and research papers to identify discrepancies between them. Researchers can also consult people knowledgeable in the field to see if what the bot found and summarized reflects reality. Finally, researchers can and should cite bots as a source (it's increasingly allowed, even in academic circles), but rewrite their "botly" prose into their own words and check and cite other sources too. Finally, educators and business leaders would do well to stop pearl-clutching about bot technology's misuse and minor failings and spend more time exploring how to use it to help their students learn and their employees become more productive.
Conclusion
Understanding primary, secondary, and peer-reviewed sources is essential to good research. Primary sources are original information. Secondary sources can give more details about the original information. Academic groups check peer-reviewed sources carefully to ensure they are reliable and trustworthy, so peer-reviewed papers tend to be the most reliable of all. When using AI-generated research, verifying the accuracy of the information generated by cross-checking and referencing primary, secondary, and human sources is critical. Knowing when and how to use these sources makes your research more trustworthy. Some day, AI will become so powerful and intelligent that its replies will rival the accuracy and transparency of peer-reviewed articles. In the meantime, researchers should cross reference and investigate as many types of sources as possible to verify what their sources and the bots tell them. Even with their imperfections, Chatbots offer business and academic researchers a way to increase productivity.
References
Alba, D. (2022, December 8). ChatGPT, Open AI's Chatbot, Is Spitting Out Biased, Sexist Results. Bloomberg.com. https://www.bloomberg.com/news/newsletters/2022-12-08/chatgpt-open-ai-s-chatbot-is-spitting-out-biased-sexist-results
Fowler, G. A., & Merrill, J. B. (2023, April 14). The AI bot has picked an answer for you. Here's how often it's bad. Washington Post. https://www.washingtonpost.com/technology/2023/04/13/microsoft-bing-ai-chatbot-error/
Giuven. (n.d.). GitHub - giuven95/chatgpt-failures: Failure archive for ChatGPT and similar models. GitHub. https://github.com/giuven95/chatgpt-failures
Hensell, H., & Cameron, C. (2023, September). AI Chat GPT Should We Reject It Like We Did Wikipedia? Remington College Refresher Summit 2023, Online Summit. https://remingtoncollege.brightspace.com/d2l/le/lessons/21191/topics/317396
How to Develop Chatbots With Real-Time Sentiment Analysis. (2022, October 27). Vonage.com. Retrieved October 2, 2023, from https://www.vonage.com/resources/articles/develop-chatbots-real-time-sentiment-analysis/
Kooli, C. (2023). Chatbots in Education and Research: A Critical Examination of ethical implications and solutions. Sustainability, 15(7), 5614. https://doi.org/10.3390/su15075614
Lee, S. E., Ju, N., & Lee, K. (2023). Service chatbot: Co-citation and big data analysis toward a review and research agenda. Technological Forecasting and Social Change, 194, 122722. https://doi.org/10.1016/j.techfore.2023.122722
LibGuides: AI Chatbots (ChatGPT): Teaching & Learning: Introduction. (2023, March 15). https://guides.westoahu.hawaii.edu/chatgpt
Pilgrim Library: Finding Primary Sources: Evaluating primary & secondary sources. (2023, July). https://library.defiance.edu/primarysources/evaluating
OpenAI. (2023). ChatGPT (September 25 Version) [Large language model]. https://chat.openai.com
Singer, N. (2023, September 1). Ban or embrace? Colleges wrestle with A.I.-Generated admissions essays. The New York Times. https://www.nytimes.com/2023/09/01/business/college-admissions-essay-ai-chatbots.html#:~:text=chatbots%20to%20generate%20ideas%20or,might%20have%20a%20democratizing%20effect.