Lesson 4: Evaluation

4.1 Evaluation through a Disciplinary Lens

As a graduate student, you probably already have some methods you use to evaluate the sources that you find. For example, you might consider whether the source is relevant to your research. You might explore the authority of the source—the author’s credentials, affiliations, and expertise. Or you might consider how current the source is and whether that is important for your research. Finally, you might examine the purpose for the source—whether it was created to inform, persuade, or entertain, and what biases exist. Within your specific discipline, however, there may be additional criteria you need to consider when evaluating sources. For example, if your discipline conducts original research in labs, evaluating the particular methodology used in an experiment may be important. If you’re conducting historical research, you may have criteria to consider when evaluating the authenticity or provenance of a document or artifact. In health-related fields, it may be important to consider the authority that a professional organization holds when evaluating best practice documentation. Regardless of your field, take the time to consider what matters in your discipline when it comes to evaluating sources.

For a quick guide to evaluating sources, check out our Evaluating Sources Tips.

Criteria

The specific criteria you use to evaluate sources will vary depending on your discipline and your purpose for using the source, but as a starting place, here are some questions you might consider:

  • Is the date of publication important? Does a source’s value decrease if it is 5 years old? 10 years old? 50? What if the source is considered an important work in your field? How will you know it is important?
  • Is the method in which research was conducted an important factor in your field? If so, what factors distinguish a sound methodology from a poor methodology? Are certain methods valued more than others?
  • Is research in your field funded by grants or outside funding? If so, who is providing this funding, and could the funders introduce bias into the research or results?
  • How is prestige or impact determined in your field? Does the number of times a source is cited determine its impact on the field? Are certain journals or publishers more highly valued than others? Are certain institutions or geographic areas more highly valued than others?
  • Within your field, whose voices are dominant? Have any groups been historically, or are currently, excluded from your field (e.g. racial and ethnic minorities, women, LGBTQ community)?

You might also consider talking with instructors or peers in your field to see what matters to them when evaluating sources. While each field tends to have its own approach, you will also find that perspectives differ from individual to individual. Take the time to develop your own criteria, and be open as these will likely change over time.


4.2 Information Has Value

Authority is Constructed and Contextual

Just as it’s important to evaluate the sources you use for research, you should also think about the ways in which the value of a given source is constructed. That is to say, not only should you consider which journal published an article but also the reasons that one journal is valued over another. Authority can be constructed in a variety of ways, including subject expertise (e.g., scholarship), societal position (e.g., public office or title), and special experience (e.g., participating in a historic event). As one example of how this can play out, an oral history from someone who participated in American Indian Movement actions may have just as much, if not more, authority on the topic than a respected historian who is an expert on the same movement; the primacy of lived experience may provide more authority than the educational qualifications others may possess.

Information Has Value

Similar to the way authority is constructed, the value of a particular source of information is shaped by a variety of contexts, from actual financial value, to intellectual property, to professional prestige. As a member of a scholarly community, you are undoubtedly aware of the value that various sources of information possess within your field. Copyright, intellectual property and patent laws, and plagiarism guidelines all serve to preserve this value and ensure that the person or persons who developed an idea or work are given due credit.

As one example of the way this value plays out, many scholarly journal databases require a paid subscription to access their articles. Journals with high impact factors that have established prestige and value within their field can sometimes charge significant fees for access. That is to say, the journal’s professional value is translated into financial value for the publisher. While someone who is affiliated with a large research university may be able to access journals through their institution’s library, which pays the subscription fees on behalf of faculty, students, and staff, someone who works at a small institution, an institution outside the U.S., or someone who is not affiliated with any institution, may not have access to this literature, thus preventing them from engaging fully in the scholarly conversation. In this situation, the financial value of the information has created a barrier for access.

For this reason, among others, open access journals have become an important aspect of scholarly communication. Open access journals typically do not charge readers for access, and many open access journals in a variety of fields—especially STEM fields—have established professional and academic value among researchers.

As you navigate the wealth of resources within your field, you will need to remain aware of both the established authorities—those who have shaped schools of thought and shifted discipline paradigms—as well as those voices that may have been less valued or even excluded given the way that authority and value are constructed. Maintaining a healthy skepticism and checking your own assumptions and biases will ensure you’re considering the full breadth of scholarship available to you as a researcher.

Financial standing can help an author achieve an education or to gain a life experience, but their wealth is not usually taken into account when determining if they have the authority to write about a topic.


4.3 Lesson Review

Test your knowledge and comprehension of evaluation strategies by answering the three questions below. Complete the Lesson 4 Quiz before moving on to Lesson 5: Institutional Library Services.

Question 1

Which of the following are ways an author’s authority is typically established? Select all that apply.

Correct! While financial standing can help an author achieve an education or to gain a life experience, their wealth is not usually taken into account when determining if they have the authority to write about a topic.
Incorrect. The answers are A, B, and C. Financial standing can help an author achieve an education or to gain a life experience, but their wealth is not usually taken into account when determining if they have the authority to write about a topic.

Question 2

Which of the following statements is true?

Correct! Open access journals have established professional and academic value among researchers for eliminating barriers to access that traditional journals create.
Incorrect. The answer is D) Many open access journals have established professional and academic value among researchers.

Open access journals have established professional and academic value among researchers for eliminating barriers to access that traditional journals create.

Question 3

True or False: Anyone who wants to can access important literature in their discipline.

Correct! There are many barriers that can prevent a person from accessing important literature in their discipline, such as the financial cost of an article or journal subscription.
Incorrect. The answer is False. There are many barriers that can prevent a person from accessing important literature in their discipline, such as the financial cost of an article or journal subscription.