Keywords

Why is it important to evaluate resources?

Along with accessing, searching, and finding information, evaluating that information is vital.

When you research, you want to find the best information to support your ideas. This requires careful consideration of the information you find.

There are some key questions one should take into account:

  • Is it a scholarly source and a peer review process is evident?

  • Is this a primary, secondary, or tertiary source?

  • Is this source current, or is the information it presents outdated?

  • Is this source based on solid academic background research?

  • Do you detect that the information in this source is biased?

Given the dramatic increase in the amount of scientific information, tools for assessing the quality of information have been created. We will present two kinds of scientific information: that, published in journals and that found in web resources.

7.1 Evaluation of Scientific Series Publications (Journals)

The scientific community around the world considers serial publications as the:

  • Main means of scientific communication;

  • Main documentation base; and

  • Means of assessing scientific research.

Database makers are the publishers of serial publications, but also information intermediaries buying copyrights of the journals, organising them in a system and selling them as new information products. These are called databases (Biagioli 2002).

7.2 Peer Review Process

Peer-review (also known as refereeing) is the process journal editors use to ensure that the articles they publish meet standards of good scholarship. Academic papers (Table 7.1)

Table 7.1 Evaluate sources: evaluating articles (Libguides.uleth.ca. 2019)
Full size table

Journal articles, research papers and such like are examined by a panel of other scholars in the field (the author’s peers). The panel may decide to accept the paper, recommend revision or reject it completely.

Any resource that passes the peer-review process can be considered to have the highest level of academic credibility—although of course you still need to consider other elements such as age, and relevance. Not all journals use peer-review. Some lecturers may specify that only peer reviewed resources are to be used in assignments. Some databases allow you to narrow your search to return only peer-reviewed results. Unfortunately this process differs from database to database. Sometimes this is as easy as simply ticking a box that says “Peer-reviewed”.

This table outlines many of their distinguishing characteristics (Fig. 7.1):

Fig. 7.1
figure 1

Literature review—finding the resources: evaluating sources (Libguides.library.cityu.edu.hk. 2019a, b)

Full size image

7.3 Bibliometric Databases

The most widely used bibliometric databases are: Web of Science (WoS) (1970) and Scopus (2000)—payable (through subscription); Google Scholar—Free.

They are called “bibliometric” because they index all bibliographic references and may be used to calculate a citations index.

WoS includes five databases: Science Citation Index Expanded (SCI-EXPANDED); Social Sciences Citation Index (SSCI); Arts and Humanities Citation Index (A and HCI); Conference Proceedings Citation Index-Science (CPCI-S); and Conference Proceedings Citation Index-Social Science and Humanities (CPCI-SSH). Books and a large variety of journals are not included in the WoS.

SCImago Journal and Country Rank is a portal that presents the visibility of scientific journals; it includes indicators on the scientific development of countries. These indicators are generated by the Scopus database and show the visibility of the journals included in the 1996 database. The portal is developed by SCImago, a research group in Spain (Scimagojr.com 2018).

Publish and Perish (Harzing 2007) is free software that enables you to perform bibliometric analyses in Google Scholar and Microsoft Academic, https://academic.microsoft.com/.

Download from the page: https://harzing.com/resources/publish-or-perish.

The results obtained from the three databases can be very close, although only the publications included in that database are taken into account.

7.4 Clarivate Analitycs—WoS

7.4.1 Impact Factor

Impact factor, or Journal Impact Factor, is a measure of the frequency with which the “average article” published in a given scholarly journal has been cited in a particular year or period and is often used to measure or describe the importance of a particular journal to its field.

Impact factor was originally developed by Eugene Garfield, the founder of Institute of Scientific Information, which is now a part of Clarivate Analytics.

Journal Impact Factor can be found in the Journal Citation Reports or the JCR, as it is commonly known (Libguides.exeter.ac.uk. 2019).

Journal Impact Factors are the measure of the frequency with which the average article in a journal has been cited in a particular year.

For example, the Journal Impact Factor for a journal in 2015 is calculated by taking the number of citations made in 2015 to the articles published in the journal in 2013 and 2014 and then divided by the total number of articles published in that journal in 2013 and 2014.

Impact Factors are:

  • Used to measure the importance or rank of a journal by calculating the times its articles are cited; and

  • A quantitative tool for evaluating the relative importance of a journal.

7.4.2 LibGuides: Journal Impact Factors: Home

Libguides.exeter.ac.uk. (2019). LibGuides: Journal Impact Factors: Home. [online] Available at: https://libguides.exeter.ac.uk/journalimpactfactors [Accessed 5 Feb. 2019].

7.4.3 H-index, HIRSH Index

The H-index is the largest number of articles per author, with at least one of them (Hirsh 2005).

Methods of calculation:

Automatic: the bibliometric platforms Web of Science and Scopus, and “Publish or Perish” software for Google Scholar.

7.5 Conclusion

The consequences of applying this bibliometric indicator may, depending on the context, be beneficial or, on the contrary, destructive.

Eugene Garfield concluded in (2005) on how its application: “Like nuclear power, the impact factor is a mixed blessing. I expected it to be used in a constructive way, but I admit at the same time that in the wrong hands it can be abusive.”

7.6 Web Resources Evaluation

Along with accessing, searching, and finding information, evaluating that information is vital. You can use the CRAAP Test to ask questions about websites, books or articles.

The Search Tips page suggests ways to limit your searches to reliable sources (Fig. 7.2).

Fig. 7.2
figure 2

CRAAP test (Google CRAAP test 2019)

Full size image

Top-level domain tells you from what type of institution the information comes (Libguides.library.cityu.edu.hk 2019a, b):

  • .com—a commercial site (may be trying to sell a product)

  • .edu—an educational institution (usually reliable but may not if it is a personal web page of a member of the institution)

  • .gov—a government department or agent

  • .net—network access provider

  • .org—a non-profit organisation (may or may not be biased).

  • Suggested Teaching

  • Warm-up

  • The message of the wise men

Students will be distributed into four groups. They will receive a secret message to decipher. They will also receive the codes. There will be a competition between the four groups created.

  • Message for Group 1.

figure a

  • Why is it important to evaluate resources?

Along with accessing, searching, and finding information, evaluating that information is vital.

When you research, you want to find the best information to support your ideas. This requires careful consideration of the information you find.

  • Message for Group 2.

figure b

There are some key questions you should take into account:

  • If it is a scholarly source and a peer review process is evident?

  • Is this a primary, secondary, or tertiary source?

  • Is this source current, or is the information outdated?

  • Is this source based on solid academic background research?

  • Did you find biased information in this source?

  • Message for Group 3.

figure c

  • Message for Group 4.

figure d

The consequences of applying this bibliometric indicator may, depending on the context, be beneficial or, on the contrary, destructive. Eugene Garfield concluded in (2005) on its application: “Like nuclear power, the impact factor is a mixed blessing. I expected it to be used in a constructive way, but I admit at the same time that in the wrong hands it can be abusive.”

Message Code:

A

¥

N

Π

Ă

Ɣ

O

Â

Ƴ

P

⎕

B

Ƿ

Q

‡

C

ȣ

R

₤

D

Ɋ

S

₴

E

ɵ

Ș

⎕

F

Ξ

T

⎕

G

Ω

Ț

⎕

H

⎕

U

⎕

I

ξ

V

ﻜ

Î

ζ

W

Ħ

J

Ґ

X

Ʃ

K

Ѧ

Y

Ʊ

L

Ӿ

Z

Э

M

  

After they decipher the messages and present their message to the other groups, students will try through discovery learning to find connections between the messages and make inferences about the next lesson.

  • Review of previous knowledge

Concepts such as: Library catalogues; information sources; classification will be briefly defined and described. The teacher will ask students to make connections between these concepts and the messages of the wise men.

  • Teaching activities

Through a heuristic conversation, the teacher will guide the students’ activity to discover the curricular content of the lesson.

To compare three types of journals (Scholarly Journal, Trade Journal, and Popular Magazine), the teacher will divide students into three groups. Groups will receive worksheets that will contain the comparative table structure and analysis criteria. A group will receive information about the three types of journals in sophisticated and complex language, another group will receive written material in a more accessible manner, from an Information Literacy manual, and the third group will use the Internet as a source of information.

After completion of the tasks, students’ results will be compared and difficult elements will be discussed. A metaphor will be used as a pedagogical approach to understanding the content of the learning unit.

To make the transition to the next sequence of the lesson, in which the criteria for evaluating sources of information will be discussed, the next exercise will be done. Students will be asked to draw a glove as beautifully as possible. Students will not be rushed in drawing. After each student finishes, they are asked to draw as many gloves as they can, as similar they can, on a new sheet of paper. The task will have a time limit of 45 s. The two products of each student will be compared, using criteria such as: aesthetic, number of products (gloves), and clarity. Analogy with sources of information will be made and the advantages and disadvantages of the main types of information will be discussed. The teacher together with the students will develop criteria to evaluate the sources of information and will analyse, according to these criteria, some sources of information. Further, by lecturing, the teacher will teach the content task related to bibliometric databases.

  • Review

Students will compare types of information depending on the type of electronic information source (.com, .edu, .gov, .net, or .org).

  • Applications

Each student will choose a relatively common illness/affection and will seek methods and treatment schemes using different sources of information.

They will compare the results obtained from research and compare the degree of congruence between the information found.

  • Evaluation

figure gfigure g

Correct answers: 1D, 2A, 3B, 4C, 5D, 6D, 7C, 8D, 9C, 10C, 11C, 12C, 13B, 14C, 15D, 16 D