GOOGLE SCHOLAR AND ISI WOS COMPARED A USER PERSPECTIVE - SUSANNE MIKKI UNIVERSITY OF BERGEN LIBRARY

Google Scholar and
 ISI WoS compared
 A user perspective

      Susanne Mikki
University of Bergen Library

Database policies
•   WoS has a thorough journal selection process based on
         •   quality of citation data (metadata),
         •   publication standards
         •   expert judgements
         •   regular appearances
     – more than 7000 journals (peer reviewed), most English language
     – few books, proceedings …
     – metadata is carefully controlled

•   Google Scholar samples a wider variety of publications. Its
    collection is based on agreements of use with
         • journal publishers
         • database vendors
         • scholarly societies
     – metadata is either offered by its partners or automatic recognized by its
       software

Google Scholar criticisms
• Google Scholar include (P. Jacsó, 2006).
   –   non-scholarly sources
   –   incomplete content
   –   items apparently not matching the search expression
   –   inflated search results,
        • due to not exact matches
        • insufficient grouping
• Criticism is related to
   – inaccurate notification of content
   – inefficient use of metadata.

Google Scholar and the users
•     Students are enabled to retrieve full text peer-reviewed documents,
      relevant for their assignment (Haya et al., 2007)
•     Advanced researchers extensively use Google (Scholar) for
      searching.
       – offers easier access to full text than many library provided portals do
         (Haya et al., 2007; Webb, Gannon-Leary, & Bent, 2007, pp. 18-20).
       – academic researchers use cited reference searching or known author
         searching rather than a keyword approach to cover their information
         need (Booth 2007)
       – Researchers perform simple, aimless, unstructured searches (Haglund
         2008)
       – However, researchers IL behaviour bypasses a thorough research,
         following established ideas (Evans 2008)
•     With online searching, especially googling a new paradigm has
      established (Drewry 2007)

    Teaching is wider than instructing search techniques for locating information.
    It involves a deeper awareness about the services and how they affect
    science.

Why comparing Google Scholar
          with WoS
• WoS is the most recognized proprietary
  database for peer reviewed journal content
• Google Scholar and WoS are similar
  – multidisciplinary
  – include citation data
  – export of references
  – full-text linking

GS – Advanced Scholar Search

Of about 1840

                Book at the top

Publication
original
published
in Science

RE Garfield -
namesake?

Results 1528

Characteristic features for
          Google Scholar:
• Results are over-reported
• Books receive high citation counts (J. Bar-Ilan,
  Levene, & Lin, 2007)
• Citation counts are similar for Google Scholar
  and WoS
• Elderly articles not posted on the web, are not
  likely to be indexed (Neuhaus et al., 2006; Pauly
  & Stergiou, 2005; Walters, 2007)
• Lack of content from certain publishers, here
  Science (Jacsó, 2008; Neuhaus et al., 2006)

Databases compared
•   Overlap
•   Coverage (recall)
•   Ranking
•   Citation metrics

Neuhaus et al. 2006
The Depth and Breadth of Google Scholar: An Empirical Study
Samples of 50 randomly selected titles were drawn from 47 databases.

NP

          Research impact studies
     • Based on the number of
       – publications
       – citations.
     • Typical measures
       – mean citation counts
       – h-index.
         “A scientist has index h if h of his or her Np papers
         have at least h citations each and the other (Np-h)
         papers have h citations each”
       – ranking (Spearman’s footrule)
     • GS and WoS keep track of data useable for
       scientometric evaluation and ranking.

Statistics are based on Google Scholar search results

Items for rank > 997 ignored    h-index = 46 (32)

Impact measures - Findings
The two services return
• similar citation counts
• similar h-indexes
• similar rankings (Spearman’s footrule about 0.8)

Similarities depend on publication practice of
  subject discipline
Results may differ considerable from average
  measures. For single scientist measures may
  even be opposite.

Implications for teaching
  –   Citation data are available
  –   Citation metrics are useful for evaluating sources
  –   Metrics are used for performance measures
  –   They influence researchers’ behaviour and research
      as so

To construct an awareness about the data stored
  in the databases, will contribute to a wider
  understanding of information literacy.

Results of a study conducted
          earlier this year
       (Mikki, forthcoming)
• 29 (26) researchers from earth sciences mainly related
  to UoB
• Searched for names in author field
• Complete result list stored
   – Exported from WoS to EN
   – Cut and pasted from GS to MS Excel
• Data cleansing
   – Automatic ignorance of blank spaces and special characters
   – Cleansing some data manually
• Compare titles using Matlab (max first 50 characters)

Summing up
Comparative findings
•  WoS holds its status as the most trustful database
     –   Comprehensive citation metrics (under-reported)
     –   Advanced search features
     –   Satisfactory export features
     –   Linkage to library holdings

•   GS is a useful supplement with similar services
     –   High degree of overlap (not all disciplines)
     –   Similar citation counts and impact metrics (over-reported)
     –   offers search features (poor)
     –   offers export features (poor)
     –   Linkage to library holdings (few clicks)

Relevance for teaching
•   Build up an awareness about the services, their purpose, strength and weakness
•   Pass on the knowledge to the users
     –   Citation data are available
     –   Citation metrics are useful for evaluating sources
     –   Metrics are used for performance measures
     –   They influence researchers’ behaviour and research as so

Author’s references
• Mikki (forthcoming), Comparing Google
  Scholar and ISI Web of Science for Earth
  Sciences. Submitted and accepted to
  Scientometrics.
• Mikki (forthcoming), Google Scholar
  compared to Web of Science. A literature
  review. Submitted to NORIL

• Susanne.mikki@ub.uib.no