Login

Publications  •  Project Statistics

Glossary  •  Schools  •  Disciplines
People Search: 
   
Title/Abstract Search: 

Dissertation Information for John James Regazzi III

NAME:
- John James Regazzi III

DEGREE:
- Ph.D.

DISCIPLINE:
- Library and Information Science

SCHOOL:
- Rutgers University (USA) (1982)

ADVISORS:
- Ernest Deprospo

COMMITTEE MEMBERS:
- None

MPACT Status: Incomplete - Inspected

Title: A STUDY OF CRITICAL FACTORS AFFECTING THE DEVELOPMENT OF PERFORMANCE MEASURES IN EVALUATING BIBLIOGRAPHIC INFORMATION RETRIEVAL SYSTEMS

Abstract: This study is an investigation into critical factors which would constitute reliable performance measures for bibliographic information retrieval systems. Thirty-two (32) judges, grouped by type of judge (researcher or student), level (senior or junior), speciality (biomedicine or social science), and evaluation context (relevance or utility), were asked to rate sixteen (16) documents on alcohol studies which included four (4) different search topics and four (4) document citations and abstracts for each search topic. These searches were taken from the Alcohol Information Retrieval System (AIRS). Half of the judges were asked to rate the documents on how relevant the documents were to the search topic; the other half rated the same documents on the basis of the document's perceived utility for the individual judge. The topics and documents were presented in a controlled prescribed order, so as to test the effects of order on the evaluation process. After rating the documents, judges were also asked to rate the importance of five (5) document attributes (author, title, abstract, source of publication, and date of publication) and six (6) information attributes (accuracy, completeness, subject, suggestiveness, timeliness, and treatment). Statistical tests of analysis of variance and discriminant analysis were used to analyze the data.

The findings of the experiment indicate no significant difference in document rating, document attributes, or information attributes due to the evaluation context. Students rated the title, source of publication, and the suggestiveness of the article as significantly more important than researchers in rating the documents. Judges with a social science specialty rated the timeliness of the document as significantly more important than judges with a biomedical specialty. There were several significant differences among the four derived experimental groups (senior researcher, junior researcher, senior student, and junior student), which include variation on importance of titles, accuracy, completeness, suggestiveness, and timeliness. The effects of order of document presentation were found to be significant, particularly for source and date of publication. It was also determined through discriminant analysis that group affiliation could be predicted with 70% accuracy or better for senior researchers, junior researchers and junior students from the ratings of documents and all attributes.

The study concludes that there is no operational difference between the relevance-theoretic and the utility-theoretic model of evaluation. It further suggests the need for performance measures based upon a complex set of factors including document and information attributes, the judge, and other environmental factors such as the order of document presentation.

MPACT Scores for John James Regazzi III

A = 0
C = 2
A+C = 2
T = 0
G = 0
W = 0
TD = 0
TA = 0
calculated 2009-05-19 10:34:01

Advisors and Advisees Graph