CHRIS: Custom Holdings Ranking Information System Implementing - - PowerPoint PPT Presentation

CHRIS:

Custom Holdings Ranking Information System Implementing Value, Service, and Community in Interlibrary Loan (ILL)

Traditional ILL Custom Holdings

• Three-tiered approach to managing Custom

Holdings in Interlibrary Loan

– Cost is king – Service is second – Rely on consortia (IDS, LVIS, Rapid) to monitor cost and performance evaluation.

Custom Holdings Defined as Queries

• All libraries that will lend to us for Free.
• All of those libraries that are in New York
• All of those libraries that use LAND
• All of those libraries that are in IDS

Custom Holdings Defined as Queries

• All free libraries in proximate states
• …That use UPS

Custom Holdings Defined as Queries

• All libraries that will lend to us for $5
• That are proximate
• …Ugh. Forget it. Too many queries to repeat

regularly.

Problems

• Requires a lot of legwork and labor to set up

and maintain.

• Ideally, custom holdings would be updated

frequently to verify information

• Hours of tedious, monotonous labor on an

annual or semi-annual basis

• Limited in scope; if the custom holdings list is

short, you’re rolling the dice on cost and service

Problems (con’t)

• Even after all that labor, it’s uncertain what

you actually get for it.

– It’s hard or impossible to efficiently extract the data you need from OCLC Policies Directory – Can’t even be certain that data is correct, cost is frequently misrepresented

• Even the best custom holdings are based on

individual knowledge (craft or anecdotal information, depending on your perspective).

SYB’s Approach to Custom Holdings

• Network (Consortia or OCLC group affiliation),

Cost, Service, Billing

• IDS takes precedence
• Free is better than Not free; $10 is better than

$15

• Post receipt processing labor is discouraged

– IFM favored over Invoices – Odyssey favored over Email

SYB’s Approach (con’t)

• Limited accounting for how GOOD a service is

provided to us, only what level of service is provided

• Limited accounting for geography and

shipping methods for book borrowing

• Both factors can add days (or even weeks!) to

an ILL request

• How to assign borrowing priorities that

account for this?

Poor service to us = Poor service provided by us

• If another library takes a week to respond to

an article request, we have provided a poor service to our patrons

• If a geographically remote library uses the

slowest available shipping to get us an ILL loan, we have provided a poor service to our patrons

Introducing CHRIS

• Currently MS Access based; shareable and

adaptable

• Optimize cost, service, and community-

building

• Highly adaptable and easy to update
• Adjusts for the quality of service provided to

us

Construction Methods

• Simple scoring algorithm
• About two dozen manual searches performed

against the OCLC policies directory

• Less laborious on the front end – maybe 1-2

hours of Excel cut-and-paste

Attributes for Articles

• Delivery

Method

• Institution Type
• Cost
• EMST
• Billing
• Performance

Attributes for Loans

• Delivery Method
• Geographic

Region

• Institution Type
• Cost
• EMST
• Billing
• Performance

Attribute Collections

• Cost

– Free, then $5 increments

• Performance

– Based on Average Performance

• Institutional Characteristics

– Minimize back end labor

Supplemental Measures

• OCLC Policies supplemented with several

sources

• Custom Holdings Helper Worksheet

Supplemental data pulled from OCLC Usage Statistics

• Reciprocity Reports from OCLC Stats: do we

have a “Trade Deficit” or “Trade Surplus” with another institution

• Lender String and Transaction Reports from

OCLC Stats: real time lender performance data

• Fee Management Reports from OCLC Stats:

cost per transaction grouped by Institution, Request Type (Loan Or Copy)

• All data pulled from one year

Why OCLC Usage Statistics?

• Provided most accurate representation of

lender performance (even when they didn’t fill).

• Easily generated and updatable without

complex searching.

Scoring

• Grouped by preferred attributes
• Bins are based on empirical scrutiny: an

attempt to predict who will give good service

• Attributes are combinatorial in nature

Main Attributes in Scoring

• 50 Total Points
• Cost

– Free, then $5 increments

• Performance

– Based on Median, Average Performance

• Institutional Characteristics

– Minimize back end labor

Attributes Comprising Total Score

• 50 Total Points
• Cost

– 15 Points

• Performance

– 15 Points, Two Categories

• Institutional Characteristics

– 20 Points, Four/Five Categories

Supplemental Measures

• Prefer Actual Data > Proxy; if we have real cost

information, use that instead of Policies reports

• Don’t punish (or reward) a few bad
• transactions. Require 5 Borrowing Requests

for actual Performance Data to be Validated

Institution Type (Articles)

Type Avg Response Days Total Percent Unfilled Score Assigned Medical 1.27 0.12 1 Academic 1.67 0.28 2 Research 1.75 0.27 2 Government 2.7 0.22 3 Community 2.68 0.55 4

Reciprocity Correction

• If More than 50 Transactions between

Institutions:

• If Borrows : Total > 0.75:

– Add 5 points

• If Borrows : Total < 0.25 AND reciprocally free:

– Subtract 5 points

• I.e., attempt to correct imbalances by shifting

institutions to a different CH group

Cost is Still King

• But performance is just as important
• Cost: 30% of Total, Radical jumps in score to

reflect preference for low cost

• Performance = Response Days + Accuracy =

30%

• Institutional Characteristics: 40% Accumulated

Results

• 8753 Ranked Institutions
• SYB has transacted with 2473 of these
• Most institutions are transacted infrequently

Number of Transactions

NumRequests Loans Articles < 5 1775 2201 5-10 350 104 10-100 289 133 100 > 59 35

5 > 607 245

Summary of Transactions

• Loans, max requests for a single institution:

1404 (NYP). Second: 1158 (BNG)

• 75 Percentile for number of loan requests: 5
• Articles, max requests: 620 (BNG), Second:

565(BUF)

• 75 Percentile for number of article requests: 1

Institution Types

Institution Type Count Academic 3312 Community 2497 Government 771 Medical 36 Association of Research Libraries 102 Total Extracted 6718

Cost

Range Count Free 2590 $0-$10 5934 $10-$15 142 Over 15 87 Total Extracted 8753

Top Ten Ranked Institutions

Symbol Cost InstType DelMethod Region EMST IFM Performance Accuracy Rank VYT 2 2 1 1 1 3 15 YGM 2 2 1 1 1 3 15 ZGM 2 2 1 1 1 3 15 YJM 2 2 1 1 1 1 16 XNC 2 2 1 1 1 2 18 VVV 2 2 1 1 1 2 18 BNG 2 2 1 1 1 2 18 VVB 2 2 1 1 1 2 18 YJA 2 2 1 1 1 2 18 ZVM 2 2 1 1 1 2 18 YPM 2 2 1 1 1 2 18

Missing Data

• Use aggregate averages for missing values (i.e.

incomplete policies directory info.)

• Finer grained picture of our custom holdings

set

• Much larger set than is possible by coding

Custom Holdings by hand

• Missing Data can be reduced further by

collaborating with other IDS institutions

Low-Lying Outliers, <$10 + Email

symbol Cost InstType DelMethod EMST IFM Performance Accuracy RANK CHM 6 1 3 2 1 26 XII 6 2 3 2 1 28 CAI 6 3 3 2 1 30 PFM 6 2 3 2 2 30 TXA 6 2 3 2 2 30 CQL 6 3 3 2 1 30 UBY 6 2 3 2 2 30 YAM 6 3 3 2 1 30 YUH 6 2 3 2 2 30

Initial General Analysis of Custom Holdings

• Major Consortial Partner Libraries still Highly

Represented, but allows for other high performers to blend with IDS.

• Comfort in Prioritizing IDS Libraries in Custom

Holdings—it’s based on data.

Updating Values

• OCLC Policies Directory subject to change on a

day to day, seasonal, or other random basis

• Can pull institutional characteristics on a

quarterly basis: about an hour cut and paste; data translation SQL stored for quick re-use

• Transaction, Performance, and Fee Reports

show a rolling aggregation of the past year with monthly pulls from OCLC

Updating Values

• Scoring system can be altered as easily as

changing digits in a query; what works for SYB may not work for all institutions

• But the model and underlying data can still be

uniform, allowing for data sharing

Other Advantages

• No more manual checking of IFM reports—

CHRIS can provide a report of what libraries have costs different than “expected”.

Assessing CHRIS

• Turnaround Time (excluding parts of workflow

affected by borrower performance—i.e. time from awaiting req. processing to sent).

• Cost-reducing (IFM and shipping) while

improving turnaround time.

• Variety of lenders used and reciprocity
• Percentage within consortia
• Reducing “outlier” transactions
• “End Labor” Processing needed

Further development

• CHRIS does videos and other formats
• Analyze loan periods, and significantly factor

in longer loan periods for loans.

• Move from Access to online database that can

be easily shared and configured

Why CHRIS Now?

• New OCLC 15 lender workform will make

custom holdings performance even more important.

• Knowledgebase and Direct Request, paired

with 15 lenders can be powerfully useful or powerfully problematic.