Objective: In a large, decentralized library system, lacking a cohesive culture of assessment, it can be challenging to encourage colleagues to embrace evidence based practice. Our objective is to develop an effective collection assessment program that librarians throughout the system will readily integrate into their daily workflows.
Methods: Collection Development and Assessment partnered to create workshops, training programs, advisory groups, and consistent messaging to promote integration of data analysis, assessment, and other evidence based practices into librarians’ daily workflows.
Results: Our case studies demonstrate that librarians have embraced evidence based approaches in areas such as ebook program assessment, collection development training for new librarians, and increasingly sophisticated engagement with electronic resources data.
Conclusions: Multipronged, ongoing work is required to change the culture. We can see the program’s efficacy when we see our colleagues request and use data to inform their daily work; when they expect library leadership to demonstrate evidence based decision-making, and when they begin to build the expectation of evidence based approaches into job descriptions and training programs.
Overview: Decisions regarding compensation structures are critical, and support a myriad of important organizational functions, including recruitment, retention, and equity. But as the literature shows salary structures are only effective when reflecting external and internal equity, and distributive and procedural fairness. Historic issues with librarian salaries at the University of Florida included compression and ad hoc salary decisions inconsistently based upon rank, and non-validated assumptions regarding job worth and market demand. The structure lacked transparency. This presentation provides a case study and real life examples depicting the development of pay structures that avoid these common issues and supports the organization’s goals.
Methods: Three common policy issues impact compensation programs: establishing pay levels in relation to other libraries; evaluating individual jobs and determining pay relationships among them; and determining pay relationships among individual workers within the same job.
Based on the literature and an in-depth analysis of the Association of Research Libraries Salary Survey, the researcher identified and established locally defined compensable factors. External equity is derived from analysis of data for librarian job types, and reflect regional variances. Internal equity is systematically determined based the individual’s experience, educational credentials, and performance; and the special requirements of the position, including skills, knowledge and ability, and administrative responsibilities.
Results: The result is an effective and equitable system. The system has been maintained by the establishment of policies and practices for recruiting, counter offers, promotions, lateral moves, and system-wide pay increases.
Conclusion: The presented model is grounded in current management theories regarding compensation. Although based on one institution, this model has been referenced at others and has proven it can be deployed, with desired modifications, to any library. The presentation attendees will leave with the ability to implement a sophisticated compensation system that is also straightforward enough to be transparent to library staff.
Engagement, Retention, and GPA: Creating a Library Index to Explore Impact
Beth Martin, University of North Carolina, Charlotte
Anne Cooper Moore, , University of North Carolina, Charlotte
Rachael Winterling, University of North Carolina, Charlotte
Objective: The J. Murrey Atkins Library will explore the relationship between student engagement, library use, student retention, and GPA to better understand our impact on our constituents. To do this the library, in collaboration with the Division of Student Affairs, participated in the National Survey of Student Engagement (NSSE) for 2016 which examines the curricular and co-curricular activities of our new freshmen, new transfer students, and seniors.
Methods: The library assessment team will create a service index using the library activities we can obtain the individual level. The index will assign engagement indicators that range from building use to attendance in instruction classes and will be a useful tool in further research around our services. The service index will be analyzed in conjunction with NSSE as well as student GPA/retention information obtained from the Division of Student Affairs.
Results: The service index should be complete by the end of December 2016 and the NSSE analysis will be complete in late February. We will analyze the data to identify explore the relationship between a high library service index score, student engagement (based on NSSE) and other student information as a part of the research.
Conclusion: The assessment team will report the findings with our internal and campus constituents to provide insight into all aspects of student engagement and the library.
Embracing the Generalized Propensity Score Method: Measuring the Effect of Library Usage on First-Time-in-College Student Academic Success
Jingying Mao, Florida State University Libraries
Kirsten Kinsley, Florida State University Libraries
Does library usage help first-time-in-college students stay in college and achieve better grades? According to Astin’s Input-Environment-Output (I-E-O) Model (1993), input variables such as pre-college high school grades and college entrance exam scores (e.g., SAT) collectively impact whether a student succeeds in college. In addition, collegiate environmental factors, such as involvement in student activities or engagement with faculty, also influence student outcomes. Visiting the library in that critical first year of college may also be a vital environmental factor contributing to student success outcomes. But, how does one isolate that independent variable and control for the other inputs and environmental variables in a scientifically rigorous way? This study investigates the effects of student library usage on student academic success, such as first-term grade point average (GPA) and freshman retention for first-time-in-college (FTIC) students using the generalized propensity score method for analyzing causal effects in observational studies. The student cohorts include those who visited two libraries between fall 2014 and spring 2016 semesters on the campus of a large southeastern university. Frequency and duration of individual library usage will be measured each semester. To adjust for self-selection bias, the generalized propensity score method in a continuous treatment setting will be used. We evaluate the effects of library usage on student academic success after adjusting for covariates imbalance. Covariates will include gender, race, age, parents’ income and education level, high score GPA, ACT score, department, etc. We hypothesize that there exists a positive effect of student library usage on academic success for FTIC students.
Library Visitor Metrics: What They Mean, Why They Matter?
Nisa Bakkalbasi, Columbia University (Twitter: @nisabakkalbasi)
Objective: The popularity of Columbia University Libraries is readily apparent in its use statistics. In the past ten years library visits across the campus rose 25% amidst library closings. In the past year, the Libraries hosted over 4.7 million visitors. This study focuses on the analysis of and practical uses for swipe-card data, emphasizing its strengths and weaknesses in arriving broad conclusions about library use.
Methods: Since 2007, Columbia University community has been required to swipe their ID cards in order to enter to library facilities, where Lenel Access Control system is installed. The card swipe verifies that the individual is a Columbia University affiliate and enables access into the library. Although the swipe-card system operates primarily as a security measure, each card swipe provides demographic information associated with the respective visitor; this information is stored and maintained in a database managed by the University’s Security and Identity Access Management division. Once all the personally identifiable information is removed from the data, transactional-level data becomes available to the Libraries, detailing various attributes such as ‘school,’ ‘program,’ ‘academic role,’ and ‘timestamp’ about each visit.
Results: Using descriptive statistics analysis of entrance data collected over a three-year period, we are able to track year-to-year changes, and time and frequency of library use (such as repeat/unique visitors). Analysis of demographic characteristics (such as school or academic role) helps us capture metrics such as ‘visit per enrolled student’ and compare library use trends across various schools.
Conclusion: Library entrance data is part of a universe of library statistics that must be systematically collected and analyzed, and critically evaluated. While descriptive analysis of entrance data produces a rich understanding of use patterns, it doesn't give us information about users’ experience. However, this data can be augmented by qualitative studies to gain insight into user experience.
Objective: Implementation of even simple assessment tools can be important for decision making. Many academic libraries are approached by student organizations about providing 24/7 library operations especially at finals time. In the fall of 2013, Rowan University Libraries’ new administration was approached mid-semester about providing 24/7 hours during finals. The administration didn’t have sufficient information to know if use would justify the cost. The compromise was to offer extended hours during finals that semester and track usage. One snapshot of usage is not enough to make permanent decisions. The administrators committed to collecting data and optimizing use and costs.
Methods: Initially a tally sheet was created that split the 130,000 square foot building into a list of easily identifiable locations. The person doing the headcount walked the building every 2 hours counting the number of individuals working alone with and without computers and in groups with and without computers. The next semester 6 overnights were added. During the 2015 fall semester, a mid-semester hourly count was added so a comparison could be made between the regular hours and finals. This meant hourly counts all day for an entire week mid-semester AND hourly counts all day during finals including the extra hours.
Results: The usage information has allowed the administration to adjust the hours with more confidence that the building will be used especially during finals. Student use of the building during finals varies between the fall and spring semester. The data also demonstrated a noted drop of use on Thursday nights, Fridays and Saturdays. The information gathered has also helped the administration make space planning decisions
Conclusion: Assessment is a very important tool for decision making. A basic headcount activity has helped improve services for students, provided information for scheduling hours and informed administrators on space planning decisions.
Objective: How do research libraries consider usage, overlap, costs, and user satisfaction in making decisions about the retention of subscription-based content packages in which there are no post-cancellation access rights? It is not uncommon for even the largest of research libraries to spend upward of 70% of their acquisitions dollars on electronic content. The ownership and licensing models for these materials remains uncertain, with some fully embracing institutional ownership and others focusing on leased models with no post-cancellation access rights. One example of such a model is the Safari ebook package. Drawing on publisher usage data, overlap studies, and other institutional data, this paper provides an evidence-based framework for analyzing the usage of the Safari ebook package, explains its implementation on the campus, and details decisions made in the fall of 2016/spring of 2017 about continued support for the comprehensive suite of titles.
Methods: This study examines the use of the Safari ebook package over a multi-year period; reviews turn-away data, pricing, and content overlap with other, ownership based ebook acquisition options; and, details the institution’s
recommended path forward in a resource-constrained environment.
Results: In a resource constrained environment, the institution needed to make decisions about the relative value/merit of continuing many resources. Initial reviews of A&I services provided one path forward. However, the influx of students in the fall of 2016 rekindled a continued point of frustration on campus – the excessive turn away problem among users of the Safari platform – and led to an examination of the product in the local environment and a recommended path forward.
Conclusion: The impact of the ‘serials crisis’ and diminished resources have led many academic libraries to review current expenditures. This study details the results of a robust review of one platform and details a part forward for similar examinations.
Comparing DDA Ebook Programs of 8 Large Academic Libraries
Kay Downey, Kent State University
Yin Zhang, Kent State University
Demand-Driven Acquisitions (DDA) represents a shift away from the traditional “just in case” (JIC) collection development model. In recent years, many academic libraries implemented DDA on a “learn-as-you-go” basis with little empirical data to guide them. For example, it was difficult to predict usage, and the most cost-effective scenarios: short-term loan only, short-term loan with purchases after meeting the threshold, straight purchase with no short-term loans, etc. In this study, DDA ebooks programs from 8 large academic libraries, will be examined in order to further research on DDA use and efficacy. Reports examined will include ebook Firm Orders and DDA Orders, DDA Trigger Reports, standard BR2 reports, usage for purchased and non-purchased ebooks. With multiple years of data this will be the first large scale study of this kind. Comparison and analysis of this data on a help discover similarities and differences between the programs. It may be a predictor of future use and benchmark for return on investment. Programs will be assessed based on DDA model variances. In conclusion the presenters will address some key questions facing libraries with a DDA ebook program such as 1) How may a DDA program be evaluated? 2) What elements to consider in setting up or modifying the DDA plan, e.g. STL or Not STL? 3) What actions to take post-evaluation? This study will provide insight into practical approaches for DDA program assessment and offer perspectives and considerations for implementing and evaluating a DDA program in large academic libraries. It will also analyze DDA ebook commonalities and future directions that help librarians in choosing the ebook DDA business model that works best for their library.
Evidence of Patron Library Space Usage: A Pilot Project using Web GIS in an Academic Library in the United States
Rick Stoddart, University of Idaho
Bruce Godfrey, University of Idaho
Objective: Academic libraries are creating and experimenting with developing more diverse learning spaces for students. Measuring how students use these new spaces can be a challenge.This paper will report on a pilot project that uses web GIS to gather evidence of library space usage. The pilot project seeks to answer these questions: What evidence can GIS provide about library space usage? What challenges and opportunities does GIS provide for library space assessment?
Methods: The pilot project uses a mobile tablet and a program called ArcGIS Collector to gather information on how students are using library spaces. Data was gathered during the Fall 2016 and Spring 2017 terms. Library space usage was mapped to different learning space types and supplemented with additional quantitative and qualitative data.
Results: Initial data and participant feedback indicates that this GIS pilot project gathers meaningful evidence to begin talking about how students use library spaces.
Conclusion: Tablet and desktop based GIS programs such as ArcGIS Collector are an innovative way to gather and display evidence of library space usage.
User-focused, User-led: Space Assessment to Transform a Small Academic Library
Christina Hillman, St. John Fisher College
Kourtney Blackburn, St. John Fisher College
Kaitlyn Shamp, St. John Fisher College
Chenisvel Nunez, St. John Fisher College
Objective: By collecting and analyzing evidence from three data points, researchers will understand how library spaces are used. Results will be used for evidence-based decision-making regarding library physical spaces.
Methods: Undergraduate researchers, sociology faculty, and librarians used mixed-methods to triangulate findings. Seating sweeps were used to map patrons’ activities in the library. Student-led focus groups discussed patterns of library use, impressions of facilities, and library features and services. The final step will be a campus survey developed from seating sweeps and focus group findings.
Results: Seating sweeps showed consistent use of the Library's main level Learning Commons and upper level quiet spaces; the library’s multipurpose lower level is under-utilized. Students use the main level of the library for collaborative learning, socializing, reading, and computer use. Students use the upper level for quiet study and group work in study rooms. Focus group findings found library use is task-specific. For example, a student may work with classmates on a project using the main level Learning Commons during the day, and then come back at night to use the quiet floor for test preparation. Participants shared additional spaces on campus they use for study and the characteristics of those locations.
Conclusion: These data offer empirical evidence for library space needs. Some data aligns with previous space studies: access to power outlets, lighting, noise, and outdated environment. New concerns included: crowding, graduate students lacking designated study space, and needing quiet study space away from group study space. Data from seating sweeps and focus groups will be used to create a campus survey to capture other information related to library use and space needs. Survey findings will offer a richer understanding of how the library is viewed and used by all of campus.
Library Use Contributes to Student Success. So What?
Dana Thomas, Ryerson University
Weina Wang, Ryerson University
Overview: Beginning in 2013, the authors embarked on a research project to prove statistical correlation between various types of library use and student grades. This paper will differentiate our work from other library impact projects conducted in the United States and Canada. We will also share how we use our data to make decisions and inform our work in an academic library.
Objective: To explore relationships between library use and student grades in order to inform decision making and to support advocacy efforts.
Methods: Working with relevant campus partners, the authors obtain all active students’ age group, disciplinary affiliation, patron type (undergrad, grad, continuing education), year of study, and grade information at the end of each term. Data collected by the library and by the registrar's office is sent to the University's Campus Computing Services (CCS) department, who then merges the data to match grade information with library usage data. The data is given non-identifiable ids by CCS and is then sent back to the library where it is used to analyze and identify patterns and relationships among library resource use (collections and services) and grades.
Results: To date, we have found that undergraduate students who use the library have on average 0.32 higher GPAs than those who do not. Graduate students who use the library have on average 0.19 higher GPAs than those who do not. 79.18% of undergraduate students and 87.62% of graduate students had used at least one measured library service in the fall 2013. Several notable differences exist across fields of study. 2014 and 2015 data are currently being analyzed.
Conclusion: While recognizing the limitations of this type of research, specifically that it proves only correlation and not causation, the data which results from it has proven valuable for advocacy efforts, marketing and decision making.
Learning about Student Research Practices through an Ethnographic Investigation
Kimberly Mullins, Long Island University
Natasha Tomlin, Long Island University
Eamon Tewell, Long Island University
Objective: Student research habits and expectations continue to quickly change due to technological advances, complicating the design of library spaces and the provision of research support. This project’s intent was to improve the library's understanding of student research and study needs, with the ultimate goal of identifying specific ways to improve the library experience.
Methods: The project utilized a robust mixed-methods design that spanned from fall 2012 to summer 2013, consisting of a survey, observations, and interviews. The first step was a survey questionnaire consisting of 51 items that received 1182 responses. Second, 32 hours of unobtrusive observations were conducted by taking ethnographic “field notes” in a variety of library locations, and during different times and days of the week. The final method was in-depth interviews with 30 undergraduate and graduate students.
Results: Several strategic actions have been completed or are being pursued based on the findings. Regarding information literacy efforts, basic library instruction and an information literacy exam were integrated within the first semester curriculum. Librarians became involved in Learning Communities to communicate directly with students early in their academic careers. New services, such as a Library App for mobile devices and Skype research appointments, were developed. Both libraries are undergoing extensive renovations that include soft seating, natural light, and the addition of “lounge environments” and “Genius Bar”-inspired workstations.
Conclusion: Understanding student research processes and preferences will result in the ability to design learning environments and research services that are more responsive to their needs. This paper will focus on takeaways related to both the study’s process and its results. It will be of interest to librarians considering a similar study at their institution as well as those wishing to learn about what students do or do not appreciate about academic library services and spaces.
Analyzing and Assessing IL in the Framework Era: A Rhetorical Analysis Followed by Exploratory Factor Analysis
April Cunningham, Palomar College
Michelle Dunaway, Wayne State University
Objective: We are creating a test of students’ information literacy (IL) knowledge and dispositions, inspired by the Framework for Information Literacy in Higher Education. Though it is controversial, we see many benefits in having a fixed-choice test for large-scale assessment. This paper describes our team’s efforts to measure students’ IL dispositions.
Methods: We began with rhetorical analysis of the Framework to identify latent affective variables. We then had experts map the variables to four IL dimensions: Evaluation, Searching, Inquiry, and Valuing Information. Using these maps we identified observable variables that we wrote into IL scenarios and strategies, which formed the basis for two sets of metacognitive knowledge scales. We then conducted an exploratory factor analysis (EFA) of students’ responses. For the first set of scales we had 170 students’ responses and for the second we had 308. The EFA allowed us to examine how students’ responses to individual strategies reveals dimensions of the test as a whole. We used EFA despite having a theory of IL dispositions because we had not yet tested the latent variables identified through the rhetorical analysis using any other methods.
Results: The rhetorical analysis resulted in defining four latent affective variables: Mindful self-reflection, Productive persistence, Responsibility to community, and Toleration for Ambiguity. The results of the EFA showed strong goodness-for-fit among the strategies we wrote for each variable. It revealed, however, that we could not discriminate between self-reflection and persistence in the dimension of Searching.
Conclusion: We established preliminary construct validity for the four IL affective variables. Following revision we will resume analysis to check for internal consistency and ensure the scales continue to discriminate between the dispositions. We are embracing evidence-based strategies to engage with the Framework, making it possible to have constructive conversations about embedding IL assessment during this time of transition.
Evidence-based Library Instruction Development for Spatial Literacy
Ningning Kong, Purdue University
Objective: Spatial literacy is defined as the competent and confident use of maps, mapping, and spatial thinking to address ideas, situations, and problems within daily life, society, and the world around us (Sinton 2012). Spatial literacy is especially needed for graduate students in many disciplines when exploring their research questions. Therefore, many students look for resources in academic libraries to support their spatial literacy skills. The objective of this research practice is to develop effective library instruction programs that support spatial literacy by using the existing models, campus students’ needs, and faculty expectations.
Methods: To develop appropriate library instruction formats and contents, we have collected evidences in three basic areas, including existing literature, campus needs, and faculty expectations. The literatures focused on existing library support for spatial literacy, as well as spatial information teaching experiences from other disciplines. The campus needs were assessed from library consultation topics and library webpage visits statistics. And the faculty expectations were collected from interviews. We analyzed the evidences collected from these information sources, and provided corresponding library instructions according to what we found.
Results: A series of spatial literacy related library instructions were developed based on collected evidences. These instructions include workshops, class visits, intensive instructions, as well as semester long courses. The evaluations from these instructions have shown great improvement of participants’ spatial literacy skills.
Conclusion: Evidences collected from multiple sources can be great references for developing library instructions. The evidences collected from literature review can serve as guidelines in developing the instruction in general. The community specific evidences collected from library usage and interviews can help to tailor the instruction toward the users’ needs. The process of developing the library instruction program also help us to collect more evidences for future improvement.
Understanding Research Productivity: Fresh New Evidence for Librarians
Kristin Hoffmann, Western University (Twitter: @KMHlibrarian)
Selinda Berg, University of Windsor (Twitter: @librarysushi)
Denise Koufogiannakis, University of Alberta (Twitter: @dkouf)
Objective: Research-informed evidence is a key component of evidence-based library and information practice. As such, it is important to understand what factors affect library and information professionals’ ability to carry out and disseminate research. Contributing to the literature about research by librarians, this project is unique in that it looks for correlations between research productivity factors and the research outputs of academic librarians. The study examined factors related to institutional supports, which have been commonly described in the literature, and also
evaluated factors related to the individual and their professional community.
Methods: An online survey was distributed to 1,683 librarians from 75 Canadian academic institutions. It included questions about the participants’ research output, and questions that measured factors related to three categories: Individual Attributes, Peers and Community, and Institutional Structures and Supports. The survey asked pointed questions about respondents’ research attitudes, behaviors, and environments in a yes or no format. It also included open-ended questions inviting respondents to indicate other factors and publication formats not included in the survey.
Results: Over 400 librarians, representing 70 of the 75 institutions, responded to the survey. At the time of submission, data analysis is still in progress. Statistical analyses are being conducted in order to determine which factors show a significant correlation with research outputs. Factors will be analyzed within the broad categories listed above, as well as in the groupings of demographics, education and experience, personal commitment to research, personality traits, motivations for research, collaboration, peer support, mentoring, institutional characteristics, and institutional supports.
Conclusion: By highlighting factors that empirically correlate with research productivity, our results may challenge assumptions and inspire new ways to think about building research communities in libraries. Focusing our energy on evidence-based strategies will help to maximize the effect of libraries’ precious resources and efforts to support research.
Using Data for Assessment and Research: The Institute for Research Design in Librarianship (IRDL)
Marie Kennedy, Loyola Marymount University
Kristine Brancolini, Loyola Marymount University
Objective: This presentation reports on three interconnected summaries of data gathered from the participants of the Institute for Research Design in Librarianship (IRDL) that were used to both improve the program over three years and
to contribute to the body of knowledge regarding success factors for librarian-researchers.
Methods: In the presentation we will fully describe the data gathering and analysis of three measures used to inform the development of this continuing education program for academic and research librarians in the United States: a measurement of the developing research network of the participants; research self-efficacy as described through a scale developed by the researchers; and the scores of the pre- and post-workshop research proposals, as modeled using a rubric designed by the IRDL assessment team.
Results: The data gathered for the three measures were used to influence the continuing development of the year-long research experience as well as an aid to further understanding what practicing librarians need in order to also be successful researchers. The summaries of data in this presentation will highlight the clear effect of intentionally developing one’s peer group for support in the research process, the achievement of mastery in the areas of the curriculum addressed in the IRDL program, and the confidence gained as a result of achieving that mastery.
Conclusion: By working with a diverse group of novice researcher-librarians over three years we learned about the power of knowledge and community to unleash creativity and productivity. Through the summer workshop that begins the year-long IRDL experience, librarians added to their research toolkit and to their network of peers who are also committed to improving their own research. We learned that confidence is powerful and that it is related to both the mastery of the research process and the ongoing development of a community of peers.
Citation Analysis of Communication Studies Journals: Actionable Knowledge for Librarian
Lisa Romero, University of Illinois
Shrinking library budgets and increasing costs of library materials have made it more challenging for librarians to maintain robust library collections that meet the needs of researchers. This is especially apparent in extremely multi-disciplinary areas of research like communication studies. To provide librarians with the data necessary to make evidence-based collection development decisions, journal citation data representing 30 years of research from 118 communication studies journals in advertising/public relations, communication/media and journalism were collected and analyzed. Data were organized and presented according to the 3 areas within communication studies to accommodate the needs and collections of a variety of libraries. This presentation will provide insight on the multidisciplinary nature of communication studies research (advertising/public relations, communication/media, journalism), discuss the most relevant scholarly journals in communication studies scholarship, and provide librarians with the data necessary to make responsible collection management decisions that accurately reflect the research needs of their users. The author will provide evidence-based recommendations regarding collection management including purchase and de-selection, format and access and provide librarians with more productive alternatives to collection management that are less vulnerable to the marketing whims of publishers. Instead of simply following the leads publishers provide, librarians can begin to focus more consciously on their collections and the fields represented.
Reading Ghosts – Monitoring In-library Usage of ‘Unpopular’ Resources
Stacey Astill, Keyll Darree Library (Twitter: @KeyllDarree)
Jessica Webb, Keyll Darree Library
Objective: This paper discusses our monitoring of library resources used in the library for reference purposes, but not loaned. In order to explore this we will discuss the difficulties of tracking this type of usage and the issues this can cause, the process of “ghosting” which Keyll Darree Library have employed, and the value of the statistics which this provides for us, and also the limitations of the method. In a time where budgets are being reduced, and stretched to their limit stock assessment is extremely important, therefore gathering an accurate picture of resources which are most popular for users is ever more valuable.
Methods: Users are asked to leave all items which they are not taking out on loan on workspaces. Library staff collect these resources daily and issue to a “ghost” account on the library system they are then returned, and re-shelved. The data is then collated and recorded on a spreadsheet which is updated weekly.
Results: Since we have used the process of “ghosting” we have had no more user complaints that regularly used stock has been removed, and we have updated issues which may not appear well used when focusing only on user circulation history. We have identified one text which has been issued repeatedly to “ghost”, but never to a user account.
Conclusion: In a small scale academic library like Keyll Darree, users will often share resources without removing them. A specific cubby is often used to store items, which are then used for reference purposes without being removed. As the decision to withdraw stock often correlates with text usage, ensuring that we have an accurate insight into which items are wanted and most used by our users is integral to ensuring that we offer the best and most useful resources for our customers.
“How do I Know if it’s Useful if I Can’t Even Get it to Open?”: Assessing Information Interaction to Improve Library Collections and Services
Erin Dewitt Miller, University of North Texas
Susan Smith, University of North Texas
Emily Billings, University of North Texas
Xin Wang, University of North Texas
Objective: New information formats like e-books and online streaming video are now widespread in education (Matusiak, 2013). However, the usefulness of these platforms to supporting academic library users’ information needs has not been sufficiently researched (Albertson, 2013; Richardson & Mahmood, 2012). Two usability studies at the University of Texas were undertaken this year in order to better understand user and librarian interactions with e-books and streaming media. Additionally, these studies attempt to correlate the technology acceptance model constructs to usability metrics. The design of each study embeds research into library assessment and services.
Methods: This research is composed of two separate but related usability studies, both of which were conducted in the libraries at the University of Texas. The streaming media usability study began in spring 2016 and involves two separate online video platforms. Twelve students and twelve librarians participated. The e-book usability study began in summer 2016 and involves 6 undergraduate students and 6 graduate students as well as a comprehensive survey designed to measure student e-book preferences and reading habits and to determine the correlation between usability and the usefulness and ease-of-use of e-books. Morae software was used to record usability tests and to analyze the results. Recordings were coded to make a thorough analysis possible.
Results: Measuring the usability of streaming media among both undergraduates and librarians allows us to better assess whether or not library instruction and library resources are truly serving patron needs. The results can be integrated into collection management decisions, library instruction and user services. The results also make it possible to develop a model correlating the technology acceptance model constructs (perceived usefulness, perceived ease-of-use, attitude and behavioral intention) with usability metrics (efficiency, effectiveness and satisfaction.
Conclusion: The conclusion from this study will be reached in spring 2017 once data analysis is completed.
Mixed Methods Not Mixed Messages: Improving LibGuides with Student Usability Data
Junior Tidal (Twitter: @JuniorTidal)
Nora Almeida (Twitter: @nora_almeida)
Ursula C. Schwerin Library, New York City College of Technology (CUNY)
Objective: This paper will discuss a project to improve LibGuides version 2 research guides at an urban, public university library and to understand student design and learning modality preferences. We will present findings of a mixed methods usability study and discuss how this data translates into executable design principles.
Problem: In spite of the ubiquitous use of LibGuides and educational technology platforms in libraries, too few librarians consider user design preferences when designing tools for student use. Additionally, many librarians adopt non-critical approaches to instructional design that don’t adequately address how students learn.
Method: User-centered design demands that stakeholders participate in each stage of an application’s development and that assumptions about user design preferences are validated through testing. Our usability test combines paper prototypes, an advanced scribbling technique modeled on the work of Linek and Tochtermann (2015), and semi-structured interviews. We opted to use this methodology because: paper prototypes are expedient to develop and assess; the advanced scribbling technique promoted close analysis of individual design elements without the interference of an interviewer; lastly, semi-structured interviews introduced more opportunities for student reflection on modality preferences.
Results: We will present student feedback on navigation layouts, terminology, and the amount of content displayed on Libguides. Beyond design preferences, we will outline students’ impressions of multimedia, text-based, and interactive instructional content in relation to specific research scenarios defined at the outset of the usability test.
Conclusion: We will discuss how study finding can be translated into best practices and incorporated into custom LibGuide templates. Our findings will be useful for librarians and designers working in academic and public libraries that utilize LibGuides or other educational technology platforms. Our research methodology may be of interest to those who want to incorporate evidence-based decision making into their web design practices.
Children’s Information Practices at a School Library Makerspace – Evidence from Video Recorded Re-enactment and Interviews
Xiaofeng Li, Rutgers University
Objective: This study aims to understand children’s information practices at a school library makerspace, and explore the roles of arrangement of the space, available technologies, and materials in their information and social practices at makerspace.
Methods: This study applies some sociocultural approaches to learning as the framework. Data was collected through semi-structured interviews that incorporated the video recorded re-enactment method (Pink & Mackley, 2014). Twelve participants were asked to show the researcher around the makerspace and re-enact what and how they usually do in this space, including the technologies and materials that they usually used and the areas where they usually went.
Results: The materials at the makerspace enabled the participants to share ideas, to play, to be inspired, to tinker, to imagine, and to build their identities. The makerspace itself provided space for them to hang out with their friends, meet new people, and allowed equal access to materials and games. With the computers available at makerspace, the participants could code, encounter ideas serendipitously, transfer knowledge learned from other classes, and create 3D models. Their friends and teacher provided them helps when needed. Emotional aspects such as feeling empowered, fun, amazed, and sometimes stressed during these practices were also identified. Key information practices involved information sharing, ideas generating, help seeking, and information use. The participants engaged in sharing information practices through the Lego shelf - an old book shelf that was re-purposed for displaying Lego creations. Cognition was distributed among these displayed Lego creations which were created by different students, at different time, and from school clubs.
Conclusion: Children’s information practices and the roles of technologies and materials at library makerspace was well captured through interviews that incorporate video recorded re-enactment. Practical implementations pertaining to the design and implementation of makerspaces in libraries will also be discussed.
Studying the Effectiveness of a Storytelling/Story-Acting Activity on Ugandan Preschoolers’ Emergent Literacy in Two Rural Ugandan Community Libraries
Valeda Dent, St. John’s University
Geoff Goodman, Long Island University
Objective: Using two rural village libraries in Uganda as a backdrop, the study explores the effectiveness of a library-based intervention known as the STSA (storytelling/story-acting) activity on preschool children’s school readiness skills related to emergent literacy. Half of the children at each library participated in STSA twice per week for six months and were compared with preschool children who did not receive the STSA intervention.
Methods: 123 children ages 3 to 5 and their caregivers from the remote villages of Mpigi (n = 61) and Kabubbu (n = 62) participated. Participants were randomly assigned in to active (STSA) and inert (story reading only) conditions. The researchers conducted baseline assessments in January, 2014, and returned in August, 2014, to conduct post-intervention assessments on these same children and caregivers.
Results: The STSA play intervention group did not perform better than the control group on the child outcomes of emergent literacy, oral language, and theory of mind skills. Children in both groups improved dramatically on all three indices of school readiness. Boys with the lowest receptive vocabulary levels did not improve as much as girls with the same levels. Caregiver’s educational level predicted the child’s emergent literacy, oral language, and theory of mind skills at Time 1. Caregiver’s total possessions also predicted the child’s oral language skills at Time 1.
Caregiver’s frequency of reading aloud to the target child predicted the child’s theory of mind skills at Time 1 and was positively correlated with the total number of stories the child told during the STSA play intervention.
Conclusion: Literacy-based activities implemented in a rural community library setting facilitate school readiness skills such as receptive vocabulary ability in Ugandan preschool children ages 3 to 5. These outcomes provide strong evidence to implement the STSA activity in more rural community libraries, which are ideally suited to host such programs.
Innovating with Evidence Collaboratively and Cross-Institutionally: Ithaka S+R’s Religious Studies Project
Danielle Cooper, Ithaka S+R
Objective: What resources and services do faculty members need to be successful in their research? The answer likely depends on their discipline. By embracing multi-institutional approaches to evidence collection academic libraries can develop more expansive approaches to innovating services for their scholars. This paper shares the methods, findings and outcomes from Ithaka S+R’s Religious Studies project, which enabled academic libraries to collaboratively collect evidence about their scholars’ evolving research activities in that discipline.
Methods: Ithaka S+R coordinated library research teams at 18 participating academic libraries and provided research instruments and methodological training. The participating libraries conducted qualitative research with their institution’s religious studies scholars through semi-structured interviews and photography. Each institution analyzed their own data and created local reports with the option of making those reports publicly available. Ithaka S+R analyzed a sample of the transcripts from across the participating institutions to create a publicly available capstone report.
Results: Ithaka S+R’s findings across the participating institutions highlight the disjunction between religious studies scholars’ research practices and emerging priorities in higher education that necessitate new approaches to library services pertaining to: international research, new digital research methods, and open access approaches to publishing. The local reports corroborated these findings and led to institution-specific outcomes that complimented the insights developed from Ithaka S+R’s discipline-wide research.
Conclusion: The project’s model is an example of how a sustainable and collaborative multi-institutional library research project can be successfully designed and implemented. By concurrently providing opportunities for participating institutions to conduct research on the needs of their own scholars while also creating aggregate analysis that provides a discipline-wide perspective, the project provides opportunities for libraries to embrace dynamic forms of evidence when developing their services. The open, collaborative approach also fosters a wider culture of evidence sharing across academic libraries more widely.
Deciding and Designing with Data: Analyzing User Data and Behaviour Trends in Online Learning Objects
Melanie Parlette-Stewart, University of Guelph
Introduction: This paper presents the analysis of online learning objects usage at the University of Guelph Library. In an effort to “meet students where they are at” and develop engaging learning content, the library has created online learning objects such as videos and online help guides that support learning, writing and research skill development. Evidence based library and information practice (EBLIP) offers an opportunity to translate usage data into evidence based decisions and best practices for the creation of online learning objects.
Objective: This study seeks to provide an overview of approaches to data analysis and user behaviour assessment for online learning objects:
Methods: Mixed method approaches were used to carry out this study. User data from Google Analytics and Springshare Libguides were used to assess usage data from YouTube and LibGuides from September 2015 to August 2016. In addition, questionnaires and user experience testing were conducted to provide additional context and understanding of the data.
Results: Online learning object usage data provides an opportunity to discover how students are using online help content. Collected data allows us to engage in ongoing evaluation of trends and enhances best practices. Key findings include the need to connect content explicitly to related courses and to create smaller, bite-sized content, acknowledging the low amount of time spent with objects.
Conclusion: This study will help librarian and student support practitioners build awareness of data available to them and understand how they may use these tools to support evidence based decision making.
Embedding Resource Sharing Evidence in Cooperative Collection Development
Thomas Teper, University of Illinois at Urbana Champaign
Joe Lenkart, University of Illinois at Urbana Champaign
Esra Coşkun, University of Illinois at Urbana Champaign
Mara Thacker, University of Illinois at Urbana Champaign
Objective: Collaborative resource sharing sustains communities of students and scholars across regions and countries. This vital component of research libraries has many dimensions, which include local and national constituencies, consortia agreements, budgetary and administrative support, subject experts, and international publishing trends. Resource sharing through interlending and duplication services empowers community building by supporting academic institutions lacking strong area studies collections by providing regional access to unique collections. Based on resource sharing data from a single research library, this paper aims to provide an evidence-based framework for cooperative collection development within area studies collections, and addresses the following questions:
How should research libraries balance local needs, minimize duplication of effort, and maximize collaboration in collection development?
Methods: This study examined the use of area studies materials by assessing five years of outgoing lending data and local use from a single research library. Moreover, this case study also investigated the correlation between outgoing Less Commonly Taught Language materials and titles published abroad.
Results: Based on the initial examination of the dataset revealed that a lending program, supported by a strong emphasis on collection building and support services, does indeed support communities of researchers nationally as well as local constituents. In addition, the preliminary findings do emphasize that language is a problematic value for assessing area studies collections.
Conclusion: As the role of resource sharing continues to growth nationally and internationally, identifying key areas for collaborative collection development becomes vital to the long term mission of research libraries. This study is a first step in utilizing resource sharing evidence from a single lending location to formulate possible models for robust cooperative collection development model for area studies.
Embracing Evidence Based Collecting in the Health and Behavioral Sciences
Carolyn Schubert, James Madison University
Michael Mungin, James Madison University
Objective: Evidence based collection development requires combining multiple sources of information, such as usage data, subject librarian expertise, user values and preferences, and external evidence from librarianship research. Through triangulation, librarians can build a collection that is responsive, balanced, and relevant to address both established and emerging curricular trends. In the field of health and behavioral studies, growth in interprofessional education supports collaborative collecting practices. In 2014-2015, health and behavioral studies librarians at James Madison University (JMU) piloted a collapsed monographic selection project to facilitate better interdisciplinary collecting. This project discusses an evidence-based approach to evaluating and updating approval profiles for
collaborative monographic purchasing and how the pilot affects collecting practices.
Methods: Subject librarians reviewed the literature regarding approval profile performance to determine an objective benchmark. Next, they reviewed the accepted and rejected approval titles from 2012-2015 to compare performance to recommended standards. Then, they retrieved the circulation and download statistics for approval items purchased in 2012-2013 to review actual usage of accepted titles.
Results: The literature indicated that approval profiles should have around a 10% rejection rate. At JMU, on average, 25% of books were rejected across disciplines and across each year. Psychology and Health Sciences had the largest areas of book rejection in comparison to Communication Sciences & Disorders, Kinesiology, Nursing, and Social Work. Of the 507 titles selected in 2012-2013, 42% circulated or were downloaded by users.
Conclusion: Assessment of the approval profile found that the profile did not save librarians time with accurate collecting nor clearly meet user needs. When working in a collaborative model, librarians accept and reject titles across a broader array of LC classifications than when previously working in siloed approval profiles. The librarians applied what they learned from these disparate data sources to develop a single approval profile for more efficient collecting.
Using Peer Budget Allocations as Benchmarks for Local Expenditures
Brian Keith, University of Florida
Joseph Salem, Pennsylvania State University
Objective: This paper presents an analysis of financial data from dozens of North American academic libraries. The purpose was to search for patterns in the allocation of funds to different types of library expenditures: labor, materials and other operational expenditures. The relationships would form the basis for assessing individual library funding allocations in these categories in order to identify variance from model expectations. Additionally, these relationships serve as a basis for comparing strategic decisions amongst peer institutions. The analysis also investigates shifts in budget allocations across years before, during and after the recent recession. The overall value of this research is to provide normative evidence for reference in strategic decision making.
Methods: The study began with a comparative analysis of the statistical relationships in expenditures of universities and their libraries. This evolved to a comparison of expenditures within libraries between categories: labor, materials and other operational expenditures. The researchers determined peer groups of comparable institutions, based on a number of institutional characteristics, and relied upon linear regression to investigate correlations. In the subsequent analysis, the researchers apply hierarchical linear regression to 10 years of data in order to assess the trends over time.
Results: The analysis of the relationship between expenditures of universities and their libraries yielded high correlations which were consistent over time. The analysis of expenditures within libraries across budget categories resulted in a useful model for comparative purposes. In both series, predictive relationships between factors exist.
Conclusion: Strategic decision making is preferably based upon statistical evidence. An important set of reference points is presented in these findings. Relationships exist between funding of universities and their libraries, as do norms between allocations of funds for different purposes within libraries. These both serve as useful reference in informing resource allocation decisions.
Finding the Silver Lining… in the Serials Budget Crisis
M. Brooke Robertshaw, Oregon State University Libraries & Press
Kerri Goergen-Doll, Oregon State University Libraries & Press
Michaela Willi Hooper, Oregon State University Libraries & Press
Objective: During the 2015-2016 academic year an R1 university in the Pacific Northwest region of the United States foresaw a potential shortfall in the serials budget in upwards of $1 million dollars. The collections department felt the need to promote transparency around the issue. Initially, they met with library faculty and staff to discuss potential scenarios for approaching cuts if the shortfall came to fruition. As the year progressed it became apparent that the discussion needed to be taken beyond the walls of the library.
Method: During the last two weeks of the school year a team, that included teaching & engagement, scholarly communications, collections, and assessment, held a series of focus groups with disciplinary faculty. The focus groups asked disciplinary faculty questions about how library collections fit into their workflow as researchers, their thoughts on the financial realities facing library collections, and what they thought librarians should be communicating to their colleagues about the economics of scholarly communications. The team used an inductive constant comparative method (Corbin & Straus, 2007) to analyze the data and develop themes.
Result: The team heard three overarching themes through the focus groups: concern about how library collections impact promotion & tenure and grant writing processes, awareness of ingrained practices creating barriers to access, and the need to develop sustainable solutions to this unremitting problem.
Conclusion: The evidence collected through this process is guiding further, more targeted, focus groups with stakeholders. The intention is that the evidence from the focus groups will be embraced by library partners and stakeholders as confirmation of the need to develop long-term solutions to rising journal costs for the university.
Envisioning the Future: Using Evidence to Shape Library Spaces
Rachel Sarjeant-Jenkins, University of Saskatchewan
Troy Smith, Group2 Architecture Interior Design
Objective: This presentation will describe an evidence based project undertaken to develop a Master Plan for an academic library that serves a Canadian medical/doctoral university. The plan focuses on library spaces to support the evolving teaching, learning, and research needs of faculty, staff, and students.
Methods: Evidence was collected using a number of mechanisms: a review of academic library spaces through websites, blogs, and articles; focused faculty and student group feedback sessions; randomized surveys of students, faculty, and staff; consultation booths situated across campus; targeted discussions with library users; email and website feedback; workshops with library employees; and the forming of a representative stakeholder group to serve as a guide and sounding board.
Results: Academic library spaces are focusing increasingly on study, collaborative work, visualization, knowledge creation, and the overall student experience. Knowledge of what was happening in other libraries around the world helped shape ideas for the master plan, inform the work of the stakeholder group, and stimulate discussion at consultation sessions and pop-up booths. Regardless of disciplinary background, student feedback was consistent around space and technology. Faculty focused primarily on resources and services. Library employees spoke of space in its various aspects.
Conclusion: Having evidence of successful library space projects helped expand clients’ views on what a library can be as well as provided support for the inclusion of new spaces within the library. Connecting with library clients through multiple venues helped to build confidence in the final master plan. Library employees brought professional understanding of how spaces are used for research, teaching, learning, and community-building. A master plan that is shaped by clients, library employees, and the broader library community can stand up to scrutiny and questioning, supported as it is by the components of evidence-based library and information practice.
From Library to Center for Learning and Innovation
Deb Ponting, Ormiston College (Twitter: @debponting)
Objective: This paper reviews the redesign of the library at a traditional Preparatory to Year 12 (P-12) college, to transition to a modern Centre for Learning and Innovation that supports collaborative learning and is an information hub for students and staff. The paper focuses on how student and staff perceptions of the new design have impacted on their use of the spaces.
Methods: An ethnographic approach was taken to capture perceptions of the use of the new spaces. Interviews with staff and students were analysed to gain insights that helped inform the design of a new centre.
Results: Findings revealed that the comfort and flexibility of the furniture, as well as the metaphorical and physical spaces, have led to more students and staff using the centre both in free and timetabled hours. The study was limited by lack of time during the semester to interview staff and students without disrupting their studies. Insights gained will be used to inform the next stage of the library redevelopment. In addition, incidental comments from staff, students and visitors indicated that the new spaces were inviting and were keeping pace with changing user needs and expectations.
Conclusion: The study contributes to research concerning the use of library space for student learning in P-12 pre-tertiary education. In addition, it provides information that can inform and guide future development of library spaces in other P-12 institutions.
Designing Service Models through Embedded Evidence: Area Studies Reference Services in Research Libraries
Joe Lenkart, University of Illinois at Urbana Champaign
Jen-Chien Yu, University of Illinois at Urbana Champaign
Objective: Area studies reference services play an integral role within the community of academic libraries, by supporting referral infrastructures, training students and faculty, providing research consultations, and filling the gap between chat and email reference. These services have already undergone service realignments, changed their scope, and capabilities. Within the last ten years, significant growth in resources from around the world has rejuvenated area studies services. These developments in resources and technology are facilitating innovation in reference services. As academic institutions unveil plans to internationalize college campuses, the ability to design services to reflect short-term and long-term goals of community building is essential to sustaining scholarly communities. The objective of this study is to utilize evidence in shaping future models for area studies units and services in research libraries.
Methods: This study examined area studies reference services at a four-year institution. Using reference transactional data from three academic years, this paper will focus on duration of the transaction, language, question type, patron type, subject and geographic areas, staffing, and expertise at these service points.
Results: By conducting transactional analysis on the dataset from area studies service points, this study made several observations, which will benefit service managers and library administrators in designing and allocating resources for research services. These include: a functioning referral system, staffing models at service points, data-input training, and subject expertise.
Conclusion: As specialized service environments continue to take on more complex research inquiries, which in turn demands expertise and training. Service coordinators are increasingly faced with the need to maintain existing services, while facing funding uncertainties. The evidence from this study will encourage library administrators to invest in specialized service points by embracing innovative staffing models and evidence generated at these service points.
A Mixed Methods Approach to Iterative Service Design of an In-person Reference Service Point
Kyla Everall, University of Toronto Libraries
Judith Logan, University of Toronto Libraries
Objective: To implement an evidence-based service design framework for a public service point at the University of Toronto’s largest library.
Methods: After completing a literature review of service models in academic libraries, we analyzed service point user interaction data from a staffing and service provision perspective to determine how efficiently the current model was deploying staff expertise. We conducted focus groups with reference service providers to identify the strengths and weaknesses of the current service model. We extended previous user experience studies that identified needs, pain-points, and areas for improvement regarding our in-person library services.
Results: Based on our evaluation, we determined that our current location, configuration, and staffing model were not meeting user needs or making the best use of staff time. As a result, we moved our reference desk to a more central location, rebranded it, and integrated it with other library services in the building. Once the service launched, we made incremental changes in response to data we continuously collected. We saw an increase in users served and time spent helping University of Toronto affiliated users. Intra and interdepartmental staff communication greatly improved resulting in better, holistic user services.
Conclusion: In order for reference services to be most effective, we must shift to a user-focused, evidence-based design model. This project reinforced that service points do not exist in a silo. Instead, they should be seen and designed as part of the library’s overall service ecology.
A Study of Foreign-Born Students and Super-Diversity: Embracing the Evidence
Frans Albarillo, Brooklyn College, City University of New York (Twitter: @Fransnyc)
Objective: This presentation aims to examine and create discussion around some key qualitative and quantitative findings of a study on foreign-born students and library use at a large public liberal arts university in New York City using the framework of super-diversity, a concept developed by social anthropologist Steven Vertovec (http://www.mmg.mpg.de/research/all-projects/super-diversity/) that has not been written about in the field of library and information sciences. As a framework, the concept works well to explain the data gathered in the study.
Methods: The data gathered from this study is based on a survey and small group interviews conducted in 2014 as part of a pilot study to create a large dataset focused exclusively on foreign-born students and their library use patterns. The survey data (N=92) looks at language use, academic and public library use, format preferences, and cultural perceptions around an American style of research.
Results: Some interesting results include:
These suggestive results are cross-tabulated by gender, age, immigration status, and race and ethnicity. Qualitative response text gathered in the survey is coded and analyzed.
Conclusions: The data gathered in this study is very useful in assembling and exploring patterns of usage and cultural and linguistic perceptions and preferences about the academic library. This is especially important if a library is looking to innovate or improve services around the ACRL 2012 diversity standards. The study had several limitations, including missed variables, which the researcher will also discuss. The primary goal of this research is to create a qualitative and quantitative tool to administer to several libraries so that the data can be compared analyzed for two-year and four-year City University of New York institutions.
Understanding Strategically Important Populations: Assessing the Information Practices, Needs, and Perceptions of International Graduate Students
Christine Wolff-Eisenberg, Ithaka S+R
Lisa Hinchliffe, University of Illinois
Objective: International student enrollment in the United States has grown considerably in the last decade. International graduate students now represent a significant portion of graduate students in the United States and graduate students represent a significant portion of the total number of international students. Colleges and universities increasingly rely on international students for tuition and fee revenue, which is often significantly higher than the tuition and fees paid by domestic students. International enrollment is not evenly distributed across institutions or geographic areas and indeed international enrollment is quite prominent at some institutions. Only eight universities – one of which is the University of Illinois at Urbana-Champaign – have more than 10,000 international students and as such, international students represent a strategically important population at the University of Illinois Library at Urbana-Champaign.
Methods: To gain insights into the information practices, needs, and perceptions of the international graduate students at Illinois, the University Library partnered with Ithaka S+R to develop an international module for the Ithaka S+R Graduate/Professional Student Survey to gather additional data from the international students.
Results: Some of the findings from the survey confirm results previously reported in the literature, while other findings raise questions about commonly-held beliefs about language difficulties and prior library experience shaping library use and research practices. In addition, this is the first known library user study to investigate the impact of whether students completed their undergraduate degrees in the United States or in another country on their information behavior and perceptions.
Conclusion: This session will share findings from the survey and how those findings can inform the Library’s service strategy for a strategically important population. Session attendees have the opportunity to reflect on their own institutions’ strategically important populations and their libraries’ efforts to meet their needs.
Let us Count the Ways: How Evidence-based Decision Making is Being Embedded in a Public Library
Soleil Surette, Edmonton Public Library
Objective: This paper presents a case study of evidence-based practice in a Canadian public library. Edmonton Public Library (EPL) is located in Edmonton, Alberta, Canada. Over the past 8 years EPL has championed evidence-based practice through the establishment of a Planning, Research and Assessment department and leadership at the executive level. How does EPL practice evidence-based decision making? At EPL, EBLIP begins with understanding the goals of the library and the need for accountability as a publicly funded institution. We ask the question what is the evidence of our success? How do we demonstrate that we are doing is what we said we would do? How do we ensure that our decisions are driving our success?
Methods: Evidence-based decision making is treated as a multi-faceted undertaking at EPL, incorporating experience of other libraries both published and unpublished, internal data collection through usage (visits, circulation, program attendance) surveys, interviews/focus groups, and observational data most commonly.
Results: Evidence-based decision making is becoming well established within EPL. A culture of evidence-based decision making has spread relatively rapidly, but has also exposed areas where more work is needed. EPL is excellent at understanding the need for evidence-based decisions, is good at collecting data/looking for existing evidence in order to make decisions, but has challenges in staff capacity for interpreting data, and recognizing how much evidence is needed for a decision.
Conclusion: EPL has been quite successful in embedding evidence-based library and information practice within its culture, but we need to ensure that staff have the skill set needed to connect the data to the decision. At EPL, we also now perceive a need to evaluate how much time should be spent gathering evidence for various decisions. We are beginning to ask the question – what is the impact of the decision.
We Are The Evidence: Critical Reflection as Personal Evidence
Rick Stoddart, University of Idaho Library
Objective: Librarians are the personification of librarianship. As such, evidence of effective library practice is embodied in librarians, in the lives they lead, and in the actions they take.This paper will explore how critical reflection informs meaningful and purposeful library practice through creating new forms of personal evidence.This personal evidence can then be incorporated into evidence-based decisions about individual library practice.
Methods: This paper will report on two North American studies that the researcher has helped facilitate. The first used a critical reflective methodology called currere with a cohort of practicing academic librarians to examine their personal connections to librarianship. The second study focused on gathering autoethnographies from a group of North American academic librarians. The paper will compare how participant reflection experiences inform their library practice and align to the evidence cycle of Assess, Ask, Acquire, Appraise, and Apply.
Results: Participant experiences in these studies indicate that generating and critically examining forms of personal evidence can have profound effects on library practice and attitudes towards librarianship.
Conclusion: Reflection creates forms of personal evidence that can be incorporated into evidence-based decision making as to how each librarian chooses to embody librarianship.
Research Ethics That Support Innovation: A Multi-Method Study of Academic Librarians’ Researching Human Subjects Using Social Network Sites
Laura Spears, University of Florida
Brian Keith, University of Florida
Overview: The use of observable information and readily available digital data about student user behavior is undeniably attractive as academic library value, return on investment and impact on student learning outcomes are increasingly scrutinized. The drive for increased assessment activity is the result of a push for evidence-based decision-making, increased higher education accountability, cohesive institutional and national criteria; and, demonstrations of library impact on student success and achievement (ACRL, 2016). In an age of vast electronic systems collecting big data on system users, the application of decades old research standards and protections for human subject studies such as informed consent, has been widely debated in mainstream media, academic journals and social media for its effectiveness. However, despite the increasing demand for scholarly research in libraries, academic librarians may lack the research training and support to effectively meet evidence-based decision making requirements and to balance human subject research standards.
Methods: This study examines both the library workplace and professional education landscapes relative to 1) librarian research education and training, organizational support and professional development; and 2) institutional infrastructure for research review. This multi-method approach includes a survey of academic librarians; and, an examination of IRB standards and digital data collection research practices in ARL libraries.
Results: As a work in progress, this study’s preliminary findings suggest librarian research competency and institutional support that lag behind today’s increasingly complex digital data landscape. This study will interest librarians, researchers and scholars interested in or required to produce scholarship; and, library administrators interested in research compliance and sound research method practices.
Conclusion: This study contributes to the literature documenting academic library scholarly research and assessment development, and introduces ethical considerations of digital human subject data into the library scholarship and graduate library education.
Evidence of Impact: Embracing Data in Strategic Directions
Kaitlin Springmier and Elizabeth Edwards, University of Chicago
Michelle Bass, Stanford University, Lane Medical Library (Twitter: @mbbassdrlib)
Objective: In 2016, the University of Chicago Library developed a set of strategic directions that would guide its work for the next three years. Having identified these directions, the Library then needed to articulate benchmarks and communicate these plans to its stakeholders. In order to identify best practices or exemplary models for this type of communication, the Library’s Assessment Planning Team performed an environmental scan of over 70 peer institutions to explore how an academic institution can and should communicate impact to stakeholders.
Methods: Peer institutions were identified using ARL ranking, consortial memberships, and attendance at assessment conferences. Team members performed content analyses on the websites of these institutions in order to identify trends in staff dedicated to library assessment and data communication practices in public-facing strategic plans, annual reports, or assessment summaries.
Results: Team members were surprised that the presence of a robust library assessment program did not guarantee that library data would be available for external audiences. Only 8% of libraries reviewed made benchmarking or assessment data available on their websites in annual reports or strategic planning documents. The majority of institutions either made no data available, or the available data was not integrated into the library’s strategic planning documents or annual reports.
Conclusion: While libraries actively collect data about the use of collections and services, the actions and behaviors of patrons and staff in physical and virtual spaces, and expenditures from operations to materials, many do not share these data with their stakeholders. It is crucial that libraries find ways to use these data to communicate our value, impact, and success. Through the investigatory project, we have identified a few ways that academic libraries use data to advocate for the library.
Exploring a Triage Reference Model for a Chat Service in an Academic Library
Hae Min Kim, Drexel University
Chat is an important online reference service in Drexel University Libraries with no reference desk and low email requests. The purpose of this study is to analyze chat data in order to explore a triage reference model in terms of service accessibility and flexibility, on its usage decrease after modifying the service access points and staff operation. It aims to assess the model and to suggest improvement for the chat service management. This case study examined chat data from January 2014 to December 2015. For the service accessibility, chat metadata were used to find access time of day, waiting time, and referral. For flexibility in responding to users, chat transcripts were analyzed to identify READ (Reference Effort Assessment Data) scale, operators, and transfer from one staff member to another. The chat service usage has decreased with implementing a triage reference model as of June 2014. Referral data showed access points on ‘About’ web pages, which were removed, had about 20% of chat requests, and the usage of new access points have increased. The READ scale and operator data showed liaison librarians served higher scaled questions. Transfer analysis revealed 19% of chat requests transferred. Of that, 53% of chats shifted from library assistants to liaison librarians, and the higher READ scaled questions were transferred to liaison librarians. Quantitative and qualitative analysis of chat data found areas for service improvement in a triage reference model. This study has identified some access points in library websites are needed to expose the service to users efficiently. The results have shown that service operators have been flexible by transfer which connecting users to appropriate staff members. This supports the idea of a triage reference model. This study will be of value to reference managers, operators, and administrators as they consider ways to quantify chat data.
Individualized Research Consultations in Academic Libraries: Useful or Useless? Let the Evidence Speak for Itself
Karine Fournier, University of Ottawa (Twitter: @karoufe77)
Lindsey Sikora, University of Ottawa
Objectives: Academic librarians consistently offer individualized help to students and researchers. However, few studies have empirically examined the impact of individualized research consultations (IRCs). For many librarians, IRCs are an integral part of their teaching repertoire. However, without any evidence of its effectiveness or value, one might ask if it’s worth investing so much time and effort? Our study embraces the evidence by exploring how IRCs have an impact on students' search techniques and self-perceived confidence levels.
Methods: Our population included students in the Faculties of Health Sciences and Medicine at the University of Ottawa, completing an undergraduate or graduate degree, and undertaking a research or thesis project. We used a mixed methods approach, with pre- and post-testing, as well as interviews, being utilized. Participants were invited to complete two questionnaires, before and after meeting with a librarian. The questionnaires consisted of open-ended and multiple choices questions, which assessed students' search techniques, their self-perceived search techniques proficiency, and their confidence level. A rubric will be used to score students' open-ended questions, and self-reflective questions will be coded and analysed for content.
Results: It was decided that 30 completed pre- and post-tests would allow us enough for statistical significance, however, data collection took longer than anticipated. As such, data analysis is not yet completed. We hypothesize that students’ searching techniques would improve once they had met with a librarian, and their confidence levels would also increase, though the range of these levels may vary on many factors
Conclusion: Our hope for this study is to provide empirical evidence that individualized research consultations are an added value to the services provided by librarians. Future research may explore specific techniques to improve search strategies across various disciplines, tips to improve confidence levels, and exploring the viewpoint of librarians.
Exploring Student Use of the Information Commons through Mixed Methods
Susan Archambault, Loyola Marymount University
Alexander Justice, Loyola Marymount University
Objective: Six years ago, a medium-sized academic library built the “perfect” Information Commons space designed to support the millennial student. Modern day usage suggests students are utilizing the Commons in unexpected ways and deviating from traditional “millennial” behavior. Which areas are really used and not used, what furniture configurations are preferred, and what mix of technology, resources, and activities are students engaged with inside of the space?
Methods: 2,443 “direct observations” were made on students who were physically present in the Information Commons during a designated two-week period at selected hours. To better facilitate the observations, the Commons was mapped and divided into five “observation zones.” Each person’s activity and use of library resources and technology was recorded through a Qualtrics form using iPads within each zone. The observations were supplemented with 248 patron surveys and 46 whiteboard poll questions. Data from the three methods was analyzed and synthesized. Results were visualized using Tableau, with filters for zone and variable.
Results: Independent study dominated the space usage. Students valued spaciousness, quiet, privacy, and a clean environment. Students frequently multi-task with additional devices as they simultaneously use the library computers, including cell phones, headphones, and laptops. Also, unattended belongings were frequently observed along with broken electrical outlets. Since the Commons was designed as a social learning space for collaboration, the findings have significant implications for redesign.
Conclusion: This methodology explored usage of a designated space of the library. Evidence from the study paved the way for improvements to the Commons, including study tables to replace underused reference books, a disinfecting wipes dispenser, and a map of all electrical outlet locations to be tested regularly. New furniture to promote noise control is being investigated, including high back arm chairs, sofas with high acoustic side panels, office booths, and privacy booths.
Planning a Collaboration Commons: A Mixed-methods Approach to Inform Design
Steve Borrelli, Pennsylvania State University Libraries
Chao Su, Pennsylvania State University Libraries
Zoe Chao, Pennsylvania State University Libraries
Objective: In preparation for a planned expansion, and renovation transitioning a traditional news and microforms library at Penn State University into a collaboration commons estimated to cost approximately $20,000,000, researchers were charged with investigating the physical workspace needs of students and to assess the need for soft seating to inform final design recommendations.
Methods: The multiple methodologies utilized included student focus groups informed by local results of the Ithaka Survey of Undergraduates, interviews with library personnel and students, an observational study of soft seating usage within existing Knowledge Commons, flip chart prompts, and results of recent space studies.
Results: The majority of Penn State students come to the Libraries to be productive, often working on multiple assignments in one sitting. They desire a variety of spaces and select workspaces based on a number of factors including variety of work, convenience, food availability, and workspaces equipped to meet their needs. Personal work surfaces were described as “spread out,” having multiple devices, snacks, and their cell phone out. Observation data showed an average of 2.28 devices out per observee (n=480). Soft seating was noted as comfortable with aesthetic appeal but little productive value. Observation data showed soft seating used for productive activities at a rate of 2 to 1 over non-productive activities and utilized by individuals over groups at a rate of 15 to 1.
Conclusion: Findings were determined using a process of corroboration across employed methodologies and integrated into final design recommendations. Students come to the Libraries to be productive, but report a general lack of seating to meet productivity needs. Participants want large tables as work surfaces. Facility enhancements for the collaboration commons should include well-designed and equipped spaces for productivity over comfort and the design and furnishings should communicate the types of intended activities and expected behaviors.