? WELCOME, LYNN.
 >> THANK YOU SO MUCH, JENNIFER.
 HELLO, EVERYONE.
THANK YOU FOR JOINING US TODAY.
IT'S MY PLEASURE TO BE HERE WITH YOU.
 I THOUGHT I'D JUST GIVE A LITTLE OVERVIEW OF ASSESSMENT.
AND EVALUATION.
 AND HAVE YOU CONTEMPLATE AND THINK ABOUT SOME OF THESE TERMS.
BUT BEFORE I START, I WANT TO TALK A LITTLE BIT ABOUT OCLC AND OUR RESEARCH AREA OF USER STUDIES.
I'M A SENIOR RESEARCH SCIENTIST HERE.
AND I'M ALSO THE DIRECTOR OF USER RESEARCH.
AND SO ANY RESEARCH THAT ANY OF US DO SHOULD ACTUALLY BE CENTERED ON INDIVIDUALS, THE USERS OR POTENTIAL USERS OF OUR SERVICES, OUR PRODUCTS, ANY OF OUR PROGRAMS THAT WE MAY HAVE FOR THEM BECAUSE IT SHOULD BE ALL ABOUT REACHING OUT TO OUR COMMUNITIES AND ENGAGING INDIVIDUALS.
AND YOU'LL SEE THAT FROM THIS SLIDE THAT WE TALK ABOUT EMBEDDING THE LIBRARY AND COMMUNITY WORK FLOWS.
AND ONE OF THE THINGS THAT -- 
ONE OF THE REPORTS THAT WE DID WAS PUTTING THE LIBRARY IN THE LIFE OF THE USERS.
AND NOW I WOULD ADD AND POTENTIAL USERS.
AND ALSO, ENGAGING INDIVIDUALS IN CONTEXT.
WHY?
BECAUSE THE WAY WE RESPOND, THE WAY WE ENGAGE, THE WAY WE REACT IS VERY DEPENDENT UPON THE CONTEXT AND THE SITUATION AT THAT MOMENT.
AND SO, YOU KNOW, MANY OF YOU MAY BE SITTING IN YOUR OFFICES ALONE.
YOU MAY BE MULTITASKING RIGHT NOW.
 YOU MAY HAVE YOUR WORK STATION, YOUR LAPTOP, BUT YOU MAY ALSO BE CHATTING TO SOMEONE ON YOUR PHONE.
 THIS WILL BE VERY DIFFERENT IF YOU'RE IN YOUR CAR.
HOPEFULLY YOU'RE NOT TEXTING.
BUT YOU MAY HAVE YOUR PHONE ON SPEAKER.
AND BE TALKING TO SOMEONE OR ASK SIRI FOR DIRECTIONS OR GOOGLE MAPS.
SO WE HAVE TO BE THINKING ABOUT THOSE CONTEXT AND SITUATIONS SO IT MAKES OUR LIFE MUCH MORE COMPLEX AND DIFFICULT WHEN TRYING TO ASSESS AND EVALUATE SOME OF OUR SERVICES AND PROGRAMS.
SO THESE ARE SOME OF THE QUESTIONS THAT I WANT YOU TO THINK ABOUT THROUGHOUT THIS -- 
THIS WEBINAR.
BUT THROUGHOUT THE SERIES.
 AND SO THINK ABOUT WHAT YOUR DEFINITION OF ASSESSMENT REALLY IS.
 AND WE'LL TALK ABOUT OUR DEFINITIONS BUT YOU NEED TO BE THINKING ABOUT ASSESSMENT IN TERMS OF YOUR LIFE, YOUR LIBRARY.
 WHAT COMES TO MIND WHEN YOU HEAR THE TERM "ASSESSMENT"?
NOW, I HOPE IT'S NOT THIS ARGHH, I DON'T WANT TO DO THAT!
I HOPE IT COMES TO MIND IS, WOW, LEARNING WHAT PEOPLE ARE DOING, WHAT THEY WANT.
HOW THEY ENGAGE WITH WHAT WE'RE PROVIDING.
 WHAT BENEFITS DO YOU SEE FOR ASSESSMENT AND WHAT ARE YOUR CONCERNS?
 AND I HAVE A FEELING I MAY KNOW SOME OF YOUR CONCERNS BUT WE'LL HIT ON THOSE TODAY AS WELL.
 SO WHY ASSESSMENT?
 WELL, NUMBER ONE, THAT WE HAVE TO REMEMBER THAT THE TRADITIONAL WAY THAT WE HAVE USED TO TALK ABOUT OUR IMPACT OR OUR VALUE TO OUR COMMUNITIES DON'T TELL THE WHOLE STORY.
 AND SO ASSESSMENT WILL ANSWER, YOU KNOW, WHAT OUR USERS, POTENTIAL USERS, STAKEHOLDERS WANT/NEED, HOW OUR SERVICE, OUR EVENTS, OUR PROGRAMS, OUR INITIATIVES MIGHT BETTER MEET THESE INDIVIDUALS' WANTS AND NEEDS.
 IS WHAT WE DO WORKING?
WE ALL -- I HAVE NEVER HEARD ANYONE IN ANY LIBRARY OR AS A MATTER OF FACT IN ANY WORK SITUATION SAY -- "WE HAVE ALL THE FUNDS, THE RESOURCES WE NEED.
WE HAVE ALL THE TIME IN THE WORLD SO WE CAN JUST DO EVERYTHING AND ANYTHING."
HOWEVER,  WHAT I USUALLY HEAR IS -- WE HAVE LESS STAFF THAN WE USED TO.
 OUR BUDGET IS CONSTANTLY BEING REVISED AND SCRUTINIZED.
AND I HAVE TO DO MORE WORK SO I DON'T HAVE ENOUGH TIME.
THOSE ARE THE THINGS THAT I HEAR.
SO WHY PUT OUR TIME AND EFFORT AND RESOURCES INTO INITIATIVES THAT AREN'T WORKING.
 OR THAT PEOPLE REALLY DON'T WANT?
 SO IT'S TELLING US WHAT WE CAN DO BETTER.
WHERE WE MAY HAVE PROBLEM AREAS AND AGAIN, IT'S NOT JUST GIVING THESE NUMBERS AS I'M GOING TO GET INTO IN THE NEXT SLIDE.
 NUMBERS ABOUT HOW MANY PEOPLE WALK THROUGH THIS DOOR.
THAT DOESN'T TELL A STORY.
IT MIGHT SAY THAT YOU NEED A LARGER DOOR, I DON'T KNOW.
 THE IMPORTANCE OF ASSESSMENT AND THIS IS -- AND I USE THE QUOTE FROM 2010 AND THERE ARE EARLIER AND LATER QUOTES.
BUT I WANT TO DEMONSTRATE THAT THIS IS NOT NEW.
 AND IT'S SOMETHING WE HAVE BEEN THINKING ABOUT FOR QUITE A LONG TIME.
 AND MANY PEOPLE HAVE BEEN DOING.
 BUT WE NEED TO DO MORE.
 AND ASSESSMENT IS NEVER DONE.
 THERE'S A NEIL YOUNG SONG THAT SAYS "RUST NEVER SLEEPS."
AND I WANT TO CHANGE THAT TO ASSESSMENT AND EVALUATION NEVER SLEEPS.
 THIS CAME FROM THE ACRL VALUE OF ACADEMIC LIBRARIES REPORT THAT WAS PUBLISHED IN 2010.
 AND AT THAT TIME, IT WAS SAYING LIBRARIANS ARE INCREASINGLY CALLED UPON TO DOCUMENT AND ARTICULATE THE VALUE OF ACADEMIC AND RESEARCH LIBRARIES AND THEIR CONTRIBUTION TO INSTITUTIONAL MISSION AND GOALS.
 THIS IS NOT JUST LIMITED TO ACADEMIC AND RESEARCH LIBRARIES, AS MANY OF YOU KNOW.
THIS IS ANY ORGANIZATION, INSTITUTION OR LIBRARY.
SO THE STEPS IN THE ASSESSMENT PROCESS AND WE WILL GO THROUGH THIS AND MY COLLEAGUE, KARA REUTER IS HERE TODAY AND SHE WILL BE TALKING ABOUT THE WHOLE STEPS AND THIS ASSESSMENT PROCESS AND WHAT SHE DID AT THE WORTHINGTON PUBLIC LIBRARY.
SO TO PUT THIS MORE PRACTICAL APPROACH TO THIS AND TO HELP YOU UNDERSTAND WHAT CAN BE DONE, MAYBE WHAT SHOULD BE DONE.
 SO NUMBER ONE, WHY ARE WE DOING THIS?
 YOU NEED TO HAVE A PURPOSE AND YOUR LIBRARIANS, YOUR STAFF, YOU NEED TO IDENTIFY WHY YOU ARE DOING THIS.
SO WHAT DO YOU WANT TO KNOW?
 WHAT DO YOU ALREADY KNOW?
 WHAT ARE YOU GOING TO DO WITH THESE RESULTS?
AND WHAT OUTCOMES DO YOU WANT TO ASSESS?
THOSE ARE THE QUESTIONS THAT YOU NEED TO ASK.
 YOU ALSO NEED TO ASK WHO IS GOING TO DO THIS?
 BECAUSE YOU NEED TO HAVE A TEAM OF PEOPLE WHO WILL WORK WITH YOU.
 ALSO, HOW.
YOU NEED TO CHOOSE THE MODEL, THE APPROACH, THE METHOD OF GETTING TO THE INFORMATION THAT YOU NEED TO ANSWER YOUR QUESTIONS.
NOW, I OFTEN HAVE PEOPLE COME TO ME AND SAY I WANT TO DO A SURVEY AND I'LL SAY OH, FINE.
YOU WANT TO DO A SURVEY.
BUT TELL ME WHAT IS YOUR QUESTION?
WHAT DO YOU WANT TO KNOW?
AND THEN WHEN THEY TALK ABOUT THEIR QUESTION AND WHAT THEY WANT TO KNOW, I'LL SAY, YOU KNOW, DO YOU THINK THE SURVEY IS REALLY THE MOST APPROPRIATE DATA COLLECTION METHOD FOR THAT?
AND SO WE NEED TO THINK IN THOSE TERMS OF FIRST, WHAT DO YOU WANT TO KNOW?
WHAT ARE YOUR QUESTIONS?
AND THEN WORK FROM THAT.
AND THEN THAT LAST ONE WITH THE EXCLAMATION POINT, COMMIT!
THIS IS SOMETHING THAT IS VERY IMPORTANT.
IT'S THAT TRAINING, THAT PLANNING, AND I WAS ONCE TOLD AS A DOCTORAL STUDENT THAT IF YOU ASK A QUESTION AND YOU GET AN ANSWER, BE SURE THAT YOU'RE WILLING TO DO SOMETHING AND TO MAKE A CHANGE BASED ON THAT ANSWER, IF NECESSARY.
IF NOT, DO NOT ASK THE QUESTION.
SO THAT'S THAT WHOLE COMMITMENT.
SO THINK ABOUT THAT AS WELL.
 NOW, I'M NOT GOING TO READ THESE QUOTES TO YOU.
BUT YOU'LL SEE THAT THIS WAS ANOTHER QUOTE FROM ALA/ACRL IN 1998.
AND SO AGAIN, TALKING ABOUT HOW INDIVIDUALS CAN CHANGE IN SOME WAY BECAUSE OF SOMETHING THAT WE HAVE OFFERED, INITIATIVES THAT WE HAVE HAD IN OUR LIBRARIES.
AND THAT'S WHAT WE WANT TO IDENTIFY.
 THAT'S WHAT WE WANT TO ARTICULATE.
WHY?
BECAUSE WE'RE VYING FOR BUDGET DOLLARS.
WE'RE VYING FOR ATTENTION.
 NOT ONLY FROM OUR BOARD, OUR UNIVERSITY ADMINISTRATORS, BUT WE'RE VYING FOR ATTENTION FROM OUR USERS AND POTENTIAL USERS.
AND SO WE NEED TO REALLY BE THINKING ABOUT THESE OUTCOMES.
 NOW, I JUST WANT TO BRIEFLY OUTLINE OUTPUTS AND INPUTS.
 I ALLUDED TO THIS EARLIER.
YOUR OUTPUT.
HOW MANY OF YOU THINK ABOUT THIS, HOW MANY OF YOU CALCULATE AND DOCUMENT THE NUMBER OF REFERENCE QUESTIONS THAT YOU HAVE ANSWERED.
PROBABLY MANY OF YOU.
 HOW MANY OF YOU EVALUATE THE NUMBER OF REFERENCE QUESTIONS YOU'VE ANSWERED CORRECTLY?
 OR TO MEET THAT NEED OF THAT INDIVIDUAL WHO ASKED THAT QUESTION.
 DID YOU ANSWER THE QUESTION THAT WAS ASKED?
 SOMETIMES WE DON'T.
AND WE HAVE DATA ON THAT FROM VIRTUAL REFERENCE SERVICES.
SO YOU REALLY -- THIS COUNTING IS GREAT IF YOU WANT TO SAY, IF YOU WANT TO PRODUCE SOME NUMBERS.
BUT IT DOESN'T, AGAIN, TELL THE WHOLE STORY.
AND WHAT DOES IT MEAN?
WHAT'S THE IMPACT OF ANSWERING THOSE REFERENCE QUESTIONS TO THE PARENT ORGANIZATION OR INSTITUTION?
 AND SO ONE OF THE THINGS THAT YOU REALLY SHOULD BE THINKING ABOUT IS WHAT IS THE ORGANIZATION'S MISSION, GOALS.
WHAT'S THE INSTITUTION'S MISSION GOALS?
AND THEN START LOOKING AT THE INITIATIVES THAT YOU SHOULD HAVE TO MEET THESE GOALS AND TO HAVE AN IMPACT ON ME.
YOU CAN QUANTIFY, YOU KNOW, THE WORK THAT YOU'VE DONE AS FAR AS COLLECTIONS AS WELL.
YOU KNOW, HOW MANY RESOURCES HAVE WE ADDED, HOW MANY EVENTS HAVE WE HAD?
HOW MANY PEOPLE WALK THROUGH OUR DOOR?
BUT AGAIN, HOW HAS THAT CHANGED SOMETHING?
HOW HAVE THOSE INDIVIDUALS CHANGED AND WALK OUT THAT DOOR AND SAID MY LIFE IS BETTER NOW?
SO THOSE ARE THE THINGS WE HAVE TO TALK ABOUT AND THINK ABOUT AND DEMONSTRATE.
THE OUTPUTS ALONE DO NOT RELATE TO THE OVERALL EFFECTIVENESS OF WHAT WE'RE DOING.
SO THE INPUT.
THAT'S THAT TIME WE PUT INTO THAT.
THOSE RESOURCES THAT WE PUT IN.
 AND JUST ARTICULATING THAT ALSO GIVES A VERY SKEWED PICTURE OF WHAT WE'RE DOING AND THE VALUE OF WHAT WE'RE DOING.
 SO I GUESS I SHOULD SAY THESE ARE THE PRINCIPLES FOR APPLYING OUTCOMES ASSESSMENT ACCORDING TO CONNOWAY.
THESE ARE THINGS THAT I'VE COME UP WITH OVER THE YEARS.
NUMBER ONE, CENTER ALL AROUND INDIVIDUALS.
ABOUT THE PEOPLE.
HERE WE HAVE USERS BUT AGAIN, I WANT TO SAY THINK ABOUT THOSE PEOPLE WHO DON'T USE OUR SERVICES.
OR USE YOUR SERVICE INFREQUENTLY.
AND CENTER ON THEM.
TARGET ONE OF THESE GROUPS.
 YOU ALSO NEED TO ASSESS THE CHANGES IN THE WAY PEOPLE USE THE SERVICES.
THE RESOURCES.
BECAUSE THINGS HAVE CHANGED AND THEY CONTINUE TO CHANGE.
RELATE THESE INPUTS, THE HOURS, THE PROGRAMS, RELATE THESE AND PUT THESE BACK ON TO THESE ARE BEST PRACTICES.
WHY?
WE DID THIS.
 AND THIS IS HOW IT CHANGED SOMETHING WITHIN OUR COMMUNITY OR WITH INDIVIDUALS.
YOU NEED TO USE A VARIETY OF METHODS TO COME UP WITH SOME CONCLUSIONS.
AND SO IT'S GOOD TO COLLECT INFORMATION IN DIFFERENT WAYS.
 SO, YOU KNOW, YOU CAN SELECT INFORMATION BY WALKING THROUGH YOUR LIBRARY AND OBSERVING.
THAT'S ONE WAY OF COLLECTING INFORMATION.
OBSERVE HOW PEOPLE WERE USING YOUR SPACES.
WHAT THEY'RE DOING IN YOUR SPACE.
THAT SHOULD HELP YOU AS WELL.
WHEN YOU ARE PLANNING YOUR ASSESSMENT ACTIVITY, YOU NEED TO CHOOSE A SMALL NUMBER OF OUTCOMES.
AND WE WANT TO KNOW IT ALL.
AND THAT'S THE PROBLEM.
 WE KNOW A LOT.
 SO ONCE WE SIT DOWN AND ARTICULATE WHAT WE ALREADY KNOW, LOOK AT THE GAP.
AND THEN DECIDE WHAT YOU SHOULD REALLY CONCENTRATE ON.
YOU DO NOT HAVE TO ADDRESS EVERYTHING.
 THEN YOU NEED TO ADOPT A CONTINUOUS PROCESS FOR DOING THIS.
WHY?
BECAUSE THINGS CHANGE.
 SO IT'S AN ONGOING PROCESS.
AND IT'S THIS PROCESS THAT WE BETTER WANT TO UNDERSTAND, IDENTIFY AND IMPROVE INITIATIVES.
 WE ARE VERY BUSY WITH THE DAY-TO-DAY WORK.
AND OFTEN, PEOPLE WILL TELL ME, OH, ASSESSMENT IS JUST ONE MORE THING.
IT'S A BURDEN.
AND IT SHOULD NOT BE.
IT SHOULD BE SOMETHING THAT'S PART OF OUR STRATEGIC PLANNING.
IT SHOULD BE SOMETHING EXCITING AND SOMETHING THAT WE LOOK FORWARD TO.
AND YOU CAN -- YOU SHOULD ALWAYS BUILD ON WHAT YOU HAVE ALREADY DONE.
 SO YOU'RE NOT STARTING OVER EVERY YEAR FROM ZERO.
 AND I THINK THAT'S ANOTHER THING THAT'S VERY IMPORTANT.
 NOW, I'M GOING TO TURN OVER THIS SESSION TO MY FRIEND AND COLLEAGUE, KARA REUTER.
AND KARA IS THE DIGITAL LIBRARY MANAGER AT WORTHINGTON LIBRARIES IN WORTHINGTON, OHIO.
SO KARA, I'M TURNING IT OVER TO YOU.
 >> HI, EVERYONE!
 SO LET ME JUST DIVE RIGHT IN HERE.
 SO IF YOU CARE ABOUT CUSTOMER SERVICE, YOU'RE PROBABLY INFORMALLY ASSESSING YOUR LIBRARY SERVICE ALL THE TIME.
YOU'RE HEARING ANECDOTES FROM COLLEAGUES ABOUT INTERACTIONS WITH PATRONS, WHAT THEY'RE SAYING ABOUT MAYBE HOW YOU SHIFTED A PORTION OF YOUR COLLECTIONS, YOU'RE CASUALLY OBSERVING PATRONS YOURSELF, AS LYNN SAID, AS YOU'RE MOVING THROUGH THE LIBRARY.
YOU'RE NOTICING WHERE THEY CHOOSE TO SIT OR WHAT SPACES THAT THEY USE AND AS YOU SEE ON THE SLIDE HERE, YOU PROBABLY HAVE A WAY FOR PATRONS TO LEAVE FEEDBACK OR SUGGESTIONS.
NOW, I'M SURE THIS NEVER, EVER HAPPENS AT ANY OF YOUR LIBRARIES BUT I HAVE HEARD THAT IN SOME LIBRARIES A SINGLE PATRON COMPLAINT CAN TURN INTO A FLURRY OF ACTIVITY FOR LOTS OF STAFF MEMBERS.
AND AGAIN, THIS IS PROBABLY NEVER HAPPENED TO YOU.
BUT I CALL THIS PHENOMENON THAT I'VE HEARD OF THE MAJORITY OF ONE.
 AND SO ACTUALLY, INFORMAL ASSESSMENT ISN'T BAD.
THERE'S NOTHING WRONG WITH IT.
IT'S CERTAINLY FINE FOR CERTAIN CIRCUMSTANCES.
SO YOU MIGHT GET AN IDEA FOR ADDITIONAL SIGNAGE FOR SOMETHING THAT YOU HEAR FROM STAFF MEMBERS.
YOU MIGHT REARRANGE SOME CHAIRS FROM SOMETHING YOU OBSERVE.
OR HERE'S A MAJORITY OF ONE THAT IS AN IMPORTANT ONE TO LISTEN TO.
IF A PATRON TELLS YOU THAT SOMEBODY HAS DISABLED THE SMOKE DETECTOR IN THE BATHROOM AND PEOPLE ARE SMOKING IN THERE, YOU PROBABLY OUGHT TO ACT ON THAT.
SO THERE'S NOTHING  INHERENTLY WRONG WITH INFORMAL ASSESSMENT.
YOU WANT TO BE STRATEGIC TO MAKE DECISIONS WITH BIGGER CONSEQUENCES AND WITH IMPACT, YOU'RE GOING TO WANT TO UNDERTAKE SOMETHING MORE FORMAL.
 AND SO TO CONTRAST WITH INFORMAL ASSESSMENT, I'VE LISTED A FEW CHARACTERISTICS OF FORMAL ASSESSMENT HERE.
SO FIRSTLY, FORMAL ASSESSMENT IS DRIVEN BY DATA AND IT'S BASED ON EVIDENCE.
AND SO THAT MEANS IT'S NOT ANECDOTES.
YOU'RE NOT GOING ON INTUITION OR PERSONAL EXPERIENCE OR SECONDHAND EXPERIENCE.
IT'S NOT CASUAL.
THE DATA AND EVIDENCE THAT YOU USE IS INTENTIONALLY AND SYSTEMICALLY GATHERED OR COLLECTED.
AND THEN FOR ME, THE KEY POINT ON THIS SLIDE IS THAT THESE ARE ACCEPTED METHODS.
AND WHAT I WOULD SAY IS FORMAL ASSESSMENT IS QUITE FORMULAIC.
THERE ARE TONS OF BOOKS WRITTEN ABOUT THE FINE POINTS OF RESEARCH METHODOLOGY.
THERE ARE PEOPLE WHO MAKE THEIR ENTIRE CAREERS ON THINKING ABOUT RESEARCH METHODOLOGY.
 AS LYNN SAID, RESEARCHERS, YOU SELECT THE PARTICULAR TRADITION AND YOU FOLLOW THE METHODS OF THAT TRADITION.
SO THAT COULD BE EXPERIMENTAL DESIGN.
IT COULD BE CASE STUDY.
 SO YOU'RE NOT INVENTING IT WHEN YOU'RE DOING ASSESSMENT.
THERE'S A MODEL THAT YOU'RE FOLLOWING.
 AND THEN FOR EMPIRICAL RESEARCH, THE KIND THAT SCIENTISTS AND ACADEMICS DO, THESE ACCEPTED METHODS ARE ESPECIALLY IMPORTANT, ONE, BECAUSE YOU NEED TO BE TRANSPARENT ABOUT WHAT YOU'VE DONE.
AND TWO, THE GOAL IN EMPIRICAL RESEARCH IS TO REPLICATE OR REPEAT STUDIES.
AND THEN THAT IS HELPING BUILD KNOWLEDGE AND TRUTH.
UNDERSTANDING THE TRUTH VALUE OF WHAT YOU FIND.
SO THAT'S THE IMPORTANCE OF THE ACCEPTED METHOD.
NOW, DEPENDING ON YOUR ROLE, YOU MIGHT NOT NEED TO ACHIEVE THAT STANDARD IN YOUR LIBRARY.
BUT WHEN YOU USE THESE ACCEPTED METHODS, THAT'S GOING TO LEND YOUR WORK CREDIBILITY.
AND I THINK OF THESE METHODS AS LIKE SCAFFOLDING FOR YOUR PROJECT.
IT GIVES YOU SHAPE AND IT GIVES YOU RIGOR.
AND LASTLY, WHAT I WOULD SAY IS THE GOAL OF ASSESSMENT IS GENERALIZEABILITY.
YOU'RE GOING TO START WITH LOTS OF LITTLE DATA POINTS AND YOU WANT TO MAKE A BIG GENERAL STATEMENT.
YOU'RE GOING TO COME TO A CONCLUSION.
MAKE AN INTERPRETATION.
SETTLE ON A DIRECTION.
MAKE A DECISION.
AND SO FORMAL ASSESSMENT IS A PROCESS THAT GETS YOU THERE.
AND I'M GOING TO ILLUSTRATE THAT PROCESS A LITTLE BIT FURTHER IN A MOMENT.
BEFORE WE GET THERE, I WANT TO SAY LYNN SAID, I HOPE YOUR RESPONSE TO ASSESSMENT ISN'T ARGHH!
AND IF THAT IS YOUR RESPONSE, THAT'S OK.
 I HOPE I CAN CHANGE YOUR MIND BECAUSE ASSESSMENT IS A CREATIVE ENDEAVOR.
THAT MIGHT NOT BE THE FIRST WAY YOU THINK ABOUT IT.
BUT LET ME TRY AND HAVE YOU SEE MY POINT.
YOUR END GOAL IN ASSESSMENT IS TO MAKE AN INTERPRETATION AND TO COME TO CONCLUSIONS.
YOU'VE HEARD THE EXPRESSION "LIES, LIES AND STATISTICS."
STATISTICS ALONE, NUMBERS ALONE, THOSE AREN'T TELLING YOU ANYTHING.
DATA DOESN'T REALLY SPEAK FOR ITSELF.
 THAT'S NOT TRUE.
 WHAT HAPPENS IS YOU AND YOUR TEAM ARE GOING TO INTERPRET THE DATA AND COME TO A CONCLUSION.
AND THAT WORK IS CREATIVE.
THAT IS ADDING SOMETHING, ADDING YOUR OWN PERSPECTIVE, ADDING YOUR OWN UNDERSTANDING TO THE DATA THAT YOU COLLECT.
AND ANALYZE.
 AND THIS IS, AGAIN, ONE OF THOSE THINGS THAT AS IN THE SLIDE HERE, WHERE THE WHOLE IS GREATER THAN THE SUM OF ITS PARTS.
AND SO THE GOAL OF ASSESSMENT DEPENDING ON YOUR PROJECT MIGHT BE UNDERSTANDING A PHENOMENON.
SO AN EXAMPLE WOULD BE UNDER WHAT CIRCUMSTANCES DO PEOPLE ASK FOR HELP FROM LIBRARIANS?
YOU MIGHT BE LOOKING AT BUILDING MEANING.
THIS IS SOMETHING THAT ESPECIALLY SCIENTISTS THINK ABOUT DOING.
SO WHAT FACTORS INFLUENCE PEOPLE'S DECISION TO SEEK HELP?
THAT'S ANOTHER APPROACH YOU MIGHT TAKE.
YOU MIGHT BE TAKING A SNAPSHOT AND THIS IS MORE LIKE EVALUATION.
HOW SATISFIED ARE PEOPLE WITH OUR ON-LINE REFERENCE SERVICE?
OR TELLING A STORY.
LYNN TALKED ABOUT THIS, TOO.
HOW HAS INTERACTING WITH LIBRARIANS IMPACTED PEOPLE'S LIVES?
THAT'S LOOKING AT OUTCOMES.
THAT'S TELLING A STORY.
SO IT'S NOT THIS DRY REPORTING OR SOMETHING THAT YOU MIGHT THINK ABOUT IF YOUR RESPONSE IS ARGHH.
IT ACTUALLY BE VERY CREATIVE AND IMPACTFUL.
 SO AS I SAID, THE PROCESS IS ACTUALLY QUITE FORMULAIC.
YOU'LL NOTICE A TEMPLATE.
THERE ARE SECTIONS ON A RESEARCH QUESTION, A LIT REVIEW, METHODOLOGY, FINDINGS, CONCLUSIONS.
AND REALLY, THE SECTIONS OF A RESEARCH PAPER MATCH ALMOST EXACTLY TO THE STEPS AND ASSESSMENT PROCESS.
THAT'S WHY RESEARCH PAPERS ARE WRITTEN THE WAY THAT THEY ARE.
BECAUSE THIS IS THE WAY WE DO RESEARCH.
 AND SO THIS PROCESS, THIS FORMULA IS DESIGNED TO MOVE YOU FROM THE PARTICULAR TO THE GENERAL.
AND THE SLIDE HERE I CHOSE IT FOR TWO REASONS.
IT'S A GOOD METAPHOR.
ONE, YOU TAKE STEPS ALONG A PROCESS.
BUT ON THE OTHER HAND, YOU'RE MOVING HIGHER AND GAINING A DIFFERENT VANTAGE POINT AND SO TODAY, WHAT I'M GOING TO TALK TO YOU ABOUT THE FIRST THREE STEPS IN THIS PROCESS.
DEFINING THE PROBLEM.
ENVIRONMENTAL SCANNING.
AND PREPARING THE PLAN.
THE NEXT TWO SESSIONS ARE GOING TO LOOK AT THE REMAINING STEPS.
COLLECTING DATA, ANALYZING DATA, AND INTERPRETING AND REPORTING IECHLT  -- I'M GOING TO GIVE YOU THE OVERVIEW OF THE PROCESS AND I'LL TOUCH ON SOME POINTS OF DATA COLLECTION AND ANALYSIS.
BUT YOU'RE GOING TO TAKE A MUCH DEEPER DIVE INTO THAT IN THE COMING MONTHS.
 SO STICKING WITH THE SUBJECT OF STEPS.
I HAVE ALWAYS LOVED THIS CONCEPT OF THE LADDER OF ABSTRACTION.
AND THIS COMES OUT OF THE TRADITION OF QUALITATIVE DATA ANALYSIS.
AND THE IDEA HERE OR WHAT'S BEING ILLUSTRATED HERE IS THE PROCESS OF MOVING FROM THE PARTICULAR TO THE GENERAL.
AND I'M NOT GOING TO GO THROUGH EVERY LITTLE WORD ON THIS IN THIS GRAPHIC.
BUT WHAT YOU'RE LOOKING AT IS YOU START WITH RAW DATA AT THE BOTTOM.
AND SO IN THIS CONTEXT, WE'RE THINKING ABOUT A TEXT AND SO THAT MIGHT BE THINGS LIKE INTERVIEW TRANSCRIPTS, OPEN ENDED SURVEY RESPONSES, FIELD NOTES, THAT KIND OF THING.
 AND WHAT YOU'RE DOING IS YOU'RE MOVING UP THESE STEPS, YOU CAN SEE SUMMARIZING AND PACKAGING THE DATA.
THAT MIGHT MEAN CATEGORIZING IT.
YOU MIGHT HEAR PEOPLE TALK ABOUT CODING THE DATA.
THAT'S PUTTING IT INTO BUCKETS, CATEGORIZING IT.
AND YOU'RE JUST PUTTING LIKE CONCEPTS TOGETHER.
AND YOU KEEP DOING THIS SUCCESSIVELY.
YOU'RE CONTINUING TO CODE IT INTO BROADER AND BROADER CATEGORIES.
YOU THINK OF A THEME -- SORRY, I'M SEEING THE CHAT AND WORRIED A LITTLE BIT WE WERE LOSING AUDIO.
SOUNDS LIKE THAT MAY BE LIMITED.
SORRY ABOUT THAT.
 >> YEAH, THIS IS JENNIFER.
REAL QUICK, SORRY TO FOLKS THAT ARE HAVING AUDIO ISSUES FOR THE MOST OF US, IT'S THERE.
AND KENDRA CAN ASSIST YOU WITH WJ SUPPORT.
SHE'S ALSO POSTED THE OPTION FOR A PHONE NUMBER.
SO IF YOU HAVE A PHONE AVAILABLE, THAT MIGHT BE THE MOST SECURE WAY TO CONNECT THE AUDIO RIGHT NOW.
THANK YOU.
 >> TERRIFIC.
 SORRY TO EVERYBODY WHO IS HAVING ISSUES.
 SO YEAH, THE IDEA IS YOU'RE STARTING WITH THE RAW DATA AND TURNING IT INTO BROADER AND BROADER CATEGORIES AND ULTIMATELY, YOU END UP WITH WHAT'S CALLED HERE AN EXPLANATORY FRAMEWORK.
SO A CONCLUSION.
SOMETHING BIGGER THAN WHAT YOU STARTED WITH.
 NOW, I LOVE THE LADDER OF ABSTRACTION BUT MAYBE THAT METAPHOR DOESN'T DO IT FOR YOU.
THIS IS REALLY THE SAME CONCEPT BUT IT'S AUTO FORM SO THE DATA ANALYSIS SPIRAL.
AND AGAIN, YOU'RE MOVING FROM THE PARTICULAR TO THE GENERAL. 
 YOU'RE STARTING WITH RAW DATA AT THE BOTTOM.
THIS IS TEXT AND IMAGES.
AND YOU CAN -- THERE'S CERTAIN BEHAVIORS THAT YOU UNDERGO THROUGH THE PROCESS SO YOU'RE READING AND TAKING NOTES.
SOMETHING YOU HEAR IN THE QUALITATIVE RESEARCH TRADITION IS YOU IMMERSE YOURSELF IN THE DATA.
YOU JUST READ IT AND I LOVE THAT CONCEPT.
IMMERSING YOURSELF.
YOU LET IT POUR OVER YOU.
YOU REALLY ARE SWIMMING IN IT TO TRY TO UNDERSTAND IT.
YOU'RE CATEGORIZING IT.
YOU'RE DESCRIBING IT.
YOU'RE INTERPRETING IT.
AND AS YOU COME OUT AT THE TOP OF THIS PROCESS, YOU HAVE WHAT'S KIND OF BORINGLY CALLED AN ACCOUNT.
IT'S A STORY.
IT'S WHAT YOU'VE LEARNED THROUGH THE PROCESS.
AND SO THE REASON I WANT TO SHARE THESE WITH YOU IS BECAUSE WHEN YOU'RE DOING ASSESSMENTS, I THINK IT'S REALLY EASY TO THINK ABOUT DATA, DATA, DATA, DATA.
YOU GET CAUGHT UP IN COLLECTING THE DATA, MANAGING THE DATA, INTERACTING WITH DATA AND IT MIGHT FEEL LIKE IT'S ALL DATA ALL THE TIME.
AND ESPECIALLY I JUST TALKED ABOUT HOW GREAT IT WAS TO IMMERSE YOURSELF IN THE DATA.
BUT THE TRUTH IS YOUR ULTIMATE GOAL IS A PROCESS OF MOVING BEYOND THE DATA.
GETTING THE BIG PICTURE.
AND SEEING THE FOREST, NOT THE TREES.
 AND SO THIS PROCESS OF ASSESSMENT IS ABOUT CLIMBING THAT LADDER OF ABSTRACTION.
MOVING UP THE SPIRAL. 
 AND LEAVING BEHIND THE PARTICULAR AND AIMING FOR THE GENERAL.
 AND SO NOW, WE'RE GETTING TO THIS FIRST STEP IN THE PROCESS.
THIS IDEA OF DEFINING THE PROBLEM.
 AND I SEE THAT AS FINDING FOCUS.
WHAT ARE YOU FOCUSING ON?
AS LYNN SAID, YOU CAN'T ADDRESS EVERYTHING IN ONE ASSESSMENT.
WE'RE TALKING ABOUT USER CENTERED ASSESSMENT TODAY.
YOU SHOULDN'T BE SURPRISED OUR FOCUS IS THE USER.
IN THIS STEP, WE WANT TO LEARN ABOUT OUR USERS AND OUR COMMUNITIES AND WE'RE GOING TO NEED TO FIGURE OUT WHAT ARE THE RIGHT QUESTIONS TO ASK?
 SO IT SHOULD BE OBVIOUS THAT WE'RE THINKING ABOUT USERS.
BUT I'M GUILTY OF THIS SOMETIMES.
WE GET SO CAUGHT UP IN THINKING ABOUT LIBRARIES AND LIBRARY SERVICE ALL THE TIME, WE GET SO EXCITED ABOUT WHAT'S HAPPENING IN THE PROFESSION, IT'S EASY TO BECOME TOO FOCUSED ON OURSELVES, OUR ORGANIZATIONS AND OUR PROCESSES.
BUT WHAT WE NEED TO REMEMBER IS WE WANT TO SERVE OUR PATRONS, OUR STUDENTS, OUR CUSTOMERS.
SO WE NEED TO BE FOCUSING ON THEM.
 AND THAT MEANS THINKING ABOUT THEIR NEEDS, THEIR PERSPECTIVES, THEIR BEHAVIORS.
 AND AT THE SAME TIME, YOU DON'T WANT TO FORGET ABOUT THE CONTEXT.
PEOPLE EXIST IN COMPLEX CONTEXT.
JUST THINK ABOUT YOURSELF.
 YOU LIVE IN A COMPLEX WORLD.
EVERYTHING THAT IS, IN FACT, AROUND YOU IS COMPLEX.
I'M SURE NOBODY LISTENING RIGHT NOW HAS A SIMPLE, STRAIGHT FORWARD LIFE.
THERE ARE A LOT OF COMPETING INTERESTS AND THINGS THAT ARE INFLUENCING EVERYTHING YOU DO.
 SO, YOU KNOW, THAT CAN BE DIFFERENT FAMILIES AND TRADITIONS, DIFFERENT BACKGROUNDS, DIFFERENT CHALLENGES.
AND SO WHAT I JUST SAID BEFORE MEANS PERSPECTIVES AND BEHAVIORS REALLY DIFFER.
THINK ABOUT YOURSELF DAY TO DAY.
ALL OF THESE THINGS DIFFER DAY TO DAY AND MOMENT TO MOMENT AND SO ON.
NOT JUST BETWEEN PEOPLE BUT, YOU KNOW, WITHIN A SINGLE PERSON OVER TIME.
AND SO WHAT I WOULD SAY, USER IN CONTEXT SOUNDS SO FORMAL.
BUT WHAT WE WANT TO BE THINKING ABOUT IS PEOPLE AND THEIR LIVES.
 PEOPLE AND THEIR LIVES.
 AND I WOULD PUSH EVEN FURTHER AND SAY, YOU ONLY WANT TO THINK ABOUT PEOPLE AND THEIR LIVES.
TRY TO TAKE YOUR LIBRARY OUT OF THE PICTURE AND THINK ABOUT THE FACT THAT HOW IT PLAYS IN PEOPLE'S LIVES BEYOND THE LIBRARY.
REALLY TRY TO DECENTER THE LIBRARY AND THE PROCESS.
 SO IN ADDITION TO PERSONAL CONTEXT, THERE'S COMMUNITY CONTEXT.
SO YOUR LIBRARY IS NOT THE ONLY SERVICE IN YOUR COMMUNITY.
 YOU MIGHT HAVE THE PARKS AND REC IF YOU'RE A PUBLIC LIBRARY.
THERE ARE MUSEUMS.
OTHER KINDS OF GROUPS IN THE COMMUNITY THAT PEOPLE ENJOY.
IN AN ACADEMIC LIBRARY, YOU ALSO ARE THINKING ABOUT DIFFERENT SOURCES THAT PEOPLE ARE USING, INTERNET, FAMILY, FRIENDS AND SO ON.
>> PUBLIC LIBRARY.
>> YES!
IF YOU'RE AN ANNINGNING -- ACADEMIC LIBRARIAN AND VICE VERSA.
I KNOW HERE IN OHIO A LOT OF COMMUNITIES ARE INTERESTED IN HOW THEY CAN HELP INTERVENE IN RELATION TO THE OPIOID CRISIS.
UNEMPLOYMENT MIGHT BE AN ISSUE IN YOUR COMMUNITY.
THERE ARE POLITICAL ISSUES.
IMMIGRATION MIGHT BE SOMETHING THAT'S A CONCERN IN YOUR COMMUNITY.
FUNDING IS PROBABLY A CONCERN IN EVERYBODY'S COMMUNITY!
 SOMETIMES MORE THAN OTHERS.
 SO YOU REALLY ALSO WANT TO BE THINKING ABOUT WHAT'S GOING ON IN THE COMMUNITY, HOW ARE COMMUNITY ISSUES IMPACTING PEOPLE'S LIVES AND THEN WHERE DOES THE LIBRARY FIT INTO THAT PICTURE?
 SO AS WE'RE DEFINING THE PROBLEM, YOU MAY BE ASKING YOURSELF QUESTIONS LIKE -- WHEN SHOULD WE STAFF OUR REFERENCE DESK?
HOW SHOULD WE FURNISH OUR STUDY ROOMS?
WHAT NEW INITIATIVE SHOULD WE PURSUE?
AND WHEN I'M TALKING ABOUT THE RIGHT QUESTIONS, I'M SORT OF THINKING ABOUT IT IN TWO DIFFERENT WAYS.
THE QUESTIONS I JUST LISTED OFF ARE POTENTIAL RESEARCH QUESTIONS.
THESE ARE THE THINGS THAT YOU'RE ASKING PROFESSIONALLY AND THAT ARE NECESSARY TO GUIDE YOUR ASSESSMENT PROCESS AND I THINK SOMETIMES WE CAN THINK ABOUT THESE QUESTIONS AS THE QUESTIONS WE ARE GOING TO ASK OUR USERS.
AND THESE ARE ABSOLUTELY NOT THE RIGHT QUESTIONS TO POSE TO YOUR USERS.
AS I JUST SAID, THERE'S A LOT HAPPENING IN PEOPLE'S LIVES AND THE LIBRARY AS SAD AS IT IS TO SAY IS NOT THE CENTER OF MOST PEOPLE'S LIVES.
 AND WHEN WE'RE THINKING ABOUT THIS DIFFERENCE BETWEEN THE RESEARCH QUESTION AND THE QUESTIONS THAT WE ASK OUR PATRONS, REMEMBER, YOU ARE A LIBRARY EXPERT.
YOU HAVE A LOT OF EXPERTISE.
YOU KNOW WHAT'S HAPPENING IN THE PROFESSION THAT YOU'RE GOING TO BRING TO BEAR.
SO WHEN YOU'RE THINKING ABOUT STAFFING THE REFERENCE DESK, YOU KNOW ABOUT STAFFING MODELS AND LIMITATIONS.
WHEN YOU'RE THINKING ABOUT FURNISHING STUDY ROOMS, YOU KNOW ABOUT FURNITURE ON THE MARKET AND TRENDS THAT ARE HAPPENING.
WHEN YOU THINK ABOUT NEW INITIATIVES YOU HEAR THE BUZZ ABOUT WIRELESS HOT SPOTS OR MAKER SPACES OR 3D PRINTERS.
WHATEVER IT IS THAT'S COOL AND NEW RIGHT NOW.
BUT YOUR USERS ARE THE EXPERTS IN THEIR OWN LIVES.
 AND SO IT'S REALLY IMPORTANT TO SEPARATE YOUR OVERARCHING RESEARCH QUESTION.
WHAT YOU CARE ABOUT PROFESSIONALLY FROM THE QUESTION THAT YOU POSE TO YOUR USERS WHICH IS WHAT YOU REALLY NEED TO KNOW AT THE END OF THIS ASSESSMENT PROCESS.
 SO THE RIGHT QUESTION IS GOING TO FOCUS ON YOUR AUDIENCE IN THE CONTEXT OF THEIR EVERYDAY LIVES AND WITHIN YOUR COMMUNITY.
 YOU SHOULD NOT BE ASKING THEM HOW TO DO YOUR JOB.
WHAT YOU LEARN FROM YOUR ASSESSMENT PROJECT SHOULD HELP YOU DO YOUR JOB.
BUT YOU'RE THE ONE.
THIS IS CREATIVE.
YOU'RE THE ONE THAT WILL BRING YOUR CREATIVE AND PROFESSIONAL EXPERTISE TO BEAR ON WHAT IT IS YOU LEARNED FROM YOUR AUDIENCE.
 I KNOW THAT THIS SLIDE PROBABLY ALL OF US RECOIL A LITTLE BIT AT THE IDEA OF A STUPID QUESTION.
BUT I'M GOING TO GIVE YOU AN EXAMPLE OF A STUPID QUESTION I MYSELF ASKS THAT BREAKS ALL OF THE RULES I'VE JUST GIVEN YOU.
 BEFORE THAT, AFTER WE DEFINE THE PROBLEM, THE NEXT STEP IS ENVIRONMENTAL SCANNING.
AND THAT'S STARTING FIRST WITH WHAT'S OUT THERE?
 SO ASSESSMENT IS TIME CONSUMING.
WELL, IF DONE WELL.
AND RESOURCE INTENSIVE.
SO YOU DON'T WANT TO REINVENT THE WHEEL.
SO YOU REALLY WANT TO THINK HARD ABOUT WHAT YOU ALREADY KNOW.
AND THE TWO THINGS I'M GOING TO MENTION TODAY ARE WHAT EXISTING DATA SOURCES DO YOU HAVE ACCESS TO?
AND WHAT OTHER RESEARCH IS OUT THERE?
 SO WHAT DATA DO YOU ALREADY HAVE?
FIRST OF ALL, DON'T BOTHER TO ASK YOUR USERS A QUESTION THAT YOU CAN FIND AN ANSWER TO OTHERWISE.
THAT'S A WASTE OF EVERYBODY'S TIME.
 IT'S A WASTE OF YOUR TIME.
AND YOUR USER'S GOOD WILL.
 I WOULD SAY IF WHAT YOU'RE LOOKING AT IS BEHAVIOR, PEOPLE CAN BE REALLY UNRELIABLE WHEN SELF-REPORTING THEIR BEHAVIOR.
CONSIDER WHETHER WHAT YOU WANT TO KNOW, WHETHER ASKING YOUR USERS DIRECTLY IS GOING TO GET YOU GOOD RESULTS.
 SO I'VE GOT SOME EXAMPLES HERE OF THE KINDS OF DATA YOU MIGHT ALREADY HAVE.
IF YOU LOOK IN THE TOP LEFT THERE, I'D BE FLABBERGASTED IF THERE WAS ANYBODY LISTENING RIGHT NOW THAT DIDN'T HAVE A SHEET WITH HASHMARKS ON IT SOMEWHERE IN THEIR LIBRARY.
SO IN THIS CASE, THESE ARE, I KNOW YOU CAN'T SEE THE DETAIL BUT IT'S THE TYPE AND NUMBER OF QUESTIONS AT THE INFO DESK AT DIFFERENT TIMES OF DAY.
WE DO SOMETHING AT MY LIBRARY THAT I'M SURE OTHER LIBRARIES DO, TOO, WE HAVE WHAT QUESTIONS HAVE PEOPLE ASKED THAT WE HAVE TO SAY NO TO?
THAT'S A GREAT WAY TO THINK ABOUT WHAT PEOPLE -- UNFULFILLED NEEDS THAT PEOPLE HAVE.
WHAT YOU'RE LOOKING AT THERE IN THE UPPER RIGHT IS PROGRAM ATTENDANCE.
THAT'S A CALENDAR.
PROGRAM ATTENDANCE.
AND IN THE LOWER LEFT, THAT'S ACTUALLY FROM GOOGLE ANALYTICS AND THAT'S THE BUSIEST DAYS AND TIMES ON THE LIBRARY WEBSITE.
THE DARKER THE COLOR, THE MORE SESSIONS ON THE WEBSITE AT THE TIME.
 IN THE CENTER BOTTOM, THAT'S SOME MARKET RESEARCH DATA FROM A COMPANY.
IF THAT'S SOMETHING THAT YOU USE ON YOUR WEBSITE, IT CAN PEOPLE WHO VISIT YOUR WEBSITE.
AND THEN LASTLY, THIS IS ONE I HAVE TO SAY I HAVE NOT DONE MY DUE DILIGENCE TO FIND OUT HOW IT IS GOOGLE DISCOVERS THESE POPULAR TIMES.
BUT IF YOU'VE NEVER LOOKED AT THIS FOR YOUR LIBRARY, IF YOU JUST GOOGLE YOUR LIBRARY, CHANCES ARE IN THE RIGHT SIDE BAR, YOU'RE GOING TO GET A CHART LIKE THIS THAT SHOWS YOU THE POPULAR TIMES.
 AND, YOU KNOW, MAYBE YOU DON'T HAVE A WAY TO TRACK FOOT TRAFFIC IN THE LIBRARY OR MAYBE YOU WANT TO LOOK AT THAT IN ADDITION TO SOME KIND OF WAY THAT YOU'RE DOING HEAD COUNTS.
BUT THAT MIGHT BE A RESOURCE YOU MIGHT NOT IMMEDIATELY THINK OF.
OTHER THINGS THAT AREN'T PICTURED THAT PROBABLY YOU'RE THINKING OF BUT IN CASE YOUR MIND DOESN'T GO THERE, CERTAINLY YOU HAVE CIRCULATION STATS AND RESERVE LISTS.
YOU HAVE PHONE LOGS.
 I'M NOT TALKING ABOUT RECORDING THE CALLS BUT JUST THE NUMBER OF CALLS AND THE TIME OF DAY THAT THEY'RE COMING IN.
 YOU WOULD HAVE REFERENCE INTERACTIONS IF YOU HAVE CATALOGS AND E-MAILS.
WHAT ABOUT COMPUTER SESSIONS AND WI-FI SESSIONS?
WHAT ABOUT ROOM RESERVATIONS?
AND CATALOG SEARCH QUERIES?
THERE ARE SO MUCH THAT IF YOU JUST START LOOKING AROUND, YOU HAVE A LOT OF POTENTIAL INFORMATION THAT YOU CAN START TO BASE YOUR QUESTIONS ON AND FOCUS IN ON SPECIFICS THAT YOU REALLY NEED TO SPEAK TO YOUR USERS ABOUT.
 SO THE OTHER PIECE OF THIS IS OTHER RESEARCH.
 SO I'M PERSONALLY A BIG FAN OF THE RESEARCH CENTER ON LIBRARIES AND, OF COURSE, OCLC REPORTS AND THEN THERE'S THE ALPHABET SOUP OF OTHER DATA SOURCES SO IMLS, PLA, ACRL, NCES.
I'M NOT GOING TO GO INTO DETAILS ON THESE.
THE LINKS ARE IN THE LEARNER GUIDE AS I THINK THAT JENNIFER JUST POSTED SO IF YOU'RE NOT FAMILIAR WITH THESE, TAKE A LOOK.
THERE'S VALUABLE DATA AND REPORTS THERE.
AND WHAT I WOULD SAY IS I DON'T KNOW WHAT YOUR FIRST INCLINATION IS WHEN YOU'RE LOOKING AT THESE STUDIES, YOU MIGHT BE FOCUSED ON RESULTS AND FINDINGS.
BUT A LOT OF TIMES WHEN I LOOK AT THIS RESEARCH, I'M REALLY LOOKING AT METHODS AND INSTRUMENTS.
SO LOOK AT WHAT THEY'RE DOING AT OCLC, OR PEW.
HOW ARE THESE RESEARCHERS COLLECTING DATA?
WHAT CODING SCHEMES HAVE THEY USED?
THINK ABOUT THE INSTRUMENTS.
WHAT INTERVIEW PROTOCOLS ARE THEY USING?
WHAT KIND OF SURVEY QUESTIONS ARE THEY ASKING?
WE HAVE MIRRORED OCLC AND PEW SURVEY QUESTIONS AND WHAT'S GREAT ABOUT THAT IS YOU GET A SENSE OF HOW YOUR COMMUNITY MIGHT DIFFER NATIONALLY.
AND  THAT'S REALLY AN INTERESTING PERSPECTIVE.
 I'M GOING TO GO BACK TO WHAT LYNN SAID, WHEN YOU TALK ABOUT A PROCESS.
LOOK AT PRIOR RESEARCH.
IF YOU OR SOMEONE ELSE IN YOUR ORGANIZATION HAS DONE A SURVEY OR HAS DONE SOME KIND OF DATA COLLECTION, EXERCISE, THAT HAS REALLY WORKED, I MEAN, IF IT HASN'T WORKED, I DON'T KNOW.
IF IT'S REALLY WORKED, REPEAT IT.
AND SEE HOW YOUR AUDIENCE IS CHANGING OVER TIME.
AND AS YOU'RE DOING YOUR NEXT ASSESSMENT PROJECT, IT SHOULDN'T BE ONE AND DONE.
THINK ABOUT DEVELOPING AN INSTRUMENT, PROTOCOL OR QUESTION THAT YOU CAN KEEP USING AGAIN AND AGAIN.
 SO THAT YOU CAN SEE LONGITUDINAL RESULTS.
I WOULD SAY, TOO, ONE THING TO BE CONSISTENT WITH IF YOU EVER LOOK AT YOUR QUESTIONS OR LOOK AT YOUR POPULATIONS BY DEMOGRAPHICS, MAKE SURE YOU ASK THOSE DEMOGRAPHIC QUESTIONS THE SAME WAY EVERY TIME.
SO YOUR AGE BREAKDOWNS IF THAT'S WHAT YOU CARE ABOUT, I KNOW THERE'S A LOT OF TALK ABOUT LOOKING AT HOW YOU ASKED A QUESTION ABOUT GENDER.
HOW YOU MIGHT ASK A QUESTION ABOUT WHERE THEY LIVE.
LIKE WHETHER YOU'RE ASKING ZIP CODE OR NEIGHBORHOOD OR SOMETHING LIKE THAT.
SO ASK THOSE THE SAME WAY ALWAYS.
 SO THAT YOU CAN COMPARE.
 THOSE.
THAT'S MY LITTLE PRO TIP ON INSTRUMENTS.
 SO LYNN TALKED ABOUT COMMITTING.
HERE'S WHERE I THINK THE RUBBER MEETS THE ROAD.
AND THIS IS THE SINGLE MOST IMPORTANT PART ABOUT ASSESSMENTS.
HAVE TO SEE WHETHER OR NOT THE SUBSEQUENT SPEAKERS THINK WHAT THEY'RE GOING TO SAY IS THE MOST IMPORTANT PART.
I WOULD SAY THIS IS WHERE YOU DON'T WANT TO SKIMP.
AND MY TRICK IS TO START AT THE END.
I DON'T KNOW IF YOU KNOW THE EXPRESSION IN SOFTWARE DEVELOPMENT, YOU TALK ABOUT GARBAGE IN AND GARBAGE OUT.
I THINK THAT IS 100% TRUE OF ASSESSMENTS.
THE CONCLUSION THAT YOU COME TO DEPENDS UPON YOUR ANALYSIS WHICH GROWS OUT OF THE DATA THAT YOU COLLECT WHICH IS SHAPED BY THE QUESTIONS THAT YOU ASK.
 ASSESSMENT ISN'T MAGIC.
YOU'RE NOT GOING TO TRANSFORM A LUMP OF COAL INTO A DIAMOND.
SO THAT'S WHY I SUGGEST STARTING BACKWARDS AND THINKING ABOUT THE IMPLICATIONS FOR THE STEPS AHEAD.
SO WHEN I ASK WHAT KIND OF CONCLUSION DO YOU WANT TO COME TO?
THIS IS ACTUALLY FAIRLY MAYBE MORE STRAIGHT FORWARD THAN YOU THINK.
I'M NOT SAYING WHAT YOU THINK THE FINDINGS ARE GOING TO BE.
IT'S ARE YOU GOING TO MAKE A DECISION?
ARE YOU DOING AN EVALUATION?
ARE YOU LOOKING FOR A STRATEGIC DIRECTION?
 AND THEN WHEN YOU ANALYZE, ARE YOU GOING TO USE OPEN CODING?
DO YOU HAVE AN EXISTING CODING SCHEME THAT YOU WANT TO USE?
ARE YOU CALCULATING MEANS AND MEDIANS AND RANKING A LIST OF OPTION?
AND WHEN YOU COLLECT, ARE YOU GOING TO DO A SURVEY, INTERVIEWS, OBSERVATION?
AND HERE'S WHERE YOUR TEAM MATTERS.
 A SURVEY IS PRETTY EASY TO IMPLEMENT.
INTERVIEWS, OBSERVATIONS CAN BE A LITTLE BIT MORE DIFFICULT.
SO YOU HAVE TO BE REALISTIC, TOO, IF YOU DON'T HAVE A LARGE TEAM OF PEOPLE WHO CAN DO THE WORK, YOU NEED TO TAILOR YOUR ASSESSMENT TO REALITY.
AND THEN WHAT IS IT THAT YOU'RE GETTING AT?
SO DO YOU CARE ABOUT BEHAVIORS?
ATTITUDES, BELIEFS, DO DEMOGRAPHICS MATTER?
SO AS I ROUND OUT -- TURN A CORNER HERE, WHAT I'M GOING TO DO IS GIVE YOU SOME EXAMPLES.
THIS IS ALL REALLY TRICKY.
IF YOU'VE NEVER DONE ASSESSMENT AND NEVER DONE RESEARCH, IT'S HARD TO KNOW WHAT CAN GO WRONG.
WHEN YOU'VE DONE THIS A FEW TIMES, YOU START TO ANTICIPATE IT AND SORT OF BUILD SOME CONTROLS IN THE PROCESS.
THERE'S JUST SO MANY MOVING PARTS AND SOMETHING ALWAYS GOES WRONG.
SO WHAT I'M GOING TO DO -- OOPS.
OH, NO!
 I JUMPED AHEAD.
 THERE WE GO.
I'LL GIVE YOU A CRASH COURSE IN THE KIND OF THING THAT CAN GO WRONG SO A TEST OF THE ASSESSMENT EXPERIENCE.
I'LL SHOW YOU AN EXAMPLE ASSESSMENT PROJECT TO DEMONSTRATE HOW IN PARTICULAR THE WRONG QUESTION COLLECTS INADEQUATE DATA RESULTING IN DULL ANALYSIS AND LEADING TO USELESS CONCLUSIONS.
AND YOU CAN SEE IT'S KIND OF A DOMINO EFFECT.
HERE'S THIS GARBAGE IN, GARBAGE OUT CONCEPT.
AND I'M GOING TO CONTRAST THAT, THEN, WITH A MUCH MORE EFFECTIVE ASSESSMENT EXAMPLE.
SO KIND OF THIS SCENARIO AND THESE ARE REAL EXAMPLES FROM MY LIBRARY.
 SO YOU ALREADY SAW MAYBE AS PART OF STRATEGIC PLANNING SEVERAL YEARS AGO, WE WANTED TO GET A SENSE OF WHAT PEOPLE WERE LOOKING FOR FROM THE LIBRARY?
SO WE POSTED THIS QUESTION IN OUR LOBBIES WITH STICKY NOTES AND MARKERS FOR PEOPLE TO GIVE ANSWERS.
AND YOU MAY ALREADY BE CRITIQUING THIS QUESTION BASED ON WHAT I'VE ALREADY RECOMMENDED.
BUT I'M GOING TO RESERVE MY EDITORIALIZING FOR NOW.
WALK YOU THROUGH THE PROCESS A LITTLE BIT.
 SO HERE'S A WORDLE OF THE RESULTS.
IF YOU'RE NOT FAMILIAR WITH IT, THE SIZE OF THE WORD SHOWS THE FREQUENCY OF MENTION.
THIS IS FOR ILLUSTRATION PURPOSES.
WE DIDN'T USE IT FOR ANALYSIS.
NOW, ONE THING IS PROBABLY JUMPING RIGHT OUT AT YOU HERE.
THAT'S BOOKS.
OTHER THINGS THAT MIGHT BE GRABBING YOUR ATTENTION, MUSIC, GAMES, MOVIES, READING, EBOOKS, THOSE ARE THE THINGS THAT ARE PROMINENT HERE.
 WE CODED THE DATA AND WE ENDED UP WITH SOME CATEGORIES.
WE ENDED UP WITH COLLECTION MATERIALS, TECHNOLOGY, FACILITIES, PROGRAMMING WHICH PRETTY MUCH SOUNDS LOOK A LIBRARY.
 AND SO OUR ANALYSIS OF THE FINDING IS THEY WANTED MORE OF WHAT WE ALREADY DO WHICH IS NOT THE MOST VISIONARY STATEMENT IN THE WORLD.
BUT THE TRUTH IS THIS WAS OUR FAULT.
WE SHOULD HAVE SEEN THIS COMING.
WE KNEW THAT USE OF OUR TRADITIONAL SERVICES WAS STEADY FROM WEB STATS AND SO ON.
WE KNEW THAT PEOPLE LACKED AWARENESS OF OUR OTHER SERVICES.
AND FROM OCLC'S PERCEPTIONS REPORT BACK IN 2010, WE KNOW THAT BOOKS ARE OUR BRAND.
 SO WE DIDN'T LEVERAGE WHAT WE KNEW AND DIDN'T FOCUS ON FILLING IN THE GAPS.
THIS IS WHAT I MEAN.
WE ASKED A STUPID QUESTION AND DIDN'T LEARN ANYTHING NEW.
THIS IS WHAT I MEAN.
WHAT WE DID HERE IS ASKED OUR USERS TO DO OUR JOBS.
WE NEEDED TO ANSWER THE QUESTION WHAT'S YOUR VISION FOR THE LIBRARY'S FUTURE?
BUT WE TRIED TO PASS THE BUCK AND HAVE OUR PATRONS ANSWER THAT.
AND THEY DON'T HAVE THE EXPERTISE.
THAT'S NOT FAIR TO THEM.
SO WE'RE GOING TO FAST FORWARD A FEW YEARS WHERE WE FIXED IT.
 LIKE I SAID, OUR PREVIOUS QUESTION WAS MORE LIKE A RESEARCH QUESTION.
IT'S WHAT WE CARED ABOUT AS PROFESSIONALS, NOT SOMETHING THAT SHOULD HAVE SEEN THE LIGHT OF DAY WITH OUR PATRONS.
FOR OUR NEXT STRATEGIC PLANNING CYCLE, WE USED THE SAME PROCESS.
A QUESTION IN THE LOBBIES WITH STICKY NOTES.
AND WORKING BACKWARDS, WE WANTED TO COME TO A DIRECTION OR VISION.
WE WANTED IDEAS ABOUT NEW SERVICES OR INITIATIVES.
WE WANTED TO BE OPEN TO ANYTHING.
SO WE EXPECTED TO DO OPEN CODING TO LET THE DATA SPEAK TO US AND TO FIND SOMETHING WE WEREN'T NECESSARILY LOOKING FOR.
AND FOR THAT, WE NEEDED TO ASK AN OPEN ENDED QUESTION AND WHAT WE CARED ABOUT WAS PEOPLE'S NEEDS, PARTICULARLY UNMET NEEDS.
AREAS WHERE THEY WERE UNFULFILLED.
AND SO THE PROMPT WE CAME UP WITH HERE IS ONE THING THAT WOULD MAKE MY LIFE BETTER..
SO THIS NEW QUESTION FOCUSES NICELY ON THE USER AND CONTEXT.
PEOPLE ARE USEERS IN THEIR OWN LIFE AND THEY CAN TELL US WHAT THEY NEED.
THIS ISN'T ABOUT THE LIBRARY.
IT'S SOMETHING WE GENUINELY DON'T ALREADY KNOW AND HERE'S A PRO TIP.
THE PHRASING IT WAS IS TANTALIZING.
YOU PROBABLY HAVE AN ANSWER TO THIS RIGHT NOW.
SO PEOPLE WANT TO ANSWER.
THEY'RE EXCITED TO ANSWER.
 AND ANOTHER EXAMPLE THAT I WOULD GIVE THAT'S WORKED WELL FOR US IS BEFORE I DIE, I WANT TO..
IT'S LIKE ASKING PEOPLE TO TELL YOU WHAT'S ON THEIR BUCKET LIST.
 SO HERE'S THE RESULTS.
AND YOU KNOW, HOW DIFFERENT IS THAT?
BIG THING THERE IS FAMILY.
YOU SEE MONEY JUST ABOVE MONEY IS TIME.
THERE'S FRIENDS, HAPPINESS, PEACE, LOVE, PUPPIES.
FOOD, TELEPORTER.
YOU CAN GET LOST IN SOME OF THESE ANSWERS.
JUST TO POINT OUT, HOMEWORK, WORDLE SEPARATES WORDS THAT ARE IN PHRASES.
THIS WAS PART OF THE PHRASE LESS HOMEWORK.
WE DIDN'T HAVE SUPER STUDENTS THAT SAID MORE HOMEWORK WOULD MAKE THEIR LIVES BETTER.
 SO WHEN WE CODED THE DATA, WE GOT A HIGHER LEVEL OF ABSTRACTION.
THINK OF THAT LADDER.
WE WENT A COUPLE OF STEPS UP ON THAT LADDER HERE.
WE GOT AWAY FROM THE WORDS OUR PATRONS USED TO END UP WITH THESE BROAD THEMES.
SO TO GIVE AN EXAMPLE OF THE FIRST TWO HERE, TIME AND MONEY WERE WORDS THAT OUR PATRONS USED BUT WE CONCEIVED OF THAT AS FREEDOM.
WHAT PEOPLE WANT IS FREEDOM IN THEIR LIVES.
AND PETS AND FAMILY BECAME AFFECTION.
AND SO WHAT I LOVE ABOUT THIS IS THIS DOESN'T INHERENTLY DESCRIBE A LIBRARY.
SO THERE'S A LOT OF OPPORTUNITIES.
WITH THESE CONCEPTS.
AND SO WE CAME TO THIS REALLY MEANINGFUL AND I FIND PERSONALLY INSPIRING CONCLUSION.
THAT PEOPLE ARE LOOKING FOR WARMTH AND AFFECTION.
THEY'RE LOOKING FOR FREEDOM, WHIMSEY AND THEIR LIVES AND SO ON.
SO THIS HAS BECOME THE BACKDROP OR TOUCH STONE THAT WE AS LIBRARY PROFESSIONALS CAN USE WHEN WE'RE CONSIDERING NEW LIBRARY INITIATIVES TO MEET PEOPLE'S NEEDS.
AND SO THIS IS MY EXAMPLE OF WHAT A DIFFERENCE A QUESTION MAKES.
 SO JUST TO BRING THIS FULL CIRCLE, I WANT TO CLOSE BY SHOWING YOU JUST SOME OF THE OUTCOMES AT MY LIBRARY.
WE'VE CREATED A NUMBER OF PROGRAMS AND SERVICES THAT FOLLOW OUT OF THESE BROAD THEMES.
SO WHEN WE THINK ABOUT AFFECTION, WE'RE THINKING ABOUT CREATING OPPORTUNITIES FOR CONNECTION OR TOGETHERNESS.
LOVELY PHOTO OF A YOUNG MAN READING TO A DOG.
WE DO THIS FAIRLY REGULARLY AND WE HAVE UNDERNEATH, IT MAY BE A LITTLE BIT HARD TO SEE.
THOSE ARE ACTUALLY BLANKETS SPREAD ACROSS TABLES AND CHAIRS.
THIS IS FAMILY FORTNIGHT.
SO WE'RE REALLY TRYING TO DO A LOT MORE FAMILY PROGRAMS.
NOT JUST FOR CERTAIN AGE GROUPS.
AND SO THIS IS A WAY FOR FAMILIES TO CONNECT.
AND BY THE WAY, THEY CAN SPEND THE NIGHT IN THE LIBRARY.
 AS A PARENT, I AM NOT SURE THAT'S WHAT I WANT TO DO.
 BUT YEAH, THAT'S A REALLY GREAT WAY FOR THE KIDS AND THE PARENTS TO HAVE AN ADVENTURE TOGETHER.
FOR FREEDOM, WE CERTAINLY ARE NOT GIVING THEM MONEY ALTHOUGH WE DID RECENTLY GO FINE FREE.
BUT WE CAN IN A WAY GIVE THEM TIME.
AND SO WE THINK ABOUT THAT AS OFFERING CONVENIENCE.
 SO WE HAVE PICKUP LOCKERS SO YOU CAN GET YOUR HOLD AFTER THE LIBRARY IS CLOSED AND WE HAVE THESE LENDING MACHINES AND REC CENTERS THROUGHOUT THE COMMUNITY.
IF YOU'RE AT THE GYM WORKING OUT, YOU CAN GRAB A BOOK.
FANTASY, WE DO ESPECIALLY WITH OUR TEENS, THIS IS A ZOMBIE ESCAPE THE ROOM GAME THAT WE DO WITH TEENS SORT OF AN IMMERSESIVE EXPERIENCE.
AND THE BOTTOM THERE UNDERNEATH, YOU'RE SEEING A SCULPTURE.
WE HAVE LITTLE CRITTERS THAT WE USE AS MASCOTS FOR THE SUMMER READING PROGRAM.
AND THAT'S BEING TURNED INTO A SCULPTURE THAT'S ACTUALLY GOING TO BE UNVEILED IN JUST A COUPLE OF WEEKS.
WHICH IS SORT OF FUN.
CERTAINLY FANTASTICAL.
AND MORE STRAIGHT FORWARD IS FOOD.
PEOPLE LOVE FOOD.
WE DO A LOT OF TASTE TESTS SERIES.
AND WE DO -- WE'VE HAD REALLY POPULAR EDIBLE BOOKS SERIES THAT YOU'RE LOOKING AT THERE AND I WOULD SAY THESE ARE REALLY BECOMING SOME OF OUR SIGNATURE SERVICES IN THE LIBRARY.
I THINK THAT'S IN LARGE PART BECAUSE THEY'RE BASED ON THE NEEDS THAT WE IDENTIFIED IN THE COMMUNITY THROUGH ASSESSMENT.
 SO THAT'S IT.
I THINK WE'RE AVAILABLE TO ANSWER QUESTIONS.
OR HAVE A DISCUSSION?
>> YES.
THANKS SO MUCH, KARA!
 I REALLY ENJOYED HEARING ABOUT THE FAILURE.
SORRY.
 BUT ALSO, HOW YOU LEARNED FROM THAT AND CHANGED.
AND THAT'S SOMETHING WITH RESEARCH THAT WHEN WE FAIL OFTEN OUR RESEARCH ASSISTANTS WILL BE VERY UPSET THAT WE'VE FAILED.
WE HAVEN'T.
WE'VE LEARNED SOMETHING.
WE'VE LEARNED WHAT NOT TO DO.
WE NEED TO LET OTHERS KNOW SO THAT YOU AREN'T REPLICATING OUR -- OUR -- OUR PROBLEMS OR THE ISSUES THAT WE HAVE.
I DON'T SEE ANY QUESTIONS YET UNLESS I'M NOT LOOKING -- 
SOMEBODY JUST WROTE THE LITTLE -- THE LIBRARY.
ONE OF THE THINGS THAT I REALLY WANT TO STRESS IS THAT ASSESSMENT IS CYCLICAL AND I KNOW KARA SAID THAT AND I'M GOING TO KEEP SAYING THAT IN EVERY ONE OF THESE SESSIONS THAT WE REALLY NEED TO CONTINUE TO ASSESS.
AND SO, YOU KNOW, IT'S NOT LIKE YOU SAY OK, I'M DONE.
CLAP MY HANDS AND MOVE ON.
 THERE'S A QUESTION AND THIS PROBABLY GOES TO KARA.
YOU TAKE PEOPLE'S THOUGHTS ON IMPROVING THEIR LIVES AND TRY TO TRANSLATE THOSE THOUGHTS INTO SERVICE IMPROVEMENTS.
HOW DO YOU FIND OUT IF THE IMPROVEMENTS YOU MAKE ARE VALUED?
>> RIGHT, SO YEAH, I THINK THAT'S A TERRIFIC QUESTION AND ABSOLUTELY.
YOU NEED TO CLOSE THE LOOP ON THAT.
AND I THINK PROGRAM ATTENDANCE IS ONE.
RIGHT?
I MEAN, WE KNOW THAT PEOPLE LOVE OUR FOOD PROGRAMS BECAUSE THERE ARE WAITING LISTS TO GET INTO THEM.
AND SO THAT'S PART OF IT.
AND I THINK YOU CAN DO -- I KNOW WE GO THROUGH CYCLES OF THIS.
I DON'T THINK WE DO IT FOR EVERY PROGRAM.
BUT YOU CAN ASK PEOPLE ABOUT HOW MUCH THEY ENJOYED THE PROGRAM.
SO WE DO LITTLE MINI  EVALUATIONS TO ASK HOW PEOPLE FEEL ABOUT THEM.
I DON'T KNOW IF WE'VE DONE THIS IN PARTICULAR FOR THE LOCKERS.
IN THE LOCKERS, YOU COULD PUT A LITTLE BOOKMARK SURVEY AND ASK HOW SATISFIED ARE YOU ABOUT THIS?
IF YOU REALLY WANTED TO LOOK AT OUTCOMES AND HOW THIS HAS IMPACTED THEIR LIVES, SAY, WHAT ELSE COULD YOU BE DOING NOW?
OR, YOU KNOW, WHAT DID YOU NEED THIS BOOK FOR?
WOULD YOU HAVE BEEN ABLE TO GET THIS BOOK?
I'M THINKING OUTLOUD AND BEING A LITTLE SLOPPY IN MY THINKING.
BUT WE COULD BE ASKING, HOW HAS THIS CHANGED YOUR LIFE?
WHAT VALUE HAS THIS BROUGHT TO YOUR LIFE?
 SO I HOPE THAT GIVES YOU SOME IDEAS.
I THINK, YOU KNOW, THE EXAMPLES I GAVE YOU WERE THESE VISION, YOU KNOW, ASSESSMENT THAT'S VISION WHICH I'M PERSONALLY VERY DRAWN TO.
BUT ASSESSMENT IS ALSO EVALUATING A PROGRAM.
EVALUATING A PROGRAM SERIES AND HOW ARE PEOPLE SATISFIED WITH IT?
ARE THEY GETTING WHAT THEY WANT OUT OF IT?
>> AND KARA, DO YOU HEAR FEEDBACK FROM THE COMMUNITY WHETHER BOARD MEMBERS OR YOUR FUNDING OR -- YOU KNOW, WHAT -- 
OR INDIVIDUALS?
>> YEAH, I MEAN, SO ANY TIME WE DO A SURVEY PEOPLE GIVE US FEEDBACK.
NOT ALWAYS THE FEEDBACK THAT WE WANT TO HEAR AT THAT MOMENT.
BUT YEAH, WE DO.
NOT AS MUCH AS WE'D LIKE.
PEOPLE ARE MORE LIKELY TO, YOU KNOW, OFFER COMPLAINTS THAN THEY ARE TO OFFER KUDOS WHICH IS A LESSON THAT I HAVE IN MY OWN LIFE TO TRY TO GIVE KUDOS MORE.
WE DO HEAR FROM PEOPLE ESPECIALLY STORY TIMES I KNOW WE HEAR A LOT ABOUT PEOPLE WHO LOVE THE STORY TIMES.
>> DOES YOUR LIBRARY HAVE A FORMAL DATA ANALYSIS AND ASSESSMENT STRATEGY?
>> WOULDN'T THAT BE NICE?
 I HAVE FOR MANY YEARS SERVED ON OUR STRATEGIC PLANNING TEAM SO DO ALWAYS BRING THIS COMPONENT TO IT.
AND BY THE WAY, LYNN SAID MAKE SURE TO REPEAT THIS, I'M HOPING -- WE HAD SUCH SUCCESS OVER THAT, YOU KNOW, ONE THING THAT WOULD MAKE MY LIFE BETTER AND BEFORE I DIE, I WANT TO -- I HOPE WE DO THAT FOR OUR NEXT STRATEGIC PLANNING AND DO IT FOREVER BECAUSE WE GET SOME REALLY GOOD INFORMATION OUT OF THAT. 
 PLEASE, STEAL IT, TOO.
USE IT IN YOUR LIBRARY.
>> I THINK THAT'S GREAT.
>> FANTASTIC!
>> IT LOOKS LIKE WE'RE THERE -- 
ABOUT THERE, JENNIFER?
>> YEAH, WE ARE AT THE TOP OF THE HOUR.
SO WE WANT TO BE MINDFUL OF PEOPLE'S TIME.
JUST A REMINDER THAT I WILL LET YOU KNOW WHEN THE RECORDING IS AVAILABLE FOR TODAY'S SESSION.
AND IT WAS GREAT TO HEAR FROM SOME OF YOU THAT YOU LOOK FORWARD TO SHARING IT WITH YOUR COLLEAGUES.
HOPEFULLY YOU JOIN US FOR THE FULL SERIES.
AND WORK THROUGH SOME OF THE WORK WE'VE OUTLINED MANY OF THESE STEPS IN THE LEARNER GUIDE.
SO THERE'S WORK FOR YOU TO DO, HOMEWORK FOR THIS WEBINAR SERIES.
FOR YOU TO WORK THROUGH BETWEEN SESSIONS.
AGAIN, FEEL FREE TO CUSTOMIZE IT IF YOU HAVE SOME OF YOUR OWN RESOURCES OR QUESTIONS YOU WANT TO DISCUSS AS A TEAM, DEFINITELY ADD THOSE INTO THE GUIDE.
AND WE'LL LOOK FORWARD TO SEEING YOU NEXT TIME AND WE LOOK FORWARD TO HEARING FROM YOU HOPEFULLY IN CHAT AND DEFINITELY UTILIZE THAT HASHTAG.
THANKS TO ALL OF YOU WHO WERE ACTIVE ON CHAT AS WE MOVE THROUGH TODAY'S SESSION, THE LIVE DATA FOR IMPACT HASHTAG WILL HOPEFULLY STAY LIVELY AS WE MOVE THROUGH THE SUMMER.
AND WE LOOK FORWARD TO SEEING YOU ON AUGUST 13TH AND THANK YOU SO MUCH TO KARA AND LYNN FOR BRINGING YOUR GREAT WORK TO THIS FIRST SESSION IN THE SERIES AND AGAIN, THANKS TO ALL OF YOU WHO JOINED US TODAY.
WE WILL SEE YOU ALL NEAR THE END OF THE SUMMER.
THANKS SO MUCH.
 BYE-BYE.
 >> THANK YOU, EVERYONE!
>> BYE!