Checklist for Using LIBRA with Study Participants <http://www.cs.utexas.edu/users/libra> Copy for Reader There are several steps to complete. (1) Create your new account. (2) Choose a reading genre. (3) Rate books you have read. (4) If necessary, edit the ratings you have just given. (5) Get recommendations for new books from LIBRA. (6) Read LIBRA s explanation of your reading profile. (7) Read LIBRA s explanations of selected recommendations. (8) Rate some of the recommendations. (9) Repeat steps 4-8, if needed to improve LIBRA s recommendations. (10) Select two books for purchase. (11) Formally evaluate LIBRA s final recommendations. (12) Complete questionnaire 1 before leaving. (13) Receive and read books. (14) Complete questionnaire 2 for each selected book.
Reader: Pseudonym: Phone Number: E-mail: Date: Book 1: Date Ordered: Book 2: Date Ordered: There are several steps to complete. Checklist for Using LIBRA with Study Participants <http://www.cs.utexas.edu/users/libra> (1) Create your new account. (2) Choose a reading genre. (3) Rate books you have read. (4) If necessary, edit the ratings you have just given. (5) Get recommendations for new books from LIBRA. (6) Read LIBRA s explanation of your reading profile. (7) Read LIBRA s explanations of selected recommendations. (8) Rate some of the recommendations. (9) Repeat steps 4-8, if needed to improve LIBRA s recommendations. (10) Select two books for purchase. (11) Formally evaluate LIBRA s final recommendations. (12) Complete questionnaire 1 before leaving. (13) Receive and read books. (14) Complete questionnaire 2 for each selected book. Note: Tell readers that we are asking them to provide their names so that we can notify them when their books come in. All information about them in LIBRA is not attached to their name
but, instead, is attached to their pseudonym. Remember: Take notes about the session. Note down comments the user makes about LIBRA, especially suggestions for revising LIBRA and comments about the results. Please provide me with a summary of the session. Detailed Checklist for Using LIBRA with Study Participants (1) Create your new account. A. Give each reader a copy of the cover letter. B. Give each reader a copy of Questionnaire 1. Tell them that this Questionnaire will help us evaluate LIBRA. Tell them that they might want to answer some questions during the session and some questions after the session. C. Creating the new account. Go to the LIBRA page. Scroll down to the bottom of this screen. Click here to register as a new user. Establish a login account with a pseudonym the reader selects. Record the reader s pseudonym at the top of this sheet. The reader also selects a new password that is private to him or her. D. If the reader has logged in before, ask them to use a new login name. They should not use the same login name in the future. In other words, they are to select a new login name to use only for this test session. (2) Choose a reading genre. A. Reader will next choose a reading genre. They do so by first clicking on Change Genre or by clicking on step one on the page labeled Welcome to LIBRA! This will bring you to a screen titled, Change Selected Genre. B. The reader will click on the button preceding the genre of his or her choice. Click on Change genre, at the bottom on the list of genres. C. The reader will next click on option 1 to search for books in this genre. (3) Rate books you have read. A. Ask the reader to name one or more favorite authors or titles. Help them to input the information on
the page titled, Search for books to rate. B. Remember that LIBRA will search for authors exactly as they are listed on the Amazon page C. Encourage the reader to rate as many books as he or she can. D. Click submit ratings at the bottom of the page. E. Reader can repeat steps A-D, adding ratings for other books until the reader has rated at least 10 books. (4) If necessary, edit the ratings you have just given. A. If the reader needs to change some of his or her ratings, click Review Ratings. (Note: the reader cannot add new titles on this page. He or she would have to return to Rate Books to add more titles.) B. To save the edited ratings, click on submit ratings. (5) Get recommendations for new books from LIBRA. A. Click on the button, Get LIBRA s Recommendations. This may take a little time. B. LIBRA will give you a list of 50 titles. You can scroll down this list as far as you want to view LIBRA s personalized recommendations. C. Clicking on the title of the book will take you to Amazon s page on that book. (6) Read LIBRA s explanation of your reading profile. A. Click explain this profile at the top of the page, LIBRA s Recommendations. B. LIBRA knows six aspects of a book: author, title, description, subjects, related authors, and related titles. These variables are listed in the column labeled, Slot. C. Encourage readers to investigate some explanations for highly-rated features. D. Next, encourage readers to investigate explanations for negative profile features in the second (lower) table on the page. E. Ask the reader to answer question 6 on Questionnaire 1. This question addresses LIBRA s explanation of the reader s reading profile. (7) Read LIBRA s explanations of selected recommendations. A. Use back to return to the page, LIBRA s Recommendations. B. Encourage readers to explore the explanations of individual books by clicking, Explain this recommendation. C. LIBRA knows six aspects of a book: author, title, description, subjects, related authors, and related titles. These variables are listed in the column labeled, Slot. D. Encourage readers to investigate some explanations for highly-rated features. (8) Rate some of the recommendations. A. Use back to return to the page, LIBRA s Recommendations. B. Now, the reader will rate the books that LIBRA has recommended to them. Encourage the reader to rank books they know they probably would not like as well as those they will probably like. Don t only provide positive feedback. Negative feedback will also help improve LIBRA s recommendations in the next round.
C. Click submit ratings at the bottom of the page of 50 recommended titles. (9) Repeat steps 4-8, if needed to improve LIBRA s recommendations. A. If the reader feels that he or she is still not getting good recommendations, advise them to repeat these steps 2 or 3 times. This would especially apply if the reader has already read all of the recommended titles from the first list of LIBRA s Recommendations. (10) Select two books for purchase. A. Now you have a finalized list of recommended books from LIBRA. B. If you do not end on a page of recommendations, Go to Get LIBRA s Recommendations. C. Ask the reader select two books he or she would like to read. D. Click on the title of each book to go to the Amazon page. E. Print two copies of the Amazon page for this book. F. Click on Explain this recommendation for each of the two books the reader orders. Print out two copies of the resulting explanation. G. Make sure the total cost for the books is not more than $40.00. H. Attach the copies of the Amazon pages and the copies of Explain this recommendation to this checklist. I. Tell the user we will contact them by phone or e-mail when their books have arrived. J. Tell readers they have one month to read the book. (11) Formally evaluate LIBRA s final recommendations. A. Go to the Evaluate LIBRA s Recommendations page by typing in: http://www.cs.utexas.edu/users/libra/evalrecs.cgi B. This will allow us to evaluate the accuracy of the recommendations that LIBRA is making. C. You will see the list of books that LIBRA recommends to the reader. We will now use this list for another purpose. D. The reader now has an opportunity to tell us what he or she thinks about the recommended list. E. Encourage the reader to rate as many of the 50 titles on the list. F. You might stop at selected books to ask the reader: What do you think of this rating? G. Click submit ratings at the bottom of the list when done. H. You will then receive a thank you message. I. These ratings will be saved on the LIBRA server and will be analyzed later. (12) Complete Questionnaire 1 before leaving. A. If the reader has further comments, they can e-mail us after their session. (13) Receive and read books. A. We will order your books and contact you when they arrive. B. Attach one copy of the Amazon page and one copy of Explain this recommendation to the books and give them to the reader.
C. Give the reader two copies of questionnaire 2 and one campus routing envelope (addressed to Dr. Loriene Roy, GSLIS, SZB 564, D7000)+ (14) Complete questionnaire 2 for each selected book. A. You will receive a copy of a questionnaire to complete for each of the two books you read. B. Please complete the questionnaire and return it to Dr. Roy via campus mail. Revised: 26 June 2000.