Case Study： Usability Evaluation
The A.R.T. Website:
​Usability Report
A usability evaluation project aiming to improve event-searching experience with the A.R.T. website.
-
Sector
​​
-
My Role
​
-
Project Time
​
User Research/ Website/ Event Searching
UX Researcher
Jan 2020 - May 2020
The
​Client
NYC Archivists Round Table was founded in 1979 and is known as a nonprofit organization. Serving a diverse group of more than 400 archivists , librarians, and record managers in the NYC area. Also one of the largest local organizations of its kind in the United States. The mission of ART is to educate the public on the legal, historical, and cultural value of public and private archives and manuscript collections.
The research team understood the needs and the goal of the Archivist Roundtable team through a video conference in early March 2020. The Archivist Roundtable team wanted to optimize the website structure to not only visually attract users but also make it easy to navigate. Based on the feedback from the Archivist Roundtable team, the targeting users for the website were categorized into the following two groups: professional archivists and regular users who might be interested in archives. According to the current user habit, this project will focus on the desktop version of the website.
Maya Naunton
Director of Education
"We've noticed that our current website isn't easy to use and hence could not accomplish our goals. We hope to have a more appealing interface which gets our users what they want."
Maya Naunton
Director of Education
"Top 4 most important goals we want our website to achieve:
-
Usable
-
Well organized
-
Easy to navigate
-
Visually attractive "
Defining
Scope:
The A.R.T. website has 3 goals:
1. To introduce the Archivist Round Table to the public.
2. To provide information about what are the new events the A.R.T. is currently providing.
3. To keep A.R.T. members informed about what is new with the A.R.T.
Believing "quality over quantity", the research team decided to focus on one goal of its client. Goal No.2 was finally chosen as target of this research, because:
First of all, "event searching experience" would include most amount of interactions for the A.R.T. website.
Second, "event searching experience" covers all the features that would help the A.R.T. website to improve its other pages.
​
"Event
Searching"
Method--ology
This research used a contoured version of traditional user test method.
​
One challenge the team was facing is the COV-19. With this background, In-person interactions and controlled test environments have to be compromised for the good of participants and researchers’ health and safety.
The researchers thus used traditional user test methods via the internet. The team developed following procedures to maximize the accuracy of data collection:
1. To control the environment remotely, participants were asked to finish all the tests on their computer desks within 45 minutes.
2. All the tests were accomplished with the same online meeting tool.
3. Participants were asked to open their cameras throughout the tests and to share their screens with the researchers at the same time.( In this way, researchers can still observe how users interact with the product and wouldn’t lose certain nuances such as body language and facial expression of the users.）
"A contoured version of traditional user test"
Designing
Task
1 scenario, 3 sets of tasks, 1 pre-test questionnaire and 1 post - test questionnaire is carefully designed for selected participants*.
*The A.R.T website sees its potential users as archive enthusiasts. For this reason, this study includes 8 researchers who use archives regularly and have expressed their interest in archives. All of them have never used The A.R.T website before. In another word, all of the participants could be seen as novice users.
"Create a real scene for testers. "
Senario
"You are an archive enthusiast who heard about Archivist Roundtable from your friend. You want to know more about it."
"What do users expect?"
Pre - test questionnaires
"What would you do when trying to seek events on a specific website?"
"How users interact with the website?"
Tasks List
Timeline-Related
-
-
A friend sent you a link to the A.R.T Awards ceremony because you told her that you are interested in this event. Can you please check it out and see when this event happened？
-
You want to know more about A.R.T events.Can you find out what event took place on November 01, 2019？
-
Can you describe your overall experience during the process of finding out the event? Is there anything unclear in the website? Did you meet any difficulties?
-
Did the overall experience meet your expectations?
-
Location-Related
-
Go back to “A.R.T. Awards Ceremony” Page. Can you tell on which location this event is taking place？
-
Are you able to find 3 events that took place in Manhattan?
-
Can you describe your overall experience during the process of finding out the event? Is there anything unclear in the website? Did you meet any difficulties?
-
Did the overall experience meet your expectations?
Registration-Related
-
Can you try to register for the event “Fashion: Now & Then” and describe your experience, how did it make you feel?
-
Can you try to register for the event “2019 New York Archives Week Awards Ceremony” and describe your experience navigating to it?
-
Can you try to register the event “An evening at Poster House” and describe your feelings?
-
Could you please compare the two register processes, do you have any preference or what do you think?
"What's users' overall impression on the website?"
Post - test questionnaires
"Can you please write down two descriptive words for your overall impression about the website? "
Measurement Criteria
Following the collection of data from a total of 8 user testing sessions, the usability team collected key findings and recommendations to improve event searching experience with The A.R.T website.
Findings and recommendations were pinpointed with guidance of following questions:
1. What are users’ expectations for an event searching experience?
​
2. What information users are looking for in search of a certain event?
​
3. What information on the homepage gets the most attention?
​
4. What information on the current event searching pages gets the most attention?
​
5. Can users find all the information they are looking for on current pages？If not, what information do they want but can’t find?
​
6. What do users like the most about current pages?
​
7. What do users dislike the most about current pages?
​
8. What are users' overall impressions on their event searching experience with The A.R.T website?
Final Deliverables
Insights
&
Reccomendations
Insight 1
Insight 1
&
Rec 1
Usability test participants found it hard to read the ART website home page. Most of them have difficulty finding out the event page. 60% of our testers went directly to the search bar to search for event information.
Users tend to be confused by current homepage.
​
They kept asking the research team: "Where should I click?"
Recommendation 1
Upcoming events should be highlighted on the homepage. While building a high-usability search system could be time consuming and technically challenging, the website may lead users to the right route by restructuring its home page hierarchy. Figure 1.2 has given an example of how the A.R.T website may change its home page by putting emphasis on upcoming events.
​The A.R.T website may give users more senses of what’s happening right now by providing an section describing upcoming events on the home page.
​The extra button asking users to “check out all events with the A.R.T” provides users an opportunity to find out more information about A.R.T. events.
Shown left are 2 examples which may help the A.R.T. website to improve its usability.
Insight 2
Insight 2
&
Rec 2
90% of test participants had difficulty finding the “switch to calendarview” button. 60% of them prefer listview while 40 % of them prefer calendarview. This phenomenon could be explained by various reading habits of the users. Those who prefer listview tend to be sophisticated text readers and find it easy to follow an event list organized chronically. On the other hand, those who prefer calendarview have high priority on schedule management. They find calendarview an intuitive way to find out if a certain event fits in their personal agendas.
60% of them prefer listview while 40 % of them prefer calendarview.
Recommendation 2
Apply a small calendar with event lists and make the “switch to calendar view” button easier to notice.
Shown left are 2 examples which may help the A.R.T. website to improve its usability.
The A.R.T website may present events in a more intuitive way by providing users a small calendar with the list of events.
Meanwhile, a “switch between views” button give users a chance to choose the type of view they prefer.
Insight 3
Insight 3
&
Rec 3
All the usability test participants found time and location information important when searching for an event. Most of them require the website to highlight time and location information on the event lists pages, otherwise with “too many words on the single page”, it’s hard for users to find out what they “need”.
All the usability test participants found time and location information important when searching for an event.
Recommendation 3
Make time and location information more apparent/significant to users when signing up for an event.
This is an example of how the A.R.T. website can emphasize time and location information
on upcoming events section.
For upcoming events, the A.R.T website may provide time and location information on the side for users before they click in to an event detail page.
This is an example of how the A.R.T. website can emphasize time and location information
on event list page.
Time and location information could be organized in a unified way so that users could find it easily.
This is an example of how the A.R.T. website can emphasize time and location information
on event detail page.
Time and location information should also be highlighted on each event detail page. The A.R.T website could provide more detailed information on time and location. As shown above, location information could be presented with a link to Google Map.
Insight 4
Insight 4
&
Rec 4
Usability test participants were”asking where should I click on for registration” repeatedly during the usability test, which indicates that users have high priority on finding out how to register when they are led to event detail pages. In addition, while some participants reported that they really liked to be led to a third-party registration platform, others were uneased by it.
Usability test participants were”asking where should I click on for registration”
Usability test participants have different preference on registration methods.
Recommendation 4
Since users have put high priority on registration, the “register” buttons should be in bigger sizes and look more eye-catching.Moreover, since users have various preferences on third-party registration platforms, there should be a notification sign before users are led to another platform to register.
This is an example of how the A.R.T. website can make "register'" button more significant when registration is still open.
The register button should be highlighted everywhere. In this example, “Register” button is considered the most import signal in the section. For this reason, the “Register” button here is highlighted with color and font size.
This is an example of how the A.R.T. website can make "register'" button less significant when registration is closed.
This example shows a way to present a registration button when registration is currently closed.In this example, registration button is less highlighted sending the message “registration closed”.