Summary of Evaluation Techniques

After carrying out our task analysis, cognitive walk through and heuristic evaluation we identified several weakness in our proto-type. These weak points will now be listed:

From the Task Analysis:
- If the user forgets their PIN number
- If the user can't find the desired product on search
- If the user does not follow the on screen navigation instructions
- What happens if the user is refused credit
- Conflict between the cashier and the user between prices


From the Cognitive Walk-through:
- Language Support
- Annoying recommendations keep repeating itself
- An on screen guide showing how to enter the card
- Intelligent search
- Display photos of the items
- Display where in the shopping process she is - what's next to be found on the shopping list
- Show an animation when an item is being detected in the trolley
- Allows the user to set the fat, sugar and salt content intake
- Shows all the offers that are applicable to the user


From the Heuristics Evaluation
- The system does not tell the user what part of the shopping process they are at, for instance does not tell the user that they are logging in
- The measurements are in metric, but the old people understand imperial
- Change font size control
- Change 'Begin Route' to 'Start Shopping' a phrase they are more familiar with
- Too much Red is used
- Help screens need to available in necessary places

Heuristic Evaluation

I'm going to write an HCI expert, and using Jakob Neilsens "Ten Usability Heuristics" I'll perform a Heurisitic Evaluation on the ShopNav system. I'll be using the diagrams of the screens that were provided by Baber earlier on in the blog to make comments on.

To make this as least painful as possible I'm going to write down the Neilsens Heurisitics and underneath each heading try and make a comment on HCI point about our software.

Visibility of system status.
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
The ShopNav unfortunately fails terribly at Visibility of System Status. The Log on Screen does not mention that is actually logging the person on, but just to insert the store card and then to enter a password. When first logged in the user does not know that they need to search for an item before they can start ShopNav. When the ShopNav is giving directions it does not state what item it is taking the user to.

Match between system and the real world.
The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
When directing the user, the ShopNav displays in metric, old people would prefer imperial measurements.
The word 'Begin Route' could cause confusion, but renaming the button to 'Start Shopping' would clarify to old people that start shopping is exactly that.
The text can be too small sometimes for the old people with poor eye sight to read, need a font size changer feature.


User control and freedom.
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
The system does allow the user to remove items so if anything does get entered incorrectly it can be removed easily.
The system however does not allow the user to log in because there is no keyboard to enter the pin number!
There is a lot of use of red which is not good because for old people most of them could experience colour blindness which would leave them not seeing much on the screen.

Consistency and standards.
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Again, the word 'Begin Route' could cause confusion, but renaming the button to 'Start Shopping' would clarify to old people that start shopping is exactly that.
Predictive word like Open Office where if the user types in C, then all the ingredients listed that begin with C appear and it carries on filtering with every key press so that way the user can then select from the final 10 in the list or something of that type of UI.

Error prevention.
Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
The final screen shows that before the user is about to pay it does have a confirmation screen that allows them to confirm the price of what they have just bought.

Recognition rather than recall.
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
As everything is on screen the user has all of the information to hand which means that they don't have to remember anything.

Flexibility and efficiency of use Accelerators
Unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
The more experience the user gets the more they will learn that they can add to their shopping list favourites to save them having to re-enter data into the system. Thanks to the store card the device can also monitor what items the person usually buys and asks them if they would like to add it to the shopping list.

Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
The page that confirms the user is not needed, at this point anyone that has stolen a card and knows the pin number is going to confirm the person. The help button on the user confirmation page is not needed, however a help button on the login pages would be more helpful.

Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
If the store card is inserted incorrectly then the device should display some sort of message

Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
There is no help for the main screen, some interactive help to make the user take the neccessary steps to start the shopping experience, such as searching for an item before hitting 'Begin Route'.

Cognitive Walkthrough by Tony

I park up to the closest parking space available… saving me walking distance as I have troublesome knees. I enter the store and head straight for the brand new super duper trolley.

Task 1: Entering Store card into ShopNav device
The screen tells me to “Insert Card”, so I pull out my wallet and search out my store card. I look around for a split second, finding the place to insert the card, and put it into the device. The screen then informs me to “Enter PIN”, as I deal with a lot of cards, I change all my PIN numbers to the same number, so I don’t have any trouble remembering.

Positives• I like dealing with the Store Card and not with cash
Negatives• Another card that is clustering up my wallet• As I have to insert the card, I am worried I may forget to take it out when I leave the store

Task 2: Using the product search facility
My weekly diet consists of pizza, so it’s the first thing that I want to search for. The search button on the interface is very visible, and it then takes me to the search screen. I type in pizza, and a list of pizzas comes up, I select the 1 that I want and select the “add to shopping list” button, which consequently adds it to my personal shopping list. I also search for Tagliatelle Pasta, but I get a spelling mistake, and it does not come up… so I have to result in having to walk to the pasta section myself to find it.

Positives• I can search for a product without having to search up and down isles
Negatives• If I get a spelling mistake, my search doesn’t return results• Having to type long names of products is time consuming

Task 3: View Recommendations
After searching for pizza, there is an alternative products list displayed next to it, displaying recommendations of complementary products. I love this feature, as I am the sort of person that likes to experience new things, the recommendations offers me a whole new world of food choices that I never knew about.

Positives• Recommends complimentary foods that I never knew about.
Negatives• Sometimes it can be annoying clustering up the screen with lots of products I don’t want

Task 4: Navigate user around the store
After creating my shopping list, I select the “Confirm List” button, which then provides me with the most efficient route around the store. As an individual that finds it hard walking long distances due to my troublesome needs, I found this aspect of the device revolutionary. As I walked around, I did want to veer off sometimes just to see some offers that are down certain isles.

Positives• Saves a lot of time and walking up and down the supermarket and isles searching for my product
Negatives• Sometimes I like to walk down isles viewing something new, or any great new offers that are on

Task 5: Adding product to Trolley
Whilst following the ShopNav’s route around the supermarket, I pick up the product and place it into the trolley. The product gets automatically added to my purchase list on the device, and informs me of it, and displays the total cost of the shopping so far. I also add Coca Cola into the trolley, the device beeps, and inform me on the screen that high level of sugar content is not recommended for my high blood pressure. As a person that hasn’t taken care of himself over the years, I didn’t realise how much sugar was actually in coca cola.

Positives• Thinks its great that he can see a running total of the cost of all goods in my trolley• It’s fantastic the way the device informs me of the health issues that come with products. Its invaluable information that everybody should be educated on.
Negatives• As I am overweight, I am looking to reduce my fat content, but when I added sausages into the trolley, it didn’t highlight that fact. Maybe a system, where I could put in a maximum % of fat content, so the system could highlight the issue just as it did for the coca cola.

Task 6: Removing a product from the trolley
I subsequently remove the coca cola from the trolley, and put it back onto the shelf. The device informs me that the product has been removed from the purchase list and the running cost total is recalculated and the cost of the coke taken off.

Positives• Great that the system automatically deals with removing the product from the purchase list, so I don’t have worry about purchasing goods that I have taken out.

Task 7: Paying for shopping
After a long shop, the thing I hate most is waiting in a long shopping queue waiting to pay for my goods, also standing their waiting for the checkout person to process all my bank details and enter my PIN number. This automatic purchasing attribute is fantastic, as I walk over the RFID readers, a “Confirm Purchase” buttons pops up showing my total cost and the list of products, I press confirm and the cost is automatically billed to my bank. My shopping time has halved, and can look forward to the evening not worrying about my knees hurting as much.

Positives• I love the fact that I don’t have to wait in queues for longs times
Negatives• Not to sure about the reliability as I have never used this before. What if it charges me twice or the RFID readers fail?

Task 8: Paying for shopping by cash I walk over to the checkout and purchase my goods in the traditional way.

Positives• Tried and trusted
Negatives• Have to wait in long queues, and there’s nothing worse for my knees

Cognitive Walkthough by Maureen

Despite it being a delicate, warm, spring day, Maureen insists on her coat as she leaves the house for the supermarket.

Having made her store card for the ShopNav during her previous visit with the assistance of a store employee, she’s now ready to give it a go.

She already feels a bit uncomfortable by breaking her old routine; having to collect the trolley from a new area of the store, but she’s willing to give it a go.

Task 1: Entering Store card into ShopNav device
“Swipe Card” reads on the computer screen of the ShopNav (or NavShop as Maureen remembers it). She swipes the card several times with the barcode facing the wrong way, however eventually proceeds to the next screen. It worked on her final attempt before she approached a store assistant – the only reply she got was “try again”.
“Enter Pin”. Knowing a strong memory isn’t the best of her qualities, she wrote the PIN code when she made her store card last week. In her purse, she retrieves the small slip of paper and carefully enters. A sigh of relief it worked!

Improvements
- Insert card instead of swiping
- Another solution for pin? Is it actually required?

Task 2: Using the product search facility
Maureen’s shopping list:
Milk - Bread - Baked Beans - Oranges - Murray Mints - Tomato Soup x2 cans - Eggs - Orange Squash -Teabags

The instructions on the screen clear and she knows what she must do. However typing is very time consuming and Maureen’s becomes a bit frustrated with her slow typing speed and having to stand in one spot for so long.
Selecting products on the whole is fine, however, there is some confusion where Maureen is unable to remember the exact name of the product. She’s always remembered and gone by the pictures and colours on the label.

Improvements
- Intelligent search
- Display photos of items


Task 3: View Recommendations

“Ahhh finally - Can I just get my food now?” She mumbles to the screen. Having clicked done in the shopping list, a recommendations list appears. Maureen is slightly shocked, and reads the screen intently. After reading a bit, Maureen is a bit confused and decides to ignore it all by pushing “Begin Route”.

Improvements
- Display where in the shopping process she is – what’s going to happen next
- Show on what basis the product recommendations are coming from


Task 4: Navigate user around the store
Very happy to be moving Maureen carefully follows the large arrows directing her to the appropriate aisles. She becomes a bit concerned about being put off track when she has to move around obstacles. However this concern is eliminated by the “distance to item” reading on the screen.


Task 5: Adding product to Trolley

She places a bottle of milk into the trolley… a beep… she immediately looks at the screen for any changes. “Ah, the milk has appeared on the side of my screen – How did that happen?”.

Improvements
- Explain when you place a item in the trolley the barcode is automatically read, perhaps show an animation


Task 6: Removing a product from the trolley
Walking away from the ‘Juice’s Aisle’, Maureen becomes aware of an offer on a bottle of squash that she would like to swap with the bottle currently in her trolley. Apprehensive to experiment with this new piece of technology, she decides to ignore the 10p saving.

Improvements
- As with Task 5 – show an animation of the correct action/how barcode reader works


Task 7/8: Paying for Shopping
A notification appears on the screen informing her she has all the items she entered at the start. An option to pay by “card” or “at the counter” appears. Instantaneously she pushes “at the counter” for she is not keen on credit cards.
She goes straight to the cashier and hands the total displayed on the screen. The cashier inspects the trolley and screen and helps Maureen pack her bags.
“Do you not trust me?” questions Maureen “Do you suspect me of not declaring all the items in my trolley?”

Improvements

- Payment without the need of a cashier - being able to walk directly to packing counter

Cognitive Walkthrough by Major Singh

Cognitive Walkthrough by Major Singh

Tony will answer the following 3 questions when performing a cognitive walkthrough using the task analysis

Q1: Is the correct action made evident to the user?
Q2: Do I understand a device action description with what I am trying to do?
Q3: Do I know if I’ve made a right or a wrong choice?

After the cognitive walkthrough, Singh's criticisms were:

- No Punjabi language support

- Annoying recommendations repeating themselve eg. Whiskey prohibition!

- An onscreen guide is needed for entering store card

Task 1: Entering Store card into ShopNav device
Q1: Is the correct action made evident to the user?
The device screen informed me to insert store card into device.
The store card reader was visible, but I wasn’t sure on which way to enter the store card.
An onscreen guide to entering the card would be useful

Q2: Do I understand a device action description with what I am trying to do?
I understand that the store card is reading my card and loading my details.

Q3: Do I know if I’ve made a right or a wrong choice?
Yes, my card was rejected when I entered it the wrong way.

Task 2: Using the product search facility
Q1: Is the correct action made evident to the user?
The search button was very visible on the main screen

Q2: Do I understand a device action description with what I am trying to do?
I understood as I entered my search, the system was looking for the boneless chicken breast meat I wanted

Q3: Do I know if I’ve made a right or a wrong choice?
I found it difficult to enter my product as my English is not brilliant; I want a Punjabi version!

Task 3: View Recommendations
Q1: Is the correct action made evident to the user?
I am given a list of recommended foods depending on my medical conditions; I was recommended salt free peanuts after picking up KP Salty Nuts.
This ensured my blood pressure wouldn’t rise.

Q2: Do I understand a device action description with what I am trying to do?
I understand the system is looking out for my best interests

Q3: Do I know if I’ve made a right or a wrong choice?
I am repeatedly told not to pick up any Whiskey! This is very annoying!
I want Whiskey and this phencaude computer can’t stop me!

Task 4: Navigate user around the store
Q1: Is the correct action made evident to the user?
This was a great feature, I’m get annoyed at walking round the store aimlessly and part time teenage workers not helping you find products.
The device navigates me around the store in a very efficient manor
Q2: Do I understand a device action description with what I am trying to do?
Yes, I simply follow the on screen directions
Q3: Do I know if I’ve made a right or a wrong choice?
I intentionally took a wrong turn and the system did not notice. However, there is a ‘Stop’ feature I could have used when I fancied taking a detour to the ‘MEAT’ aisle.

Task 5: Adding product to Trolley
Q1: Is the correct action made evident to the user?
This was a great surprise, I didn’t realise the trolley scanned products. As I put an Organic Chicken leg into the trolley, a price was displayed! Bruuahhh!!
Q2: Do I understand a device action description with what I am trying to do?
Yes, I realised the product was being scanned as it was dropped into the trolley
Q3: Do I know if I’ve made a right or a wrong choice?
Yes, the product information was displayed as it was scanned.

Task 6: Removing a product from the trolley
Q1: Is the correct action made evident to the user?
I accidentally put some Quorn into my trolley mistaking it for Meat! As I removed the fake meat, it was taken off my running total.
Q2: Do I understand a device action description with what I am trying to do?
Yes
Q3: Do I know if I’ve made a right or a wrong choice?
Yes

Task 7: Paying for shopping
Q1: Is the correct action made evident to the user?
I was aware that I could pay by store card, but I pay with hard cash only. I didn’t use this feature
Q2: Do I understand a device action description with what I am trying to do?
NA
Q3: Do I know if I’ve made a right or a wrong choice?
NA

Task 8: Paying for shopping by cash
Q1: Is the correct action made evident to the user?
I went to checkout as usual, the cashier confirmed my shop and asked me to give it her the money. I paid for it.
Q2: Do I understand a device action description with what I am trying to do?
Sure
Q3: Do I know if I’ve made a right or a wrong choice?
I’m always right, don’t worry bout it

Task Analysis

Cognitive Walkthrough and Task Analysis

Cognitive walkthroughs are performed at any stage of design using a prototype. This approach is intended especially to help understand the usability of a system for first-time users, that is, for users in an exploratory learning mode.

A cognitive walkthrough starts with a task analysis that specifies the sequence of steps or actions required by a user to accomplish a task, and the system responses to those actions. The designers and developers of the software then walkthrough the steps as a group, asking themselves a set of questions at each step.

Data is gathered during the walkthrough, and afterwards a report of potential issues is compiled. Finally the software is redesigned to address the issues identified.

I will blog our task analysis and then each of our personas can carry out a cognitive walkthrough.