Jax Wechsler

Operationalising In-house Usability Testing

I spent some time contracting this year at a large organisation as part of an internal experience architecture team. One of my tasks was to operationalise usability testing within the online team who were responsible for the design and development of both online and mobile sites and apps.

In the past I have done a fair amount of usability consultancy for large organisations as an external consultant. Ahhh the joy of being a consultant, where the fact that clients are paying a high fee for your services makes your work seem valuable.

Selling in and implementing usability testing in this context (as a “free” internal service) was an interesting experience and I thought I would share some of my learnings.

1. Naming the initiative

When I started this initiative I realised that some education was required. I presented an approach to the heads of online (experience architecture, development and creative) which went fairly well however a curious thing occurred afterwards whereby I got a meeting request from the test lead (part of the development team) requesting involvement in my ‘testing’ process. It seemed that the term ‘usability testing’ seemed to suggest an association with UAT (user acceptance testing). So we changed the name.

Learning 1: Use the term ‘user review’ instead of ‘usability testing’

2. The importance of framing the research

My first session involved one on one sessions with users testing a site on a staging server. Parts of the business were very keen to launch this version, whilst others considered it still in need of substantial front-end fixes before launch. Irrespective of ones opinion, the site did have some bugs, the severity of which was debatable.

When presenting my findings, I realised that some meeting participants saw this ‘user review’ exercise as an exercise to catalogue bugs rather than a way of gaining information about how actual customers interacted with the site. Instead of wire-frames and recommendations for improvements, they seemed to expect a catalogue of existing bugs. From this first presentation I learned the importance of a bit of educational material at the beginning of the presentation and the importance of framing the objectives of the research.

Learning 2: Focus on user perception and the users’ journey on a site rather than just cataloguing bugs. Video is a great way to illustrate usability issues and illustrate customer delight or confusion.

3. Reporting Language

The aim of these ‘user reviews’ is ultimately to help improve the experience of customers with online and mobile products. Often this involves the unlocking of additional funds. With this in mind, I realised that it was important to communicate my findings using business terminology with the projects’ associated KPI’s in mind.

For executives and business people, numbers have power and numbers with $ signs even more power. Being able to set up tests from which I could benchmark the performance of existing properties compared to the performance of new ones was also valuable to frame the findings and quantify possible improvements. Business people usually value “conversion” more than the provision of “superior customer experiences” (I confess I have found that unfortunately not everyone yet gets the relationship between the two!). For example, after some remote testing I found that with a proposed IA that 40% of people were more able to find support pages about a common problem using the proposed IA than they did using the current sites IA. Speaking to the call centre manager, I supplemented my report with statistics as to what % of calls were currently made in relation to this matter. Through relaying this information in my report in a concise and easily digestible format manner  the improvements were made more tangible and relevant.

 

Additionally, as I am sure you are aware, execs are really busy and don’t have time to read long winded reports. Particularly reports that they have not specifically commissioned on topics that they may not yet be 100% convinced about. Making your reports scannable with graphs highlighting efficiencies, savings and potential profits ratifies both the importance of fixing usability issues as well as your work.

Lesson 3: Pictures speak a 1000 words…but projected percentages speak 1000 pictures.

 

In Summary: Complimenting your usability research work with education and well considered reporting can really help businesses engage with your in-house usability initiatives.