Creating a Culture of Testing, Part 2 of 3: how to do build a simple test.

Struggling to improve your website? Do it fast and easy.

The Culture of Testing: Part 2

Last week we covered the value of creating a culture of testing. This week we’re going to show you how to test simply, effortlessly, with incredible impact that will galvanize the troops and be super-fun in the process.

Start the cycle of testing – with 1 question.

What if you had 50 of the most burning questions about your business operations – that includes customer service, website functionality, and product development – answered by the end of the year? Would your sales go up? Yep. Even 20 questions per year would help you in big ways. And yet, in my experience, very little time and energy is spent testing. Why? Most people

  • Think that they don’t know how to test,
  • Think that they don’t know what to test for,
  • Think that testing is too hard,
  • Don’t know how to start testing,
  • Think they need a lot of money to test.

Yet, when you think about it, testing is nothing more than asking questions. One simple question per week, to as many customers as you ask without interrupting workflow, is the ticket.

The informality of the testing is key.

Again, it’s important to understand that this is ad hoc testing; testing on the fly. This is not a formal process. All you have to do is make sure that employees actually ask questions.

Just get started.

It’s also more important to just get started, rather than do a great deal of pondering. Start small. Have all your employees ask the most pertinent question to the company for one week. Just have them ask the question, then let go. This is a great question:

“If there was one thing that you would like to see
on the website, what would it be?”

It’s a great starter question because it’s open-ended; it’s not specific enough to give you precise answers, but it is very good at generating questions.

Here’s what you need:

  • Pushpins
  • Pushpin board
  • Clipboards
  • Pens
  • Paper (or use the MS Word printout below)
  • 3×5 cards, splurge and get different colors

During the week it’s always on the employee’s mind. They might not want to do it at first. But when they start to get answers, they feel like they’re part of something.

The testing cycle is 6 steps, repeated each week.

Start testing.

The cycle.

They cycle is broken down into 6 steps. Each step is described below.

Since this is a cycle, it repeats itself. That means that at the end of the cycle, the primary goal is to ask a more pertinent (or valuable) question. That’s the way testing works: test, examine, retest, examine, retest, rinse, repeat.

1. Ask the right question.

The first question will be the one that you want answered most.

Get everyone in a room and spitball the first question. Remember we’re only asking one question, here; try to find a single common problem, company-wide.

Go around the room and ask each person what the biggest problem customers have encountered with the website. If they can’t come up with a common problem, use the ‘starter question’ above. By the way, this question is easily adapted for customer service or product development questions).

Click on image above for MS Word Testing Page.

Create a single piece of paper with the question at the top (or printout our easy Testing page). Photocopy and distribute with clipboards. Have each of the employees carry their clipboard with them. Don’t rely upon computers; computers have to be walked over to, turned on, programs have to be opened – argh. Just simple pen and paper is fine. Clipboards are at the ready and allow employees to doodle ideas on them.

2. Start collecting answers.

Now, starting Monday morning, have your employees grab their clipboards, to carry with them at all times. They might object or think it’s goofy; don’t give up! Once they get into the habit, it’s not a problem. Note that it might be 3 or 4 weeks before they internalize the habit.

Every time they have an opportunity to ask the question, they should ask it. Remember, this is an open-ended question, it’s supposed to generate new questions. Allow employees to explore the question from different angles, and allow them to go off-topic and ask more questions if they can.

3. Spot assessments, each person.

Note that the printout has a line of text in each answer; A better question would be:. This is a good thing. Each employee should reassess the question after getting an answer. The more they ask the question, the more the right question will emerge. This reassessment is done independently of others.

If you’re the boss, please don’t jump in and ask them what they’re finding. Just let it go. Wait until the end of the week to discuss the findings. Interruption will lead to trying to change the question in mid-week. This will drive your employees crazy (and ruin the data set.)

4. Collect/collaborate with a few others.

However, around Wednesday you should allow the employees to start talking with each other on what they’re finding. They might or might not talk about the question between them – some might still find it silly, but most will find it (perhaps reluctantly) intriguing.

The trick is for YOU as a boss not to interrupt the employees’ workflow – that’s just annoying. As the boss, you color their reactions to things – this you do not want. But if you let them talk (when they can) with each other, they’re going to spot commonalities between the answers. This will intrigue them (and give you a different perspective than your own about the website.)

If they don’t talk with each other, that’s fine. They will at the end, anyways in the group meeting. But they might get very excited about what they’re finding, and really get into it. It gives them power to change the business for the better. And that excitement is contagious (the ‘this is silly’ employees will engage more often.)

5. Pool the data.

Start around Friday morning to pool the data. Appoint someone to tally the answers.

Once that person starts to a pattern emerge in the answers, have them write that particular problem onto a 3×5 card and post it on the pushpin board. Add a pushpins for each answer given that falls into that problem. For instance, if one of the answers was, “I don’t like the color,” then put a 3×5 card that says, “color” onto the pushpin board, and if there are 7 ‘color issues’, then put 7 pushpins into it.

Keep doing this until there are no answers left. Then the fun begins!

6. Deep assessment, whole group.

Friday afternoon, brainstorming mode: Start with the 3×5 that has the most pushpins. Then ask questions. Try to define the answer further. In the example above, did they suggest another color? Was the color described in a certain way (icky, glaring, too bright, couldn’t read the text on top of it, etc.)?

Repeat until all answers are discussed. Now, something magical will probably happen. They employees might seem very excited; they’re asking to contribute to something outside their normal work. That’s creativity for you.

(7.) Start again.

Now, there are two ways you can start the cycle again, next week:

  1. The fix is simple. So fix it! Hand it off to your web develop and discuss it with them. Ask for their input (if they were not part of the discussion already). If there was a related answer (but not on the same 3×5), fix that too. This means you have to come up with an entirely new question. But from the answers generated from the ‘A better question would be:’, this will probably be easy to do.
  2. The fix is not simple or cloudy. Then brainstorm from the A better question would be: ideas for the next question.

How’s that for simple?

Then reward your employees.

Oh, and do yourself a favor: order pizza for your awesome testers! They deserve it; they just uncovered very real data for your business!


You might get some objections to the process. Don’t let it stop you!

  • “But how do I ask the question?” This is not something you can do much about; if they’re having problems asking the question, give them some insight on how you ask the question.
  • “But what do I ask?” The question – it’s right there in front of them.
  • “It’s too hard to ask questions if I’m helping them!” Some people know how to ask the question without distracting the customer. Find out who does it best, then bring it up in the meeting next week on how that person ‘slips in’ the question.
  • “But how do I start?” You just do. There’s not getting around this. After a few times, they’ll develop a pattern of asking the question that is not annoying to the customer.
  • “But shouldn’t a professional be doing this testing?” No, they shouldn’t. These types of ad-hoc questions can often only be asked by employees. Formal testing is multiple questions from a particular segment of customers. The thing to remember is that this is a weekly cycle of testing that can be adapted and shaped week to week – not something a testing company can do easily (or cheaply). Besides, the ‘low hanging fruit’ of problems are easily picked by the employees, and that will build morale.

Next week: more than one test per week.

Part 3 of 3 will cover doing more than one test per week. If you’re lucky enough to have a large company, there are other types of tests you can do. A simple one is to come up with a single test per department. This targets each department’s biggest problem every week. You can also do a split test where you reward the customer in different ways for asking the same question. Tune in next Friday for the finale!

Greg Cox

Analytics Geek. Gregory likes Motorcycles, Cooking, and Sciencey stuff.