Evaluation Plan
How can we know that our design solution worked? The final, but equally crucial, stage of the process was to objectively evaluate our own design. But to do that, we needed a plan.
This section was informed by the works of Brown & Green (2015).
How do you know that what you are doing is effective? Some of it probably isn't, but if you don't have feedback coming into the system, you'll never be sure what is working and what isn't.
Dirksen, 2016, p. 286
Formative Evaluation
Evaluation Method
Now that we had a working prototype, it was time to put it into the hands of our potential users and see what they thought about it. Using the Dick et al. (2011) evaluation method, we sought out at least 3 users of differing learning abilities to conduct usability tests with.
Phase 1: Usability Testing
The Method
Step 1
Explain to the learner that you would like his or her reactions to new instructional materials.
Step 2
Gives a pre-test on the content.
Step 3
Encourage the learner to talk about the materials as he or she goes through it.
Step 4
Give a post-test.
Step 5
Note the time it takes for the learner to complete the
Next Steps
Using the feedback, reiterate on the prototype, then repeat this process.
How We Applied It
Step 1
We created Usability Test Questions first. Going by the Dick et. al method, we organized our audiences by their prior knowledge levels (low, middle, high). The link to our Evaluation Plan and its questions can be found here.
Step 2
We each tested with a volunteer usability tester 1-on-1. We asked them to use the Think Aloud Protocol and took notes on what they said and how they behaved.
Phase 2: Data Analyzation
Affinity Mapping
Just like we did with our initial research phase, we decided to analyze our usability testing data by affinity mapping on Miro.
Evaluation Questions
To help guide our Evaluation methods (above), we created the following Evaluation Questions which were designed to determine who it was for, which methods we used, and why we chose to use those methods.