User testing result and evaluation of the Video Prototype

First of all, thank you all for helping me do the user evaluation especially when I submitted late. I'm very appreciated for this. :)

Outcome and Reflection

As I did the user evaluation online instead of in the lab. I could only ask them to watch my video and did a survey designed by me. Which means I could not observe how they watch the video and how many times did they watch it. Also have no idea are they distracted or engaged while watching the video? However, you can tell that they watched the video carefully from the constructive feedback.

The survey contains two types of questions, one is multiple choice with the points from 1-5, to simply grade the communicating effects such as the rules of the game and the design rationale, the concept itself, as well as the quality of the video and audio. The other one is the text question, in order to get their opinions of the concept, rules, and potential problems of the game.

Based on the feedback, I can tell that I did get the basic game idea across to them and I'm so proud of this. Besides, not all people thought it is an interesting game. The average score of whether it is interesting is 4. This is fine, after all it depends on personal interest, what really matters is why they thought it is not interesting. There are several common reasons:
- hard to use body to control
- energy-consuming
- not significantly different from Space Invader 

To me, body control is really hard and energy-consuming. I also concerned this issue while shooting the video, even did a short time shooting tortured me a lot. Nevertheless, when I thought of the games I played with Kinect and they are also like this, I don't think it's a big issue. Certainly, if I can think of a way to minimize the negative effect of the body control system, I will. Besides, I have to admit the game mashup idea really just change the rules of Twister and interaction of Space Invader rather than the rules of Space Invader. 


As for the potential problems, there are also some brilliant and farsighted comments from them.
- challenging in the high level when the aliens move faster
- the sensibility of the control pat 
- vague gestures: if the feet could only be in the middle squares. 

The faster speed of the aliens bothers me a lot as well, it took me a while to crash them by using the keyboard, let alone the body control. I could decrease the difficulty to tackle this problem by limiting the maximum quantity of the lights to 2. The user thus could just use both hands to shoot, which is much easier. I didn't take much care of the control pat at this stage, but the feedback can be a good reminder. The last reason, vague gestures, I didn't communicate it well in the video because I only did one possible move of it. Actually, the user could also step on the lit up squares by feet or press the direction arrows by hands. It's quite arbitrary that depends on how could it be easier to do a specific shooting.

The suggestions to refine the game:
- using the color system Twister, assume there are different colors of bullets and aliens, and only if the bullet color matches the alien's, it'll be destroyed.
- don't make the time for each round too long
- more than 1 person to play
- refer to the dancing blanket and use the hands to show direction rather than physical contact with the control pad.

The first suggestion is a great idea somewhat like another game called Puzzle Bobble. Overall it's an excellent idea to change the rules of Space Invader a little bit. But to some extent it'll further increase the difficulty of the game.

As for the questions of the video and audio quality, the feedback received is positive. I'm surprised and happy with it because I really spent a lot of time on it. However, it's a pity that I forgot to put a text question for them to comment, otherwise the class might give me some useful suggestions about shooting, recording or editing skills. 

Effectiveness

The video prototype and the evaluation session are both quite effective. From the feedback, you can see that the video does communicate the game idea clear enough. As for the feedback itself, it not only gives me constructive suggestions to refine the design but also the confidence to do video in the future. Thanks for all the criticism and compliments.

The testing protocol is much more detailed in the statement of delivery rather than the real one as I can't do it in class. Hence, I grasped the key aspects of the concept and video of which I want to get the feedback and put it into the survey. Somehow I thought It was incomplete but better than nothing. But later it proved me wrong because it was really useful.

Constraints

Luckily my idea is simple, I used paper prototyping and montage shots in the video prototype to communicate the idea and it seems most of the people could understand it. But still, I found it's hard to apply dynamic elements in the video. I drawed some sketches on Illustrator but limited by my own skills, I can't make it dynamic and put it into the video. 

As for the testing session, sometimes when I have no idea of the one of the two games, it's even difficult to understand the mashup game and comment.

Implications

Some suggestions, especially the color bullets idea, are quite good and I'm now seriously thinking of whether to iterate them in my following design.  There is actually a goof in the video that nobody discovered so far. This kind of mistake can't be accepted in the future prototypes. Moreover, I'll try to learn how to make the animation to help present dynamic elements in the video. 

Future testing sessions will surely be offline and face to face, this time is just an accident. I can and will ask detailed and in-depth questions to them. Those questions are all in the statement of delivery. Besides, if I could, I would try to run a workshop for the user evaluation as well.