Project Duration: 1 week
Team: Tim Kanellitsas, Andy Jevsevar and Pilar Lindal
Overview
An Education game revolving around language adoption. This particular proof of concept used Spanish as the exercised language, but the same process can be applied to any language.
The example scene takes place in a small town in Spain, where the user needs to go to the market to pick up ingredients for the evening.
Before entering the scene, the user can see their profile, which gives them updates on key metrics metrics monitoring engagement and progress. These metrics are not live, but could be by simply integrating Unity Analytics and tracking custom events.
Project Constraints:
Exercises/challenges must be able to be used with other languages.
Experience needs metrics to track student engagement and progress.
Exercise must be able to engage both systems of thinking (1 & 2). See Education Design below for more details.
Experience must be valuable without a community, but can be enhanced with more users.
Tools Used:
Whiteboard for wireframes and storyboarding, white boxing in the Unity game engine to lock in interaction functionality, Unity for level design, post-processing effects for polish.
Usability Testing:
Usability testing revealed players having a hard time figuring out the controls. Future iterations need a proper onboarding component (UI pointing to the key buttons).
Users also recommended incorporating positive feedback when the player adds the correct food to the box. More details below.
User Testing:
User testing revealed a huge bottleneck in language adoption due to the natural habit of translating from the native language into the foreign language (English to Spanish, in the case of this project). Studies show that kids see an object, and immediately associate the new word with that object. That is one of the reasons why they can adopt a new language so quickly, in addition to their higher level of neuroplasticity.
We need an experience that reinforces that direct association between object and foreign language word, and reduce the opportunity to translate.
Challenges:
The physics system proved to be a challenge in this project. By incorporating a large number of rigidbodies, Unity needed to track the physics of quite a few objects, resulting in a ton of lag. To compensate, I made a script that only engaged the rigidbody on objects when they detected a collision. The same script also made the objects kinematic until they were released (onCollisionExit).
Education Design:
Simplifying language adoption:
We know that students typically go through a 3-step process before they truly speak with fluency. This three step process involves recognizing an object, thinking of the object’s name in their native language, and then searching for the word in the foreign language.
Alimentos en España aims to remove this middle step, where the student sees and picks up the object, and immediately sees the word in Spanish. This reinforces the object-to-Spanish word association, which I suspect leads to rapid language adoption.
Designing for Friction:
Based on Daniel Kahneman’s book, Thinking Fast and Slow, we have two systems for thinking: System 1 and System 2. System 1 refers to a mode of thought, where we rely on our gut instincts to make a decision. This is efficient to execute tasks and work quickly, but is not ideal for strategic thinking. Strategy typically needs deep, methodical, slow thought. This is where System 2 comes in, which refers to said methodical, slow thinking. This is the mode we use when we search for something specific, to attempt to learn something.
To organize this distinction further, System 1 is used for actions and tasks, where as System 2 is used for problem solving and learning.
In order to make the user learn, we need to engage System 2 as much as possible. But to keep the user engaged, we need to keep them busy with actionable tasks.
Alimentos revolves around a simple proof-of-concept, where the user searches for food from the market. The searching for the correct food, based on the words they hear and see reinforces the learning by engaging System 2, while the task of moving around the market and picking up the food engages System 1 and keeps them interested.
Ideation
Vocab Category Brain dump
Work
Weather, Time
Colors, clothing
Body and health
Family Relationships
At a gas station
Renting an apartment
At the doctor’s office
At the dints
Buying a computer
Food
Likes & Dislikes
Eating at restaurants
Making a phone call
At a jewelry store
At a theater box office
Bargaining
Looking for a job
At the post office
At the beach
At the pharmacy
Daily Activities
Traveling by bus
Traveling by train
Renting a car
At the bank
At the currency exchange
At the police station
Watching television
At the movies
Greetings
Asking for information
Clock Time
At the market
Basic Grammar and uses
Sports
At the hotel
The media and communications
Holidays and festivals
Politics and rights
Storyboard
.
Storyboarding on the whiteboard led to crucial interaction revelations, which ultimately enhanced the equality of education:
1. Use a laser pointer with UI to increase the academic feel, theoretically increasing focus.
2. Picking up the food object, hearing the food object, and seeing the object's name (displayed above the food) would increase retention.
The learning triangle is a repeated pattern of discovery, reiteration, and award which reinforces the learning. This is a critical part of the learning process, which avoids translation all together (an inefficient part of language adoption, in my experience).
Wireframes
The "home scene's" UI revolved around the user's profile, online tutors, and different lessons (levels). The user can choose different levels to work through, reference their progress and statistics on the home screen, or see which online tutor is available to work on their Spanish in real time (conversational fluency).
Curved UI is both more comfortable to read and use in Virtual Reality.
Prototypes & Testing
1st Usability Tests
After coding a “grabbing script” to interact with grabbable objects in the scene, I tested the interaction to see which Oculus Touch Controller button users typically press to grab an object.
I tested this interaction on 3 different people, which led to two key insights.
Insight #1: All three users used both their pointer fingers and middle fingers, and pressed both the trigger and grip buttons in sync.
Insight #2: All three users expected gravity to pull down the objects upon letting go of the objects. Originally the objects were kinematic, and had to be set down (rather than dropped).
Action Taken: Though one button would suffice, I modified the code so that both buttons would need to be pressed to grip an object. I also switched the objects to use gravity, rather than being kinematic (unaffected by the Unity physics system).
Prototyped Scene Design
Snug Grips: Adjusting colliders
Many of the game objects reacted strangely upon adding the rigidbody component, enabling gravity. By adjusting the colliders to more "snuggly" fit the game objects, they responded more realistically to the gravity and the player grab.
Reiterating Vocab: Adding UI To Game objects
Two key part of the "Learning Triangle" was discovery and reiteration. As the player grabbed different objects, she could see each object's name, in Spanish, hovering above. This not only confirmed they had discovered the right food, but also reiterates the food's name.
Too many physics calculations!
Originally every object had a rigidbody with gravity turned on, not only did this make a few buckets of food explode and roll around the ground, I dropped through ground when I would "run after" the rogue fruit and vegetables.
Solution: add a script that only adds a rigidbody component and enables gravity after we interact with the object.
Silver Lining: This revealed a cool opportunity for a skydiving simulation. We just need a big fan and a computationally expensive experience!
2nd Usability Testing: contextual task completion
The testing phase revolved around two parts, and involved three new users. Two had learned a very little amount of Spanish, and the third had never taken a single class.
Actions From Insights:
Spawn the user right in front of the box so the box and its UI is the first point of attention.
Include a voice recording explaining what the user needs to do and which foods they should grab. This also aids pronunciation.
Add a “ding” when they add the correct object to the box, and add a barrier prohibiting them from adding the wrong food to the box.
Polish
Goal:
Make a scene with an “old, authentic feeling” to increase immersion.
Increase fidelity of objects, scenery, and structures as much as possible.
Techniques:
Antialiasing - smooths the edges of objects. Objects, when brought close to the camera, can look rough and jagged on the edges.
Ambient Occlusion - darkens edges, holes, and other areas that cause shadows.
Bloom - adds a glow to brightly lit objects and lights.
Analysis:
Antialiasing is crucial to increase fidelity of the scene, but can also make an object look a little blurrier if overused. Ambient Occlusion is a great tool to combat the blurriness, by further highlighting darker details of the scene and objects.
Adding the Bloom effect lightened the mood of the scene. With an increase in shadows and their contrast, it’s easy to bring the mood in a more negative direction. Bloom increases the effect of light reflection, adding a lighter feel to the scene.
Project final (V1)
Next Steps
1) Adding additional levels for each chapter in the above research.
2) Adding voice recordings of each object's name, to further reiterate the pronuncation.
3) Dynamic "shopping list:" each item is checked off as it's placed in the bucket.
4) Onboarding. Every subject tested needed help figuring out what to do and how to do it. Once the initial goal was established, they had no problem navigating through the level.
5) Voice recognition and interaction, to further reinforce vocabulary and help develop accurate pronunciation.
6) Adding custom grips for each object to increase immersion, and n theory increasing learning time.
7) Conduct tests around efficacy of this simulation as a learning platform. The theory above is all based on my experience studying abroad, articles I have read, and teachers I consulted. Hard data is still needed to support claims around "the Learning Triangle," and other techniques.