Conducting research about a product, either with interviews or directly observing your target audience using it has proven to give valuable insights and lead to significant improvements. At Oppia, that is how we validate the effectiveness and enjoyability of our Basic Math lessons, and make sure that students in different regions have a positive experience learning through the platform or the Android app.

As an UX Researcher at Oppia, I conduct research with the rest of the Learner Feedback Team to gain an understanding of our lessons comprehension and platform usability. We call this research we do in Oppia as Learner Feedback Sessions, because it is basically a way to get constant feedback from potential users.

What is a Learner Feedback Session?

A learner feedback session is aimed to help us get feedback for making the lessons better for Oppia’s students around the world. These sessions may focus on the Oppia lessons, the Oppia website, the mobile app, or on the general situation of students, including parents sometimes too.

These sessions are focused on understanding students’ expectations, behaviors, and needs. Local volunteers conduct studies with students by working through lessons and gathering feedback that allows us to create content that is fun and easily digestible.This feedback is critical in helping us better align our lessons with the core needs and goals of students. It also enables us to help future students more effectively ( Oppia Foundation).

Additionally, we hope that the students we work with will learn a lot from that experience, and improve their understanding of key mathematics concepts that will help them in the future!

How is a Learner Feedback Session conducted?

First, the facilitator of the study needs to choose one of the available Research Plans for conducting a learner feedback session. At the moment, Oppia has 3 active main types of plans: Android App Baseline Research, Persona Development and Effectiveness of Basic Math Lessons, each with a different goal and instructions.

While the Android App Baseline Research is focused on determining the usability of the app, the Persona Development is aimed to understand how parents discover learning opportunities for their children outside of school and the Effectiveness of Basic Math Lessons is used to determine whether Oppia’s basic mathematics lessons are effective, and suggest improvements based on challenges that students face when playing them.

First thing to do is check if the participant matches the criteria defined for each Research Plan. That is a way to ensure that the study is being conducted with the target audience based on the research goal. Then, each student is assigned an unique ID in order to de-identify students’ personal information, such as their name/email.

After scheduling the study, we also schedule a setup meeting 2 days prior to the actual session, to get parents’ consent, remind them about the actual study session in 2 days and give technical instructions if the study is virtually conducted. For the Effectiveness of Basic Math Lessons research, it’s important to also get information about the background of participants, as well get them tested in their mathematical skills. This is critical because it helps to decide what lesson should be assigned for them, and also it’s how we measure how much the student managed to learn from the lesson, that is the effectiveness of the lesson, as he is tested in the same skills 2 days after the study.

During the actual sessions, we use a think aloud approach that involves observing the students think out loud as they are attempting to complete the lessons or a set of core tasks while using Oppia’s platform and App. We do that by asking the students to talk about their thoughts and actions as they try to complete scenarios. This approach helps to identify user misunderstandings about the lessons or the system, and to better understand why these errors and confusions occur.

As a general rule, we try to intervene as minimally as we can during the study. Guidance is not offered until the user is seeking assistance. When intervention is needed, it should be detailed in the report to help identify what the baseline gaps are, and how they should be addressed in the future.

After the session, a detailed report is written based on a template we use to ensure quality/consistency of information given, and submitted through a Google Form so specific teams can use it to make the appropriate improvements.

In the case of Effectiveness of Basic Math Lessons Research, we try to understand the strengths and Inefficiencies on each math topic, so we get feedback about the students experience and test again their mathematical skills on the topic they learned, in order to compare with the results on the same skills of the pre-test and access the effectiveness of the lesson.

Can I conduct a Learner Feedback Session for Oppia?

As an open-source and volunteer driven project, anyone interested is welcome to join our Feedback Learner Team, or any other team within Oppia, you just have to fill this form and we will contact you. You can check all the ways you can participate in Oppia by accessing this link.

This is an opportunity to practice your skills, make new friends, learn a lot of exchanging experiences with other people and specially to impact the lives of millions of students. If you are passionate about education and want to make a difference in your local community, sign up to be a Learner Feedback Program volunteer and help us create the best platform for students around the world.