2017 Udemy Spotlight: Reviews and Ratings
Over the past year, our team has been working on updating the review and rating system. We use reviews and ratings in both sides of our marketplace. First, we use ratings and reviews to ensure that you, our instructor community, receive helpful and actionable feedback on how your courses are performing. Second, we use ratings to ensure that students get accurate indications of which courses are right for them.
We gather this feedback in two places – in the course dashboard, where students can leave a review from the top of the dashboard, and within the course taking experience (the early review prompt), where students are prompted to leave a review early in the course.
Today, we’re announcing some optimizations of the early review Prompt:
- Giving students options. We’re updating the design of the Early Review Prompt to give students a clearer option for leaving a review later in the course. We’re also adding in a “Never Ask me Again” option for students who do not want to see the review pop-up.
- Removing irrelevant reviews. Our Trust and Safety team will be proactively scanning the marketplace and removing reviews that only contain comments like “Too early” or “Too soon”. The team will be removing these reviews regardless of rating. Please note that we will not be removing reviews that include “Too Early” comments in addition to other comments relevant to the content.
- Updated spam filter. Over the next few weeks we’ll also be running our periodic update of our spam review filter. You may see the number of reviews on your courses change as a result of this update.
Both the course dashboard review and the early review prompt serve different purposes. The review prompt on your course dashboard is helpful for students who feel strongly enough about your course to go out of their way to review your course. The early review prompt is for everyone else — all those students who have an opinion but don’t want to go out of their way to leave a review.
We feel that this early review prompt (and the 2 subsequent in-course review prompts) are crucial for getting a range of opinions (students who use the course dashboard prompt tend to feel either very strongly or very negatively about the course) and for generating a large volume of reviews. More reviews means more variance in ratings across the marketplace which makes it easier for students to select the perfect course for them.
Additionally, we’ve seen that feedback that students leave in the early review prompt is strongly correlated to the ratings they leave later in the course. This is showing us that the signal from students left early in the course is valuable and relevant.
While the early review prompt will not be substantially changing (we will continue to make copy and slight timing tweaks to it) we’re exploring some additional ways to provide more context for ratings and reviews:
- Display student consumption on the CLP. We’re exploring showcasing how much a student has consumed of the course to provide context for their review.
- Optimizing for action.
- Providing alternate avenues for student feedback. Right now we’re asking ratings and reviews to provide feedback for both students and instructors. We’re exploring alternate ways to get you, our instructor community, actionable feedback for improving your courses, separate from the rating and review system.
As always, when the rating and review system is updated, we’ll keep you informed over email, the Teach Hub, and within the communities.