Investments in the Review System: March 6th Update

We’re excited to share that we’ve just made two improvements to the Review System: Weighted Ratings and Removing the 0.5-Star Rating.

Weighted Ratings

Over the last few months we’ve told you about how we plan to build an infrastructure that will allow us to weight ratings differently. We’ve now taken a big step forward and have started to apply this new weighting infrastructure!

There are a number of factors we’re using to help weight ratings; however, three of the most significant are:

• Amount of the course the student consumed before leaving the rating
• How recently the rating was left
• How valuable the student’s feedback was to instructors and students

With weighted ratings, if you receive a 3-star rating from a student who consumed very little of the course and provided no feedback or context behind the rating, it won’t be weighted as heavily as it was in the past. Alternatively, if you get a 3-star rating from a student who consumed a substantial amount of the course and the rating provides valuable feedback, it will be weighted more heavily than it was in the past. We want a course’s average rating to be a better reflection of what the most engaged students thought of it. We also don’t believe a course with many 4- and 5-star ratings should be unfairly dragged down by a low-star outlier rating that provides no explanation.

Our weighted rating infrastructure is still in its infancy, but we’re excited for it to start addressing some of the frustrations we’ve heard from you about making the Review System more fair and useful.

Removing the 0.5-Star Rating

We’ve received a lot of feedback from you about half-star ratings. While it’s important for us to keep half-star ratings (4.5, 3.5, 2.5, 1.5) so that we can have a more realistic distribution of ratings, we have decided to remove the 0.5-star rating. It doesn’t provide any additional insight to an instructor about what a student thought of the course that a 1-star rating doesn’t already communicate.

Going forward, we’ve removed the ability for students to leave a 0.5-star rating on desktop and mobile web, and will soon remove this ability on our mobile apps, as well. Existing 0.5-star ratings will remain visible to students, but over time they won’t factor into courses’ “Average Rating” which relies on ratings over a “recent rolling period.”

More about the Review System

We recently hosted an Ask Me Anything (AMA) session about the Review System in our Udemy Instructor Club. You can find a recap of some of the most popular questions and answers here. If you have questions about the Review System, we encourage you to look at that resource first, and if your question isn’t answered, please also feel free to reach out to [email protected].