The latest on practice test upgrades

Español | Deutsch | 日本語  | Français | Italiano | Türkçe | Polski | Português | 한국어| 中文(繁體) | Bahasa Indonesia

Practice tests are comprehensive assessments that serve many purposes on Udemy. As demand grows for cloud computing and technical skills certifications, we’ve seen professional learners increasingly are using them to prepare for certifications. They want a simulated testing experience to gauge readiness and find knowledge gaps before the actual exam. They also want to use practice tests to receive feedback in real time and target their studying. Knowing this, we’re improving practice tests so that you can better deliver on helping your learners reach their professional goals.

This article contains the changes we’ve made to Udemy’s practice tests. Read on below for the latest.

November 2024

Over the past year, we’ve continued to invest in practice tests to boost their effectiveness for your learners.

Now, we’re excited to announce the release of a new learner-facing feature, practice mode. 

With practice mode, your learners can customize how they spend their study time and receive feedback immediately.

Here’s what’s new:

A flexible testing environment

Learners can choose between exam mode or practice mode. Exam mode mimics the real testing environment. With practice mode, learners can enter a low-pressure environment, without taking a timed test.

Immediate feedback

Learners can receive feedback by clicking “check answer” to learn from their mistakes instead of waiting to complete the full test.

Focused studying
Learners can focus on what they need to practice most and ensure questions come only from the domain they prefer.

Want to set your learners up for success? Visit our Help Center article to learn about the importance of assigning domains to your questions, plus other key tips.

We’ll be rolling out practice mode to Marketplace learners starting in late November and aim to make it available for all Marketplace and Udemy Business learners by early 2025.

October 2024

Over the past year, we’ve updated practice tests to ensure that they provide more value to you and your learners. We also heard that you wanted more insights into how learners are responding to practice tests. 

Based on your feedback, we’re excited to announce the release of a Practice Test Insights Report with data and insights into how your learners interact with practice tests. You can start using these new insights today! The Practice test insights page is currently in beta and is not available to all instructors. Our team is working to provide all instructors with published practice tests access to the page in the coming weeks.

In this report, we’re providing you with metrics that show how your learners respond the first time they see a practice test question — how much time they spend, what answer they pick, and more. We’ll also continue to show information like how many learners viewed, completed, and successfully passed your practice tests.

These new insights can help you create higher-quality practice tests based on where your learners get stuck in their certification exam journey.

Here’s what’s new:

Performance data

Get a high-level overview of learner behavior for your practice test including:

• A performance funnel of how many learners started, completed, and passed the test.

• The average number of attempts each learner takes to pass the test.

• The average time spent on the test by learners who pass.

Question analytics

Dive deeper into each question to figure out where learners got stuck. This includes:

• The question, average time spent, number of learners, and success ratio (with a visual breakdown).
• For multiple choice and multi-select question types, learner answer distribution across different answers. 
• The ability to search through your questions and sort them based on learner success.
• The capability to jump directly to the edit page of the question you viewed analytics for.

Version history

See the impact of your edits on learner behavior, including:

• All updates you have published to practice tests, along with the dates that each version of the test was available to learners.
• Practice test insights for a specific date range

Ready to check it out for yourself? You can view your report now.

May 2024

New practice tests creation flow

We’ve redesigned the practice test creation process with its own dedicated page, separate from the curriculum builder. By applying our Learning Science and Design team’s best practices and coding exercise creation flow learnings, this new builder aims to make practice test creation more intuitive and help you provide high-quality assessment experiences for your learners. Additionally, we’ve changed the bulk upload CSV template to align with the new explanation structure.

Previous

New

New “fill in the blank” question type 

As we hope to better align practice tests with 3rd party certification exams, we want to make sure you can realistically simulate the test-taking experience for learners. This includes providing similar question types, lengths, and requirements that learners would find in the actual exam. Therefore, we have added a “fill in the blank” question type, expanding beyond just “multiple choice” and “multiple selection” options.  The new “fill in the blank” format mirrors actual exam requirements by having learners fill in blank fields within statements you provide.

AI-supported explanation generation (English development and IT-related courses only for now)

We’ve added a new AI-enhanced feature to help instructors easily explain practice test answers for multiple choice and multi-select questions. Comprehensive explanations can give learners valuable insights into why they selected the wrong response, and it’s now easier for you to create them through AI. While this feature is only available in English for now, we hope to roll it out in other languages soon.

Practice test insights (Coming mid-May)

Finally, we understand the importance of data-driven decision making when it comes to creating and optimizing your practice tests. That’s why we’re excited to announce that we’ll soon be rolling out practice test summary insights on your course engagement report. This update will provide you with information about how your learners interact with your practice tests, including the number of starts (viewed), incompletes (dropped), and overall success ratio. 

The “starts” metric will show the unique learners who clicked the “Begin test” button, while “drops” will indicate the number of unique learners who started but never finished the test. The success ratio will reveal the percentage of learners who successfully completed a practice test at least once. We hope these insights begin to give you a clearer picture of your practice tests’ performance, and we plan to dive deeper and provide you with question-specific data later this year.

We encourage you to start making edits to your practice tests today and we’ll continue to keep you posted on the changes. Feel free to visit the getting started guide for more information!