The CLE uses the term “Quiz,” for the activity, but do not be mistaken – the Quiz activity is a robust online assessment tool that can be used for self-evaluations, tests, exams, mid-terms, and finals.
A quiz is made up of a combination of question types, including multiple-choice, true-false, short answer, essay, and more. The Quiz activity pulls questions from a question bank, which faculty and course staff can use to create, preview, edit, and organize questions in any number of categories.
After students have completed a quiz, faculty and course staff have access to a number of reports, including a statistical analysis of the quiz. The majority of question-types are automatically graded in the CLE, except for essay questions that require manual grading.
Audience: Managers, Instructors, Course Assistants, Content Assistants, and Grading Assistants.
Resources: Click the appropriate tab(s) above or use the buttons below to learn more about the Quiz activity.
Create a Quiz
Creating a quiz in the CLE is a two-part process. The instructions below cover the first part – adding a quiz to a CLE course and configuring the quiz settings. The second part of the quiz-making process is adding questions to the quiz, which is covered in the Add Questions tab above. Visit Moodle Docs to view an abbreviated quick guide for managing the Quiz activity.
Let’s create a final exam in our CLE course using the Quiz activity. Here is the criterion for the final exam:
- Students can start the final exam on October 12 at 8am and must finish by October 12 at 5pm
- Students have a 120-minute time limit to complete the final exam and are allowed only one attempt
- Students will be able to see their final exam score after the exam closes on October 12 at 5pm and can review their attempt for 24 hours after the close time
To add a quiz to a CLE course:
- Log in to the CLE (cle.ucsf.edu).
- Go to the CLE course where you would like to add the quiz.
- From the Administration block or from the top-right of the CLE, select Turn editing on.
- In the section where the quiz should be located, click the Add an activity or resource link (you can always move the quiz to another location later).
- From the list of Activities and Resources, select Quiz, then scroll to the bottom of the window and click Add.
Continue creating the quiz up on the Adding a new Quiz page as documented below. Click a link below to skip to a specific section on the Edit Settings page:
Configure general information for the quiz, such as a name and quiz description.
- Click the Expand all link to expand all sections on the Edit Settings page.
- The Name will appear on the home page of the course, navigation menu, logs, and other areas that link to the quiz.
- In the Description field type instructions, background information, an honor code, etc., for the quiz. Select to Display description on course page checkbox to display the text directly on the CLE course page, below the quiz name.
Configure when students can attempt the quiz, set a time limit, and configure other time-related settings.
- Choose to specify a window of time when students can attempt the quiz. Prior to the open time, students will not see a Start Attempt button and will not be able to attempt the quiz. Students must also complete the quiz by the time entered in the close time. To configure an Open and Close time, first select the enable checkbox and then enter the times and dates.
- By default quizzes do not have a time limit, which provides students with as much time as they need to complete the quiz, or within the Open and Close times if enabled. Select the enable checkbox to enter a time limit (in minutes) for the quiz. Note that the quiz will close when the time limit expires or when the student reaches the close time, whichever comes first.
- Choose what happens when time expires during a quiz. The default Open attempts are submitted automatically is recommended as this is how a student typically expect a quiz to behave when the time limit or close time approaches.
- If you have selected There is a grace period when open attempts can be submitted, but no more questions answered from the When Time Expires drop-down, enter a grace period in the Submission grade period field.
Configure grade settings for the quiz as well as where the activity displays in the gradebook.
- If you use gradebook categories in the course, assign the quiz to a Grade category using the drop-down.
- Faculty typically leave the Attempts allowed set to the default one attempt for graded quizzes. You can provide students with as many attempts as you like, even unlimited.
- If the quiz allows multiple attempts, choose a Grading method, or how the attempts will be scored and reported in the gradebook (e.g. highest grade, average grade, first, last attempt).
Choose how questions display in the quiz and other navigation options. Click the Show more… link to expand the section as shown below.
- Choose how questions display in the quiz from the Question Order drop-down. Choose the default setting Shuffled randomly or As shown on screen.
- Configure pagination for the quiz, or how many questions are displayed on each page from the New Page drop-down. You can manually override this later in the quiz-making process while adding questions to the quiz. For quizzes that contain essay questions, we recommend re-paginating at Every question so responses are saved as students move from question to question.
- When Sequential navigation is used as the Navigation method, a student must progress through the quiz question-by-question and may not return to a previous pages nor skip ahead. The default setting is Free, which allows students to move back and forth in a quiz.
Question Behavior Section
Configure whether responses are shuffled within questions, how questions behave, and whether multiple attempts build upon the last. Click the Show more… link to expand the section as shown below.
- If Shuffle within questions is set to Yes, the question responses will be randomly shuffled for students, provided the option is also enabled in the question settings. This setting only applies to questions that have multiple parts, such as multiple choice or matching questions.
- How questions behave: Most quizzes use the default setting Deferred feedback, which means students must complete the quiz before they receive any feedback or grades. There are other Question Behaviors that can be applied to CLE quizzes and you can read more about each below and on the Moodle Docs Question behaviors document.
- Deferred feedback: Students must enter an answer to each question and then submit the entire quiz before anything is graded or they get any feedback.
- Adaptive mode and Adaptive mode (no penalties): Students have multiple attempts at the question before moving on to the next question. This behavior requires that the Whether correct box is ticked under During the attempt in the Review options section, at a minimum.
- Interactive mode: Students see a Try again button next to each question during the attempt that they can click to receive hints to help answer the question. Once the student has answered the question correctly, they can no longer change their response. Once the student has answered the question incorrectly too many times, they can no longer change their answer. The number of tries the student gets is equal to the number of hints configured for the question, plus one.
- Immediate feedback: Similar to interactive mode in that the student can submit their response immediately during the quiz attempt, and get it graded. However, they can only submit one response, and they cannot change it later.
- Deferred feedback or Immediate feedback with Certainty-based marking (CBM): Students not only answer the question, but also indicate how confident they are they answered the question right. The grading is adjusted by the choice of certainty, so that students must reflect honestly on their own level of knowledge in order to get the best mark.
- If multiple attempts are allowed and Each attempt builds on the last is set to Yes, each new quiz attempt will contain the results of the previous attempt. This allows a quiz to be completed over several attempts and is helpful if students experience technical problems during an online exam.
Review Options Section
Select the information you want students to see at different times during and after the quiz attempt.
Each of the four columns highlighted in the picture above represent a different time during the quiz and are explained below (remember to reference the Open and Close times we configured in the Timing section above):
During the attempt
What students can see and review during an attempt – this column is only relevant for certain Question Behavior modes.
Immediately after the attempt
What students can see and review during a two-minute window of time immediately after submitting an attempt.
Later, while the quiz is still open
What students can see and review following the two-minute window following submitting an attempt, until the quiz close time of October 12 at 5pm.
After the quiz is closed
What students can see and review after the close time has passed on October 12 at 5pm.
Each checkbox controls what a student can see during these times and are explained below:
For the final exam, we want students to be able to review all feedback after submitting the quiz, but we do not want to provide Points (question and quiz grades) or the Correct answers until the quiz has closed on October 12 at 5pm. The Review Options shown above reflect this set up.
Configure display settings for the quiz. Click the Show more… link to expand the section as shown below.
- Select to Show the user’s picture during the quiz attempt – this is helpful when proctoring a CLE quiz.
- Choose the number of Decimal places in grades that are displayed in the gradebook for the overall quiz. This only affects the display of grades, not the grades stored in the database, nor the internal calculations, which are carried out to full accuracy.
- Choose the number of Decimal places in question grades that is displayed for individual questions.
- Select whether to Show block during quiz attempts. We recommend using the default setting No, which will prevent blocks such as the Navigation and Administration blocks from displaying during the quiz attempt.
Extra Restrictions on Attempts Section
Configure security restrictions for the quiz. Click the Show more… link to expand the section as shown below.
- Enter a password that students must enter to start the quiz. This is a good practice for high-stakes, proctored exams. We will protect the final exam with the password UCSF.
- Choose to restrict access to an IP range using the Require network address setting. We discourage using this setting due to the number of ways students can access the CLE and a quiz. Restricting access to an IP range on the UCSF WPA network would prevent students using the UCSF Guest network or cellular data from attempting the quiz. We recommend using a password to restrict access to a CLE quiz.
- If you allow multiple attempts for the quiz, choose to Enforce timed delays between attempts.
Overall Feedback Section
Enter Overall feedback for grade boundaries that is provided to students after they have completed the quiz.
- In the example above, we have entered Overall Feedback for the grade range from 100%-70%.
- We can continue to enter Overall Feedback for grade range from 69%-0%. Keep in mind we can also include multimedia such as videos, images, or links to outside resources in addition to text feedback.
Commons Module Settings Section
Configure whether the quiz is visible on the CLE course page, use a Group Mode, and restrict access to a Grouping.
- Choose whether the quiz displays on the main course page (we can always change this from the CLE course page).
- Enter an ID number to provide a way of identifying the activity for grade calculation purposes. We will leave this field blank for the final exam.
- Apply a Group mode to the quiz to organize students into Groups for grading purposes. For example, there are 150 students and 5 graders in the course. We can create 5 Groups of students based on the course graders, and apply either the Separate or Visible Groups mode (choose either) to split up grading responsibilities. Graders will see the Group names in a drop-down menu when grading the quiz, and can choose to view and grade only the students who have been assigned to their Group. Click to read more about Group modes.
- We can restrict the ability to access a quiz to a Grouping that has been created in a course. Groupings are often used when students are taking the same course for a different number of units, and activities such as quizzes may be worth more or less, depending on their Grouping. Selecting the Available for group members only checkbox will prevent students not assigned to the Grouping from seeing or accessing the quiz link on the main course page. Click to read more about Groupings.
Restrict Access Section
Restrict student access to a quiz, or when students can see the final exam link on the CLE course page, based on a number of different criterion (this is sometimes referred to as “conditional release”).
- Click the Add a restriction button to configure when students can see the quiz link on the CLE course page, based on any combination of activity completion, date, grade, user profile, and more.
- For the final exam, we will restrict access by date only. Why would we want to restrict access to the final exam in addition to having open and close times? Open and close times refer to the window of time when students can start and end the quiz, while restrict access refers to when the quiz link displays on the course page. Remember we allow students to review the final exam for 24 hours after the quiz has closed. By restricting access to the final exam after October 17 at 5pm (24 hours after the close time), we can control the length of the review time.
- For the final exam, we will configure Restrict Access settings so the quiz link will be hidden on the CLE course page until October 16 at 7:40am (20 minutes before the open time). After October 16 at 7:40am, students will then be able to access the final exam until their access to the quiz is again restricted on October 17 at 5pm.
- Further restrictions may be added by clicking the Add restriction button again, and it is possible to specify that ALL the conditions or ANY of the conditions are required before the activity is made available. Thus, it is possible to use Or as well as Any conditions. Click to read more about the improved and simplified Restrict Access setting.
Review and save your changes.
Click the Save and return to course button to return to the front page of the CLE course, the Save and display button to return to the quiz, or the Cancel button to cancel any changes made on the page and return to the course.
Add Questions to Quiz
Let’s continue with the second part of the quiz-making process, adding questions to our final exam. There are multiple workflows for creating and adding questions to a CLE Quiz. While we recommend first building a robust, organized question bank to manage quiz questions, many faculty create questions while building a quiz. Either workflow can be used to create quizzes and the instructions below highlight the latter.
Here is the criterion for our final exam:
- The final exam is worth 100 points
- There are a total of ten questions made up of two multiple choice, two essay, and two true/false question types
- All ten questions are weighted equally
To create a quiz question and add it to a quiz:
- Log in to the CLE (cle.ucsf.edu).
- Go to the CLE course where you would like to add the quiz and questions.
- Click on the quiz link that has been created inside of a CLE course. For this example, we will click the Final Exam link as shown below.
- While in the quiz, navigate to the Administration block and click Edit quiz to add, remove, edit or reorder questions. The Edit quiz button is also located in the middle of the page as shown below.
Continue with creating quiz questions and adding questions to the final exam on the Editing Quiz page as documented below. Click a link below to skip to a specific section on this page:
- Create a Question
- Add Questions to a Quiz
- Configure Question and Quiz Grades
- Question Ordering and Paging
- Question Bank Management and Best Practices
Create a Question
Let’s first create a multiple-choice question and add it the final exam.
- From the Question Bank Contents block on the right-side of the Editing Quiz page, click the Create a new question to add the question to the Question Bank, or click the Add a question button in the middle of the page to create and add a question to the quiz AND Question Bank.
- Choose a question type to add to final exam. Question-types that can be used in a quiz include multiple choice, true/false, matching, essay, short answer, and more. For thus example, select multiple choice and click the Add button. See the full list of quiz question types on Moodle Docs.
- Create the quiz question on the Adding Multiple Choice Question page as shown below (click images to view full-screen). Want to add video or audio to a quiz question? See tips and instructions in Use Video or Audio in a Quiz.
- After creating the question, click the Save changes button or choose to Save changes and continue editing. Previously, you had to save your question in the question bank, return to the Question Bank, and click edit again to continue editing. This new feature saves time and clicks!
You have successfully created a quiz question and are ready to add the remaining nine questions to the final exam.
Add Questions to Quiz
After creating the question, you will see that the question has been added to the final exam on the Edit quiz page. The remaining nine questions have already been created in the Question Bank, but need to be added to the final exam as shown below.
- From the Question Bank Contents block, click the double arrow to the left of the question to add it to the quiz. Alternatively, you can select the checkboxes to choose multiple questions to add to the quiz at once.
- Let’s select the checkboxes next to the nine remaining questions and click the Add to Quiz button.
We have added the ten questions to the final exam, continue below to configure the final exam grade, question weighting, and ordering.
Configure Question and Quiz Grades
After adding the ten questions to the final exam, we can configure the maximum grade for the quiz, how questions are weighted, and add or remove questions while in the Editing quiz tab. For more information on quiz grades, weighting and scoring, please see the Moodle Doc Assigning Points to Quiz Questions.
- Enter a Maximum grade for the final exam and click the Save button.
- Enter a Weight for each question and click the Save button
- Re-order or remove questions from the final exam using the up/down arrows and cross icons.
Question Order and Paging
Navigate to the Ordering and paging tab to configure how questions are ordered and display in the final exam.
If we want questions to display in a specific order for students during the final exam, we can configure question ordering on the Ordering and Paging tab. Remember that we configured the question order to be shuffled randomly on the Edit Settings page when we created the quiz, so we can leave the question ordering as is.
If you want to change the order of questions you will need to first return to the Edit Settings page to update the Questions order setting to display as shown on edit screen. Then return to the Edit Quiz page and use the arrows located next to each question to reorder as shown below.
Question Bank Management
The Question Bank is a great resource for managing CLE quiz questions. Every CLE course has a question bank that can be organized into categories and subcategories. This is helpful for managing and building quizzes from quarter to quarter, as well as for tracking objectives.
To access and mange a question bank in a CLE course:
- Log in to the CLE (cle.ucsf.edu).
- Go to the CLE course to manage the question bank.
- From the Administration block, click Question bank.
- Choose to view the course Questions, Categories, or Import/Export questions.
Clicking the Questions link takes you to the question bank and displays all of the questions in the course. From this page you can create a new question, edit questions, and move questions from one category to another (click image to view full-screen).
Clicking the Categories link takes you to the question bank categories. From this page you can organize questions into categories that can shuffled, randomized, and managed (click image to view full-screen). Categories and sub-categories are very powerful when combined with randomized quiz questions that can be selected either from a single category or from a collection of sub-categories
Question Bank Tips:
- Put descriptive information in the question name so you can quickly identify the question. The question name is never displayed to students before, during, or after a quiz. This is especially useful if you are managing a large question bank.
- Create question categories and sub categories – it is better than keeping all your questions in one large category.
- Use special characters in the question name to indicate the status of question. For example, an * may indicate that the question is currently being developed and should not be used in a current quiz. Remember that students never see the question name!
- Questions can be assigned to a category while being created and can be moved from one category to another any time after the question has been added to the question bank.
- Ensure that each category contains a variety of good questions, allowing you to randomize questions, asking different questions to each student for the same quiz.
Question Bank Resources:
- Best Practices: Three Tips for Making the Most of Moodle’s Question Bank (Moodlerooms)
- Moodle Docs
- Attend a Tech Clinic with the Learning Tech Group
Grade a Quiz
The majority of question types used in the Quiz activity are graded automatically in the CLE. This is one of the major benefits of using the CLE for online assessment. Essay questions, however, are not automatically graded by the system and must be manually graded by instructors or grading assistants.
To access Grades, Responses, Statistics, and Manual Grading:
- After all students have submitted their attempts, log in to the CLE (cle.ucsf.edu).
- Go to the CLE course where a quiz has been attempted by students.
- Click on the quiz link on the course page. For this example, we will click on Final Exam as shown below.
- While in the quiz, navigate to the Administration block and select Results to access Grades, Responses, Statistics, or Manual grading for the quiz attempts. You can also click the Attempts link from the middle of the page to access reports as shown below.
- Grades shows all the students’ attempts, along with their overall grades, and the grade for each question for the final exam. There are links to review details related to each student’s attempt.
- Responses is very similar to the Grades report, except it shows the responses the students submitted, rather than the marks (or points) they earned. It is also possible to show the question text or the right answer in this report, to compare with the other student’s response. This is helpful when the question is randomized.
- Statistics gives a statistical (psychometric) analysis of the quiz, and the questions within it. The front page of this report gives a summary of the whole test. Additionally there are links to drill down into a detailed analysis for each question. Please refer to Quiz statistics report or watch the Grading a Quiz video for more detailed information about the different types of statistics that can be collected in the CLE.
- Manual grading is used to grade essay questions and override scores for other question-types. The first screen of the report lists all the questions in the quiz that need to be, or have been manually graded. Additionally there is an option to show the questions that have been automatically graded, which is useful if you ever need override a grade for a multi-choice question for example.
To manually grade essay questions in the final exam, navigate to the Administration block, and click Manual grading (all other questions types are graded automatically by the system).
Click Also show questions that have been graded automatically to show all questions. Each row represents a different question.
To grade student responses for a question, click grade in the To grade column or click grade all in the Total column. Selecting either options will display all student responses for the essay question.
Configure the viewing options for grading essay questions. Increasing Questions per page can be helpful in large classes.
On the grading screen, you will see the following:
- The text of the essay question.
- The student’s response, shaded in white.
- A field to enter comments for the students.
- A field to enter the number of points a student earned for the question in the Point box. The maximum number of points a student can earn on a single question is displayed to the right of the Point box.
Proceed to grade each response on the page in this manner. When you are done, click the Save and go to next page button to save your work and report the grade to the gradebook.
From the Administration block, click Results, and Grades to confirm that students received a grade for the question.
Override a Grade
To change a grade or comment on a question that has already been manually graded by the CLE, click the quiz link, go to the Administration block, click Results, and then Manual grading.
You will be returned to the Manual Grading page. Click update grades from the Already graded column to update previously graded questions.
When grading is finished, you can export student quiz grades to enter in another UCSF system. Choose to export grades as an Excel spreadsheet or other file type.
Need to make a change to a student’s quiz grade? You can manually override a student’s score or response for a quiz question any time after the attempt. Visit the Moodle Doc for more information on regrading CLE quizzes.
Are students not seeing their final exam grades in the gradebook? Visit the Quiz FAQ page for answers!
The User Overrides feature allows instructors to provide a student, or group of students, with a specific amount of time, multiple attempts, or other options that differ from those configured on the Edit Settings page. Instructors need to configure accommodations for individual students for every quiz; it is not currently possible to apply this setting at the course level and have it affect all quizzes within that course.
Let’s create a User Override for our final exam using the following criterion:
- Harold Test Student contacted the faculty to let them know they have an accommodation that requires they receive double-time for the final exam
- Harold Test Student also contacted the faculty letting them know they were sick for the final exam, and would like to make-up the attempt on October 15
To set up a user override for a Harold Test Student:
- Log in to the CLE (cle.ucsf.edu).
- Go to the CLE course where you need to provide a user override.
- Click on the quiz link in the CLE course. Accommodations are set up after the quiz has been created. For this example, we want to set up a user override for the Final Exam to provide a student with more time.
- While in the quiz, navigate to the Administration block and click User overrides to provide a student with accommodations. Choose Group overrides to set up an override for a group of students in a CLE course (please note Groups must first be created a CLE course before setting up a Group override). For this example, we will setup a user override.
- Click the Add user override button to continue.
- Configure the user override for the student using one (or a combination) of the following criterion:
- Use the Search field to find the student in the CLE course who requires an accommodation or select their name from the list of students.
- To protect the quiz, enter a password in the Require password box. This password can be different than the password configured on the quiz settings page.
- To set a different access time for the exam since Harold Test Student was absent the day of the final exam, select the enable box and enter the open and close dates and times.
- Provide more or less time for the user override in the Time limit field.
- Choose to allow the student to attempt the quiz more than once using the Attempts allowed drop-down.
- If you made a mistake while configuring the accommodation, click the Revert to quiz defaults button to remove the accommodation settings.
- Click the Save button to return to the User override page or Save and enter another override to provide another student with an accommodation.
- Review the User Override to confirm it is configured correctly. Choose from the Action column to edit the override, duplicate the override settings to apply to another student, or delete and remove the override. Add another override by clicking the Add another user override button.
Add a Quiz
Watch this video to learn how to add a Quiz to your CLE course.
Add Questions to a Quiz
Watch this video to learn how to create and add questions to your CLE Quiz.
Grade a Quiz
Watch this video to learn how to grade a CLE Quiz.