On this page
- Review and Submit Ratings
- Consensus
- Reviewing Final Ratings for an Individual Candidate
- Reviewing Final Ratings for All Candidates
Review and Submit Ratings
During the Review Ratings phase, the Interview Guide displays all of interview questions and their associated rating criteria, rating scale, and notes sections.
Finish rating each competency and review your notes for each question. Interviews may include interviewer-only questions. These questions were not presented during the interview and are visible to the interviews only after the candidate has been dismissed. Rating and note-taking for these questions is the same as it is for non-interviewer-only questions.
Once you have completed your review, select the “Submit” button in the bottom, right-hand corner of the screen.
If consensus is required for the interview, these will be considered your initial ratings for the interview. You will not be able to edit your ratings or notes until the panel conducts consensus.
If consensus is not required for the interview, these will be considered your final ratings for the interview. You will not have a future opportunity to edit your ratings or notes.
Once you receive the confirmation page, you can close out of the tab in your browser or close the window to exit the interview. Remain on the confirmation page to proceed directly into consensus.
Consensus
Consensus can be conducted once all interviewers have submitted their initial ratings. Even if all questions are within consensus, the interview panel will need to go through the consensus process to submit final ratings for the panel.
Starting Consensus
Once all interviewers have submitted their initial ratings, the Moderator must select the “Start Consensus” button at the bottom of the Interview Guide to start the consensus process.
Delayed Consensus
If the interview panel does not have time to conduct consensus right after the interview, they may leave the interview and return to complete consensus at a later, scheduled time. In this instance, to access consensus, navigate back to the candidate record and select the “Join Consensus” button on the right-hand side of the screen below the “Actions” header.
Raters will need to complete the Device Check to access the consensus page. Select the “Run Test” button in the bottom, right-hand corner of the screen to run the Device Check. Once the Device Check is passed, select the “Join Interview” button in the bottom, right-hand corner of the screen to join consensus.
Consensus Interface
The interview panel will be able to use audio and video to communicate during consensus. Use the “Camera” and “Mic” buttons at the top of the screen to turn audio and video on and off as needed.
The consensus interface presents one interview question at a time. For each question, the system displays the question content and the competency name, rating scale, and rating criteria for each associated competency. It includes your notes section for that question and any notes you documented during the rating process. You can add additional notes or edit your existing notes during consensus. On the rating scale, each interviewers’ initial rating is represented by a circle overlayed with their initials. Your initial rating circle is blue, the other interviewers’ circles are gray.
To determine if a question is within consensus, look at each competency name associated with that question. If there is a check mark next to the competency name, the ratings for that competency are within consensus. If there is not a check mark next to a competency name, the ratings for that competency are not within consensus. Next to the competency name, the system will display a message detailing the number of points ratings must be within to achieve consensus.
Consensus leverages an auto-save functionality, so should you leave consensus, edits to your notes or ratings will be automatically saved.
Changing a Rating
To change your rating for a competency, select the number in the rating scale that corresponds with the rating you would like to give for that competency. Once selected, your circle will move to hover over the new rating.
Moderator Role
During consensus, one interviewer or Observer will have the responsibility of serving as the Moderator. The Moderator is responsible for moving the panel from one interview question to another and submitting final ratings on behalf of the panel.
The first panelist to join consensus is automatically assigned as the Moderator. To make yourself Moderator, select the blue star icon above the raters’ video panels. The Moderator role can be switched at any time.
During consensus, an Observer serving as the Moderator will be able to view the ratings submitted by the interviewers, the rating criteria, and a notes section for each question. The Observer will be able to use the notes section to take consensus notes.
To move the panel forward from one question to another, select the “Next” button in the bottom, right-hand corner of the screen. To move the panel backward from one question to another, select the “Previous” button in the bottom, left-hand corner of the screen.
Submitting Final Ratings
Once all questions are within consensus, the blue “Submit Final Ratings” button will appear to the Moderator in the bottom right-hand corner of the Interview Guide.
Confirm that all interviewers are comfortable with their final ratings and do not need to add any additional notes. Based on your protocols, consensus notes may be documented by each individual rater in their notes section or a single individual may be responsible for taking consensus notes for the interview panel in their notes section.
To submit final ratings, the Moderator must select the “Submit Final Ratings” button. An “Are You Sure” pop-up will appear. Select the “Confirm” button to end consensus.
Reviewing Final Ratings for an Individual Candidate
Final ratings for a candidate are viewable on the candidate record for 180 days, at which time they will be archived. Interviewers as well as Interview Coordinators and Interview Admin are able to review final ratings.
To view a candidate’s final ratings, follow the below steps:
- Find your Interview Template by selecting the Interview Templates tab under “Interview Management”. Use the search bar to search for the name of your Interview Template.
- Once identified, open the template by selecting the pencil icon in the same row as the name of your Interview Template.
- Select the Interviews tab and find the name of the candidate.
- Once identified, open the candidate record by selecting the pencil icon in the same row as the name of the candidate.
- Confirm in the Status field that the interview is in “Complete” status.
- A snapshot of the final ratings are displayed in the Results section. This section includes an overall average score for the candidate and average scores across each competency.
- To export additional details, including each interviewers’ final ratings per competency and notes, select the “Download Results” button on the right-hand side of the screen below the “Actions” header to download an excel file of the interview results.
Reviewing Final Ratings for All Candidates
Final ratings for all candidates associated with an interview can be exported into a single excel file. Only Interview Coordinators and Interview Admin have the permissions to download results for all candidates at once.
To export final ratings for all candidates associated with an interview, follow the below steps:
- Find your Interview Template by selecting the Interview Templates tab under “Interview Management”. Use the search bar to search for the name of your Interview Template.
- Once identified, open the template by selecting the pencil icon in the same row as the name of your Interview Template.
- Select the Interviews tab.
Select the “Results” button in the top, right-hand corner of the screen to download an excel file of the results across candidates.