005-150 Consensus Scoring

005-150 Consensus Scoring ITS utilizes a Consensus Scoring methodology for proposal evaluation. ITS has found that a consensus rating arrived at by the evaluation team after consideration and discussion of all information provided by a vendor represents a more accurate assessment of the vendor's offering than does a mathematical averaging of individual evaluators' scores. In a consensus scoring approach, individual evaluators read the assigned proposals, or sections of proposals, prior to evaluation work sessions and make notes of proposed scoring, observations of strengths and weaknesses, and questions regarding the vendor's proposal. During consensus scoring sessions, the evaluation facilitator directs the team's attention to each item in the specifications. The evaluation team considers one proposal at a time, comparing the vendor's proposed offering against the specifications in the underlying RFP or LOC. Consensus scoring sessions encourage open discussions and questions among members of the evaluation team. Evaluators discuss the relative strengths and weaknesses of a vendor's proposal in each area. Open debate about a vendor's statement or response is encouraged to help ensure nothing proposed by a vendor in response to a requirement is overlooked. This discussion may provide additional insight into the vendor's offering and/or correct misperceptions of individual evaluators, so that the consensus score arrived at by the team may differ from the initial score of the majority of evaluators and from the mathematical average of the individual scores. The most important factor in assigning a final consensus score to any item is that the score accurately reflect the merits and value of the vendor's proposal for that item. Once the team has arrived at a consensus score for an item in the proposal, the evaluation facilitator captures the consensus score along with documentation of the team's observations of noted strengths and weaknesses of the vendor's proposal for that item. Narrative documentation is required for items that either exceed the specification or do not meet the specification in some manner. Only the consensus score sheet and comments are official and become part of the project file. Individual team member evaluation worksheets and notes are collected and destroyed once the consensus scoring is completed. If an item or area in a vendor's proposal is not clear or cannot be scored due to ambiguity or missing information, the evaluation team may decide a written clarification from the vendor is appropriate for a given item. In this case, a written request for clarification will be generated and sent to the vendor. The area or item under review may be suspended from scoring until the response is received from the vendor or a temporary score may be assigned. When the clarification is received, the evaluation team once again addresses the item and determines a final consensus score.