Testing Methods

20.4 - Testing Methods (Rev. 13001, Issued: 12-13-24, Effective: 10-01-24, Implementation: 01-15-25) Testing the policies and procedures involves ensuring that the documented policies and procedures are actually being used as designed and are effective to meet a control objective. Evaluating and testing the effectiveness of policies and procedures is important to determine if the major areas of risks have been properly mitigated and provide reasonable assurance that the control objective is met. Testing and evaluating the policies and procedures consists of five (5) steps: Step 1: Select the policies or procedures to be tested It is both impractical and unnecessary to test all policies and procedures. The policies and procedures to be tested are those that primarily contribute to the achievement of the control objectives. A policy or procedure may be eliminated from testing when it does not meet the control objective to be tested due to being poorly designed, unnecessary or duplicative, or not performed in a timely manner. However, if this justification is invoked, other policies and procedures should be tested to validate meeting the control objective. Another justification for testing elimination is due to the cost of testing the policy or procedure exceeds the value of the control objective to be tested. If a policy or procedure is eliminated from testing, the reasoning should be documented. Step 2: Select test methods Once the policies and procedures to be tested are determined, test methods shall be determined. A combination of tests can be used depending on risk or type of activity. The following would be considered acceptable tests: 1. Inquiry: Asking responsible personnel if certain controls are functioning as intended (e.g., “Do you reconcile your activity or do you review a certain report each month?”). 2. Inspection: Analyzing evidence of a given control procedure (e.g., searching for signatures of a reviewing official or reviewing past reconciliations). 3. Observation: Observing actual controls in operation (e.g., observing a physical inventory or watching a reconciliation occur). 4. Re-performance: Conducting a given control procedure more than once (e.g., recalculating an estimate or re-performing a reconciliation). Observation and inquiry are less persuasive forms of evidence than inspection and re-performance. Step 3: Determine how much testing is needed The next sub-step is to determine the extent of the testing efforts. In most cases, it is unrealistic to observe each policy and procedure or to review 100 percent of all records. Instead, policies and procedures are tested by observing a selected number of controls performed or by reviewing a portion of the existing records. This selection process is called sampling. A representative sample provides confidence that the findings are not by chance by considering the factors of breadth and size. 1. Breadth: Breadth of the sample assures that the testing covers all bases and is a representative cross section of the universe being tested. This will provide confidence that the sample will lead to a conclusion about the situation as a whole. 2. Size: Size is the number of items sampled. The size should be large enough to allow a conclusion that the findings have not happened by chance and provide confidence in the conclusion. The size of the sample should not be so large that testing becomes too costly. When selecting the size of the sample consider: a. Experience: Reducing the size of the sample when controls have operated satisfactorily in the past and no major changes have occurred. b. Margin of Error: Increase the size of the sample when only a small margin of error is acceptable. c. Importance: Increase the size of the sample when an important resource is at stake. d. Type: Increase the size of the sample when the control to be tested requires judgment calls. Decrease the size of the sample when the control is routine. Step 4: Plan data collection The sampling plan gives an idea of the "who, where, what, when, why, and how" (see Section 20.1) aspect of the tests to be conducted. A data collection plan can be used to determine how the test results will be recorded. The accurate recording of test results is an extremely important part of the test documentation. Planning data collection prior to beginning the testing can be very helpful to ensure the information collected will provide conclusive data from which to evaluate the controls. Step 5: Conduct the tests The final step of testing and evaluating controls consists of actually effectuating the testing protocol and documenting the results. At the conclusion of the testing, the results are analyzed and evaluated. Evaluating involves reviewing the information collected and making an overall judgment on the adequacy of the internal control system as a whole. Deficient areas are to be categorized into Control Deficiencies, Significant Deficiencies, and Material Weaknesses and should be considered for inclusion in the CPIC submission (see Section 30.6). End Section 20.4 – Testing Methods: Back to Table of Contents