The Secretary may limit the loan by written notice to the borrower to those amounts, if any, already disbursed under the loan, if the Secretary has determined that the borower has failed to comply with a material term or condition set forth in the loan agreement.