Name Degradation Testing

Name Degradation Testing for Watchlist Validations

Name degradation testing is a crucial technique for assessing model performance in response to real-world efforts to conceal identities or structure payments. This process includes altering watchlist names to replicate spelling variations or format adjustments that criminals might employ to avoid detection. For instance, systematically adjusting watchlist samples by shortening first names, altering spellings, and varying punctuation. A significant drop in match scores on degraded variants signals insufficient model flexibility. Retraining with expanded name data can enhance real-world adaptability.

Best practices

Consider below best practices for integrating effective name degradation tests:

• Review BSA/AML risk assessment: Identify areas of higher vulnerability, such as products/services, customer segments, or geographic locations.

• Analyze past suspicious activity reports (SARs) and transaction monitoring alerts: Look for name-related red flags or anomalies in previous reports and alerts.

• Assess system exposure:

  • Evaluate systems with more exposure, including onboarding, consumer ID validation, ongoing transaction monitoring, wire transfers, etc.
  • Identify weak points in current models.

• Test performance on synthesized name variations: Evaluate the system's performance on synthesized name variations to gauge its effectiveness.

• Analyze false negatives/positives: Evaluate instances of false negatives and positives to identify areas for improvement.

• Gather feedback from BSA Officers and Analysts: Obtain input from BSA officers and analysts regarding system performance and potential vulnerabilities.

• Implement tiered testing:

  • Focus on testing in the highest vulnerability areas initially.
  • Expand testing scope over time.

• Regularly review and update testing: Periodically review testing protocols and update the focus based on evolving risk postures.

👍

Tip:

We recommend applying name degradation methods proportionate to your risk profile, beginning with high-need areas for validating identity and transaction monitoring. Targeted testing aligned with inherent risks efficiently enhances fraud and AML detection capabilities. This approach provides significant insights into your model boundaries, ensuring readiness for any compliance audits.

Need more?

For customized requirements, reach out to your Account Manager, and we can provide hands-on guidance as needed.