Threshold optimization¶

Supported for: AI-ADC EfficientNet

This process calculates the best confidence thresholds per class based on a model, a dataset with annotations, and a target unknown rate.

When a prediction’s confidence falls below the threshold, the system automatically adds metadata Unknown: true to the sample. You can easily view or filter these low-confidence predictions by creating a custom metadata filter with the condition Unknown is true.

Purpose¶

1. Labeling: Use these thresholds on unlabeled data to identify samples that are good candidates for labeling and training because the model is uncertain about them. You can use predictive labeling in the label center or tag samples in the test center to generate these unknown predictions.
2. Unknown rate monitoring: Track how the unknown rate changes over time with inference monitoring. If the rate diverges from the initial target, your training data may no longer represent the inference data, indicating the need for model retraining.

To generate class thresholds¶

1. Open an AI-ADC EfficientNet project.
2. Go to the Optimization center, and then click Set up optimization.
3. In Datasets, select one or more datasets as ground truth.
4. In Models, select the model for generating predictions.

5. Configure optimization parameters:

   • Target unknown ratio: Desired percentage of samples marked as unknown.
   • Tolerance: Allowable deviation from the target unknown ratio.
   • Trials: Number of optimization attempts.

6. Click Run optimization.

Output¶

• The optimization generates confidence threshold parameters for each class.
• These thresholds can be used in predictive labeling, tests, and inference.