Optimize parameters¶

The optimization process calculates the best confidence thresholds per class based on a model, a dataset with annotations, and a target unknown rate.

Predictions with a confidence below this threshold will get the "unknown" class.

Purpose¶

1. Labeling: Use these thresholds on unlabeled data to identify samples that are good candidates for labeling and training because the model is uncertain about them. You can use predictive labeling in the label center or tag samples in the test center to generate these unknown predictions.
2. Unknown rate monitoring: Track how the unknown rate changes over time with inference monitoring. If the rate diverges from the initial target, your training data may no longer represent the inference data, indicating the need for model retraining.

To generate class thresholds¶

1. Open an AI-ADC EfficientNet project.
2. Go to the Optimization center, and then click Set up optimization.
3. In Datasets, select one or more datasets as ground truth.
4. In Models, select the model for generating predictions.

5. Configure optimization parameters:

   • Target unknown ratio: Desired percentage of "unknown" classifications.
   • Tolerance: Allowable deviation from the target unknown ratio.
   • Trials: Number of optimization attempts.

6. Click Run optimization.

Output¶

• The optimization generates confidence threshold parameters for each class.
• These thresholds can be used in predictive labeling, tests, and inference.

What's next?¶

• Test model