...
| Table of Contents | ||||||
|---|---|---|---|---|---|---|
|
📑 Case 1 - An image or
...
a region of interest contains annotations, but the app prediction doesn’t overlap with them.
Your results
Intersection over Union (IoU) | Precision [%] | Recall [%] | Average Precision (AP) | Number of False Positives (#FP) | |||
|---|---|---|---|---|---|---|---|
0.00 | 0.00 | 0.00 | 0.00 | n | |||
Meaning | there is no true-positive
|
there is no true-positive
all predicted instances are false-positive
| there is no true-positive
all annotated instances are false-negative
| there is no true-positive
the AP curve stays at zero | There are n (n - number) predictions in the image(s) or the ROI(s), but none of them |
correspond to |
annotations, so all of them are false-positive
|
What does this mean?
This is a valid case, where the model is unable to identify the correct instances, which could be due to insufficient training data preparation, i.e. incomplete or incorrect annotationannotations.
What can you do?
Check if all instances/objects areas have been annotated and labeled correctly as mislabeling will confuse the model. Check the training and validation images/ROI(s).
If this issue occurs on individual images or ROI(s) only, try to add more annotated training images or ROI(s) with a similar appearance to your training dataset.
📑 Case 2 - An image or
...
a region of interest contains no
...
annotations, and
...
the app doesn’t predict anything.
Your results
Intersection over Union (IoU) | Precision [%] | Recall [%] | Average Precision (AP) | Number of False Positives (#FP) | |
|---|---|---|---|---|---|
- | - | - | - | 0 | |
Meaning | without |
annotated |
instances and predicted |
instances, the IoU |
value cannot be estimated | without predicted instances, the Precision |
value cannot be estimated | without annotated |
instances, the Recall-value cannot be estimated | without defined values for Precision and Recall, |
the AP curve |
cannot be generated | without predicted instances, the number of false-positive instances is per definition zero |
What does this mean?
This is a valid case, where the model shows that it does not recognize anything in an image due to the lack of without annotations.
What can you do?
In practice, you want to find out how well your application performs on images showing actual object instancesobjects. If your validation
...
set doesn’t contain enough images with annotations, add more
...
of them. To do so, select a manual split between the training
...
and the validation images and increase the number of validation images with annotations.
📑 Case 3 - An image or
...
a region of interest contains annotations, but
...
the app doesn’t predict anything (Recall equals 0.00 or is unexpectedly low).
Your results
Intersection over Union (IoU) | Precision [%] | Recall [%] | Average Precision (AP) | Number of False Positives (#FP) | |
|---|---|---|---|---|---|
0.00 | - | 0.00 | - | 0 | |
Meaning | without |
predicted |
instances, there is no intersecting area | without predicted instances, the Precision |
value cannot be estimated | without predicted instances, the number of true-positive
all annotated instances are false-negative
| without a defined value for Precision, |
the AP curve |
cannot be generated | without predicted instances, the number of false-positive instances is per definition zero |
What does this mean?
This is a valid case, suggesting that your model is not yet able to sufficiently recognize your objects of interest.
On occasions where the Recall-value is greater than 0.00, but still unexpectedly low, your trained application model seems not to perform optimally poorly when recognizing your annotated instances.
...
First, you need to check, if the annotations on your validation images are correct and complete (i.e. whether all target instances have been annotated). If annotations are not correct and complete, then you may have omitted annotations in your training images. This can be the reason for the low Recall-value, as the application will also mimic your annotation behavior and omit instances.
Further, you can add more annotated regions annotations with the desired label of to your interest to the training set to improve your performance. You will get the optimal improvements by adding images or ROI(s) containing instances with a similar appearance to the validation images that have not been correctly classified by the trained model.
If annotations are correct and complete, then you can improve your app by adding new images or ROI(s) containing annotations to the training set. You will get the optimal improvements by adding images or ROI(s) containing instances with a similar appearance to the regions in the validation images that have not been correctly classified by the trained modelapplication.
📑 Case 4 - An image or
...
a region of interest contains no
...
annotations, but there is a prediction (Precision equals 0.00 or is unexpectedly low).
Your results
Intersection over Union (IoU) | Precision [%] | Recall [%] | Average Precision (AP) | Number of False Positives (#FP) | |
|---|---|---|---|---|---|
0.00 | 0.00 | - | - | n | |
Meaning | without |
annotated |
instances, there is no intersecting area | without annotated |
instances, there is no true-positive
all predicted instances are false-positive
| without annotated |
instances, the Recall value cannot be estimated | without a defined value for Recall, |
the AP curve |
cannot be generated | there are n (n - number)predictions in the image or the ROI, but none of them corresponds to |
annotations, so all of them are false-positive
|
What does this mean?
This is a false-positive prediction made by your trained application, meaning it predicts areas that shouldn’t be predicted.
On occasions where the Precision - value is greater than 0.00, but still unexpectedly low, your trained model application seems to predict instances in your validation images that have not been annotated. This can either be due to the fact that because the application has made false-positive predictions or that because some annotations in your validation images do not include all target instances.
What can you do?
Verify if there are really no instances of the target labels present in the image(s) or ROI(s). Try to add images or ROI(s) that have a similar appearance to the ones containing false-positive predictions.
Have a look at the annotations in your validation images in cases where the app has performed a segmentation task and check whether the annotations are complete. If annotations are correct and complete, then you can improve your trained model by adding new images or ROI(s) without any instances to the training set.
You will get the optimal improvements by adding images or ROI(s) containing instances with a similar appearance as to the areas that have been detected by the model application in the validation images.
...
If you still have questions regarding your application training, feel free to send us an email at support@ikosa.ai. Copy-paste your training ID in into the subject line of your email.
...