Semantic segmentation results improvement
If your Dice Coefficient, Precision, and Recall values are lower than expected or undefined (marked in the table as a dash “-”), don’t worry! In most cases, there are solutions to improve your results.
What to expect from this page?
- 1 Case 1 - An image or a region of interest contains annotated and labeled areas, but the app prediction doesn’t overlap with them.
- 2 Case 2 - An image or a region of interest contains no annotated and labeled areas, and the app doesn’t predict anything.
- 3 Case 3 - An image or a region of interest contains annotated and labeled areas, but the app doesn’t predict anything (Recall equals 0.00 or is unexpectedly low).
- 4 Case 4 - An image or a region of interest contains no annotated and labeled areas, but the app predicts something (Precision equals 0.00 or is unexpectedly low).
- 5 Case 5 - Dice Coefficient is unexpectedly low
- 6 Related articles
The Dice Coefficient, Precision, and Recall values range between 0 and 1. However, sometimes these values cannot be defined. This is not necessarily a bad sign, although it is critical to identify the root cause of this issue.
The following sections contain indications that can be causing such issues. However, possible solutions are suggested to improve the app’s performance after re-training.
Case 1 - An image or a region of interest contains annotated and labeled areas, but the app prediction doesn’t overlap with them.
Your results
| Dice Coefficient | Precision [%] | Recall [%] | Specificity [%] |
|---|---|---|---|---|
| 0.00 | 0.00 | 0.00 | 0 ≤ x < 100 |
Meaning | there is no true-positive GREENpredicted area | there is no true-positive GREENpredicted area all predicted areas are false-positive RED | there is no true-positive GREENpredicted area all annotated areas are false-negative blue | If all the background is predicted, the specificity is 0%. All predicted areas are false-positive RED and lower the amount of true-negative area (and thus specificity) |
What does this mean?
This is a valid case where the application cannot segment the correct area. It could be due to insufficient training data preparation, i.e. incomplete or incorrect labeling and/or annotation.
What can you do?
Check if all areas in your training and validation images/ROIs were annotated and labeled correctly. Wrong annotations and mislabeling negatively affect the application.
If this issue occurs on single images/ROIs only, add more annotated and labeled images/ROIs with a similar appearance to the ones with wrong predictions to your training set.
Case 2 - An image or a region of interest contains no annotated and labeled areas, and the app doesn’t predict anything.
Your results
| Dice Coefficient | Precision [%] | Recall [%] | Specificity [%] |
|---|---|---|---|---|
| - | - | - | 100% |
Meaning | without annotated area and predicted area, the value could not be defined | without a predicted area, the value could not be defined | without an annotated area, the value could not be defined | specificity is at its maximum, as there is no predicted area in the background |
What does this mean?
This is a valid case where the application does not recognize anything in an image/ROI without labeled annotations.
What can you do?
If this is an unexpected situation:
Add more annotated and labeled images/ROIs to your validation set.
Select manual data split between the training and the validation images/ROIs and increase the number of validation images.
Case 3 - An image or a region of interest contains annotated and labeled areas, but the app doesn’t predict anything (Recall equals 0.00 or is unexpectedly low).
Your results
| Dice Coefficient | Precision [%] | Recall [%] | Specificity [%] |
|---|---|---|---|---|
| 0.00 | - | 0.00 | 100% |
Meaning | without a predicted area there is no true-positive GREEN area | without a predicted area the value could not be defined | without a predicted area there is no true-positive GREEN area all annotated areas are false-negative blue | specificity is at its maximum, as there is no predicted area in the background |
What does this mean?
This is a valid case where the application cannot recognize the targeted areas.
On occasions where the Recall value is greater than 0.00, but still unexpectedly low, your trained application seems to perform poorly when recognizing your annotated area.
What can you do?
Add more images/ROIs containing annotated and labeled areas of the labels that were poorly recognized to your training set.
If the annotated and labeled areas are correct and complete, include new annotated and labeled images/ROIs in the training set.
Check if all areas in your validation images/ROIs were annotated and labeled correctly. If not, annotate and label all missing areas and correct any inaccuracies in your validation set.
Case 4 - An image or a region of interest contains no annotated and labeled areas, but the app predicts something (Precision equals 0.00 or is unexpectedly low).
Your results
| Dice Coefficient | Precision [%] | Recall [%] | Specificity [%] |
|---|---|---|---|---|
| 0.00 | 0.00 | - | 0 ≤ x < 100 |
Meaning | without an annotated area there is no true-positive area GREEN | without an annotated area there is no true-positive area GREEN all predicted areas are false-positive RED | without an annotated area the value could not be defined | If all the background is predicted, the specificity is 0%. All predicted areas are false-positive RED and lower the amount of true-negative area (and thus specificity) |
What does this mean?
This is a false-positive prediction by your trained application, meaning it predicts areas that shouldn’t be predicted.
If the Precision value is greater than 0.00 but still unexpectedly low, your trained application seems to predict areas in your validation images that have not been annotated.
This can either be because:
the app has made false-positive predictions, or
not all areas are annotated and labeled on your validation images.
What can you do?
Check if all areas in your validation images/ROIs were annotated and labeled correctly:
If not, annotate and label all missing areas and correct any inaccuracies in your validation set.
If yes, add background images/ROIs to the training set.
Add images/ROIs containing areas with a similar appearance as the false-positive predictions to the training set.
Case 5 - Dice Coefficient is unexpectedly low
What does this mean?
A low dice coefficient means that the overlap between your labeled annotations and the predicted areas is low. A low dice coefficient can be due to poor Recall, poor Precision, or both.
Important: Inaccurate annotations make it very hard for the application to properly recognize the desired regions. Just as the annotations, the predictions will be “inaccurate“ and affect Precision, Recall, and of course - the Dice value.
Learn more about annotating How to annotate an image?
What can you do?
Check whether values for Recall and Precision are also low and have a look at the sections Case 3 - Recall equals 0 or is unexpectedly low and Case 4 - Precision equals 0 or is unexpectedly low, respectively.
Having successfully alleviated all frustrating false predictions, you can now use the potential of your trained applications to full advantage!
If you still have questions regarding your application training, feel free to send us an email at support@ikosa.ai. Copy-paste your training ID in the subject line of your email.
Related articles
Copyright 2016-2021 KML Vision GmbH. IKOSA® is a registered EU trademark.