Understanding the Evaluation Tool #926
Unanswered
Kalachuchi07
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I've recently been exploring the Evaluation tool in BirdNET GUI and tested it with one of my audio clips containing nightjar calls. All of BirdNET's detections were true positives, though it missed some calls within the recording. Below is a screenshot of my ground truth labels against BirdNET's selections:
When I tried running these two in the evaluation tool, it gave me a precision of 0.75. This confused me because every prediction has an annotation in the same timeframe within the minimum overlap (0.2s). Do the ground truth labels have to align exactly with the 3-second segment of the prediction in order for it to output 100% precision?
Additionally, I want to know whether this tool also evaluates recordings that have positives from ground truthing but no detections by BirdNET? I tried running the evaluation for a few audio recordings together, with the recording that didn't have a detection from BirdNET having an assigned "prediction" which was an empty file that only had column headings. However, only the recordings that had actual predictions from BirdNET were considered. I was hoping the recordings without any predictions would be considered as well, especially when calculating recall and F1 score.
All reactions