Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Add evaluation scripts (Updated on 2023年11月07日) #119

Open
Assignees
Labels
evaluationadding tools for evaluation or bugs of eval scripts featureNew feature or request
@fengyuentau

Description

We now have over 15 models covering more than 10 tasks in the zoo. Although most of the models are converted to ONNX straightly from its original format, such conversion may potentially lead to drop of accuracy, especially for FP16 and Int8-quantized models. To show the actual accuracy for our users, we now already have some evaluation scripts with the following conditions met in https://github.com/opencv/opencv_zoo/tree/master/tools/eval:

  1. Reproduce the claimed accuracy with the converted FP32 ONNX model using OpenCV DNN as inference framework. The claimed accuracy is either from the source repository or paper on the same dataset, which needs to be specified in the first comment of the pull request.
  2. Once it is reproduced, apply the same evaluation script on FP16 and Int8-quantized models.

Take a look at the task list below for current status. Feel free to leave a comment for application or discussion before you start to contribute.

Status Task Dataset Models Notes
✅ Done in #70 Face Detection WIDERFace YuNet -
✅ Done in #72 Face Recognition LFW SFace -
❗️ Need Contribution License Plate Detection ? LPD-YuNet -
❗️ Need Contribution Object Detection COCO YOLOX & NanoDet Refer to #91
❗️ Need Contribution Text Detection ? DB -
✅ Done in #71 Text Recognition ICDAR2003 & IIIT5K CRNN (EN & CN) -
✅ Done in #69 Image Classification ImageNet PP-ResNet50 & MobileNet V1 / V2
✅ Done in #130 Human Segmentation Mini Supervisely Persons PP-HumanSeg -
❗️ Need Contribution QR Code Detection / Parsing ? WeChatQRCode -
❗️ Need Contribution Person Re-identification ? YoutuReID -
❗️ Need Contribution Palm Detection ? MP-PalmDet -
❗️ Need Contribution Hand Pose Estimation ? MP-HandPose -
❗️ Need Contribution Person Detection ? MP-PersonDet -
❗️ Need Contribution Pose Estimation ? MP-Pose -
❗️ Need Contribution Facial Expression Recognition RAF-DB FER -
❗️ Need Contribution Object Tracking ? VitTrack Could be done via #205

Metadata

Metadata

Labels

evaluationadding tools for evaluation or bugs of eval scripts featureNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      AltStyle によって変換されたページ (->オリジナル) /