create tools for model_summary parse and mock operators execution#2

Open

baominghelly wants to merge 1 commit into

master from

add_new_tools

Open

create tools for model_summary parse and mock operators execution #2
baominghelly wants to merge 1 commit into
master from
add_new_tools

Conversation

@baominghelly

@baominghelly baominghelly commented Nov 21, 2025

Copy link

Copy Markdown

Collaborator

Description

create tools for model_summary parse and mock operators execution

Test

Unit test

(base) ➜ InfiniMetrics git:(add_new_tools) python -m unittest discover tests
.....
============================================================
🚀 Starting operator performance test (2 total)...
============================================================
--- Running (1/2): test.layer1 ---
 > Status: SUCCESS, Latency: 32.4055 ms
--- Running (2/2): test.layer2 ---
 > Status: SUCCESS, Latency: 13.4484 ms
============================================================
✅ All operator tests completed.
============================================================
.
----------------------------------------------------------------------
Ran 6 tests in 0.453s
OK

Test with real data

(base) ➜ benchmark git:(add_new_tools) python parse_summary.py -i model_summary
--- 1. Parsing Phase ---
Parsing model_summary ...
Parsing complete! Found 76 operator instances in sequential order.
Test cases saved to: operator_test_cases.json
(base) ➜ benchmark git:(add_new_tools) ✗ python run_benchmark.py -i operator_test_cases.json
--- 1. Loading Test Configuration ---
Successfully loaded 76 test cases (from operator_test_cases.json)
============================================================
🚀 Starting operator performance test (76 total)...
============================================================
--- Running (1/76): transformer.wte ---
 > Status: SUCCESS, Latency: 62.6148 ms
--- Running (2/76): transformer.wpe ---
 > Status: SUCCESS, Latency: 94.1032 ms
--- Running (3/76): transformer.h.0.ln_1 ---
 > Status: SUCCESS, Latency: 82.2264 ms
...
--- Running (72/76): transformer.h.11.ln_2 ---
 > Status: SUCCESS, Latency: 65.5149 ms
--- Running (73/76): transformer.h.11.mlp.c_fc ---
 > Status: SUCCESS, Latency: 65.9573 ms
--- Running (74/76): transformer.h.11.mlp.c_proj ---
 > Status: SUCCESS, Latency: 36.6459 ms
--- Running (75/76): transformer.ln_f ---
 > Status: SUCCESS, Latency: 30.699 ms
--- Running (76/76): lm_head ---
 > Status: SUCCESS, Latency: 95.2294 ms
============================================================
✅ All operator tests completed.
============================================================
--- 3. Saving Final Report ---
Final assessment report (including latency) saved to: operator_test_results_20251121_105423.json

@baominghelly


 create tools for model_summary parse and mock operators execution

7f5297c

@baominghelly baominghelly requested a review from Chamberlain0w0

November 21, 2025 08:04

@baominghelly baominghelly self-assigned this

Nov 21, 2025

@Chamberlain0w0

Chamberlain0w0 commented Nov 24, 2025

Copy link

Copy Markdown

Collaborator

这个 infinimetrics/tools/ 文件夹是什么定位?跟放在 infinimetrics/ 下面其他路径的 py 文件有什么关系?

Chamberlain0w0

Chamberlain0w0 reviewed

Nov 24, 2025

View reviewed changes

infinimetrics/tools/benchmark/model_summary

@@ -0,0 +1,131 @@

======================================================================================================================================================

@Chamberlain0w0 Chamberlain0w0 Nov 24, 2025

Copy link

Copy Markdown

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

输入文件不提交到 git repo 内

infinimetrics/tools/benchmark/op_library_mock.py

It pretends to run a test and returns a simulated latency.

(In the future, we can replace this with a real call like:

`actual_latency = infiniCore.benchmark(...)`)

@Chamberlain0w0 Chamberlain0w0 Nov 24, 2025

Copy link

Copy Markdown

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个信息最好在对应要替换的代码行上面加一个 TODO,这样清楚一些,便于之后定位

tests/tools/benchmark/test_op_library_mock.py

@@ -0,0 +1,49 @@

import unittest

@Chamberlain0w0 Chamberlain0w0 Nov 24, 2025

Copy link

Copy Markdown

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

单元测试的部分目前感觉不是特别必要,因为逻辑都是输入输出格式的检查,但之后如果格式还有变化的话,还需要再重新改这部分

@Chamberlain0w0

Chamberlain0w0 commented Nov 24, 2025

Copy link

Copy Markdown

Collaborator

我感觉还有一个点就是现在的逻辑过于"流式"了,观感上很像很多零碎的脚本。我感觉如果要形成一个"框架"的话,实现上最好还是面向对象一点,比如说负责读入的定义一个 class Parser,然后固定几个操作接口;中间定义一个 Excecutor,结果输出再定义一个 Reporter,中间涉及的一些处理过的数据也可以定义为结构化的数据,这样的话所有测试都可以用一套逻辑调用,扩展性更强,并且 main 的脚本里会显得更清楚一些。晚点可以再讨论一下。

Labels

None yet

2 participants

@baominghelly @Chamberlain0w0

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

create tools for model_summary parse and mock operators execution#2

create tools for model_summary parse and mock operators execution #2
baominghelly wants to merge 1 commit into
master from
add_new_tools

Conversation

@baominghelly baominghelly commented Nov 21, 2025

Description

Test

Unit test

Test with real data

Uh oh!

Chamberlain0w0 commented Nov 24, 2025

Uh oh!

@Chamberlain0w0 Chamberlain0w0 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

@Chamberlain0w0 Chamberlain0w0 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

@Chamberlain0w0 Chamberlain0w0 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

Chamberlain0w0 commented Nov 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants