Skip to content

Navigation Menu

Sign in
Sign up

data: add monitor category — 760 monitors - #105

Merged
Seungpyo1007 merged 4 commits into
main from
data/import-staging
Jul 8, 2026
Merged

data: add monitor category — 760 monitors #105
Seungpyo1007 merged 4 commits into
main from
data/import-staging

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jul 8, 2026

Copy link
Copy Markdown
Member

What & why

Adds the dataset's third new category and second non-mobile class: monitor — with 760 monitors. This is the TechAPI half; the engine half (model/schema/router/seed/dump/validate) merged in TechEngine PR #40 and the submodule pin was bumped to bfdd4d9, so monitors now validate, serve and dump like any other category.

  • Added: 760 monitor records (Acer 532, Asus 109, LG 31, Samsung 27, Dell 19, MSI, Lenovo, Gigabyte, HP, BenQ, NEC, Philips)
  • All verified: false, brand required FK. Display specs — size_inch, resolution, aspect_ratio (100%), refresh_hz (85%), panel_type, curvature, hdr, rating (100%), msrp_usd (100%) — extracted from the product titles.

Commits (each Refs #1): validator rule, 2 brand-group data commits, and the surgical dump refresh (new monitors collection).

Closes #1

Source

durjoychandrapaul/amazon-products-sales-monitor-dataset (Kaggle, CC BY 4.0). Resolution labels mapped to WxH (with explicit title WxH preferred); refresh_hz/panel_type/curvature/hdr parsed from the title; MSRP is USD. release_date is a placeholder (2023年01月01日) — the source is current Amazon listings with no dates. 187 rows skipped (brands not in the catalog: ViewSonic, AOC, Sceptre, ...).

Dump

Regenerated via TechEngine app.dump at the monitor-supporting pin bfdd4d9. Monitors are unscored — the dump adds v1/monitors/** + a manifest entry; only the new collection + manifest are committed.

Checklist

  • python -m app.validate — ✅ passed (full dataset incl. 760 monitors)
  • python TechEngine/integrity_check.py data --strict — ✅ no hard anomalies
  • git diff --check — ✅ clean
  • Correct data/monitor/<brand>/<year>/<base>/<slug>.json paths, kebab-case unique slugs, source_urls present
 

Mirror the TechEngine monitor validator: MONITOR_REQUIRED fields,
ranges (size_inch 5-120, refresh_hz 24-1000, ppi 20-1000, rating 0-5,
msrp 10-50000), brand FK, and the variant path convention.
Refs #1 
New `monitor` category seeded from the Amazon monitor dataset
(durjoychandrapaul, Kaggle, CC BY 4.0). Acer line with size, resolution,
aspect ratio, refresh rate, panel, curvature, HDR, rating and MSRP
(extracted from the product titles). verified:false.
Refs #1 
...l and others
Remaining Amazon monitor records (Asus, LG, Samsung, Dell, MSI, Lenovo,
Gigabyte, HP, BenQ, NEC, Philips). verified:false.
Refs #1 
Add the 760 monitor records to the static v1 dump (new `monitors`
collection, unscored) and register it in the manifest. Regenerated via
TechEngine app.dump at the monitor-supporting submodule pin bfdd4d9;
only the new monitor collection + manifest are committed.
Refs #1
@Seungpyo1007 Seungpyo1007 added enhancement New feature or request data Dataset changes labels Jul 8, 2026
@github-actions github-actions Bot added the app Validator or application code changes label Jul 8, 2026
@Seungpyo1007 Seungpyo1007 moved this from Todo to In Progress in TechAPI-Project Jul 8, 2026

Copy link
Copy Markdown
Member Author

🤖 Review — PASS

Check Result
python -m app.validate ✅ passed (full dataset incl. 760 monitors)
python TechEngine/integrity_check.py data --strict ✅ no hard anomalies
git diff --check ✅ clean
app.dump @ bfdd4d9 ✅ 108,366 records incl. monitors:760

Data-change summary

  • Added: 760 monitors (new category) · Modified/Deleted: 0
  • Wired end-to-end: engine landed in TechEngine PR data(mobile): import sady36 market variants #40 (merged); submodule pinned to bfdd4d9.
  • Specs extracted from Amazon titles — aspect/rating/msrp 100%, refresh 85%, plus panel/curvature/HDR where present.

Example records

  • data/monitor/acer/2023/acer-eda320q/...-amazonmon-90.json (Acer Nitro 31.5" 1920x1080, 165Hz, 1500R curved, 170,ドル 4.4★)
  • data/monitor/lg/2023/... UltraGear QHD 240Hz IPS.

Site build

  • No site/src changes. Surgical dump — only the new v1/monitors/** collection + manifest (monitors are unscored).

Copy link
Copy Markdown
Member Author

📊 Dataset stats (post-merge projection)

Metric Count
Total records 108,366
verified: true 10,028
verified: false 98,332
missing verified 6
Verified ratio 9.25%

monitors: 760 — the dataset now spans 10 categories (brand, soc, smartphone, tablet, watch, pda, gpu, cpu, laptop, monitor).

Seungpyo1007 merged commit 4fc49b8 into main Jul 8, 2026
5 checks passed

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

No records scored.

Full-dataset baseline

105671 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
 title Verification bands — all records
 "Green" : 10029
 "Yellow" : 93177
 "Red" : 2465
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 189 10 179 0 5.3%
soc 🟩🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥🟥 2104 123 680 1301 5.8%
smartphone 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93343 8453 83770 1120 9.1%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3455 174 3253 28 5.0%
watch 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 433 11 412 10 2.5%
pda 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 140 27 107 6 19.3%
gpu 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 2030 245 1785 0 12.1%
cpu 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨 3977 986 2991 0 24.8%
All 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 105671 10029 93177 2465 9.5%

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
python integrity_check.py TechAPI/data --strict PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0

Changed record examples

  • No data file changes detected.

Heuristic review

  • Heuristic warnings: none found.

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 189 10 179 0 189 5.3%
soc 2104 123 1981 0 2104 5.8%
smartphone 93343 8453 84890 0 93343 9.1%
tablet 3455 174 3281 0 3455 5.0%
watch 433 11 422 0 433 2.5%
pda 140 27 113 0 140 19.3%
gpu 2030 244 1786 0 2030 12.0%
cpu 3977 986 2991 0 3977 24.8%
all 105671 10028 95643 0 105671 9.5%

Warning

Tracked verified coverage is below 50% for watch 2.5% (11/433), tablet 5.0% (174/3455), brand 5.3% (10/189), soc 5.8% (123/2104), smartphone 9.1% (8453/93343), all 9.5% (10028/105671), gpu 12.0% (244/2030), pda 19.3% (27/140), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py --strict
loaded CPU=3977 GPU=2030
✅ integrity gate: no hard anomalies.
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 152
GPU cross-source ratio outliers + sanity 18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Labels

app Validator or application code changes data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

AltStyle によって変換されたページ (->オリジナル) /