This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[BUG] fix SymbolBlock constructor from user-defined graph #21044

Open

khaotik wants to merge 5 commits into apache:master

from khaotik:fix-symbolblock-init

Open

[BUG] fix SymbolBlock constructor from user-defined graph #21044

khaotik wants to merge 5 commits into apache:master from khaotik:fix-symbolblock-init

+52 −14

Conversation

@khaotik

Copy link

Contributor

@khaotik khaotik commented May 29, 2022

Description

Fixes a bug where SymbolBlock does not copy user-given parameter attributes such as lr_mult. Simple reproducer:

#!/usr/bin/env python
import numpy as np
import mxnet as mx
import mxnet.symbol as mxs
DTYPE = np.float32
LR_MULT = 0.555
WD_MULT = 0.444
s_x = mxs.var('x', shape=(1,256,), dtype=DTYPE)
s_w = mxs.var('W', shape=(256,192), lr_mult=LR_MULT, wd_mult=WD_MULT, dtype=DTYPE)
s_b = mxs.var('b', shape=(1,192,), dtype=DTYPE, init=mx.init.Zero())
s_y = mxs.linalg.gemm(s_x, s_w, s_b)
fn = mx.gluon.SymbolBlock([s_y], [s_x])
fn.initialize()
fn.forward(mx.nd.random_uniform(-1., 1., shape=(1,256), dtype=DTYPE))
param_di = fn.collect_params()
assert param_di['W'].lr_mult == LR_MULT
assert param_di['W'].wd_mult == WD_MULT
assert not param_di['b'].data().asnumpy().any()

Checklist

Essentials

PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage
Code is well-documented

Changes

SymbolBlock now copies user-given wd_mult lr_mult init attributes from symbol.
stype attribute

Comments

stype attribute is still not copied. There's no MXSymbolInferStorageType C API. I think a complete fix would require some C API mods.

khaotik added 2 commits

May 29, 2022 08:26

@khaotik


 Let SymbolBlock copy lr_mult wd_mult init attributes from symbol

274d883

@khaotik


 add test

8544c05

@khaotik khaotik requested a review from szha as a code owner

May 29, 2022 01:46

@mxnet-bot

Copy link

mxnet-bot commented May 29, 2022

Hey @khaotik , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

To trigger all jobs: @mxnet-bot run ci [all]
To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [unix-cpu, unix-gpu, centos-gpu, centos-cpu, sanity, miscellaneous, windows-cpu, edge, clang, website, windows-gpu]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.