[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

Zhang-Zhenning wants to merge 1 commit into DeepRec-AI:main

from Zhang-Zhenning:inference_zzn

Open

[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

Zhang-Zhenning wants to merge 1 commit into DeepRec-AI:main from Zhang-Zhenning:inference_zzn

Conversation

@Zhang-Zhenning

Copy link

@Zhang-Zhenning Zhang-Zhenning commented Jul 15, 2022

I have finished exporting savedmodel for models in modelzoo. Under Deeprec/modelzoo there is a Get_SavedModel.md for new user to learn how to export and where the result model will locate.

@CLAassistant

Copy link

CLAassistant commented Jul 15, 2022

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

@shanshanpt shanshanpt self-requested a review

July 17, 2022 10:07

@shanshanpt

Copy link

Member

shanshanpt commented Jul 17, 2022

for #232

shanshanpt

shanshanpt reviewed

Jul 17, 2022

View reviewed changes

modelzoo/BST/pb_to_pbtxt.py Outdated

@@ -0,0 +1,13 @@

from tensorflow.python.saved_model import loader_impl

from tensorflow.python.lib.io import file_io

from tensorflow.python.platform import tf_logging as logging

Copy link

Member

@shanshanpt shanshanpt Jul 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useless file.

modelzoo/BST/prepare_savedmodel.py Outdated

@@ -0,0 +1,738 @@

import time

import argparse

import tensorflow as tf

Copy link

Member

@shanshanpt shanshanpt Jul 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不需要修改原文件,在原文件上添加代码即可。

modelzoo/BST/start_serving.cc Outdated

#include "serving/processor/serving/processor.h"

#include "serving/processor/serving/predict.pb.h"

static const char* model_config = "{ \

Copy link

Member

@shanshanpt shanshanpt Jul 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

除fake数据之外,需要根据实际的train data生成serving data,然后能够根据serving data进行process

@shanshanpt

Copy link

Member

shanshanpt commented Jul 17, 2022

CLA required.

@Zhang-Zhenning

Copy link

Author

Zhang-Zhenning commented Jul 25, 2022

I have finished exporting savedmodel, extracting data from the input file to generate requests and serving for DIEN DeepFM WDL under modelzoo/features/EmbeddingVariable, please check.
One thing that needs to be emphasized for models with non-string inputs like DIEN, is the model_instance.cc will automatically run CreateWarmup to accelerate real serving, which is inappropriate under cases such as the input placeholders don't have a fixed length. During my local test I comment it out, please contact me if there is any problem.

shanshanpt

shanshanpt reviewed

Jul 25, 2022

View reviewed changes

modelzoo/features/EmbeddingVariable/DIEN/generate_data.py

cat_voc = os.path.join(data_location, "cat_voc.pkl")

def prepare_data(input, target, maxlen=None, return_neg=False):

# x: a list of sentences

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DeepRec中一般使用两个空格锁进,这些文件都稍微改一下

modelzoo/features/EmbeddingVariable/DIEN/generate_data.py Outdated

f.close()

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

多余的空格删除

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc Outdated

::tensorflow::eas::ArrayShape array_shape;

::tensorflow::eas::ArrayDataType dtype_f =

::tensorflow::eas::ArrayDataType::DT_FLOAT;

int num_elem = (int)cur_vector.size();

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不用(int)强转,会隐式转换的,或者定义为 size_t num_elem也可以

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc Outdated

int num_elem = (int)cur_vector.size();

array_shape.add_dim(1);

if((int)cur_vector.size() < 0){

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if((int)cur_vector.size() < 0) ,cur_vector.size()不会小于0的,类型是size_t本身是正值。

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc Outdated

return input;

}

array_shape.add_dim((int)cur_vector.size());

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cur_vector.size()前面已经赋值了num_elem

modelzoo/features/EmbeddingVariable/DeepFM/train.py Outdated

if tf_config:

print('train steps : %d' % train_steps)

print("-----------")

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useless code

modelzoo/features/EmbeddingVariable/SavedModel_Serving.md

@@ -0,0 +1,58 @@

## How to use prepare_savedmodel.py to get savedmodel

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

文件名叫README.md就可以

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc Outdated

}

::tensorflow::eas::ArrayProto get_proto_f(float char_input,int dim,::tensorflow::eas::ArrayDataType type){

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

函数重复

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc Outdated

while (record != NULL) {

// only 1 label and 39 feature

if (j >= 40) break;

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc Outdated

struct input_format39 inputs;

inputs.I1 = (float)(atof(all_elems[start_idx]));

inputs.I2 = (float)(atof(all_elems[start_idx+1]));

inputs.I3 = (float)(atof(all_elems[start_idx+2]));

Copy link

Member

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个搞一个循环吧

@shanshanpt

Copy link

Member

shanshanpt commented Jul 28, 2022

先删除掉不必要的修改。

shanshanpt

shanshanpt reviewed

Jul 28, 2022

View reviewed changes

Copy link

Member

@shanshanpt shanshanpt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

先把所有的文件整理一下

@Zhang-Zhenning


 reorganize the file structure

576652e

@Zhang-Zhenning Zhang-Zhenning force-pushed the inference_zzn branch from b52d097 to 576652e Compare

July 28, 2022 08:59

shanshanpt

shanshanpt reviewed

Aug 8, 2022

View reviewed changes

modelzoo/features/serving/README.md

## How to use prepare_savedmodel.py to get savedmodel

- Current support model: \

BST, DBMTL, DeepFM, DIEN, DIN, DLRM, DSSM, ESMM, MMoE, SimpleMultiTask, WDL

Copy link

Member

@shanshanpt shanshanpt Jul 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now support DIEN, DeepFM and and WDL.

modelzoo/features/serving/DIEN/generate_data.py

maxlen,

data_location=data_location)

f = open("./test_data.csv","w")

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_data.csv 怎么得到的呢,可以在readme中写清楚

modelzoo/features/serving/DIEN/prepare_savedmodel.py

with tf.Session() as sess1:

model = Model_DIN_V2_Gru_Vec_attGru_Neg(n_uid, n_mid, n_cat,

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个Model_DIN_V2_Gru_Vec_attGru_Neg是哪里import的?

modelzoo/features/serving/DIEN/start_serving_dien.cc

input.set_dtype(dtype_f);

switch(dtype_f){

case 1:

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

两个case中大部分代码都在重复的,可以只在
input.add_float_val((float)atof(cur_vector.back()));
input.add_int_val((int)atoi(cur_vector.back()));
代码加上if判断。

modelzoo/features/serving/DIEN/start_serving_dien.cc

temp_ptrs.clear();

// traverse current line

record = strtok(line, delim);

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

下面这个可以搞成类似 split(...) 函数吗,现在这样写比较hack。

modelzoo/features/serving/DeepFM/start_serving_deepfm.cc

// input setting

::tensorflow::eas::ArrayProto I1 = get_proto_cc(&inputs.I1_13[0],1,dtype_f);

::tensorflow::eas::ArrayProto I2 = get_proto_cc(&inputs.I1_13[1],1,dtype_f);

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

搞一个数组存储?

modelzoo/features/serving/DeepFM/start_serving_deepfm.cc

int main(int argc, char** argv) {

// PLEASE EDIT THIS LINE!!!!

char filepath[] = "/home/deeprec/DeepRec/modelzoo/features/EmbeddingVariable/WDL/test.csv";

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

文件如何生成的呢

modelzoo/features/serving/README.md

For every model listed above, there is a prepare_savedmodel.py. To run this script please firstly ensure you have gotten the checkpoint file from training. To use prepare_savedmodel.py, please use:

```

cd [modelfolder]

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

training的文件位置需要写清楚,用户可能找不到具体的training的文件所在。
或者你把training文件copy到你的目录下也可以。

modelzoo/features/serving/WDL/start_serving_wdl.cc

int main(int argc, char** argv) {

// PLEASE EDIT THIS LINE!!!!

char filepath[] = "/home/deeprec/DeepRec/modelzoo/features/EmbeddingVariable/WDL/test.csv";

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

文件如何生成的呢

modelzoo/features/serving/WDL/start_serving_wdl.cc

::tensorflow::eas::ArrayDataType::DT_INT64;

// input setting

::tensorflow::eas::ArrayProto I1 = get_proto_cc(&inputs.I1_9[0],1,dtype_f);

Copy link

Member

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

搞一个数组存储

@liutongxuan liutongxuan changed the title ~~(削除) savedmodel generation (削除ここまで)~~ (追記) [Modelzoo] Add serving for DIEN, DeepFM and WDL. (追記ここまで)

Aug 11, 2022

@gumanchang

Copy link

gumanchang commented Oct 25, 2023

请问下,这个代码还是没有合并嘛? 现在仍然只有训练,没有推理部署,文档也没说

@candyzone

Copy link

Collaborator

candyzone commented Oct 25, 2023

请问下,这个代码还是没有合并嘛? 现在仍然只有训练,没有推理部署,文档也没说

DeepRec 本身具备推理的能力,这个PR只是这三个model 的推理例子,你可以参考

Labels

None yet

5 participants

@Zhang-Zhenning @CLAassistant @shanshanpt @gumanchang @candyzone

[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

Are you sure you want to change the base?

[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

Uh oh!

Conversation

@Zhang-Zhenning Zhang-Zhenning commented Jul 15, 2022

Uh oh!

CLAassistant commented Jul 15, 2022

Uh oh!

shanshanpt commented Jul 17, 2022

Uh oh!

@shanshanpt shanshanpt Jul 17, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 17, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 17, 2022

Choose a reason for hiding this comment

Uh oh!

shanshanpt commented Jul 17, 2022

Uh oh!

Zhang-Zhenning commented Jul 25, 2022

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

shanshanpt commented Jul 28, 2022

Uh oh!

@shanshanpt shanshanpt left a comment

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Jul 28, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

Uh oh!

@shanshanpt shanshanpt Aug 8, 2022

Choose a reason for hiding this comment

Uh oh!