HugeCTR：面向点击率(CTR)估计训练的高效GPU框架

HugeCTR：面向点击率(CTR)估计训练的高效GPU框架

HugeCTR

HugeCTR is a high-efficiency GPU framework designed for Click-Through-Rate (CTR) estimation training.

Design Goals:

Optimized for recommender systemEasy to be customized

Please find more introductions in our HugeCTR User Guide and doxygen files in directory docs\

Requirements

cuBLAS >= 9.1Compute Capability >= 60 (P100)CMake >= 3.8cuDNN >= 7.5NCCL >= 2.0Clang-Format 3.8OpenMPI >= 4.0 (optional, if require multi-nodes training)

Build

Init Git

$ git submodule init$ git submodule update

Build with Release

Compute Capability can be specified by -DSM=XX, which is SM=60 by default. Only one Compute Capability is avaliable to be set.

$ mkdir -p build$ cd build$ cmake -DCMAKE_BUILD_TYPE=Release -DSM=XX ..$ make

Build with Debug

Compute Capability can be specified by -DSM=XX, which is SM=60 by default. Only one Compute Capability is avaliable to be set.

$ mkdir -p build$ cd build$ cmake -DCMAKE_BUILD_TYPE=Debug -DSM=XX ..$ make

Build with Mixed Precision (WMMA) Support

To use mixed precision training, enable USE_WMMA and set SCALER to 128/256/512/1024 by:

$ mkdir -p build$ cd build$ cmake -DSM=XX -DUSE_WMMA=ON -DSCALER=YYY ..

Run

Please refer to samples/*

Coding Style and Refactor

Default coding style follows Google C++ coding style (link). This project also uses Clang-Format(link) to help developers to fix style issue, such as indent, number of spaces to tab, etc. The Clang-Format is a tool that can auto-refactor source code. Use following instructions to install and enable Clang-Format:

Install

$ sudo apt-get install clang-format

Run

# First, configure Cmake as usual $ mkdir -p build$ cd build$ cmake -DCLANGFORMAT=ON ..# Second, run Clang-Format$ cmake --build . --target clangformat# Third, check what Clang-Format did modify$ git status# or$ git diff

Document Generation

Doxygen is supported in HugeCTR and by default on-line documentation browser (in HTML) and an off-line reference manual (in LaTeX) can be generated within docs/.

Install

Download doxygen

Generation

Within project home directory

$ doxygen

File Format

Totally three kinds of files will be used as input of HugeCTR Training: configuration file (.json), model file, data set.

Configuration File

Configuration file should be a json format file e.g. simple_sparse_embedding.json

There are four sessions in a configuration file: "solver", "optimizer", "data", "layers". The sequence of these sessions is not restricted.

You can specify the device (or devices), batchsize, model_file.. in solver session;and the optimizer that will be used in every layer.File list and data set related configurations will be specified under data session.Finally, layers should be listed under layers. Note that embedders should always be the first layer.

Model File

Model file is a binary file that will be loaded for weight initilization. In model file weight will be stored in the order of layers in configuration file.

Data Set

A data set includes a ASCII format file list and a set of data in binary format.

A file list starts with a number which indicate the number of files in the file list, then comes with the path of each data file.

$ cat simple_sparse_embedding_file_list.txt10./simple_sparse_embedding/simple_sparse_embedding0.data./simple_sparse_embedding/simple_sparse_embedding1.data./simple_sparse_embedding/simple_sparse_embedding2.data./simple_sparse_embedding/simple_sparse_embedding3.data./simple_sparse_embedding/simple_sparse_embedding4.data./simple_sparse_embedding/simple_sparse_embedding5.data./simple_sparse_embedding/simple_sparse_embedding6.data./simple_sparse_embedding/simple_sparse_embedding7.data./simple_sparse_embedding/simple_sparse_embedding8.data./simple_sparse_embedding/simple_sparse_embedding9.data

A data file (binary) contains a header and data (many samples).

Header Definition:

typedef struct DataSetHeader_{ long long number_of_records; //the number of samples in this data file long long label_dim; //dimension of label long long slot_num; //the number of slots in each sample long long reserved; //reserved for future use} DataSetHeader;

Data Definition (each sample):

typedef struct Data_{ int label[label_dim]; Slot slots[slot_num];} Data;typedef struct Slot_{ int nnz; T* keys; //long long or uint} Slot;

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。