Improving transformer-based speech recognition using unsupervised pre-training

Dongwei Jiang, Xiaoning Lei, Wubo Li, Ne Luo, Yuxuan Hu, Wei Zou, Xiangang Li, (2019), “Improving transformer-based speech recognition using unsupervised pre-training.” arXiv preprint arXiv:1910.09932.

Download paper here