Recent advances in deep learning and automatic speech recognition have enhanced the accuracy of end-to-end speech recognition to unprecedented levels. However, recognizing personal content like contact names still poses a challenge. In this study, we propose a personalization solution for an end-to-end system based on connectionist temporal classification. Our solution utilizes a class-based language model where a general language model captures the context for named entity classes, while personal named entities are compiled in a separate finite state transducer. Additionally, we introduce a phoneme-to-wordpiece model to map uncommon named entities to more common homophonic wordpieces. We also incorporate wordpiece prior normalization to prioritize rare wordpieces, resulting in a further 48.9% relative improvement in the accuracy of personal named entities, in addition to the improvements achieved through personalization. This research enables our systems to compete on par with highly competitive personalized hybrid systems in terms of personal named entity recognition.
