Learn more about technologyandthefuture with this collection
Understanding machine learning models
Improving data analysis and decision-making
How Google uses logic in machine learning
The transcription feature in the Google Translate app may be used to create a live, translated transcription for events like meetings and speeches, or for a story at the dinner table. In such settings, it is useful for the translated text to be displayed promptly to help keep the reader engaged.
Early versions of this feature the translated text suffered from multiple real-time revisions. The non-monotonic relationship between the source and the translated text, in which words at the end of the source sentence can influence words at the beginning of the translation.
The new version of the Google Translate app that significantly reduces translation revisions and improves the user experience. The research enabling this is presented in two papers. The first formulates an evaluation framework tailored to live translation and develops methods to reduce instability. The second demonstrates that these methods do very well compared to alternatives, while still retaining the simplicity of the original approach. The resulting model is much more stable and provides a noticeably improved reading experience within Google Translate.
Erasure: Measures the additional reading burden on the user due to instability. It is the number of words that are erased and replaced for every word in the final translation.
Lag: Measures the average time that has passed between when a user utters a word and when the word’s translation displayed on the screen becomes stable. Requiring stability avoids rewarding systems that can only manage to be fast due to frequent corrections.
BLEU score: Measures the quality of the final translation. Quality differences in intermediate translations are captured by a combination of all metrics.
It is important to recognize the inherent trade-offs between these different aspects of quality. Transcribe enables live-translation by stacking machine translation on top of real-time automatic speech recognition. For each update to the recognized transcript, a fresh translation is generated in real time; several updates can occur each second. This approach placed Transcribe at one extreme of the 3 dimensional quality framework: it exhibited minimal lag and the best quality, but also had high erasure. Understanding this allowed us to work towards finding a better balance.
One straightforward solution to reduce erasure is to decrease the frequency with which translations are updated. Along this line, “streaming translation” models (for example, STACL and MILk) intelligently learn to recognize when sufficient source information has been received to extend the translation safely, so the translation never needs to be changed. In doing so, streaming translation models are able to achieve zero erasure.
In our paper, “Re-translation versus Streaming for Simultaneous Translation”, we show that our original “re-translation” approach to live translation can be fine-tuned to reduce erasure and achieve a more favourable erasure/lag/BLEU trade-off. Without training any specialized models, we applied a pair of inference-time heuristics to the original machine translation models — masking and biasing.
The end of an on-going translation tends to flicker because it is more likely to have dependencies on source words that have yet to arrive. We reduce this by truncating some number of words from the translation until the end of the source sentence has been observed. This masking process thus trades latency for stability, without affecting quality. This is very similar to delay-based strategies used in streaming methods such as Wait-k, but applied only during inference and not during training.
The combination of masking and biasing, produces a re-translation system with high quality and low latency, while virtually eliminating erasure. The table below shows how the metrics react to the heuristics we introduced and how they compare to the other systems discussed above. The graph demonstrates that even with a very small erasure budget, re-translation surpasses zero-flicker streaming translation systems (MILk and Wait-k) trained specifically for live-translation.
The solution outlined above returns a decent translation very quickly, while allowing it to be revised as more of the source sentence is spoken. The simple structure of re-translation enables the application of our best speech and translation models with minimal effort. However, reducing erasure is just one part of the story — we are also looking forward to improving the overall speech translation experience through new technology that can reduce lag when the translation is spoken, or that can enable better transcriptions when multiple people are speaking.
MORE LIKE THIS
Ready for the next level?
Read Like a Pro
Explore the World’s
Save ideas for later reading, for personalized stashes, or for remembering it later.
# Personal Growth
Take Your Ideas
Just press play and we take care of the words.
No Internet access? No problem. Within the mobile app, all your ideas are available, even when offline.
Ideas for your next work project? Quotes that inspire you? Put them in the right place so you never lose them.
2 Million Stashers
Best app ever! You heard it right. This app has helped me get back on my quest to get things done while equipping myself with knowledge everyday.
Great interesting short snippets of informative articles. Highly recommended to anyone who loves information and lacks patience.
This app is LOADED with RELEVANT, HELPFUL, AND EDUCATIONAL material. It is creatively intellectual, yet minimal enough to not overstimulate and create a learning block. I am exceptionally impressed with this app!
Don’t look further if you love learning new things. A refreshing concept that provides quick ideas for busy thought leaders.
I have only been using it for a few days now, but I have found answers to questions I had never consciously formulated, or to problems I face everyday at work or at home. I wish I had found this earlier, highly recommended!
Great for quick bits of information and interesting ideas around whatever topics you are interested in. Visually, it looks great as well.
Even five minutes a day will improve your thinking. I've come across new ideas and learnt to improve existing ways to become more motivated, confident and happier.
Brilliant. It feels fresh and encouraging. So many interesting pieces of information that are just enough to absorb and apply. So happy I found this.
Read & Learn
Access to 200,000+ ideas
Access to the mobile app
Unlimited idea saving & library
Unlimited listening to ideas
Downloading & offline access
Claim Your Limited Offer
Get Deepstash Pro
Supercharge your mind with one idea per day
Enter your email and spend 1 minute every day to learn something new.
I agree to receive email updates