Generative AI tools are changing how we break down language barriers. These advanced systems can now process not just words, but also videos, images, and audio. This allows them to understand context better and deliver more accurate translations in real time. It’s a big step forward from traditional translation tools that only handled plain text.

More Than Just Words: The Power of Multimodal AI

In the past, translation mainly meant converting written words from one language to another. Now, AI agents can analyze actions in videos, tone in voices, and visuals in images to grasp what’s really being communicated. Matt Hardy, VP of linguistic AI at RWS, explains that dealing with multiple data types is essential. Modern translation isn’t just about words; it’s about understanding the full picture. That’s why these AI tools are becoming so valuable, especially in customer service and productivity tasks where nuance matters.

Why Context and Culture Matter in Translation

Languages are more than just a set of rules for replacing words. They carry cultural and situational meaning. Stefan Mesken from DeepL emphasizes that capturing this nuance requires AI to be close to the user’s context. For example, tone, expressions, and physical gestures can all influence how a message is understood. Multimodal AI can pick up on these cues, making translations feel more natural and accurate.

One interesting idea is creating multilingual AI agents that can switch seamlessly between languages. This mirrors how humans interact, switching languages based on who they’re talking to or what they’re discussing. Currently, there are over 8,000 languages worldwide, but most AI tools excel mainly with high-resource languages like English. For many other languages, quality drops off. This presents a challenge but also an opportunity for AI to make a real difference in global communication.

Bringing AI into Business and Everyday Use

Big companies like RWS, DeepL, and Grammarly are developing AI systems that handle multiple languages better than ever before. DeepL, for example, has launched a “DeepL agent” that can translate text, images, and even diagrams across 36 languages. It uses multiple AI agents working together to improve accuracy, with human experts stepping in when necessary. This approach helps maintain high quality, especially for complex or less common languages.

RWS has created Evolve, a system that first translates text and then checks its accuracy with a second AI. If the machine isn’t confident, a human translator takes over. This hybrid approach combines the speed of AI with the creativity and understanding of human linguists. Hardy from RWS notes that machines are unlikely to replace human translators anytime soon. Human expertise remains essential for ensuring translations are correct and culturally appropriate.

Grammarly has also added multilingual features, allowing users to translate in real time across 19 languages. They focus heavily on quality, with teams of linguists evaluating translations to catch subtle nuances. However, the bigger challenge is expanding these capabilities to languages with less training data. Most AI systems are trained mainly on English, which limits their effectiveness for many world languages. Hardy points out that the next billion internet users will come from regions like Africa, Southeast Asia, and India. Making AI translation work well for these languages is crucial for global connectivity.

In short, AI tools that incorporate video, audio, and images are making translation more accurate, contextual, and human-like. While there’s still work to do, especially for low-resource languages, these advances promise a future where language barriers are much less of an obstacle. As AI continues to evolve, it will help more people communicate across cultures and borders with ease.

Inspired by

• https://www.computerworld.com/article/4072296/genai-agents-are-changing-language-translation-in-the-enterprise.html

Sources

• gmpg.org
• infoworld.com
• rws.com
• deepl.com
• grammarly.com