Codenet AI Translating Computer Languages

Certainly interesting, but how good, secure is the resulting translation?

IBM's CodeNet dataset can teach AI to translate computer languages  By A. Tarantola in Engadget

AI and machine learning systems have become increasingly competent in recent years, capable of not just understanding the written word but writing it as well. But while these artificial intelligences have nearly mastered the English language, they have yet to become fluent in the language of computers — that is, until now. IBM announced during its Think 2021 conference on Monday that its researchers have crafted a Rosetta Stone for programming code.

Over the past decade, advancements in AI have mainly been “driven by deep neural networks, and even that, it was driven by three major factors: data with the availability of large data sets for training, innovations in new algorithms, and the massive acceleration of faster and faster compute hardware driven by GPUs,” Ruchir Puri, IBM Fellow and Chief Scientist at IBM Research, said during his Think 2021 presentation, likening the new data set to the venerated ImageNet, which has spawned the recent computer vision land rush.

“Software is eating the world," Marc Andreessen wrote in 2011. "And if software is eating the world, AI is eating software,” Puri remarked to Engadget. “It is this relationship between the visual tasks and the language tasks, when common algorithms could be used across them, that has led to the revolution in breakthroughs in natural language processing, starting with the advent of Watson Jeopardy, way back in 2012,” he continued.

In effect, we’ve taught computers how to speak human, so why not also teach computers to speak more computer? That’s what IBM’s Project CodeNet seeks to accomplish.”We need our ImageNet, which can snowball the innovation and can unleash this innovation in algorithms,” Puri said. CodeNet is essentially the ImageNet of computers. It’s an expansive dataset designed to teach AI/ML systems how to translate code and consists of some 14 million snippets and 500 million lines spread across more than 55 legacy and active languages — from COBOL and FORTRAN to Java, C++, and Python.

“Since the data set itself contains 50 different languages, it can actually enable algorithms for many pairwise combinations,” Puri explained. “Having said that, there has been work done in human language areas, like neural machine translation which, rather than doing pairwise, actually becomes more language-independent and can derive an intermediate abstraction through which it translates into many different languages.” In short, the dataset is constructed in a manner that enables bidirectional translation. That is, you can take some legacy COBOL code — which, terrifyingly, still constitutes a significant amount of this country’s banking and federal government infrastructure — and translate it into Java as easily as you could take a snippet of Java and regress it back into COBOL.

It is the ImageNet of code.   .... " 

Translating Lost Languages Using ML

 Most interesting, lots more at the link.  Via patterns of association in other languages.   Might this be used in ways to link with associations between other language style 'patterns'?  What are the assumptions regarding the forms of the languages?   Ideas?   Plan to pass this along to people at our  'Language Lab'.  

Translating Lost Languages Using ML

MIT News, By Adam Conner-Simons. October 21, 2020

Researchers at the Massachusetts Institute of Technology (MIT) have developed a machine learning system that can automatically translate a lost language, without advanced knowledge of its relationship to other dialects. The system applies principles based on historical linguistic insights, including the fact that languages generally evolve in certain predictable patterns. MIT's Regina Barzilay and Jiaming Luo developed a decipherment algorithm that can segment words in an ancient language and map them to words in related languages. The algorithm infers relationships between languages, and can assess proximity between languages.

Amazon Launches Live Translation for Alexa

In the enterprise we looked at efficient ways to translate consumer comments.  Just tried the Alexa real-time translation mode in German - English.     Is now on all devices it seems.  Works well ...  You could do this previously, but you had to pause and say 'translate' and was process-klunky , now if you just pause it goes ahead and translates what you said.   All I heard was right on, am sure you could trip it up, but for basic understanding - very useful.  Devices with a screen will show the text and translation.  And probably has issues with humor/satire/mood.   Will compare with Google's translator method, which had worked quite well also.   They do not do Chinese, which I understand Google does well.  

Amazon launches Live Translation mode for Alexa    VentureBeat by Kyle Wiggers 

Amazon today rolled out Live Translation, a new Alexa feature that aims to assist with conversations between people who speak two different languages. Leveraging speech recognition and machine translation technology, Amazon says that Live Translation can interpret between a number of dialects in real time including English and French, Spanish, Hindi, Brazilians Portuguese, German, or Italian.

The pandemic appears to have supercharged voice app usage, which was already on an upswing. According to a study by NPR and Edison Research, the percentage of voice-enabled device owners who use commands at least once a day rose between the beginning of 2020 and the start of April. Just over a third of smart speaker owners say they listen to more music, entertainment, and news from their devices than they did before, and owners report requesting an average of 10.8 tasks per week from their assistant this year compared with 9.4 different tasks in 2019. And according to a new report from Juniper Research https://www.mediapost.com/publications/article/350643/voice-assistant-usage-seen-growing-to-84-billion.html , consumers will interact with voice assistants on 8.4 billion devices by 2024 ... ' 

Indiegogo Translation, with Accents

With 93 accents, which sounds exciting.  How much wil this improve translation?  Are we much closer to the fabled and fictional 'Babel fish'?!  Here presented with the usual caveats regarding investment in innovation systems like Indiegogo ... they have high risks.

Timekettle M2: 1st Offline Translation Earbuds
Speak and translate completely offline. And it sounds spectacular playing music!
Timekettle
3 Campaigns | Pasadena, United States
$707,765 USD by 6,302 backers
$623,752 USD by 5,666 backers on Aug 9, 2020

Back in 2017, our WT2 campaign launched with tremendous success - the world's most intelligent live translator that enthralled nearly 300K users across the globe. It's currently one of the best-selling translators on Amazon to date.

We listened to our user's experiences to see if we could up our game. After some fine-tuning, we are back with a fully immersive and highly sophisticated pair of earbuds capable of pumping out Hi-Fi music and interpreting many different languages offline. It is a complete answer to your translation needs - the Timekettle M2.

Using WiFi or a cellular network, M2 can translate up to 40 languages, including 93 unique accents. Why 93 accents, you may ask? We understand that not one country has one generalized accent (you'll already know this if you've met a Newyorker in Cali!). To bridge the gap between dialects, Timekettle M2 ensures recognition accuracy is high, allowing you to converse effortlessly wherever you are.

Another exciting feature is our Offline Speech Translation technology, which allows you to translate between English and Spanish, French, Japanese, Chinese, Korean, and Russian:  ... " 

Sign Language Translating Glove

Consider the broad increase in functionality from these kinds of AI applications.

A high-tech glove can translate sign language with 99-percent accuracy
It could help more people learn to sign.

Steve Dent, @stevetdent
Jun Chen Lab/UCLA

Researchers at UCLA have developed an inexpensive, high-tech glove that can translate sign language into written and spoken words on a smartphone (via Fast Company). The system works in real time and can interpret 660 American Sign Language signs with a promising 98.63 percent accuracy. It could one day be used to teach more people sign language and help deaf people communicate with non-sign language users.

The gloves have stretchable sensors made of electrically-sensing yarn running up each of the five fingers. The signals travel to a dollar coin-sized circuit board placed on the back of the glove, which then transmits wireless signals to smartphone. An app can convert those into text in real time at a rate of up to a word per second (60 words per minute). The team also place adhesive sensors on testers’ faces to capture facial expressions that are part of American Sign language. ...  "

Google Reports on Recent Advances in Translate

Google reports measure improvements in their Translate, but there are still contextual challenges.  I would be tempted to try to construct a risk analysis regarding its context, but that too would be hard.  Though impressed, I woud still be cautious about autonomous use.

Recent Advances in Google Translate
Monday, June 8, 2020
Posted by Isaac Caswell and Bowen Liang, Software Engineers, Google Research

Advances in machine learning (ML) have driven improvements to automated translation, including the GNMT neural translation model introduced in Translate in 2016, that have enabled great improvements to the quality of translation for over 100 languages. Nevertheless, state-of-the-art systems lag significantly behind human performance in all but the most specific translation tasks. And while the research community has developed techniques that are successful for high-resource languages like Spanish and German, for which there exist copious amounts of training data, performance on low-resource languages, like Yoruba or Malayalam, still leaves much to be desired. Many techniques have demonstrated significant gains for low-resource languages in controlled research settings (e.g., the WMT Evaluation Campaign), however these results on smaller, publicly available datasets may not easily transition to large, web-crawled datasets.

In this post, we share some recent progress we have made in translation quality for supported languages, especially for those that are low-resource, by synthesizing and expanding a variety of recent advances, and demonstrate how they can be applied at scale to noisy, web-mined data. These techniques span improvements to model architecture and training, improved treatment of noise in datasets, increased multilingual transfer learning through M4 modeling, and use of monolingual data. The quality improvements, which averaged +5 BLEU score over all 100+ languages, are visualized below. ... " 

 And see also, on issues with translation, from the ACM:

Automatic Translators are Not Really Capable of Learning
By Herbert Bruderer    ... "

Google Creates Transcribe

Had seen and noted this before, likely quite helpful for business applications.  I remember noting its need to be embedded in the enterprise Looking forward to see it on Assistant and  IOS Devices.

Google Translate launches Transcribe for Android in 8 languages   By Khari Johnson

Google Translate today launched Transcribe for Android, a feature that delivers a continual, real-time translation of a conversation. Transcribe will begin by rolling out support for 8 languages in the coming days: English, French, German, Hindi, Portuguese, Russian, Spanish and Thai. With Transcribe, Translate is now capable of translating classroom or conference lectures with no time limits, whereas before speech-to-text AI in Translate lasted no longer than a word, phrase, or sentence. Google plans to bring Transcribe to iOS devices at an unspecified date in the future. ..... " 

Longer Form Translation for Assistants

Have mentioned some experimentation with real time longer form translation, as it can exist within the framework of voice driven assistants.  Impressive so far.

Google Assistant can now read or translate websites and Android app content   By Khari Johnson

Google Assistant is able to read web pages and news articles on Android devices worldwide today, a company spokesperson told VentureBeat. You can trigger the feature by simply saying “Hey Google, read it” or “Hey Google, read this page.”

If you land on a webpage in a language you don’t understand, Google Assistant is also able to read and translate 42 languages into your preferred language. A full list of supported languages can be seen in the video above.  ... "

'Autonomous' Translation at Hand?

A number of mobile devices how have  instantaneous translation, so we have the promise of using them in any multilingual environment.   I have a Google assistant that will translate phrases.    How is this done?  Are we there yet, what more do we need?  Good review of the current tech.s.  How is the barrier been removed?

Across the Language Barrier
By Keith Kirkpatrick
Communications of the ACM, March 2020, Vol. 63 No. 3, Pages 15-17
10.1145/3379495

The greatest obstacle to international understanding is the barrier of language," wrote British scholar and author Christopher Dawson in November 1957, believing that relying on live, human translators to accurately capture and reflect a speaker's meaning, inflection, and emotion was too great of a challenge to overcome. More than 60 years later, Dawson's theory may finally be proven outdated, thanks to the development of powerful, portable real-time translation devices.

The convergence of natural language processing technology, machine learning algorithms, and powerful portable chipsets has led to the development of new devices and applications that allow real-time, two-way translation of speech and text. Language translation devices are capable of listening to an audio source in one language, translating what is being said into another language, and then translating a response back into the original language.

About the size of a small smartphone, most standalone translation devices are equipped with a microphone (or an array of microphones) to capture speakers' voices, a speaker or set of speakers to allow the device to "speak" a translation, and a screen to display text translations. Typically, audio data is captured by the microphones, processed using a natural language processing engine mated to an online language database located either in the cloud or on the device itself, and then the translation is output to the speakers or the screen. Standalone devices, with their dedicated translation engines and small portable form factors, are generally viewed as being more powerful and convenient than accessing a smartphone translation application. Further, many of these devices offer the ability to access translation databases stored locally on the device or access them in the cloud, allowing their use in areas with limited wireless connectivity.

Instead of trying to translate speech using complex rules based on syntax, grammar, and semantics, these language processing algorithms employ machine learning and statistical modeling. These initial models are trained on huge databases of parallel texts, or documents that are translated into several different languages, such as speeches to the United Nations, famous works of literature, or even multinational marketing and sales materials. The algorithms identify matching phrases across sources and measure how often and where words occur in a given phrase in both languages, which allows translators to account for differences in syntax and structure across languages. This data is then used to construct statistical models that link phrases in one language to phrases in the second, which allows for accurate and fast translation.

In practice, this means devices can translate between languages more quickly than ever before by using such modeling. Incorporating high-powered processors, quality microphones, and speakers into the device, a person can carry on a real-time, two-way conversation with someone who speaks an entirely different language. These devices represent a significant increase in accuracy and functionality above manual, text-based translation applications such as Google Translate. ... "

Value of Using Google Assistance for Translation

What we always wanted, a seamless translation mode. That made us completely indifferent to the language in use.  Hearing and speaking.   Its not quite here yet,  I have tried it and its less than seamless or complete.   But its clearly moving in the right direction.   Its the ultimate assistance need though, so why not?   Will be taking a closer look in the new year.

Google Assistant Can Now Translate Speech Through Your Phone
Interpreter Mode comes to iOS and Android, making it easier to converse somewhat seamlessly across language barriers.Google Assistant Can Now Translate Speech Through Your Phone
Interpreter Mode comes to iOS and Android, making it easier to converse somewhat seamlessly across language barriers.

When I hopped into a cab in Barcelona last year, my taxi driver began asking me questions—you know, small talk. But his phrasing was awkward because English wasn't his native tongue. Cue a kludgy back and forth, with both of us having a hard time really understanding one another. That's when I whipped out Google Translate, and his eyes lit up as my phone conveyed his questions better than I ever could. It wasn't perfect, and he had to repeat himself a few times, but what flowed out was a proper conversation—one that wouldn't have been so easily possible a decade ago.

Translation apps like Google Translate or Microsoft Translator are familiar to world travelers. Now, Google is making it possible to have spoken conversations across language barriers without needing to download an app at all. The company has updated Google Assistant so that it supports translating languages in real time.   ... "

Assistant Translation Services by Baidu

One business use of assistants is language translation.    While translation today is very good, its still not contextually perfect.  But then neither are people.  So whats the risk of misunderstandings?  Baidu just announced such a service.   Can an element of risk transparency be provided?

Baidu announces simultaneous translation service for business users   By Mike Wheatley in Siliconangle.

Baidu Inc. today announced the availability in beta test of a new “speech-to-speech simultaneous translation service” designed to facilitate communication at business events.

The Chinese technology giant said the service can provide highly accurate, real-time translation of speaker presentations at events in both audio and text.

The way it works is interesting, too, as the translations are delivered directly to people’s personal devices. Users simply download the Baidu translation app, scan a QR code for the specific conference they’re attending, and the translations will be streamed directly to their device.  .... " 

Google Translate Does not Understand Context

There has been lots of talk about the perfection of machine translation.  It has advanced considerably since we looked at it for corporate tasks in the 80s.  In particular now I see examples of where it is touted for real-time, unchecked interactions.   This ACM article looks at a considerable test, where the results vary from 'very good to useless'.  Translation does often depend, like all AI,  strongly upon context.  Risk of outcome should drive particular usage.  Quality control essential.

Google Translate Does Not Understand the Content of the Texts
By Herbert Bruderer 

" .... The examples show that the quality of machine translations vary between fairly good and useless. It depends, among other things, on the language pair, the subject area and the available data set and its quality. In Wikipedia, there are also large differences between the different language versions.

Further tests have shown that automatic translations are often inconsistent and sometimes even nonsensical. Sometimes even words are missing. The decisions are sometimes difficult to understand. In many cases the results are "good enough" for some applications. For non-native translators, the programs can be a valuable help. For language pairs with very large data sets, machine translation may achieve the quality of a mediocre human translator. The raw translations usually have to be edited manually. In the case of poor raw translations, the effort required for an improvement can be higher than for a manual retranslation ....  "

Interpreter Mode on Google Home

Have now been using bilingual mode on a Google Home for some time.  It works well, even picks up many expressions on the edge of slang.    Now I see in the following article that you can turn on a translator mode, whether you are on audio only or visual Google devices.   Makes it easier to do translation in batch or real time.  Over 25 languages work now.  Note its early testing in hospitality settings with real time interactions with guests.  Like it, continuing to experiment.   Implications for language study. 

The following article outlines how its done, nice overview:

How to use Interpreter Mode on Google Home devices in DigitalTrends.

Language barrier? Psh. Here's how to make your Google Home an ace translator

There are more than 7,000 languages spoken around the world. In the U.S., Census Data indicates that roughly one in five people speak a language other than English in their homes.

Aware of the need for people to be able to communicate with others who might speak a different language, Google has come up with a new feature on Google Home devices and smart displays — Interpreter Mode.  The search giant piloted Interpreter Mode at big hotels like Caesar’s Palace in Las Vegas, Dream Downtown in New York City, and Hyatt Regency in San Francisco. The hotel staff use Interpreter Mode to converse with guests in real-time who speak a different language than their own.

Now, you can use Interpreter Mode at home on a Google Home smart speaker (like the Home or Home Mini), smart display (like the Google Home Hub), and on some speakers with Google Home built-in. You can use the mode to translate conversations in real-time, help you learn a new language, or even to help you out if you’re studying a second language.

On audio-only speakers, like the Google Home and Google Home Mini , you’ll get an audio-only translation.  ... "

Machine Translation vs Human

Been reexamining he process of language translation, especially as it relates to real time assistant interaction.  This points out that full document translations still are better using humans, which implies document-contextual elements. I saw this in starting my own look at assistant translations,  the context introduces a clearer focus   This could be provided by using intelligent followup questions.

Human translators are still on top—for now

Machine translation works well for sentences but turns out to falter at the document level, computational linguists have found.  by Emerging Technology from the arXiv  September 5, 2018

You may have missed the popping of champagne corks and the shower of ticker tape, but in recent months computational linguists have begun to claim that neural machine translation now matches the performance of human translators.

The technique of using a neural network to translate text from one language into another has improved by leaps and bounds in recent years, thanks to the ongoing breakthroughs in machine learning and artificial intelligence. So it is not really a surprise that machines have approached the performance of humans. Indeed, computational linguists have good evidence to back up this claim.

But today, Samuel Laubli at the University of Zurich and a couple of colleagues say the champagne should go back on ice. They do not dispute their colleagues’ results but say the testing protocol fails to take account of the way humans read entire documents. When this is assessed, machines lag significantly behind humans, they say. .... " 

Baby Translator Predicting Autism

Another example of pattern recognition ...

Can this AI Powered baby Translator Help  Diagnose Autism?   By Megan Moltini   in Wired 

When Ariana Anderson   had her first child, she was as clueless as any new parent about how to interpret her infant’s cries. Every wail, every sob sounded an urgent alarm to her postpartum brain. But by the time Anderson’s third kid came along, the UCLA computational neuropsychologist realized she had become fluent in baby. Her ear had learned which sounds meant “feed me!” which ones were “change me!” and which ones signaled something more serious: pain. Anderson wondered if she could train an algorithm to do the same thing.

Five years, thousands of howls, and more than 1,700 babies later, Anderson’s AI-powered translator is here. Called Chatterbaby, the free app analyzes changes in frequency and patterns in the sound to silence ratio to tell parents why their kiddo is crying. For now, the app’s got a pretty limited vocabulary—it can pick out hungry from fussy from pained. But the more people use it, the more it will be able to say about what’s going on in the infant brain.

You see, Chatterbaby isn’t just an app designed to help parents, be they deaf, or new, or just overwhelmed and underslept. It’s also a massive data-collecting tool to see if irregularities in cry patterns could carry signals about autism—and perhaps one day, diagnose it.  .... "

Microsoft Does Offline Translations

Looks impressive based on a demo I saw.   We are closer yet to a universal, mobile and free Babel Fish.  Was never expecting such a combination of features, the very basic idea seemed like fantasy not too long ago.  21 supported languages.  Offline packs for most popular languages.   Camera image text translation.   What other AI will be be carrying around soon?

Microsoft's AI-powered offline translation now runs on any phone
Translator should be more accurate when you're traveling abroad.

Jon Fingas, @jonfingas  in Engadget

Fast and Automatic Translation

We spent lots of time using translation services to understand how consumers interacted with product.  Now we re not far away from automating the process.  My desktop assistants do it already.     Some aspects, like Intent analysis are already being installed in natural language analysis.

Soon Talking to Strangers will be even easier.    
 by David Pierce in Wired.When translation happens quickly and accurately, we’ll be able to experience places in an entirely new way.  ... " 

Translating Predicting Language of Chemical Reactions

Intriguing, trying to convert this idea into my firmer experience in an internship in chemical engineering research.

A Translation Algorithm Can Predict the “Language” of a Chemical Reaction  in Technology Review.

By thinking of organic chemistry as words and sentences instead of atoms and molecules, researchers have found a way for artificial intelligence to predict chemical reactions.

In a paper published on arXiv by researchers at IBM and being presented at this week’s Neural Information Processing Systems (NIPS) conference, the researchers demonstrate that by treating reaction predictions as a translation problem, they could come up with the correct reaction more often than was possible with previous models.

“Intuitively, there is an analogy between a chemist’s understanding of a compound and a language speaker’s understanding of a word,” the researchers write.  .... " 

Translation in Your Ear

Saw this demonstrated yesterday.  Very impressive.   Still not completely automatic, transparent, though that may not be far away.   For the omni traveler certainly,  but perhaps not yet for technically or contextually difficult translation.

Google's Pixel Buds translation will change the world
Finally, a Babel Fish that doesn't feed on brainwave energy.    by Andrew Tarantola  ... "

Facebook Translates with Neural Nets

Another example  of how the power of this tech is expanding.  Some interesting details here.

Two versions of a Facebook post Facebook Translations Now Rely Entirely on Neural Networks in SiliconANGLE  by Eric David

" ... Facebook on Wednesday announced its translations are now wholly dependent on state-of-the-art machine-learning neural networks. A team of Facebook researchers says these networks manage more than 2,000 translation directions and 4.5 billion translations daily, generating more accurate translations than Facebook's previous system, which used phrase-based machine translation models. Neural machine translation gauges the complete content of a message together, which is more resource-intensive than phrase-based translation but typically results in a more fluent translation. Facebook also says neural machine translation can handle unknown or misspelled words with greater proficiency, as it can examine contextual clues to determine a word's intended meaning. Facebook thinks convolutional neural networks (CNN) can realize the same accuracy in translation as recurrent neural networks, but significantly faster. In May, the company announced that its CNN-based system was nine times faster than existing networks, and Facebook researchers note CNNs are a better fit for the newest machine-learning hardware. ... "