GLM-130B: An Open Bilingual Pre-Trained Language Model

 Linguists building and using AI language models.   Commented on here in Upenn's Language Log. 

GLM-130B: An Open Bilingual Pre-Trained Language Model

January 25, 2023 @ 9:10 am · Filed by Victor Mair under Artificial intelligence, Computational linguistics

Description of a General Language Model (GLM; also GLaM) project based at Tsinghua University in Beijing, but with users and collaborators around the world.

Homepage (August 4, 2022)

This prospectus is difficult for outsiders to understand because of the large number of unexplained acronyms, abbreviations, initialisms, etc. and other such participants' terminology.

GLM-130B is an open bilingual (English & Chinese) bidirectional dense model with 130 billion parameters, pre-trained using the General Language Model (GLM) algorithm1. It is designed to support inference tasks with the 130B parameters on a single A100 (40G * 8) or V100 (32G * 8) server. As of July 3rd, 2022, GLM-130B has been trained on over 400 billion text tokens (200B each for Chinese and English) and exhibits the following unique features:

Bilingual: supports both English and Chinese.

Performance (EN): better than GPT-3 175B (+5.0%), OPT-175B (+6.5%), and BLOOM-176B (+13.0%) on LAMBADA and slightly better than GPT-3 175B (+0.9%) on MMLU.  .. '   (much more) 

Chinese Drones in Restricted Spaces

Of course there are now many drones in that area, many Chinese built. But still a valid concern. 

 In American Military News: 

Hundreds of Chinese-made drones have flown into restricted airspaces over Washington, D.C., in recent months and officials are playing catch-up to stop the foreign-made tech from spying on those restricted areas. While China may not be controlling these drones directly, any Chinese-made devices could still be covertly sending data back to China.

Sources told POLITICO they don’t believe the Chinese government is directing the drones, which are made by China-based DJI. But officials are beginning to see risks in U.S. consumers buying up Chinese tech that can enter one of the world’s most secure airspaces and potentially become another government’s surveillance system, or worse.

“The reality is, people on the tech side always said, ‘Look, at any point in time the Chinese can take control of a DJI that’s flying in the air,’” an anonymous government contractor told POLITICO.

Commercial drones, including those by DJI, use “geofencing” to prohibit drones from entering into restricted airspace such as those over D.C. But a government contractor told POLITICO there are “YouTube videos that could walk your grandparents through” how to bypass those constraints and allow users to fly their drones wherever they want.  .... '