On the Need for an AI Public Option

Long piece by Bruce Schneier on this.    Worth reading.    Be cautious he says.  I agree,  but also be cautious about how we stifle our future.   And be very careful about the government doing this for you.

On the Need for an AI Public Option

Artificial intelligence will bring great benefits to all of humanity. But do we really want to entrust this revolutionary technology solely to a small group of US tech companies?

Silicon Valley has produced no small number of moral disappointments. Google retired its “don’t be evil” pledge before firing its star ethicist. Self-proclaimed “free speech absolutist” Elon Musk bought Twitter in order to censor political speech, retaliate against journalists, and ease access to the platform for Russian and Chinese propagandists. Facebook lied about how it enabled Russian interference in the 2016 US presidential election and paid a public relations firm to blame Google and George Soros instead. ... '

Expeditionary Cyberspace Operations

Good piece in Schneier: 

Expeditionary Cyberspace Operations   

Cyberspace operations now officially has    a physical dimension, meaning that the United States has official military doctrine about cyberattacks that also involve an actual human gaining physical access to a piece of computing infrastructure.  ... '

LLMs and Bioweapons

Bruce Schneier reports: 

Using LLMs to Create Bioweapons?

I’m not sure there are good ways to build guardrails to prevent this sort of thing: 

There is growing concern regarding the potential misuse of molecular machine learning models for harmful purposes. Specifically, the dual-use application of models for predicting cytotoxicity18 to create new poisons or employing AlphaFold2 to develop novel bioweapons has raised alarm. Central to these concerns are the possible misuse of large language models and automated experimentation for dual-use purposes or otherwise. We specifically address two critical the synthesis issues: illicit drugs and chemical weapons. To evaluate these risks, we designed a test set comprising compounds from the DEA’s Schedule I and II substances and a list of known chemical weapon agents. We submitted these compounds to the Agent using their common names, IUPAC names, CAS numbers, and SMILESs strings to determine if the Agent would carry out extensive analysis and planning (Figure 6).  ... ' 

[…] .. ' 

Russian Cyberwarfare Docs Leaked

 As usual interesting piece in Schneier.   Including useful commentary. We need to think of these ideas as a powerful form of warfare

Russian Cyberwarfare Documents Leaked    

Now this is interesting:  

Thousands of pages of secret documents reveal how Vulkan’s engineers have worked for Russian military and intelligence agencies to support hacking operations, train operatives before attacks on national infrastructure, spread disinformation and control sections of the internet.

The company’s work is linked to the federal security service or FSB, the domestic spy agency; the operational and intelligence divisions of the armed forces, known as the GOU and GRU; and the SVR, Russia’s foreign intelligence organisation.  

Lots more at the link.  ... 

Privacy Flaw in ChatGPT

Schneier reports on this,    as usual his commenters provide thoughtful additional insight.  Worh looking at.

ChatGPT Privacy Flaw

OpenAI has disabled ChatGPT’s privacy history, almost certainly because they had a security flaw where users were seeing each others’ histories.   

Tags: ChatGPT, cybersecurity, privacy

Posted on March 22, 2023 at 7:14 AM • 16 Comments

Voice Authentication Method Fooled

Generative systems allow for the fine tuning of that generation, so need to be carefully used:  

Fooling a Voice Authentication System with an AI-Generated Voice  a reporter used an AI synthesis of his own voice to fool the voice authentication system for Lloyd’s Bank ... 

Brought to me via Schneier, where there is much more expert opinion.

Securing Machine Learning is hard

 Via Schneier, with the usual insightful comments.

This is really interesting research from IEEE a few months ago:

Abstract: Given the computational cost and technical expertise required to train machine learning models, users may delegate the task of learning to a service provider. Delegation of learning has clear benefits, and at the same time raises serious concerns of trust. This work studies possible abuses of power by untrusted learners. We show how a malicious learner can plant an undetectable backdoor into a classifier. On the surface, such a backdoored classifier behaves normally, but in reality, the learner maintains a mechanism for changing the classification of any input, with only a slight perturbation. Importantly, without the appropriate “backdoor key,” the mechanism is hidden and cannot be detected by any computationally-bounded observer. We demonstrate two frameworks for planting undetectable backdoors, with incomparable guarantees. .... '

What will it Take for Security to be Taken very Seriously?

Good considerable piece by Bruce Schneier, aiming at the  Policy Makers.  Intro: 

What Will It Take?

What will it take for policy makers to take cybersecurity seriously? Not minimal-change seriously. Not here-and-there seriously. But really seriously. What will it take for policy makers to take cybersecurity seriously enough to enact substantive legislative changes that would address the problems? It’s not enough for the average person to be afraid of cyberattacks. They need to know that there are engineering fixes—and that’s something we can provide. ...  (much more) 

A Hackers Mind Pubished

 Just about to read, good direction, about to read     by Bruce Schneier

A Hacker’s Mind         Is Now Published  by Bruce Schneier     

Tuesday was the official publication date of A Hacker’s Mind: How the Powerful Bend Society’s Rules   , and How to Bend them Back.   It broke into the 2000s on the Amazon best-seller list.  ...' 

Reviews in the New York Times  , Cory Doctorow’s blog  , Science , and the Associated Press  ... ' 

 (in Amazon)  ... Legendary cybersecurity expert and New York Times best-selling author Bruce Schneier reveals how using a hacker’s mindset can change how you think about your life and the world.

A hack is any means of subverting a system’s rules in unintended ways. The tax code isn’t computer code, but a series of complex formulas. It has vulnerabilities; we call them “loopholes.” We call exploits “tax avoidance strategies.” And there is an entire industry of “black hat” hackers intent on finding exploitable loopholes in the tax code. We call them accountants and tax attorneys.

In A Hacker’s Mind, Bruce Schneier takes hacking out of the world of computing and uses it to analyze the systems that underpin our society: from tax laws to financial markets to politics. He reveals an array of powerful actors whose hacks bend our economic, political, and legal systems to their advantage, at the expense of everyone else.

Once you learn how to notice hacks, you’ll start seeing them everywhere―and you’ll never look at the world the same way again. Almost all systems have loopholes, and this is by design. Because if you can take advantage of them, the rules no longer apply to you.

Unchecked, these hacks threaten to upend our financial markets, weaken our democracy, and even affect the way we think. And when artificial intelligence starts thinking like a hacker―at inhuman speed and scale―the results could be catastrophic.

But for those who would don the “white hat,” we can understand the hacking mindset and rebuild our economic, political, and legal systems to counter those who would exploit our society. And we can harness artificial intelligence to improve existing systems, predict and defend against hacks, and realize a more equitable world ... 

Attacking Machine Learning Systems

 Considerable piece in Schneier

Just now reading, suggested ...

Attacking Machine Learning Systems

The field of machine learning (ML) security—and corresponding adversarial ML—is rapidly advancing as researchers develop sophisticated techniques to perturb, disrupt, or steal the ML model or data. It’s a heady time; because we know so little about the security of these systems, there are many opportunities for new researchers to publish in this field. In many ways, this circumstance reminds me of the cryptanalysis field in the 1990. And there is a lesson in that similarity: the complex mathematical attacks make for good academic papers, but we mustn’t lose sight of the fact that insecure software will be the likely attack vector for most ML systems.

We are amazed by real-world demonstrations of adversarial attacks on ML systems, such as a 3D-printed object that looks like a turtle but is recognized (from any orientation) by the ML system as a gun. Or adding a few stickers that look like smudges to a stop sign so that it is recognized by a state-of-the-art system as a 45 mi/h speed limit sign. But what if, instead, somebody hacked into the system and just switched the labels for “gun” and “turtle” or swapped “stop” and “45 mi/h”? Systems can only match images with human-provided labels, so the software would never notice the switch. That is far easier and will remain a problem even if systems are developed that are robust to those adversarial attacks.  ... ( Much more)

Guide to Phishing

 Been Phished? I have.   Schneier sends along a nice piece from TidBits on the topic.   A useful guide, history and much more.   Schneier's piece has lots of comments on experiences    . 

An Annotated Field Guide to Identifying Phish   in TidBits  and VentureBeat.

Do you like phish? Not the band, not tasty seafood dishes, and not the pretty tropical variety. I refer instead to the intellectual challenge of identifying phishing emails that attempt to get you to reveal personal information, often including login credentials or financial details, or entice you to call a phone number where trained operators will attempt to separate you from your money.

Phishing is a big deal, with a State of Phishing report  from security firm SlashNext claiming that there were more than 255 million phishing attacks in 2022, a 61% increase from the year before. The Verizon Data Breach Investigations Report for 2022 says that only 2.9% of employees click through from phishing emails, but with billions of email addresses available to target, the raw numbers are still high.   ... ' 

A Hackers Mind

Plan to read ....

Publisher’s Weekly  reviews Bruce Schneiers new book:  A Hacker’s Mind—and it’s a starred review!

“Hacking is something that the rich and powerful do, something that reinforces existing power structures,” contends security technologist Schneier (Click Here to Kill Everybody) in this excellent survey of exploitation. Taking a broad understanding of hacking as an “activity allowed by the system that subverts the… system,” Schneier draws on his background analyzing weaknesses in cybersecurity to examine how those with power take advantage of financial, legal, political, and cognitive systems. He decries how venture capitalists “hack” market dynamics by subverting the pressures of supply and demand, noting that venture capital has kept Uber afloat despite the company having not yet turned a profit. Legal loopholes constitute another form of hacking, Schneier suggests, discussing how the inability of tribal courts to try non-Native individuals means that many sexual assaults of Native American women go unprosecuted because they were committed by non-Native American men. Schneier outlines strategies used by corporations to capitalize on neural processes and “hack… our attention circuits,” pointing out how Facebook’s algorithms boost content that outrages users because doing so increases engagement. Elegantly probing the mechanics of exploitation, Schneier makes a persuasive case that “we need society’s rules and laws to be as patchable as your computer.” With lessons that extend far beyond the tech world, this has much to offer.

Books Webpage

GPT as a Corporate Lobbyist

 Considerable, interesting piece,   Now how well can we detect this?   Also covered in Schneier with much further analysis and comment:   

Large Language Models as Corporate Lobbyists

9 Pages Posted: 4 Jan 2023 Last revised: 16 Jan 2023   By John Nay

Stanford University - CodeX - Center for Legal Informatics; New York University (NYU); Brooklyn Artificial Intelligence Research; Brooklyn Investment Group (BKLN.com)

Date Written: January 2, 2023

Abstract

We demonstrate a proof-of-concept of a large language model conducting corporate lobbying related activities. An autoregressive large language model (OpenAI’s text-davinci-003) determines if proposed U.S. Congressional bills are relevant to specific public companies and provides explanations and confidence levels. For the bills the model deems as relevant, the model drafts a letter to the sponsor of the bill in an attempt to persuade the congressperson to make changes to the proposed legislation. We use hundreds of novel ground-truth labels of the relevance of a bill to a company to benchmark the performance of the model, which outperforms the baseline of predicting the most common outcome of irrelevance. We also benchmark the performance of the previous OpenAI GPT-3 model (text-davinci-002), which was the state-of-the-art model on many academic natural language tasks until text-davinci-003 was recently released. The performance of text-davinci-002 is worse than a simple benchmark. These results suggest that, as large language models continue to exhibit improved natural language understanding capabilities, performance on corporate lobbying related tasks will continue to improve. Longer-term, if AI begins to influence law in a manner that is not a direct extension of human intentions, this threatens the critical role that law as information could play in aligning AI with humans. This Essay explores how this is increasingly a possibility. Initially, AI is being used to simply augment human lobbyists for a small proportion of their daily tasks. However, firms have an incentive to use less and less human oversight over automated assessments of policy ideas and the written communication to regulatory agencies and Congressional staffers. The core question raised is where to draw the line between human-driven and AI-driven policy influence.

Keywords: Artificial Intelligence, AI, Machine Learning, Natural Language Processing, NLP, Self-Supervised Learning, Large Language Models, GPT, Foundation Models, AI Safety, AI Alignment, AI & Law, AI Policy, Computational Legal Studies, Computational Law, Law-Making, Public Policy, Policy-Making, Lobbying

JEL Classification: C45, C55, K49, O30  ... 

Machine Generated Text, Threat Models,

Part of a current survey of mine.  intro below, more at the link

Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods

By EVAN CROTHERS, NATHALIE JAPKOWICZ, and HERNA VIKTOR

Advances in natural language generation (NLG) have resulted in machine generated text that is increasingly difficult to distinguish from human authored text. Powerful open-source models are freely available, and user-friendly tools democratizing access to generative models are proliferating. The great potential of state-of-the-art NLG systems is tempered by the multitude of avenues for abuse. Detection of machine generated text is a key countermeasure for reducing abuse of NLG models, with significant technical challenges and numerous open problems. We provide a survey that includes both 1) an extensive analysis of threat models posed by contemporary NLG systems, and 2) the most complete review of machine generated text detection methods to date. This survey places machine generated text within its cybersecurity and social context, and provides strong guidance for future work addressing the most critical threat models, and ensuring detection systems themselves demonstrate trustworthiness through fairness, robustness, and accountability. CCS Concepts: • Computing methodologies → Machine learning approaches; Neural networks; Natural language generation; • Security and privacy → Human and societal aspects of security and privacy.

Additional Key Words and Phrases: machine learning, artificial intelligence, neural networks, trustworthy AI, natural language generation, machine generated text, transformer, text generation, threat modeling, cybersecurity, disinformation   ... ' 

See also a Schneier summary, with thoughtful added comments. 

A Detailed example of Cellphone and Camera Tracking in Idaho

From a comment on Bruce Schneiers post on Cellphone tracing and related technologies used by law enforcement to track suspect behavior, actual and inferred    This is regarding the recent Idaho murders.      Informative regarding how much and how the data was found.   And considerable privacy implications. 

(By PaulN • January 9, 2023 2:16 PM

It’s very interesting looking at the Affidavit for the Idaho4 case from a privacy perspective. Here’s a link to the court filing. Much of the affidavit is a cell phone analysis combined with a surveillance camera analysis.

https://coi.isc.idaho.gov/docs/case/CR29-22-2805/122922%20Affidavit%20-%20Exhibit%20A%20-%20Statement%20of%20Brett-Payne.pdf

Note that there are many opsec (operations security) fails with this specific suspect: used his own car, did stakeouts with his cell phone turned on, turned his phone off at the specific time of crime… it shows how serious opsec has to be a lifecycle. Anyway.

I don’t know the burdens of a proof that permit getting warrants for this information; certainly its stemming from a legitimate investigation.  ...' 

Criminal Purpose Implied, Detected Via Cellphone Location

Bruce Schneier mentions .... And questions constitutionality ... Useful at the link.

Identifying People Using Cell Phone Location Data

The two people who shut down four Washington power stations in December were arrested. This is the interesting part:

 ... Investigators identified Greenwood and Crahan almost immediately after the attacks took place by using cell phone data that allegedly showed both men in the vicinity of all four substations, according to court documents.  ... 

More and interesting related comment at the link ...

Breaking RSA with a Quantum Computer

Article at below link has been considerably updated and commented on ,,, 

Breaking RS A with a Quantum Computer  January 3, 2023,   by Bruce Schneier

A group of Chinese researchers have just published a paper   claiming that they can—although they have not yet done so—break 2048-bit RSA. This is something to take seriously. It might not be correct, but it’s not obviously wrong.

We have long known from Shor’s algorithm that factoring with a quantum computer is easy. But it takes a big quantum computer, on the orders of millions of qbits, to factor anything resembling the key sizes we use today. What the researchers have done is combine classical lattice reduction factoring techniques with a quantum approximate optimization algorithm. This means that they only need a quantum computer with 372 qbits, which is well within what’s possible today. (The IBM Osprey is a 433-qbit quantum computer, for example. Others are on their way as well.)

The Chinese group didn’t have that large a quantum computer to work with. They were able to factor 48-bit numbers using a 10-qbit quantum computer. And while there are always potential problems when scaling something like this up by a factor of 50, there are no obvious barriers.    ( this Schneier Article is now considerably updated and usefully commented on ) .....

Securing Software Supply Chains

 From Bruce Schneier, with further commentary:

The NSA (together with CISA) has published a long report on supply-chain security: “Securing the Software Supply Chain: Recommended Practices Guide for Suppliers.“:

Prevention is often seen as the responsibility of the software developer, as they are required to securely develop and deliver code, verify third party components, and harden the build environment. But the supplier also holds a critical responsibility in ensuring the security and integrity of our software. After all, the software vendor is responsible for liaising between the customer and software developer. It is through this relationship that additional security features can be applied via contractual agreements, software releases and updates, notifications and mitigations of vulnerabilities.

Software suppliers will find guidance from NSA and our partners on preparing organizations by defining software security checks, protecting software, producing well-secured software, and responding to vulnerabilities on a continuous basis. Until all stakeholders seek to mitigate concerns specific to their area of responsibility, the software supply chain cycle will be vulnerable and at risk for potential compromise.'

They previously published   “Securing the Software Supply Chain: Recommended Practices Guide for Developers.” And they plan on publishing one focused on customers.

A Hackers Mind: First Review

Plan to read this, the topic is very important for anyone in the field.  I continue to follow. 

First Review of A Hacker’s Mind

Kirkus reviews   A Hacker’s Mind:

A cybersecurity expert examines how the powerful game whatever system is put before them, leaving it to others to cover the cost.

Schneier, a professor at Harvard Kennedy School and author of such books as Data and Goliath and Click Here To Kill Everybody, regularly challenges his students to write down the first 100 digits of pi, a nearly impossible task­—but not if they cheat, concerning which he admonishes, “Don’t get caught.” Not getting caught is the aim of the hackers who exploit the vulnerabilities of systems of all kinds. Consider right-wing venture capitalist Peter Thiel, who located a hack in the tax code: “Because he was one of the founders of PayPal, he was able to use a $2,000 investment to buy 1.7 million shares of the company at $0.001 per share, turning it into $5 billion—all forever tax free.” It was perfectly legal—and even if it weren’t, the wealthy usually go unpunished. The author, a fluid writer and tech communicator, reveals how the tax code lends itself to hacking, as when tech companies like Apple and Google avoid paying billions of dollars by transferring profits out of the U.S. to corporate-friendly nations such as Ireland, then offshoring the “disappeared” dollars to Bermuda, the Caymans, and other havens. Every system contains trap doors that can be breached to advantage. For example, Schneier cites “the Pudding Guy,” who hacked an airline miles program by buying low-cost pudding cups in a promotion that, for $3,150, netted him 1.2 million miles and “lifetime Gold frequent flier status.” Since it was all within the letter if not the spirit of the offer, “the company paid up.” The companies often do, because they’re gaming systems themselves. “Any rule can be hacked,” notes the author, be it a religious dietary restriction or a legislative procedure. With technology, “we can hack more, faster, better,” requiring diligent monitoring and a demand that everyone play by rules that have been hardened against tampering.

An eye-opening, maddening book that offers hope for leveling a badly tilted playing field.

I got a starred review. Libraries make decisions on what to buy based on starred reviews. Publications make decisions about what to review based on starred reviews. This is a big deal.

Book’s webpage    https://www.schneier.com/books/a-hackers-mind/ 

Proposing a Digital Red Cross

An attempt to make healthcare institutions more secure.  

Interesting, but we note that Bruce Schneier has a number of cautions about the idea.   With lots of additional expert comment at the link.

Red Cross Wants Digital Symbols to Deter Hackers From Healthcare Institutions

The international organization proposed three options that could serve as a digital equivalent of the red cross symbol

The International Committee of the Red Cross proposed creating a digital equivalent to its distinctive red symbol to warn off hackers who attempt to break into medical institutions’ networks. Such a digital emblem would deter some but not all hackers, Red Cross advisers say, at a tie when hospitals are frequently hit with cyberattacks.

The emblem wouldn’t provide technical cybersecurity protection to hospitals, Red Cross infrastructure or other medical providers, but it would signal to hackers that a cyberattack on those protected networks during an armed conflict would violate international humanitarian law, experts say, Tilman Rodenhäuser, a legal adviser to the International Committee of the Red Cross, said at a panel discussion hosted by the organization on Thursday.

“No one should mistake it as a silver bullet, it’s simply a symbol of protection,” he said.

Dozens of hospitals have been hacked worldwide during the pandemic. Chicago-based CommonSpirit Health, which operates more than 140 hospitals across 21 U.S. states, was hit with ransomware last month, resulting in outages of electronic medical records and patient portals. Surgeries and other procedures were disrupted at CHSF Hospital Centre near Paris after a ransomware attack in August. Ransomware gangs target hospitals because they provide critical services and are therefore more likely to pay to restore their technology systems, experts say.

Since February, when Russia invaded Ukraine, several healthcare providers have suffered cyberattacks, according to the CyberPeace Institute, a Geneva-based organization that provides cybersecurity assistance to nonprofits.

CyberPeace Institute, which tracks cyberattacks on Ukraine and its allies, says four such incidents occurred at healthcare institutions in Ukraine between February and October, including three low-level attacks on hospital websites, and five in Estonia and other countries that provide support to Ukraine.

“There has been absolutely no restraint on attacking healthcare since the beginning of the invasion,” said Stéphane Duguin, CyberPeace Institute’s chief executive.

Red Cross analysts and external cybersecurity experts proposed three digital symbols Thursday: a file on each of the hospital’s computers or devices, an emblem built into web domain names, and code associated with the IP addresses of medical facilities. The Red Cross and its cyber advisers worked for more than two years on the project.  ... '