Google Says it Won't Scrape

Thinking of the full implications of this.

Google will continue to let sites opt out of showing in search results

They can legally scrape those sites now, but the tech titan promises that it won't.    By Mariella Moon, @mariella_moon in Engadget

Google swears it won't start scraping websites that previously opted out of web crawling just because it can now legally go back to its old ways. Back in 2012, the tech titan promised to change its practices in several areas to settle an antitrust investigation by the FTC. Those changes include removing AdWords restrictions that made it difficult for advertisers to launch multi-platform campaigns, as well as providing websites a mechanism to opt out of having their content crawled and displayed on search results. .... " 

full letter to the FTC