Watson Discovery

We took an early look at Watson Discovery, but it did not meet our needs.   It has been updated in how it can be used.  The below article shows how it can be used with data scraping to ingest online knowledge.   The below article is instructive, you can a free IBM Cloud subscription for testing.

Scraping and Querying Hacker News with IBM’s Watson Discovery
By Brandon Walker in TowardsDataScience

Watson was originally an AI system designed by IBM to win Jeopardy! the tv show. To be successful, Watson had to query a massive knowledge base to answer the questions (or question the answers) on the show. This knowledge base included DBPedia, WordNet and Yago. After the show was over, Watson was spun off into multiple products including Watson Discovery, Watson Text to Speech, Watson Speech to Text, Watson Assistant, and Watson Knowledge Studio, among others. Watson Discovery can be used to query documents to find relevant passages/articles/paragraphs that answer a question you may have. 

In this article I will be showing you how to use Watson Discovery to get information out of Y Combinator’s Hacker News, which will help answer questions about topics covered on Hacker News, but you may desire to re-purpose it for other documents you want to query.
If you’re familiar with scraping, you probably realize that even when it isn’t difficult, it can be very time consuming. Watson Discovery can take care of this task for you, whether your documents are PDF, Word, PowerPoint, Excel, PNG, TIFF, JPG, JSON, or HTML. Discovery can save a lot time when it scrapes a website, since it will scrape that site’s main page, as well as its other pages and links to external websites. This makes applying it to Hacker News an easy task.  ... "