AWS Launches Amazon Neptune Serverless

Detecting and storing complex data relationships,

AWS Launches Amazon Neptune Serverless

By Jaime Hampton in Datanami

Amazon Web Services has launched a serverless option for its Neptune graph database service used for building and running applications with highly connected datasets.

Neptune’s graph database engine is optimized for storing billions of relationships and allows low latency querying of the graph with supported languages including Apache TinkerPop Gremlin, the W3C’s SPARQL, and Neo4j’s openCypher. Use cases include recommendation engines, fraud detection, and knowledge graphs, among others.

AWS touts Amazon Neptune Serverless as a good deployment option for customers with variable or unpredictable workloads where the volume and complexity of database queries can be spiky or intermittent, leading to challenges with capacity planning. Instead of constantly monitoring and reconfiguring capacity, AWS says Neptune Serverless automatically provisions and scales graph database workloads to hundreds of thousands of queries. It also supports multiple AWS Availability Zones for high availability, read replicas for high performance, and fully managed software patching, updates, and backups.

Amazon Neptune is a fully managed database service made for building and running graph applications, according to AWS. Source: AWS

“Customers tell us that they appreciate the ability to use Amazon Neptune to understand complex relationships among highly connected data points. They have also asked us to take care of the heavy lifting associated with managing capacity and optimizing for cost and performance,” said Swami Sivasubramanian, vice president of databases, analytics, and machine learning at AWS. “Now, with Amazon Neptune Serverless, customers have a graph database that automatically provisions and seamlessly scales clusters to provide just the right amount of capacity to meet demand, allowing them to build and run applications for even the most variable and unpredictable workloads without having to worry about provisioning capacity, scaling clusters, or incurring costs for unused resources.”

AWS is continuously expanding its serverless offerings. The company unveiled serverless versions of its hosted Apache Kafka, Kinesis, Elastic MapReduce (EMR), and Redshift offerings last year, and this year brought an updated serverless database, Amazon Aurora Serverless V2, and SageMaker Serverless Inference for AI workloads. ... '

FlavorGraph: Food Pairings with AI and Molecular Science

Combines a number of interests of mine.  Food science and AI, Chemistry and Graph Analytics.   At the link see some impressive graphs that look at the connections  Nicely done approach to loooking at a complex problem.  In our own food industry area, coffee blending, we looked at some aspects of this, but just barely.  Worth a look if you are in the area.

 (Update:  Hmmm ... just acted as a tester of new spice blends for McCormick. Might this act as a means of generating potential new blends for them? )

FlavorGraph Serves Up Food Pairings with AI, Molecular Science  By Isha Salian

Tags: Data Science, featured, Machine Learning & Artificial Intelligence, News

It’s not just gourmet chefs who can discover new flavor combinations— a new ingredient mapping tool by Sony AI and Korea University uses molecular science and recipe data to predict how two ingredients will pair together and suggest new mash-ups. 

Dubbed FlavorGraph, the graph embedding model was trained on a million recipes and chemical structure data from more than 1,500 flavor molecules. The researchers used PyTorch, CUDA and an NVIDIA TITAN GPU to train and test their large-scale food graph.

Researchers have previously used molecular science to explain classic flavor pairings such as garlic and ginger, cheese and tomato, or pork and apple — determining that ingredients with common dominant flavor molecules combine well. In the FlavorGraph database, flavor molecule information was grouped into profiles such as bitter, fruity, and sweet. 

But other ingredient pairings have different chemical makeups, prompting the team to incorporate recipes into the database as well, giving the model insight into ways flavors have been combined in the past. .... " 

Overview of Graph Databases

Overview of Graph DBs and their uses.  From the CACM.  Below the intro, more at the link. 

Understanding NoSQL Database Types: Graph Databases     By Alex Williams  in CACM

While originating as a subset of NoSQL or "Not Only SQL," graph databases represent a sharp closing of SQL and NoSQL demarcations. Graph technologies are exploding in its market size as more companies and developers take up their hybrid flexibility offerings. Those offerings: Intuitivity plus scalability with a high connection and robust data pattern.

While I won't go into depth on the formation of the 'SQL vs NoSQL' debate, you could quite accurately say that SQL represents data stored in rows and tables, while high-growth NoSQL is data stores arranged via nested documents as columnar schemas or key-value pairs. One is relational, the other not so much.

Graph databases are formed from nodes, properties, and relationships—all in a very interlinked data structure. And yet it supports advanced, rich querying with scalability. In this model, relationships matter just as much as the data itself. In a sense, it combines the querying power of relational databases with the intuitive flexibility of columnar non-relational databases—supporting agile development while also letting you gain deep insights.

Why use graph databases: The benefits

The graph model is a general-purpose data technology. While many know it for its social media implementations—this 'emerging shape', as it's known amongst data scientists due to being a non-typical dataset, has become most popular with social media companies for performing social network analyses, and for creating social graphs via companies like Facebook and Twitter who are particularly focused on the Six Degrees of Separation concept—graph databases are actually found in a large variety of industries, ranging from finance to healthcare, to emergency-response networks.

The principal benefit of graph databases is using its ability to assign values to links or connections. If your data has connections, whether for offline machine learning systems or online mobile applications, implementing this emerging shape will likely be beneficial.

In short: Build high-fidelity, highly interconnected networks made of bite-sized, scalable patterns (ie. great for CI/CD dev) that can together service, query, and manage sophisticated problem domains.  ... ' 

Considering the Graph Database

A useful challenge, in general graphic databases are more indicative of what they show.    But they also introduce levels of complexity that has to be understood.

Graph databases are proven architectures for storing data with complex relationships. Why aren't more companies using them?
   
 By Isaac Sacolick in InfoWorld

Twenty years ago, my development team built a natural language processing engine that scanned employment, auto, and real estate advertisements for searchable categories. I knew that we had a difficult data management challenge. The data in some ad types were relatively straightforward, like identifying car makes and models, but others required more inference, such as identifying a job category based on a list of skills.

We developed a metadata model that captured all the searchable terms, but the natural language processing engine required the model to expose significant metadata relationships. We knew designing a metadata model with arbitrary connections between data points in a relational database was complex, so we explored using object databases to manage the model.

What we were trying to accomplish back then with object databases can be done better today with graph databases. Graph databases store information as nodes and data specifying their relationships with other nodes. They are proven architectures for storing data with complex relationships.

Graph database usage has certainly grown during the past decade as companies considered other NoSQL and big data technologies. The global graph database market was estimated at $651 million in 2018 and forecasted to grow to $3.73 billion by 2026. But many other big data management technologies, including Hadoop, Spark, and others, have seen much more significant growth in popularity, skill adoption, and production use cases compared to graph databases. By comparison, the big data technology market size was estimated at $36.8 billion in 2018 and forecasted to grow to $104.3 billion by 2026.

I wanted to understand why more organizations aren’t considering graph databases. Developers think in objects and use hierarchical data representations in XML and JSON regularly. Technologists and business stakeholders intrinsically understand graphs since the Internet is an interconnected graph through hyperlinks and concepts like friends and friends of friends from social networks. Then why haven’t more development teams used graph databases in their applications?  .... 

Using a Graph Database

Reasonable argument,but needs more thought for particular context.

Developers should consider using graph databases | InfoWorld

Graph databases are proven architectures for storing data with complex relationships. Why aren't more companies using them?
     
By Isaac Sacolick
Contributing Editor, InfoWorld 

Twenty years ago, my development team built a natural language processing engine that scanned employment, auto, and real estate advertisements for searchable categories. I knew that we had a difficult data management challenge. The data in some ad types were relatively straightforward, like identifying car makes and models, but others required more inference, such as identifying a job category based on a list of skills.

We developed a metadata model that captured all the searchable terms, but the natural language processing engine required the model to expose significant metadata relationships. We knew designing a metadata model with arbitrary connections between data points in a relational database was complex, so we explored using object databases to manage the model.

What we were trying to accomplish back then with object databases can be done better today with graph databases. Graph databases store information as nodes and data specifying their relationships with other nodes. They are proven architectures for storing data with complex relationships.

Graph database usage has certainly grown during the past decade as companies considered other NoSQL and big data technologies. The global graph database market was estimated at $651 million in 2018 and forecasted to grow to $3.73 billion by 2026. But many other big data management technologies, including Hadoop, Spark, and others, have seen much more significant growth in popularity, skill adoption, and production use cases compared to graph databases. By comparison, the big data technology market size was estimated at $36.8 billion in 2018 and forecasted to grow to $104.3 billion by 2026.   ... " 

Webinar:Graphs for Cyber Security

Brought to my attention.  Addresses my particular favorite advice:  Carefully define your intent, direction and process when planning to do something important and complex. Visual is good.   Upcoming Webinar:

Webinar:  Graphs for Cyber Security
On Thursday, August 6, 2020 07:00 PT | 10:00 ET | 15:00 BT | 16:00 CET

REGISTER NOW 

Find out how graphs can support solving Cyber Security problems.

In the first part of this webinar, we will find out what makes graph databases so unique and powerful, and how we can use them to solve complex Cyber Security use cases, such as fake accounts, workloads hacks or application control.

In the second part of the webinar, we will demonstrate live how to build a suitable data model and which algorithms best to use to solve the respective problems. And we will provide tips and tricks to answer your questions.

Hope to see you there,
Sabine Seitz, Neo4j

What is a Graph Database? 

Harnessing Graphs in a New Business Climate

Of interest, upcoming:

Tuesday, July 14    8:00 a.m. PT | 11:00 a.m. ET

Harnessing Graphs in a New Business Climate

Hi Franz,
As economies continue to reopen across much of Europe and North America, many organizations turn to graph technology to help rapidly reconfigure and reset operations.

Graphs are perfectly suited for handling connected data, like tracing connections through complex networks. Graphs can also identify complex relationships faster than human-only efforts by combining multiple isolated datasets and identifying missing data points.

Join us for this panel discussion with key Neo4j partners who are using graph technology to help organizations balance efficiency and innovation in today’s challenging times.

Presenters

Dr. Alessandro Negro, Chief Scientist, GraphAware
Demian Bellumio, Global Vice President of Augmented Intelligence, NEORIS
Weidong Yang, CEO, Kineviz
Axel Morgner, Founder and Managing Director, Structr
Martin Preusse, Founder at Kaiser & Preusse
Lance Walter, CMO, Neo4j

REGISTER TODAY

Hope to see you there,     Lance Walter    Neo4j

Amazon Announces Graph Database

Amazon continues to bring out advanced tools for AWS, with useful examples of their practical use, at low trial and initial testing cost.  I particularly like the examples given here about how and why you would use graph databases.  Note the implications for embedding 'intelligence' in a database.

AWS Announces General Availability of Amazon Neptune
Amazon Neptune, a fast and reliable graph database, makes it easy for customers to build applications on highly connected datasets

Thousands of customers, including Samsung Electronics, Pearson, Intuit, Siemens, AstraZeneca, FINRA, LifeOmic, Blackfynn, and Amazon Alexa, participated in the preview, building new graph applications and battle-testing their production workloads

May 30, 2018 04:46 PM Eastern Daylight Time
SEATTLE--(BUSINESS WIRE)--Today, Amazon Web Services, Inc. (AWS), an Amazon.com company (NASDAQ: AMZN), announced general availability of Amazon Neptune, a fast, reliable, and fully managed graph database service. Amazon Neptune efficiently stores and navigates highly connected data, allowing developers to create sophisticated, interactive graph applications that can query billions of relationships with millisecond latency. In the preview, customers used Neptune to build social networks, recommendation engines, fraud detection, knowledge graphs, drug discovery applications, and more. 

With Amazon Neptune there are no upfront costs, licenses, or commitments required; customers pay only for the Neptune resources they use. To get started with Amazon Neptune, visit https://aws.amazon.com/neptune

Also in SiliconAngle.

Graph Processing

Silicon Angle mentions Neo4j and what they are calling graph processing.  Good to understand.   Will be looking  at the reported update.  See their introductory piece.  Most interested in how this improves the use of data by analytics.

What you missed in Big Data: graph processing and machine learning   by Maria Deutscher

Most traditional data management products aren’t equipped to handle the increasingly complex and diverse information that is flowing into the corporate network these days. As a result, organizations are turning to new solutions like Neo4j. The widely used graph store, which sets itself apart by providing the ability to easily log the relationships among records, received a major update last week that promises to streamline large-scale analytics initiatives. ... " 

See also this piece by Jim Webber on the universality of the graph space.   Nicely done.