/* ---- Google Analytics Code Below */

Tuesday, August 04, 2020

Need for Data Observability

Never called it that in particular, but yes it is important, and is worth the included tag.  And I add, the changes and non-changes in data should also be 'observable'.

What is Data Observability?
Hint: it’s not just data for DevOps.  By Barr Moses in Medium

Observability is no longer just for software engineering. With the rise of data downtime and the increasing complexity of the data stack, observability has emerged as a critical concern for data teams, too.

Developer Operations (lovingly referred to as DevOps) teams have become an integral component of most engineering organizations. DevOps teams remove silos between software developers and IT, facilitating the seamless and reliable release of software to production.

As organizations grow and the underlying tech stacks powering them become more complicated (think: moving from a monolith to a microservice architecture), it’s important for DevOps teams to maintain a constant pulse on the health of their systems. Observability, a more recent addition to the engineering lexicon, speaks to this need, and refers to the monitoring, tracking, and triaging of incidents to prevent downtime.

As a result of this industry-wide shift to distributed systems, observability engineering has emerged as a fast-growing engineering discipline. At its core, observability engineering is broken into three major pillars:

Metrics refer to a numeric representation of data measured over time.
Logs, a record of an event that took place at a given timestamp, also provide valuable context regarding when a specific event occurred.

Traces represent causally related events in a distributed environment.
(For a more detailed description of these, I highly recommend reading Cindy Sridharan’s landmark post: Monitoring and Observability.   ... .'

No comments: