Extending Databricks Unity Catalog with an Open Apache Hive Metastore API


At the moment, we’re excited to announce the preview of a Hive Metastore (HMS) interface for Databricks Unity Catalog, which permits any software program suitable with Apache Hive to connect with Unity Catalog. Apache Hive is essentially the most broadly supported catalog interface within the {industry}, usable in just about each main computing platform. This characteristic lets organizations centralize their knowledge administration, discovery and governance in Unity Catalog and connect with it from a variety of computing platforms, together with Amazon Elastic MapReduce (EMR), open supply Apache Spark, Amazon Athena, Presto, Trino, and others. It additionally ensures constant knowledge governance throughout these platforms.

To hitch this preview, contact your Databricks consultant.

Hive Metastore interface for Unity Catalog
Hive Metastore interface for Unity Catalog

In right now’s quickly evolving knowledge administration panorama, many organizations run a number of computing platforms and face the problem of persistently implementing knowledge discovery and governance throughout them. This usually requires knowledge groups to juggle a number of knowledge catalogs and governance instruments, resulting in excessive operational overhead and difficulties in knowledge discovery, entry administration, and auditing.

Databricks Unity Catalog is a unified governance answer for knowledge, analytics and AI with easy options to find knowledge, handle permissions, audit accesses, observe knowledge lineage and high quality, and share knowledge throughout organizations. With the HMS interface, now you can join any software program that helps the industry-standard Apache Hive API to Unity, significantly simplifying governance throughout computing platforms.

On this weblog, we are going to discover the benefits of this characteristic and the way it can improve your knowledge administration practices.

Why we constructed a Hive Metastore interface for Unity Catalog

Openness

In a various knowledge ecosystem, openness performs a vital position in seamless knowledge integration and collaboration. Apache Hive is essentially the most broadly supported catalog API within the {industry}. Unity Catalog’s open HMS interface aligns with our open lakehouse platform technique and gives a unified and standardized method for accessing enterprise knowledge and avoiding vendor lock-in. By utilizing Unity Catalog with this open interface, organizations can simplify their knowledge administration structure and guarantee it may assist each present and future instruments.

Constant Information Governance

Managing knowledge throughout a number of platforms poses a major problem in sustaining constant governance. The HMS interface in Unity Catalog successfully tackles this problem, by extending enterprise-grade governance offered by Unity Catalog to a various vary of compute platforms. This integration ensures constant knowledge compliance, enhances safety measures, enforces strong entry controls, and facilitates centralized auditing.

Simple Path to Modernize Legacy Workloads

For purchasers who’ve lengthy standing legacy workloads working on completely different computing platforms, the HMS interface integration permits them to centralize metadata and entry controls in Unity Catalog, encompassing each their legacy and Databricks workloads. This integration provides an easy path for migrating these workloads working on platforms resembling Amazon EMR, whereas making certain consistency all through the migration course of.

Value Optimization

The HMS interface for Unity Catalog brings vital price optimization advantages to organizations. Historically, organizations needed to make investments vital time and assets into managing a number of catalogs, which not solely incurred extra prices but in addition launched complexity and potential inconsistencies. Syncing knowledge and insurance policies between a number of catalogs is fragile and error-prone. This integration eliminates the necessity to handle, keep, and synchronize separate knowledge catalogs.

Conclusion

The HMS interface for Unity Catalog brings collectively one of the best of each worlds—open interoperability and enterprise-grade governance. By connecting Unity Catalog with numerous compute platforms, organizations can unlock enhanced knowledge accessibility, improved governance, scalability, price optimization, interoperability, and future-proofing, and eradicate the excessive operational price of managing a number of knowledge platforms right now. This lets organizations give attention to benefiting from their knowledge belongings as an alternative.

Contact your Databricks Consultant to affix this thrilling preview!

Additionally, don’t miss extra thrilling updates about knowledge and AI governance and register for our classes on the Information and AI Summit.

Leave a Reply

Your email address will not be published. Required fields are marked *