Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://thenewstack.io/lakebase-is-databricks-fully-managed-postgres-database-for-the-ai-era/ below:

Lakebase Is Databricks' Fully-Managed Postgres Database for the AI Era

Disclosure: Databricks provided this reporter with travel support to attend its conference.

When Databricks announced its plans to acquire the serverless Postgres startup Neon a few weeks ago, it clearly signaled it’s interested in expanding support for Postgres on its larger data platform. Today, only a week after that acquisition closed, Databricks made it clearer what it plans for Neon are — and already launched a new product, Lakebase, based on Neon’s core technology.

“Lakebase combines the familiarity and extensibility of Postgres, the scalability of a modern serverless architecture, a modern developer experience, with the unified data experience of the lakehouse and operational maturity of the Databricks Data Intelligence Platform,” Databricks explains in today’s announcement. “By combining these elements into a single, fully managed offering, Lakebase enables teams to build intelligent, data-driven applications without the operational complexity traditionally associated with transactional systems.”

Image Credit: The New Stack/Frederic Lardinois

Built on top of open standards, Databricks argues that Lakebase can be both a relational database for enterprises, a way to serve data from lakehouses for a recommendation engine, for example, or a way to analyze that data stored in the lakehouse, too.

Lakebase is now in public preview.

“We think that there is going to be almost a new architecture going forward, almost like a new category. We call this the Lakebase,” Ali Ghodsi, co-founder and CEO of Databricks, said.

At today’s event, Ghodsi noted that the traditional databases, no matter whether they were MySQL, Postgres or anything else, were built for a different age. And, he argued, since data has so much gravity and these products were built for the on-prem era, vendors can easily lock in their users (and raise prices accordingly) and don’t have incentives to innovate. 

Image Credit: The New Stack/Frederic Lardinois

Meanwhile, enterprises tend to keep their operational databases separate from their analytics platforms, which in turn creates a disconnect between transactional and analytical data, while developers use separate databases for their development, testing, staging and production environments, adding additional complexity. And developers? They just want to use Postgres.

Image Credit: The New Stack/Frederic Lardinois

As you can guess, Lakebase is meant to make all of this easier with the help of Neon features like copy-on-write branching, which basically gives developers full copies of their data in separate branches, and Neon’s core innovation: the separation of storage and compute. 

This separation allows Neon — and now, by extension, Lakebase — to use cheap object storage as the data lake, allowing it to easily scale and build highly available systems, all while still allowing for low-latency access and high-concurrency transactions. And since Neon was built as a serverless product, it’s easy to autoscale the compute as needed.

All of this is integrated with the Databricks Platform and its observability, security and governance features. It also integrates with Unity Catalog, Databricks’ data catalog for structured and unstructured data, and Databricks Apps, which Lakebase powering the transactional database interactions for those apps. 

As for AI, Databricks notes that Lakebase can also be used as the online feature store for serving machine learning features and models (with the Lakehouse functioning as the offline data storage for training models). 

Before the acquisition, about 80% of databases on Neon had already been created by AI agents, not human developers. Databricks argues that features like branching and Lakebase’s serverless architecture make it ideal for agents who may want to try different approaches to solving a given problem. But, there is also another reason: all the large language models that sit at the core of these agents have been trained on open source ecosystems that use Postgres, so they already have an innate understanding of how to use it. 

Group Created with Sketch.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4