What’s New with Databricks Notebooks

Databricks Notebooks provides builders a managed authoring expertise the place knowledge and AI groups can effectively collaborate on initiatives collectively. The workforce right here is working exhausting as we put together to share thrilling new improvements for Notebooks on the Information + AI Summit later this month, and we hope you’ll be a part of us at our session Develop Like a Professional in Databricks Notebooks, led by Weston Hutchins and Neha Sharma.  As a heat up, we wished to take a fast look again at some latest additions in Notebooks.

Run Databricks Notebooks on SQL Warehouses

SQL is the 2nd hottest language in Notebooks behind Python, and to higher help our customers who love SQL, we’re bringing SQL warehouses to Notebooks.  SQL warehouses are the identical assets that energy Databricks SQL, and so they ship higher price-performance for SQL execution in comparison with all-purpose clusters.  This function is rolling out now, so preserve a watch out!

Whereas connected to a SQL warehouse, solely your pocket book’s SQL cells will execute. Cells utilizing different languages (like Python or Scala) might be skipped. Markdown cells will proceed to be rendered. To study extra, go to our documentation.

Share Notebooks using Delta Sharing

View knowledge from Notebooks, SQL editor, and Information Explorer in the identical expertise

The brand new unified schema browser enables you to view the entire knowledge within the Unity Catalog metastore with out leaving a pocket book or the SQL editor. You’ll be able to choose “For you” to filter the checklist to the lively tables in your pocket book. 

As you kind your search request into the filter field, the show actively updates to indicate solely these objects that comprise that textual content. This may search for objects which can be at present open or have been opened earlier within the present session. Study extra right here.

Run Databricks Notebooks on SQL Warehouses

Share Notebooks utilizing Delta Sharing

Now you can use Delta Sharing to share pocket book recordsdata with the Databricks-to-Databricks sharing stream. You get the benefit and safety of Delta Sharing. Sharing notebooks empowers you to collaborate throughout metastores and accounts. This permits individuals who share knowledge to unpack the worth of that knowledge with notebooks.

To study extra, right here is how one can add notebooks to a share (for suppliers) and the best way to learn shared notebooks (for recipients).

Debug your Notebooks with Variable Explorer

Debug your Notebooks with the Variable Explorer

The Variable Explorer shows the state of all of the Python variables in your pocket book growth session. The identify, kind, and worth are surfaced for all easy variable sorts. The Variable Explorer additionally surfaces extra metadata for Spark and Pandas DataFrames. The form and column names can be found at-a-glance, and a full view of the schema is offered on hover. 

The Variable Explorer additionally permits you to step by way of and debug Python code by leveraging the help for pdb in Databricks Notebooks. You’ll be able to set breakpoints with breakpoint() or pdb.set_trace(). If you run the cell, the execution will pause on the breakpoint and the Variable Explorer will mechanically replace with the state of the pocket book at that breakpoint. See our documentation for extra info.

View data from Notebooks, SQL editor, and Data Explorer

See you at Summit

At Information + AI Summit 2023, we’ll have deep dive classes into utilizing Notebooks. We may also discuss concerning the latest methods to be extra environment friendly whereas utilizing Databricks. We hope to see you there.

Leave a Reply

Your email address will not be published. Required fields are marked *