A notebook is a web-based user interface for a document that includes executable code, graphics, and narrative prose.

It also includes articles on making data visualizations, sharing visuals as dashboards, using widgets to parameterize notebooks and dashboards, leveraging notebook workflows to build complicated pipelines, and best practices for defining classes in Scala notebooks.

The Workspace API, the UI, and the CLI may all be used to manage notebooks. This article focuses on using the UI to accomplish notebook operations.

How to Create a notebook in Microsoft Azure

Use the Create button to begin.

We can simply click on the Create button and create the notebook in our default folder:

1. Click  Create a notebook in the sidebar by selecting it from the menu. The dialog box for creating a notebook appears.
2. Give the notebook a name and choose the default language.
3. The Cluster drop-down appears, if there are any running clusters. Choose the cluster to which we'd like to attach the notebook.
4. Press the Create button.

In any folder, make a notebook.

In order to create the notebook in any folder, we must follow the following steps (for example, the Shared folder):

1. Select Workspace from the sidebar. Choose one of the following options:
   • Click on the Create button and then on the Notebook button from the menu on the right side of the text, next to any folder.
     
     1. We can also Create a notebook just by clicking on the Create button > Notebook in the workspace or else we can use a user folder.
     2. Now further follow from the steps 2 through steep 4 in the heading in this document Use the Create button to begin.

Opening a notebook

In order to open a notebook, we must Select a notebook in our workspace. When you hover your mouse over the notebook title, the notebook path appears.

Copy notebook path

In order to copy the path of the notebook, just right-click the notebook name and select Copy File Path to copy the notebook file path and even without having to open the notebook.

Rename a notebook

Click the title of an active notebook and modify inline, or go to File > Rename to rename it.

Control access to a notebook

We even have the option in out notebook for the control of access. It means that we can provide control to the users that who can access the notebook. We have the option of workspace access control but only if our account is a azure premium plan.

Notebook external formats

External notebook formats supported by Azure Databricks include:

• Source file: A file having the extensions.scala,.py,.sql, or.r that simply contains source code statements.
• HTML: A.html extension for an Azure Databricks notebook.
• DBC archive: It is a Databricks archive.
• IPython notebook: It is a Jupyter notebookwith the extension .ipynb.
• RMarkdown: It is an R Markdown documentwith the extension .Rmd.

Import a notebook

An external notebook can be imported from a URL or a file.

1. Select Import from the menu.
   • Selecting a single notebook export, it to the current folder.
   • If you choose a DBC or ZIP archive, the current folder's folder structure is reconstructed, and each notebook is imported.

Export a notebook

In order to export a notebook, we can select File option and then click on the Export option and a format from the notebook toolbar.

Export all notebooks in a folder

To create a ZIP archive of all folders in a workspace folder, follow these steps:

1. In the sidebar, select Workspace. Choose one of the following options:
   • In order to export a notebook, we need to select Export option from the arrow that is present on the right side of the text which is next to any folder.
   • Or else we can also export the notebook by selecting the Export option from the down arrow which is present in the Workspace or a user folder.
2. Then we need to Select the export format in which we want our notebook to be exported:
   • Source File: From this option we can create a ZIP package of notebook source files that can be imported into an Azure Databricks workspace.
   • DBC Archive: It is a binary format that contains metadata and the results of notebook commands.
   • HTML Archive: Export HTML files as a ZIP package.
     The output of the notebook command is included.

Notebooks and clusters

You must first link a notebook to a cluster before you can conduct any work in it. This section explains how to add and remove notebooks from clusters, as well as what occurs behind the scenes when you do so.

A cluster can only have a certain amount of execution contexts (145). You can't attach a notebook to the cluster or establish a new execution context after the number of execution contexts reaches this limit.

Idle execution contexts

When the previous completed execution occurs after a given idle threshold, an execution context is deemed idle. The idle timer is set to 24 hours by default.

The notebook that used the context is remains attached to the cluster and shows in the cluster's notebook list even if the context is removed.

Streaming notebooks are always considered active, and their context is not evicted until their execution is completed. The UI displays a notice if an idle context is evicted, indicating that the notebook using the context was disconnected owing to being idle.

If you try to attach a notebook to a cluster with the maximum number of execution contexts and there are no idle contexts (or if auto-eviction is turned off), the UI displays a message stating that the current maximum execution contexts threshold has been reached, and the notebook will remain detached.

When you fork a process, an idle execution context remains idle until the request that forked the process completes. With Spark, it is not suggested to fork distinct processes.

Configure context auto-eviction

The Spark property spark.databricks.chauffeur.enableIdleContextTracking can be used to allow context auto-eviction.

• Auto-eviction is enabled by default in Databricks 5.0 and higher. Set spark.databricks.chauffeur.enableIdleContextTracking false to deactivate auto-eviction for a cluster.
• Auto-eviction is off by default in Databricks 4.3. Setting spark.databricks.chauffeur.enableIdleContextTracking true for a cluster enables auto-eviction.

Attaching a cluster to a notebook.

It requires a cluster level permission in order to attach it to a notebook.

In order to attach a notebook to a cluster, one need to follow the steps:

1. First, we need to click on the notebook toolbar, then we need to click on the Detached option.
2. Then from the drop-down menu, we need to select a cluster.

Determine Spark and Databricks Runtime version

Run the following command to identify the Spark version of the cluster to which your notebook is connected:

PythonCopy

spark.version

Run the following command to find the Databricks Runtime version of the cluster to which your notebook is connected:

Scala

ScalaCopy

dbutils.notebook.getContext.tags("sparkVersion")

Python

PythonCopy

spark.conf.get("spark.databricks.clusterUsageTags.sparkVersion")

Tip

Detaching unwanted notebooks from a cluster is recommended by Azure Databricks. This allows the driver to have more RAM.

All notebooks associated with a cluster can be viewed.

On the cluster details page, the Notebooks tab lists all of the notebooks associated with a cluster. The condition of each attached notebook is also displayed, as well as the last time a command was ran from the notebook.