Azure Databricks : Python Library Installation Steps

If you are struggling to load python libraries in Azure Databricks then this information is for you. There are different ways to add Python libraries to data bricks. When we are working on data science we have to work on multiple things and then we need to use different libraries. So let's begin with how to add Python libraries in azure databricks.

Please follow the below steps:

1^st Step:

Please navigate the “Clusters” tab.

2^nd Step:

Select cluster in which libraries need to be installed.

3^rd Step

1. Go to the libraries section

2. Click on “Install New”

4^th Step

This window will occur.

Now you have many options to install libraries.

1. Upload Option:

a. In this option, the library file need to be downloaded from the internet. Acceptable file types are .Jar, .Egg file or .Wheel file.

b. You can get the downloadable files from (https://pypi.org/) or there are many sites where you will find this python package file.

c. Then just drag the downloaded file on the pop window.

d. Click on install.

2. PyPI Option:

This is a very simple option to install the library on Databricks

a. Find pip command for the library which you want to install.

b. You can easily get-pip command from official PyPI site (https://pypi.org/)

c. Then only pick library name and remove ‘pip install’

i. E.g., If command is ‘pip install NumPy’ on website then only pick NumPy

ii. E.g., If command is ‘pip install numpy==1.12.1’ on website then pick only numpy==1.12.1

d. Then write that name on package section and click on install.

3. DBFS Option:

This option is very similar to the first option, the only difference is instead of uploading package from your local machine, upload file from dbfs file location.

if you find this blog really help you please like this blog. and post your questions in comment section.

Search This Blog

TechnicalQueries