18 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
0
votes
1
answer
81
views
Azure Databricks - use of parameters in SQL DDL in a Notebook
I am trying to set up a notebook to configure the basics of a catalog and its schemas, permissions etc for our developers can be "set loose"...
I have a parameter set as a widget: v_catalog ...
0
votes
1
answer
124
views
dbutils.notebook.run() Masks Specific Errors with General WorkflowException Logs
I'm using dbutils.notebook.run() in a Databricks Python file to execute another notebook. When executed notebook fails, I'm only seeing a general error log instead of the real specific error (that I'm ...
-1
votes
1
answer
104
views
Copying Missing store_id Records from Previous Month Data with Historical data Specific Grouping Logic in Databricks with copyforward logic
I'm working in a Databricks notebook using PySpark to process monthly sales data. I have one Apache Spark DataFrames: it is for the historical data from 2023 to still, and every month i received a ...
1
vote
1
answer
278
views
View Only Access to Azure Databricks Notebooks
I am trying to give only View access to the Notebooks for a user. I have created a premium Databricks Account from my Azure portal. Then I create a new user in the Azure portal and gave Reader access ...
0
votes
1
answer
40
views
How to create Pyspark code that will create a white space in Databricks Notebook cell
I would like help with Python(PySpark) that will continually create a white space or simply move the cursor in a Databricks notebook cell.
For example, in the image I have manually moved the cursor ...
0
votes
1
answer
446
views
Application logging in Executor/Worker using Azure Databricks python notebooks
I am using Azure Databricks for building and running ETL pipelines. For development, using Databricks notebooks (Python). My goal is to view the application logs via Spark UI for both codes running on ...
0
votes
2
answers
152
views
Py4JJava Error on Azure Databricks notebook
I am trying to read data from mongodb in batch
My compute cluster configurations are:
Spark Version > 3.5.0
Scala Version > 2.12
I have installed mongodb connector library version is(through ...
1
vote
0
answers
73
views
Call Databricks notebook without specifying parameters from an external notebook
I have main notebook main_notebook. I am going to call an external notebook notebook1 which has 2 parameters param1 and param2, but I do not want to specify parameters name, I know parameters orders. ...
0
votes
1
answer
726
views
Dynamically create and display ipywidgets, failing in databricks notebook
Goal: An array of ipywidgets, that can be extended through a button-click on the UI.
import ipywidgets as widgets
from IPython.display import display
# Keeps track of default and dynamically added ...
0
votes
1
answer
203
views
Get Command number in databricks
How can i get the current command number dynamically from a cell in Azure databricks notebook? Is there any dbutils command to retrieve it?
For below cell, it should return 1.
0
votes
1
answer
599
views
cat: write error: Resource temporarily unavailable
I am using cat command to merge multiple files using %sh command in Azure Databricks Notebook. I have around 1200 csv files in the data_files folder and the total size of files are around 300 GB. When ...
1
vote
3
answers
231
views
Handle error message from the cat command [duplicate]
I am using Databricks Notebook to execute Shell script using the %sh and I want to capture error message into err variable and then print the error message if the cat command fails (for eg, ...
0
votes
1
answer
188
views
i dont cant take the source from my notebook with terraform
Regards
I am creating a databricks notebook using terraform cloud. When the notebook code is in the same directory as the tf there is no problem, it finds it right away. But to organize the repository,...
0
votes
1
answer
223
views
how to get and set the error details to a variable when a command fails in a notebook
Am new to databricks . I have a notebook that has multiple commands . I want to capture the specific error message that is thrown when any of the commands fail. basically I would want to customize the ...
1
vote
1
answer
2k
views
Is there a way to turn off autocomplete for SQL in Databricks (it is awful)?
We recently got an update for our Databricks Notebooks web pages that now seems to include an autocomplete for SQL that is very distracting.
Is there a way to turn this feature off.
These articles don'...