Databricks Certified Data Engineer Associate : Databricks-Certified-Data-Engineer-Associate

A data engineer is building a PySpark ingestion pipeline in a local IDE and must execute heavy DataFrame transformations on a remote Databricks cluster for scale while stepping through business logic locally. The workspace uses Unity Catalog and a cluster running Databricks Runtime 15.4 LTS. The engineer also plans to register UDFs.
What should the data engineer do to avoid runtime issues during development with Databricks Connect?

• A. Disable Unity Catalog on the workspace because it conflicts with remote execution through Databricks Connect.
• B. Use any local Python version as long as the PySpark minor version matches, because the cluster's Python version is isolated from the client.
• C. Use serverless, because Databricks Connect does not support assigned or shared clusters.
• D. Match the local Python minor version to the cluster's Python minor version when using UDF's, and use a Databricks Connect package compatible with the cluster's Runtime version

A data engineer has developed a Python notebook in a Databricks workspace and has configured it to run as a scheduled Job to process daily sales data. How are the storage and execution of this notebook managed within the Databricks architecture?

• A. The notebook is stored unencrypted in the workspace storage bucket, and the code executes on the cluster driver node in the compute plane when the job runs.
• B. The notebook is stored securely and encrypted in the compute plane, and the code executes in the control plane when the job runs.
• C. The notebook is stored securely and encrypted in the control plane, and the code executes in the compute plane when the job runs.
• D. The notebook is stored securely and encrypted in Unity Catalog, and the code executes in Delta Lake when the job runs.

A data engineer is writing a DataFrame to a Delta table and wants to physically divide the data into directories based on a specific column such as country. Which Spark DataFrame writer option should be used?

• A. orderBy
• B. partitionBy
• C. groupBy
• D. distributeBy

The Delta transaction log for the 'students' tables is shown using the 'DESCRIBE HISTORY students' command. A Data Engineer needs to query the table as it existed before the UPDATE operation listed in the log. Which command should the Data Engineer use to achieve this?
(Choose two.)

• A. SELECT * FROM students TIMESTAMP AS OF '2024-04-22T14:32:47.000+00:00'
• B. SELECT * FROM students TIMESTAMP AS OF '2024-04-22T14:32:58.000+00:00'
• C. SELECT * FROM students FROM HISTORY VERSION AS OF 3
• D. SELECT * FROM students VERSION AS OF 5
• E. SELECT * FROM students@v4

A data engineer has been using a Databricks SQL dashboard to monitor the cleanliness of the input data to a data analytics dashboard for a retail use case. The job has a Databricks SQL query that returns the number of store-level records where sales is equal to zero. The data engineer wants their entire team to be notified via a messaging webhook whenever this value is greater than 0.
Which of the following approaches can the data engineer use to notify their entire team via a messaging webhook whenever the number of stores with $0 in sales is greater than zero?

• A. They can set up an Alert with one-time notifications.
• B. They can set up an Alert with a new email alert destination.
• C. They can set up an Alert without notifications.
• D. They can set up an Alert with a new webhook alert destination.
• E. They can set up an Alert with a custom template.