Study with Certified-Data-Engineer-Professional most valid questions & verified answers

A data engineer needs to install the PyYAML Python package within an air-gapped Databricks environment. The workspace has no direct internet access to PyPI. The engineer has downloaded the .whl file locally and wants it available automatically on all new clusters. Which approach should the data engineer use?

• A. Add the .whl file to Databricks Git Repos and assume automatic installation.
• B. Upload the PyYAML .whl file to the user home directory and create a cluster-scoped init script to install it.
• C. Set up a private PyPI repository and install via pip index URL.
• D. Upload the PyYAML .whl file to a Unity Catalog Volume, ensure it's allow-listed, and create a cluster-scoped init script that installs it from that path.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

In order to facilitate near real-time workloads, a data engineer is creating a helper function to leverage the schema detection and evolution functionality of Databricks Auto Loader. The desired function will automatically detect the schema of the source directly, incrementally process JSON files as they arrive in a source directory, and automatically evolve the schema of the table when new fields are detected.
The function is displayed below with a blank:

Which response correctly fills in the blank to meet the specified requirements?

• A.
  
• B.
  
• C.
  
• D.
  
• E.
  

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

A facilities-monitoring team is building a near-real-time PowerBI dashboard off the Delta table device_readings:
Columns:
device_id (STRING, unique sensor ID)
event_ts (TIMESTAMP, ingestion timestamp UTC)
temperature_c (DOUBLE, temperature in °C)
Requirement:
For each sensor, generate one row per non-overlapping 5-minute
interval, offset by 2 minutes (e.g., 00:02-00:07, 00:07-00:12, ...).
Each row must include interval start, interval end, and average
temperature in that slice.
Downstream BI tools (e.g., Power BI) must use the interval timestamps
to plot time-series bars.

• A. WITH buckets AS (
  SELECT device_id,
  window(event_ts, '5 minutes', '2 minutes', '5 minutes') AS win,
  temperature_c
  FROM device_readings
  )
  SELECT device_id,
  win.start AS bucket_start,
  win.end AS bucket_end,
  AVG(temperature_c) AS avg_temp_5m
  FROM buckets
  GROUP BY device_id, win
  ORDER BY device_id, bucket_start;
• B. SELECT device_id,
  date_trunc('minute', event_ts - INTERVAL 2 MINUTES) + INTERVAL 2 MINUTES AS bucket_start, date_trunc('minute', event_ts - INTERVAL 2 MINUTES) + INTERVAL 7 MINUTES AS bucket_end, AVG(temperature_c) AS avg_temp_5m FROM device_readings GROUP BY device_id, date_trunc('minute', event_ts - INTERVAL 2 MINUTES) ORDER BY device_id, bucket_start;
• C. SELECT device_id,
  event_ts,
  AVG(temperature_c) OVER (
  PARTITION BY device_id
  ORDER BY event_ts
  RANGE BETWEEN INTERVAL 5 MINUTES PRECEDING AND CURRENT ROW
  ) AS avg_temp_5m
  FROM device_readings
  WINDOW w AS (window(event_ts, '5 minutes', '2 minutes'));
• D. SELECT device_id,
  window.start AS bucket_start,
  window.end AS bucket_end,
  AVG(temperature_c) AS avg_temp_5m
  FROM device_readings
  GROUP BY device_id, window(event_ts, '5 minutes', '5 minutes', '2 minutes') ORDER BY device_id, bucket_start;

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

A data engineer wants to automate job monitoring and recovery in Databricks using the Jobs API.
They need to list all jobs, identify a failed job, and rerun it. Which sequence of API actions should the data engineer perform?

• A. Use the jobs/list endpoint to list jobs, then use the jobs/create endpoint to create a new job, and run the new job using jobs/run-now.
• B. Use the jobs/get endpoint to retrieve job details, then use jobs/update to rerun failed jobs.
• C. Use the jobs/cancel endpoint to remove failed jobs, then recreate them with jobs/create and run the new ones.
• D. Use the jobs/list endpoint to list jobs, check job run statuses with jobs/runs/list, and rerun a failed job using jobs/run-now.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

A data engineer is brining an existing production Databricks job under asset bundle management and wants to ensure that:
- The job's current configuration is captured as YAML, and all
referenced files are included in their bundle project.
- Future changes to the bundle's YAML will update the existing job in-
place (not create a new job)
How should the data engineer successfully move the production job under asset bundle management?

• A. Manually create the YAML configuration for the job in your bundle project, ensuring all settings match the existing job. Then, run Databricks bundle deploy the bundle, which will update the existing job in your workspace.
• B. Export the job definition as JSON, convert it to YAML, and place it in your bundle. Then, run Databricks bundle deploy to update the existing job.
• C. Run databricks bundle generate job --existing-job-id to generate the YAML and download referenced files. Then, run Databricks bundle deployment, bind to link the bundle's job resource to the existing job in Databricks.
• D. Run Databricks bundle generate job --existing-job-id to generate the YAML and download referenced files. Then, run Databricks bundle deploy to deploy the bundle, which will always update the existing job automatically.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).