Study with Databricks-Certified-Data-Engineer-Professional most valid questions & verified answers

A data engineer is designing a secure data sharing strategy for their organization. The company needs to share sensitive customer analytics data with two different partners. Partner A uses Databricks with Unity Catalog enabled, while Partner B uses Apache Spark on AWS without Databricks. How should the company implement secure data sharing for these scenarios?

• A. Both partners should use the same Delta Sharing approach since security requirements are identical. You should create bearer tokens for both partners and use the open sharing protocol (D2O) for maximum compatibility.
• B. Databricks-to-Databricks sharing (D2D) can only be used within the same cloud provider, so you must use open sharing (D2O) for any cross-cloud scenarios. Unit Catalog governance is not available when sharing with external platforms.
• C. Open sharing protocol (D2O) should be used for both partners because it provides better security than D2D sharing. The bearer token approach is always more secure than Unity Catalog's native authentication.
• D. For Partner A, implement Databricks-to-Databricks sharing (D2D) with Unit Catalog integration and no-token exchange system. For Partner B, use open sharing protocol (D2O) with either bearer tokens or OIDC federation for authentication, ensuring both approaches maintain robust security and governance.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

An organization processes customer data from web and mobile applications. Data includes names, emails, phone numbers, and location history. Data arrives both as batch files (from SFTP daily) and streaming JSON events (from Kafka in real-time).
To comply with data privacy policies, the following requirements must be met:
- Personally Identifiable Information (PII) such as email, phone
number, and IP address must be masked or anonymized before storage.
- Both batch and streaming pipelines must apply consistent PII
handling.
- Masking logic must be auditable and reproducible.
- The masked data must remain usable for downstream analytics.
How should the data engineer design a compliant data pipeline on Databricks that supports both batch and streaming modes, applies data masking to PII, and maintains traceability for audits?

• A. Load batch data with notebooks and ingest streaming data with SQL Warehouses; use Unity Catalog column masks on Silver tables to redact fields after storage.
• B. Use Lakeflow Declarative Pipelines for batch and streaming ingestion, define a PII masking function, and apply it during Bronze ingestion before writing to Delta Lake.
• C. Allow PII to be stored unmasked in Bronze for lineage tracking, then apply masking logic in Gold tables used for reporting.
• D. Ingest both batch and streaming data using Lakeflow Declarative Pipelines, and apply masking via Unity Catalog column masks at read time to avoid modifying the data during ingestion.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

A data engineer is optimizing a MERGE operation on an 800GB UC-managed table that experiences frequent updates and deletions. Which two actions should the engineer prioritize to improve MERGE performance? (Choose two.)

• A. Partition the table by date.
• B. Overwrite the table instead of Merge.
• C. Enable deletion vectors on the table if not already enabled.
• D. Apply liquid clustering using the merge join keys.
• E. Use ZORDER on high-cardinality columns.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

A data engineer has created a transactions Delta table on Databricks that should be used by the analytics team. The analytics team wants to use the table with another tool that requires Apache Iceberg format. What should the data engineer do?

• A. Enable uniform on the transactions table to 'iceberg' so that the table can be read as an Iceberg table.
• B. Convert the transactions Delta table to Iceberg and enable uniform so that the table can be read as a Delta table.
• C. Create an Iceberg copy of the transactions Delta table which can be used by the analytics team.
• D. Require the analytics team to use a tool that supports Delta table.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).

A data engineer is attempting to execute the following PySpark code:
df = spark.read.table("sales")
result = df.groupBy("region").agg(sum("revenue"))
However, upon inspecting the execution plan and profiling the Spark job, they observe excessive data shuffling during the aggregation phase.
Which technique should be applied to reduce shuffling during the groupBy aggregation operation?

• A. Caching the DataFrame df.
• B. Use coalesce() after the aggregation.
• C. Repartition by region before aggregation.
• D. Use broadcast join.

Explanation: Only visible for Easy4Engine members. You can sign-up / login (it's free).