Try Databricks : Databricks-Certified-Professional-Data-Engineer practice torrent pass for sure

A data engineer is designing a Lakeflow Declarative Pipeline to process streaming order data. The pipeline uses Auto Loader to ingest data and must enforce data quality by ensuring customer_id and amount are greater than zero. Invalid records should be dropped.
Which Lakeflow Declarative Pipelines configurations implement this requirement using Python?

• A. @dlt.table
  def silver_orders():
  return (
  dlt.read_stream( " bronze_orders " )
  .expect_or_drop( " valid_customer " , " customer_id IS NOT NULL " )
  .expect_or_drop( " valid_amount " , " amount > 0 " )
  )
• B. @dlt.table
  @dlt.expect( " valid_customer " , " customer_id IS NOT NULL " )
  @dlt.expect( " valid_amount " , " amount > 0 " )
  def silver_orders():
  return dlt.read_stream( " bronze_orders " )
• C. @dlt.table
  @dlt.expect_or_drop( " valid_customer " , " customer_id IS NOT NULL " )
  @dlt.expect_or_drop( " valid_amount " , " amount > 0 " )
  def silver_orders():
  return dlt.read_stream( " bronze_orders " )
• D. @dlt.table
  def silver_orders():
  return (
  dlt.read_stream( " bronze_orders " )
  .expect( " valid_customer " , " customer_id IS NOT NULL " )
  .expect( " valid_amount " , " amount > 0 " )
  )

A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non- overlapping five-minute interval. Events are recorded once per minute per device.
df has the following schema: device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT Code block:
df.withWatermark( " event_time " , " 10 minutes " )
.groupBy(
________,
" device_id "
)
.agg(
avg( " temp " ).alias( " avg_temp " ),
avg( " humidity " ).alias( " avg_humidity " )
)
.writeStream
.format( " delta " )
.saveAsTable( " sensor_avg " )
Which line of code correctly fills in the blank within the code block to complete this task?

• A. lag( " event_time " , " 5 minutes " ).alias( " time " )
• B. " event_time "
• C. window( " event_time " , " 5 minutes " ).alias( " time " )
• D. to_interval( " event_time " , " 5 minutes " ).alias( " time " )

Where in the Spark UI can one diagnose a performance problem induced by not leveraging predicate push- down?

• A. In the Query Detail screen, by interpreting the Physical Plan
• B. In the Executor ' s log file, by gripping for " predicate push-down "
• C. In the Stage ' s Detail screen, in the Completed Stages table, by noting the size of data read from the Input column
• D. In the Storage Detail screen, by noting which RDDs are not stored on disk
• E. In the Delta Lake transaction log. by noting the column statistics

A data engineer is configuring a pipeline that will potentially see late-arriving, duplicate records.
In addition to de-duplicating records within the batch, which of the following approaches allows the data engineer to deduplicate data against previously processed records as it is inserted into a Delta table?

• A. VACUUM the Delta table after each batch completes.
• B. Set the configuration delta.deduplicate = true.
• C. Rely on Delta Lake schema enforcement to prevent duplicate records.
• D. Perform an insert-only merge with a matching condition on a unique key.
• E. Perform a full outer join on a unique key and overwrite existing data.

A senior data engineer is planning large-scale data workflows. The task is to identify the considerations that form a foundation for creating scalable data models for managing large datasets. The team has listed Delta Lake capabilities and wants to determine which feature should not be considered as a core factor.
Which key feature can be ignored while evaluating Delta Lake?

• A. Delta Lake provides limited support for monitoring and troubleshooting data pipelines, so relevant partner tools have to be identified and set up for enhanced operational efficiency.
• B. Delta Lake's ability to process data in both batch and streaming modes seamlessly, providing flexibility in ingestion and processing.
• C. Delta Lake works with various data formats (Parquet, JSON, CSV) and integrates well with Spark and Databricks tools.
• D. Delta Lake optimizes metadata handling, efficiently managing billions of files and facilitating scalability to petabyte-scale datasets.