1500 Questions | Databricks Spark 3.0 Associate Developer

Data Engineering on Delta Lake (30%): Delta Lake File Formats, Data Versioning and History, Data Quality and Integration.

Data Engineering with Apache Spark (20%): Apache Spark Core and RDDs, Data Processing and Transformations, Data Ingestion and Pipelines.

Data Warehousing and ETL (20%): Data Warehousing with Spark, ETL and Data Integration, Big Data and Cloud Storage.

Option A: DataFrame. select()

Option B: DataFrame.filter()

Option C: DataFrame.count()

Option D: DataFrame.withColumn()

Option E: DataFrame.drop()

Option F: DataFrame.orderBy()

Correct Answer: Option C

Explanation: Spark uses lazy evaluation, meaning transformations are not executed until an action is called. Option A is incorrect because select is a transformation that simply returns a new DataFrame. Option B is incorrect because filter is a transformation. Option C is correct because count() is an action that forces Spark to compute the result and return it to the driver. Options D, E, and F are all incorrect because withColumn, drop, and orderBy are all transformations that only add to the logical execution plan.

Question 2: Which command allows you to query an older snapshot of a Delta Lake table specifically by using a timestamp to access historical data?

Option A: SELECT * FROM table_name TIMESTAMP AS OF '2024-01-01'

Option B: SELECT * FROM table_name VERSION AS OF '2024-01-01'

Option C: SELECT * FROM table_name HISTORY AS OF '2024-01-01'

Option D: SELECT * FROM table_name RESTORE TO '2024-01-01'

Option E: SELECT * FROM table_name SNAPSHOT '2024-01-01'

Option F: SELECT * FROM table_name TIME TRAVEL '2024-01-01'

Correct Answer: Option A

Explanation: Delta Lake supports time travel using either version numbers or timestamps. Option A is correct because TIMESTAMP AS OF is the exact SQL syntax required for querying by a specific date and time. Option B is incorrect because VERSION AS OF expects an integer version number (e.g., VERSION AS OF 5), not a timestamp string. Options C, D, E, and F are incorrect because they use invalid syntax that is not recognized by Spark SQL for reading a historical Delta table state.

Question 3: In the context of the Spark DataFrame API, which method should you use to decrease the number of partitions in a DataFrame while ensuring the least amount of data shuffling across the network?

Option A: DataFrame.repartition(n)

Option B: DataFrame.coalesce(n)

Option C: DataFrame.cache()

Option D: DataFrame.persist()

Option E: DataFrame.groupBy()

Option F: DataFrame.reduceByKey()

Correct Answer: Option B

Explanation: The goal is to reduce partitions efficiently. Option A is incorrect because repartition() performs a full network shuffle of the data, regardless of whether you are increasing or decreasing the partition count. Option B is correct because coalesce() is specifically optimized to reduce partitions by combining existing ones locally, avoiding a full network shuffle. Options C and D are incorrect as they are used to store data in memory/disk for reuse, not for altering partition counts. Options E and F are incorrect because they are wide transformations used for aggregations, which inherently cause massive network shuffling rather than optimizing partition reduction.

You can retake the exams as many times as you want

This is a huge original question bank

You get support from instructors if you have questions

Each question has a detailed explanation

Mobile-compatible with the Udemy app