Get the coupon in the end of description.
Description
Preparing for AWS Certified Data Engineer Associate DEA-C01? This is THE practice exams course to give you the winning edge.
These practice exams have been Oussama El Berhichi who bring their collective experience of passing 20 AWS Certifications to the table.
The tone and tenor of the questions mimic the real exam. Along with the detailed description and “exam alert” provided within the explanations, we have also extensively referenced AWS documentation to get you up to speed on all domain areas being tested for the DEA-C01 exam.
We want you to think of this course as the final pit-stop so that you can cross the winning line with absolute confidence and get AWS Certified! Trust our process, you are in good hands.
All questions have been written from scratch! And more questions are being added over time!
Quality speaks for itself
SAMPLE QUESTION:
A data engineer is encountering slow query performance while executing Amazon Athena queries on datasets stored in an Amazon S3 bucket, with AWS Glue Data Catalog serving as the metadata repository. The data engineer has identified the root cause of the sluggish performance as the excessive number of partitions in the S3 bucket, leading to increased Athena query planning times.
What are the two possible approaches to mitigate this issue and enhance query efficiency (Select two)?
Transform the data in each partition to Apache ORC format
Compress the files in gzip format to improve query performance against the partitions
Perform bucketing on the data in each partition
Set up an AWS Glue partition index and leverage partition filtering via the GetPartitions call
Set up Athena partition projection based on the S3 bucket prefix
What’s your guess? Scroll below for the answer.
Correct: 4,5.
Explanation:
Correct options:
Set up an AWS Glue partition index and leverage partition filtering via the GetPartitions call
When you create a partition index, you specify a list of partition keys that already exist on a given table. The partition index is sub list of partition keys defined in the table. A partition index can be created on any permutation of partition keys defined in the table. For the above sales_data table, the possible indexes are (country, category, creationDate), (country, category, year), (country, category), (country), (category, country, year, month), and so on.
Let’s take a sales_data table as an example which is partitioned by the keys Country, Category, Year, Month, and creationDate. If you want to obtain sales data for all the items sold for the Books category in the year 2020 after 2020-08-15, you have to make a GetPartitions request with the expression “Category = ‘Books’ and creationDate > ‘2020-08-15′” to the Data Catalog.
If no partition indexes are present on the table, AWS Glue loads all the partitions of the table and then filters the loaded partitions using the query expression provided by the user in the GetPartitions request. The query takes more time to run as the number of partitions increases on a table with no indexes. With an index, the GetPartitions query will try to fetch a subset of the partitions instead of loading all the partitions in the table.
Incorrect options:
Transform the data in each partition to Apache ORC format – Apache ORC is a popular file format for analytics workloads. It is a columnar file format because it stores data not by row, but by column. ORC format also allows query engines to reduce the amount of data that needs to be loaded in different ways. For example, by storing and compressing columns separately, you can achieve higher compression ratios and only the columns referenced in a query need to be read. However, the data is being transformed within the existing partitions, this option does not resolve the root cause of under-performance (that is, the excessive number of partitions in the S3 bucket).
Compress the files in gzip format to improve query performance against the partitions – Compressing your data can speed up your queries significantly. The smaller data sizes reduce the data scanned from Amazon S3, resulting in lower costs of running queries. It also reduces the network traffic from Amazon S3 to Athena. Athena supports a variety of compression formats, including common formats like gzip, Snappy, and zstd. However, the data is being compressed within the existing partitions, this option does not resolve the root cause of under-performance (that is, the excessive number of partitions in the S3 bucket).
Perform bucketing on the data in each partition – Bucketing is a way to organize the records of a dataset into categories called buckets. This meaning of bucket and bucketing is different from, and should not be confused with Amazon S3 buckets. In data bucketing, records that have the same value for a property go into the same bucket. Records are distributed as evenly as possible among buckets so that each bucket has roughly the same amount of data. In practice, the buckets are files, and a hash function determines the bucket that a record goes into. A bucketed dataset will have one or more files per bucket per partition. The bucket that a file belongs to is encoded in the file name. Bucketing is useful when a dataset is bucketed by a certain property and you want to retrieve records in which that property has a certain value. Because the data is bucketed, Athena can use the value to determine which files to look at. For example, suppose a dataset is bucketed by customer_id and you want to find all records for a specific customer. Athena determines the bucket that contains those records and only reads the files in that bucket.
Good candidates for bucketing occur when you have columns that have high cardinality (that is, have many distinct values), are uniformly distributed, and that you frequently query for specific values.
Since bucketing is being done within the existing partitions, this option does not resolve the root cause of under-performance (that is, the excessive number of partitions in the S3 bucket).
