This section will help you increase your knowledge of two of Azure open‐source products and features, as well as provide a summary of the storage of data along the Big [...]
One of the features of Azure Synapse Analytics is that it enables you to use database templates to quickly create new databases with predefined schemas and data structures. Database templates [...]
When you begin to brainstorm the storage of data for an analytical workload, terms such as hybrid transaction/analytical processing (HTAP) and online analytical processing (OLAP) might come to mind. Both [...]
Chapter 3 discussed designing partition strategies. Remember the two primary reasons for partitioning data. The first reason is to improve the speed at which a query for data responds with [...]
Chapter 7, “Design and Implement a Data Stream Processing Solution,” discusses partitioning data within one partition and across partitions. Exercise 7.5 features the hands‐on implementation of partitioning streaming workloads. Partitioning [...]
The serving layer is a component of the lambda architecture (refer to Figure 3.13). The serving layer is the place into which both the speed layer, which processes data incrementally, [...]
df = spark.read.option(“header”,”true”) \ .csv(‘abfss://@.dfs.core.windows.net/EMEA/brainjammer/in/2022/04/01/18/*’) display(df.limit(10)) FIGURE 4.21 Azure Synapse Analytics Develop hub load Notebook FIGURE 4.22 Azure Synapse Analytics Develop hub write Notebook Parquet files In this exercise you [...]
Your Azure Databricks workspace includes an associated Azure storage account and blob container. Two lifecycle management policies that delete temporary and log files are created by default. You can view [...]
Consider adding a few additional transformations and viewing the configuration details. Add an Aggregate schema modifier or a Window schema modifier and see how they can be used to transform [...]
Archiving data saves costs, improves performance, and may be necessary due to data privacy compliance. Cost savings are realized because you are charged by the amount of storage space required [...]