This section will help you increase your knowledge of two of Azure open‐source products and features, as well as provide a summary of the storage of data along the Big [...]
FIGURE 4.35 Serving layer using star schema distribution types FIGURE 4.36 Serving layer using star schema integration dataset COPY INTO [brainwaves].[DimMODE]FROM ‘https://.blob.core.windows.net/brainjammer/Tables/MODE.csv’ WITH ( FILE_TYPE=’CSV’,FIRSTROW = 2)GO COPY INTO [brainwaves].[DimSCENARIO] [...]
No single folder structure fits every solution; what is considered a logically designed folder structure is relative. You have seen the following folder structures: Those folders are well suited for [...]
Chapter 3 discussed designing partition strategies. Remember the two primary reasons for partitioning data. The first reason is to improve the speed at which a query for data responds with [...]
Unlike physical data storage structures, you cannot actually touch logical storage structures. That is because logical storage structures are arrangements like blocks, extents, segments, and tablespaces that reside on the [...]
A flowlet is a container that holds reusable activities. Consider the activity you created in Exercise 4.8, which retrieves brain wave data and converts the epoch date into a more [...]
When you select the Source item, six tabs are rendered in the configuration panel: Source Settings, Source Options, Projection, Optimize, Inspect, and Data Preview. Figure 4.25 shows the Source Settings [...]
df = spark.read.option(“header”,”true”) \ .csv(‘abfss://@.dfs.core.windows.net/EMEA/brainjammer/in/2022/04/01/18/*’) display(df.limit(10)) FIGURE 4.21 Azure Synapse Analytics Develop hub load Notebook FIGURE 4.22 Azure Synapse Analytics Develop hub write Notebook Parquet files In this exercise you [...]
Your Azure Databricks workspace includes an associated Azure storage account and blob container. Two lifecycle management policies that delete temporary and log files are created by default. You can view [...]
FIGURE 4.19 Azure Synapse Analytics Data hub ADLS directory FROM OPENROWSET( ) AS [result] FIGURE 4.20 Azure Synapse Analytics Data hub ADLS directory Notice in Figure 4.20 that the SQL [...]