This section will help you increase your knowledge of two of Azure open‐source products and features, as well as provide a summary of the storage of data along the Big [...]
–MODESELECT * FROM [dimensional].[MODE] WHERE [MODE_ID] = 2 Approaching the data discovery from the top down would indicate a desire to understand which mode is connected to which frequencies. Because [...]
FIGURE 4.35 Serving layer using star schema distribution types FIGURE 4.36 Serving layer using star schema integration dataset COPY INTO [brainwaves].[DimMODE]FROM ‘https://.blob.core.windows.net/brainjammer/Tables/MODE.csv’ WITH ( FILE_TYPE=’CSV’,FIRSTROW = 2)GO COPY INTO [brainwaves].[DimSCENARIO] [...]
One of the features of Azure Synapse Analytics is that it enables you to use database templates to quickly create new databases with predefined schemas and data structures. Database templates [...]
Congratulations on creating your first Azure Synapse Analytics pipeline. Reflect a bit on all the features it relied on and how much you have learned about them so far. You [...]
When you begin to brainstorm the storage of data for an analytical workload, terms such as hybrid transaction/analytical processing (HTAP) and online analytical processing (OLAP) might come to mind. Both [...]
df = spark.read.load(‘abfss://@.dfs.core.windows.net/in-path/file.csv’, df.write.mode(“overwrite”) \ df = spark.read.load(‘abfss://@.dfs.core.windows.net/out-path/file.parquet’, print(df.count()) from pyspark.sql.functions import year, month, col df = spark.read \ .load(‘abfss://@.dfs.core.windows.net/out-path/file.parquet’, format=’parquet’, header=True) df_year_month_day = (df.withColumn(“year”, year(col(“SESSION_DATETIME”)))) \ .withColumn(“month”, month(col(“SESSION_DATETIME”))) from [...]
No single folder structure fits every solution; what is considered a logically designed folder structure is relative. You have seen the following folder structures: Those folders are well suited for [...]
DROP TABLE brainwaves.DimELECTRODE 2. Create an SCD table, and then execute the following SQL script, which is located in the folder Chapter04/Ch04Ex09 on GitHub at https://github.com/benperk/ADE and named createSlowlyChangingDimensionTable.sql: CREATE [...]
COLLATE Latin1_General_100_BIN2_UTF8 WITH (LOCATION = ‘abfss://@.dfs.core.windows.net’) WITH (FORMAT_TYPE = PARQUET) ([Timestamp] NVARCHAR(50),[AF3theta] NVARCHAR(50),[AF3alpha] NVARCHAR(50),[AF3betaL] NVARCHAR(50),…) WITH(LOCATION = ‘EMEA/brainjammer/out/2022/04/03//.parquet/*’,DATA_SOURCE = SampleBrainwavesSource,FILE_FORMAT = SampleBrainwavesParquet) FIGURE 4.33 Building an external table You might [...]