In this video, seasoned IT professional Richard Nuckolls dives into writing and running Azure Data Lake Analytics jobs.
By Vlad Riscutia
This is an excerpt from chapter 2 of my book — Data Engineering on Azure — which deals with storage. In this article we’ll look at a few aspects of data ingestion: frequency and load type, and how we can handle corrupted data. We’ll use Azure Data Explorer as the storage solution, but keep in mind that the same concepts apply regardless of the data fabric used. Code samples are omitted from this article, but are available in the book. Let’s start by looking at the frequency with which we ingest data.
This is an excerpt from a draft of chapter 10 of my book, Azure Data Engineering, which deals with compliance. In this article we’ll look at a few techniques to transform sensitive user data into less sensitive data. In the book, this includes code samples for implementation on Azure Data Explorer, which are omitted from this article. Let’s start with a couple of definitions.