By Vlad Riscutia

This is an excerpt from chapter 2 of my book — Data Engineering on Azure — which deals with storage. In this article we’ll look at a few aspects of data ingestion: frequency and load type, and how we can handle corrupted data. We’ll use Azure Data Explorer as the storage solution, but keep in mind that the same concepts apply regardless of the data fabric used. Code samples are omitted from this article, but are available in the book. Let’s start by looking at the frequency with which we ingest data.