I'm going through a course in MVA "Introducing Azure Data Lake", and till module 2 ,in each vid it's highlighting, that we are paying only for time job is taking to execute.
This get me confused, if its has tendency or meant for storing the data in it permanently of our all transaction history data or it's meant for just analysing the portion of transaction temporarly in it and after doing computation on it with any analytical language i.e. USQL, HIVE ,PIG and drop the temporary analytical source once its done.
In other words:
Is it creating a temporary or staging table & analysing data on it and dropping the table after outcome of stats . Or Data lake has tendency or meant to store data for future use like warehouse and do analytics on it as per business requirement and doesn't cost for that storage that much historical that but only for the processing time for analysing on it.
Regards Harsimran
Data that you load into the ADL storage account or write with a U-SQL or Hive job (e.g., with an OUTPUT statement or inserting into a table), will be stored in the ADL storage account and will use the space. The space in your ADLS account is not limited by any cluster sizing and will grow as your demand increases.