ADF Copy activity is making double column in adx as "infinity" for 1.7976931348623157E308 value

Question

We are trying to convert the delta into adx in which we have delta column double type and data in it is "1.7976931348623157E308" which is double max value, and using copy activity we need to put this data into adx kusto, in kusto the values is "infinity", want to know the reason behind and how to tackle this.

but adding the value directly in kusto via ingestion works fine. strange...

Answer 1

• ADX uses a real data type for floating-point numbers, which corresponds to a 64-bit IEEE 754 format. The real type in ADX has a smaller range than the maximum value of a double in Delta.
• During the transfer, when ADF detects a value that exceeds ADX's range, it automatically converts it to infinity, following IEEE 754 standards for handling overflow. That may be the reason to get infinity value while copying data as shown below:

To copy your double column value as it is you use data flow instead of copy activity.

• ADF Data Flow processes data in-memory using its own engine and doesn't always enforce strict conversions between source and sink types unless explicitly configured.
• As a result, it can transfer 1.7976931348623157E308 as-is without triggering the overflow mechanism.

Add your delta table as source and ADX database as sink in your dataflow, create pipeline with dataflow activity, select created dataflow, after dataflow debug you will be able copy the data as it is.

['tableName']
|  project  <columnName> 

For more information you can refer to the MS document.