Search code examples
ssismappingtransformationtalendinformatica

informatica - Can anyone please explain on how mapping and transformation execute on input data


I been searching across and did not get a good post explaining how mapping and transformations process input data.

  • Does entire mapping execute for each record (I understand it can not be the case as sorter and aggregate etc.. need to see entire data (or defined batch size) in order to execute their purpose).

  • May be it depends on each transformation type - certain may hold entire input data (or defined batch size) before they release output.

May be this question apply yo all data integration tools.


Solution

  • Briefly speaking: this depends on transformation type - e.g. Expression is processing rows without caching them, Aggregator, Sorter needs to cache all input data, Joiner caches just Master pipeline.