I want to write a sub-transformatino that sorts a stream based on an arbitrary field (I'm going to need to sort several streams over the course of this entire project, and I'd like to just re-use 1 transformation and change the name of the field I'm sorting by). The problem is no matter what I try, I get an error saying that "The field ${SORT_FIELD} specified in the "Sort Fields" step is not in the steps input stream." (${SORT_FIELD} is the variable holding the name of the field I want to sort by and the "Sort Fields" is the name of the actual "Sort Rows" step.
${SORT_FIELD} is listed in the mapping input specification as a required field. I'm also listing ${SORT_FIELD} as a parameter to the sub-transformation (in addition to having it inherit all variables from the parent transformation).
Is there any way to pass a field name as a parameter to a sort rows step so I don't have to manually input the field I want to sort by?
yes, you can do that via metadata injection. I'm pretty sure that the sort rows step does support metadata injection. Check out matt casters blog on the subject..