Search code examples
azure-data-factoryapache-synapseazure-notebooks

ADF Web Activity Error: The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered

I am getting the error message in web Activity of ADF:

The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered:

Further looking into this error, I found that getting an error from notebook which is:

---------------------------------------------------------------------------\nValueError                                Traceback (most recent call last)\n/tmp/ipykernel_10465/628309876.py in <module>\n      7 \n      8 \n----> 9 split_df.columns = ['dot1', 'dot2', 'dot3', 'dot4', 'dot5']\n     10 print(split_df)\n\n~/cluster-env/env/lib/python3.8/site-packages/pandas/core/generic.py in __setattr__(self, name, value)\n   5476         try:\n   5477             object.__getattribute__(self, name)\n-> 5478             return object.__setattr__(self, name, value)\n   5479         except AttributeError:\n   5480             pass\n\npandas/_libs/properties.pyx in pandas._libs.properties.AxisProperty.__set__()\n\n~/cluster-env/env/lib/python3.8/site-packages/pandas/core/generic.py in _set_axis(self, axis, labels)\n    668     def _set_axis(self, axis: int, labels: Index) -> None:\n    669         labels = ensure_index(labels)\n--> 670         self._mgr.set_axis(axis, labels)\n    671         self._clear_item_cache()\n    672 \n\n~/cluster-env/env/lib/python3.8/site-packages/pandas/core/internals/managers.py in set_axis(self, axis, new_labels)\n    218 \n    219         if new_len != old_len:\n--> 220             raise ValueError(\n    221                 f\"Length mismatch: Expected axis has {old_len} elements, new \"\n    222                 f\"values have {new_len} elements\"\n\nValueError: Length mismatch: Expected axis has 9 elements, new values have 5 elements"

and when I am sending this error to web activity to send email, I am getting the above web activity error.

Solution Tried:

So, to overcome this error, I created a variable "ErrorMessage" using expression which will hold the error message first:

@activity('Notebook1').error.Message

and then another variable variables('CleanErrorMessage') which will hold clean message replace the \n to '', using

@replace(variables('ErrorMessage'),'\n','')

but it is not working as expected

Can you help me to resolve this and remove this special symbols and line break from the notebook error message so that I can pass the error to web activity in json format and get the result.

Web Activity Json:

{
    "Pipeline":"@{pipeline().Pipeline}",
    "ErrorMessage": "@{variables('CleanErrorMessage')}",
    "DataFactoryName":"@{pipeline().DataFactory}",
    "Runid":"@{pipeline().RunId}"
}
Solution

  • You can do the following things to get it worked.

    • You can replace the double quotes with two single quotes and convert them to string. Below is the expression to do.
    @string(replace(activity('Notebook1').error.Message, '"',''''))

    • You can paste the body content into the body text field box WITHOUT clicking on 'Add Dynamic Content' in web activity. If you are pasting the body text field in dynamic content, you can use additional double quotes or string functions as given in the similar question in Microsoft QnA.