Search code examples
shellcsvunixhivemultiline

convert multiline to single line in unix


I have file which has multi line data in one column, I wanted multi line to be converted to single line.

Here is the sample with Headers

final_date|Notes|Status
04/17/2019|"- OB Team - 
Number of Attempt(s): 1
Outcome:other
Order (RMO):0
Campaign : ABC
Additional Notes:  not a working number  
* If any call return to transfer to OB team *"|Complete
04/18/2019|"- OB Team - 

Number of Attempt(s): 3
Outcome: NO ANSWER
Order (RMO): 0
Campaign Name: ABC

*If return call, transfer to OB team* 


- OB TEAM - 
Number of Attempt(s):  1 
Outcome:  VM
Order (RMO):  0
Campaign Name:  ABC 
Additional Notes: None
*If return call, transfer to OB team*"|Complete

Above data is has two records. Which I want them to convert to single line and then load to Hive table.

Above data should be converted as below.

final_date|Notes|Status
04/17/2019|"- OB Team - Number of Attempt(s): 1 Outcome:other Order (RMO):0 Campaign : ABC Additional Notes:  not a working number * If any call return to transfer to OB team *"|Complete
04/18/2019|"- OB Team - Number of Attempt(s): 3 Outcome: NO ANSWER Order (RMO): 0 Campaign Name: ABC *If return call, transfer to OB team*  - OB TEAM - Number of Attempt(s):  1  Outcome:  VM Order (RMO):  0 Campaign Name:  ABC Additional Notes: None *If return call, transfer to OB team*"|Complete

Can someone help me regarding this.


Solution

  • Manipulate output record separator in accordance with the number of double-quotes in current line.

    awk -F\" 'BEGIN{ors=ORS} NF&&!(NF%2){ORS=(ORS!=ors)?ors:OFS} 1' file