Search code examples
linuxawksednewlinetrim

Bash trim or sed newline

Have log file that has SOME new lines to remove (not all); have tried many ways of trim and sed with no luck (usually ends up one big line).

I think i need to have a specific search, when trimming or sedding, but not sure.

example file:

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams)
   at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams)
   at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

Need out:

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

Any help greatly apprecaited!

Solution

  • You can do this with awk:

    awk 'BEGIN {ORS=""} /^ /{ print $0; next;} {print "\n"$0;}' input_file | sed 's/\r//g'

    which gives

    JOBV2|MISC: PURCHASE_DENIED|Puchasemode = sync ; iEoCode = 992 ; teminalId = XXXXXXXXXXXX; offeingId = XXXXXX; entalType = ps ; pice = 1649 ; entalWindowHs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TansactionID = IsUG=N;TId=X;VID=XXXX;Pice=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTansactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPice=16.49;DisPice=0.00;NetPice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PuchasePaams objPuchasePaams)   at STRING.JOBV3(Sting stInputPaams)|stInputPaams = stUnitAddess=XXXXXXXXXXXX,teminalid=XXXXXXXXXXXX,offeingid=XXXXXX,entalType=PS,pice=1649,entalWindowHs=0,IsAdult=false,IsRedeemTitle=false,pomotionType=,isUpgade=N,catalogpice=0,stpomotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PuchasePaams objPuchasePaams)   at STRING.JOBV3(Sting stInputPaams)|stInputPaams = stUnitAddess=XXXXXXXXXXXX,teminalid=XXXXXXXXXXXX,offeingid=XXXXXX,entalType=PS,pice=1649,entalWindowHs=0,IsAdult=false,IsRedeemTitle=false,pomotionType=,isUpgade=N,catalogpice=0,stpomotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Puchasemode = sync ; iEoCode = 992 ; teminalId = XXXXXXXXXXXX ; offeingId = XXXXXX; entalType = ps ; pice = 1649 ; entalWindowHs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TansactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pice=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTansactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPice=16.49;DisPice=0.00;NetPice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

    Alternative method:

    awk 'BEGIN {ORS=""} /\)$/{print $0; next;} {print $0"\n";}' input_file | sed 's/\r//g'