Replace New Line with blank inside the AWK

Question

#Removing single quote from file to avoid conflict with delmiter 
roledetails="${roledetails//","/"|"}" 
    awk '
    BEGIN                      { FS  = ":[[:space:]]+"
                                 OFS = ","
                               }
    $1 ~ /^name$/              { coll = $2 } #Collection Role Name
    $1 ~ /^description$/       { desc = $2} #Collection Role Desc
    $1 ~ / name$/              { name = $2 } #Single Role Name
    $1 ~ / roleTemplateAppId$/ { id = $2 } #Single Role App ID with all other field in last column
    $1 ~ / roleTemplateName$/  { print coll,desc,name,id,$2} #Single Role Template Name
    ' <<< "${roledetails}" >> "${fullfilepath}"

I have above code to extra data from array and file but I want description field has new line in some of the desc field. I like to replace new line with nothing. How do I do this inside the awk ?

I tried to add line as below and it is not working

$1 ~ /^description$/       { desc = gsub(/\n/,"",$2)} #Collection Role Desc

Here is the sample data. Second filed has data which is breaking the line with new line. I want to replace it with no break.

=============================================================

name: CLD:Z_BASIS_ADMIN_SUPER
description: Basis Administrator (Super)
readOnly: <null>
roleReferences:
  - description: Compound role Tenant Administrator
    name: AuthGroup_Administrator
    roleTemplateAppId: it!b56186
    roleTemplateName: AuthGroup_Administrator
  - description: Compound role Business Expert
    name: AuthGroup_BusinessExpert
    roleTemplateAppId: it!b56186
    roleTemplateName: AuthGroup_BusinessExpert
  - description: Compound role Integration Developer
    name: AuthGroup_IntegrationDeveloper
    roleTemplateAppId: it!b56186
    roleTemplateName: AuthGroup_IntegrationDeveloper
  - description: Operate the data transmission tunnels used by the Cloud connector
    name: Cloud Connector Administrator
    roleTemplateAppId: connectivity!b7
    roleTemplateName: Cloud_Connector_Administrator
  - description: Manage destination configurations| certificates and signing keys for SAML assertions issued by the Destination service
    name: Destination Administrator
    roleTemplateAppId: destination-xsappname!b62
    roleTemplateName: Destination_Administrator
  - description: Default role for sending messages to integration flows with an IdP user
    name: MessagingSend
    roleTemplateAppId: it-rt-merck-cpi-dev-ariba!b56186
    roleTemplateName: MessagingSend
  - description: Role for subaccount members with read-write authorizations for core commercialization operations| such as viewing subaccount entitlements| and creating and deleting environment instances.
    name: Subaccount Admin
    roleTemplateAppId: cis-local!b4
    roleTemplateName: Subaccount_Admin
  - description: Administrative access to service brokers and environments on a subaccount level.
    name: Subaccount Service Administrator
    roleTemplateAppId: service-manager!b1476
    roleTemplateName: Subaccount_Service_Administrator
  - description: Read-only access for authorizations| trusted identity providers| and users.
    name: User and Role Auditor
    roleTemplateAppId: xsuaa!t8
    roleTemplateName: xsuaa_auditor

=========================================================== Here is the output in plain text ===========================================================

CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,AuthGroup_Administrator,it!b56186,AuthGroup_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,AuthGroup_BusinessExpert,it!b56186,AuthGroup_BusinessExpert
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,AuthGroup_IntegrationDeveloper,it!b56186,AuthGroup_IntegrationDeveloper
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,Cloud Connector Administrator,connectivity!b7,Cloud_Connector_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,Destination Administrator,destination-xsappname!b62,Destination_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,MessagingSend,it-rt-merck-cpi-dev-ariba!b56186,MessagingSend
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,Subaccount Admin,cis-local!b4,Subaccount_Admin
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,Subaccount Service Administrator,service-manager!b1476,Subaccount_Service_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super)
,User and Role Auditor,xsuaa!t8,xsuaa_auditor

================================================================== Below is the expected output without CR character at the end

CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),AuthGroup_Administrator,it!b56186,AuthGroup_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),AuthGroup_BusinessExpert,it!b56186,AuthGroup_BusinessExpert
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),AuthGroup_IntegrationDeveloper,it!b56186,AuthGroup_IntegrationDeveloper
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Cloud Connector Administrator,connectivity!b7,Cloud_Connector_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Destination Administrator,destination-xsappname!b62,Destination_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),MessagingSend,it-rt-merck-cpi-dev-ariba!b56186,MessagingSend
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Subaccount Admin,cis-local!b4,Subaccount_Admin
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Subaccount Service Administrator,service-manager!b1476,Subaccount_Service_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),User and Role Auditor,xsuaa!t8,xsuaa_auditor

======================================================== My output .csv file shows below. I see extra CR character at the end of the description value after (Super)

Answer 1

You're mixing up carriage return with linefeed, newline, and linebreak terminology which is making it harder to understand your needs. Carriage Return (CR) is a character that can be present anywhere in a text file. Linefeed (LF) is a character that represents a newline in Unix files and so in Unix discussions is often used synonymously with newline. Carriage-Return LineFeed (CR-LF) is a 2-character string that represents newline in Windows files. On some other systems you might find that CR alone or LF-CR represents a newline. I expect by linebreak you mean newline.

From your comments you appear to want to remove CRs, not "Replace New Line with blank" as your Subject says. gsub(/\n/,"",$2) in your code will not remove CRs because CRs are represented by \r, not \n (LF). See Why does my tool output overwrite itself and how do I fix it? for more info on CRs and how to handle them.

Whenever you have tag-value pairs in the input as you do, I find it best to first populate an array (f[] below) mapping the tags (names) to the values and then you can do whatever you like with the values (print, change, test, etc.) just by indexing the array with the tags.

This should do what I think you want regardless if CRs are present at the end of lines or not:

$ cat tst.awk
BEGIN { OFS="," }
{
    gsub(/,/,"|")
    sub(/[[:space:]]+$/,"")
    sub(/^[[:space:]-]+/,"-")
    tag = val = $0
    sub(/[[:space:]]*:.*/,"",tag)
    sub(/[^:]*:[[:space:]]*/,"",val)
    f[tag] = val
}
tag == "-roleTemplateName"  {
    print f["name"], f["description"], f["-name"], f["-roleTemplateAppId"], f["-roleTemplateName"]
}

$ awk -f tst.awk file
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),AuthGroup_Administrator,it!b56186,AuthGroup_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),AuthGroup_BusinessExpert,it!b56186,AuthGroup_BusinessExpert
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),AuthGroup_IntegrationDeveloper,it!b56186,AuthGroup_IntegrationDeveloper
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Cloud Connector Administrator,connectivity!b7,Cloud_Connector_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Destination Administrator,destination-xsappname!b62,Destination_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),MessagingSend,it-rt-merck-cpi-dev-ariba!b56186,MessagingSend
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Subaccount Admin,cis-local!b4,Subaccount_Admin
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),Subaccount Service Administrator,service-manager!b1476,Subaccount_Service_Administrator
CLD:Z_BASIS_ADMIN_SUPER,Basis Administrator (Super),User and Role Auditor,xsuaa!t8,xsuaa_auditor

With that you can get rid of the "${roledetails//","/"|"}" you currently have outside of your awk script since the gsub(/,/,"|") takes care of that, and it will work even if you have lines that contain :<space> is it separates the tag from the value based on the first : position on each line rather than splitting into fields using FS = ":[[:space:]]+" and then relying on $2 containing the whole value (which it would not given input like description: here: we have a problem).

If you do find you have problems with CRs appearing in other places than the end of lines then just add gsub(/\r/,"") (or gsub(/\r/," ") or however you want to handle them) above the first gsub() in the above script.