How to use Bash Regex Lookbehind across whitespace in Terraform file?

Question

I'm writing a bash script to parse a bunch (a dozen or more) massive Terraform files that contain a large number of google_bigquery_dataset resources and their associated IAM access blocks. The script should take each dataset resource and copy it to another file, named for the dataset itself.

All of this is fine, except extracting the name of the dataset from the resource's "dataset_id" field. This would be easy enough, if not for the fact that some of these dataset resources have authorized view blocks that also contain "dataset_id" values.

Here is an example of such a resource:

resource "google_bigquery_dataset" "project-bigquery-dataset-RESOURCE_NAME" {
  access {
    role          = "WRITER"
    special_group = "projectWriters"
  }

  access {
    role          = "READER"
    special_group = "projectReaders"
  }

  access {
    role          = "WRITER"
    user_by_email = "user1@project.iam.gserviceaccount.com"
  }

  access {
    role          = "OWNER"
    special_group = "projectOwners"
  }

  access {
    view {
      dataset_id = "DO_NOT_WANT"
      project_id = "project"
      table_id   = "table1"
    }
  }

  access {
    view {
      dataset_id = "DO_NOT_WANT"
      project_id = "project"
      table_id   = "table2"
    }
  }

  access {
    view {
      dataset_id = "DO_NOT_WANT"
      project_id = "project"
      table_id   = "table3"
    }
  }

  dataset_id                      = "THIS_IS_WHAT_I_WANT"
  default_partition_expiration_ms = "0"
  delete_contents_on_destroy      = "false"

  labels = {
    application-name = "app-name"
  }

  location = "US"
  project  = "project"
}

Before I realized that the authorized view blocks also had a dataset_id field, I was using this to try to grab the value I wanted, assuming startIndex and endIndex are just the start and end line numbers representing a complete dataset resource block as above:

fileName=$( sed -n ${startIndex},${endIndex}p $bigFile | grep "dataset_id" | cut -d\" -f2)

Which works only when there are not Authorized View blocks contained other dataset_id values.

I then tried to use a Negative Lookbehind:

fileName=$( sed -n ${startIndex},${endIndex}p $bigFile | grep '(?<!view {]n)dataset_id' | cut -f1 -d\"

That doesn't work. I'm not sure if it's because of the newline or because of the whitespace between the end of view { and the start of dataset_id = "DO_NOT_WANT".

I've tried variations on it, such as (?<!view\s{\s)\s*dataset_id without success.

Is there any way to capture only the dataset_id that isn't in a view block?

A couple notes:

1. I can guarantee that view { will always precede the dataset_id in a block, without a line break.
2. I cannot guarantee the order. It's possible the dataset_id I'm trying to capture could be present before the view blocks, after them, or even somewhere between them.
3. Desired output for the above example would simply be THIS_IS_WHAT_I_WANT Any help would be appreciated.

Answer 1

With your shown samples only, please try following awk code. Written and tested in GNU awk.

awk -v RS= -v FS="\n" '
/^[[:space:]]+dataset_id[[:space:]]+/{
  split($1,arr,"\"")
  print arr[2]
}
'  Input_file

Explanation: Simple explanation for complete code would be:

• Setting RS(Record separator) as paragraph mode in awk program.
• Then setting FS(Field separator) as new line.
• Then in main block checking condition if line starts from 1 or more spaces followed by dataset_id followed by again 1 or more spaces, if this condition is TRUE then:
• Using split function of awk to split $1(first field) into an array named arr with delimiter of ". This basically creates an array named arr with index of 1 2 3 4 and so on depending upon how many elements it splits based on delimiter.
• Then printing array arr's 2nd element which is required output by OP.