Is there a way to instantly resolve macro variable created in a data step in the same data step?

Question

Background is that I need to use filename command to execute grep and use the result as input.

Here is my input data set named test

firstname   lastname   filename
<blank>     <blank>    cus_01.txt
<blank>     <blank>    cus_02.txt

Filename values are actual files which I need to grep because I need certain string inside those files to fill up the firstname and lastname

Here is the code:

data work.test;
   set work.test;
   call symputx('file', filename);
   filename fname pipe "grep ""Firstname"" <path>/&file.";
   filename lname pipe "grep ""Lastname"" <path>/&file.";
   infile fname;
   input firstname;
   infile lname;
   input lastname; 
run;

However, macro variables created inside a data step can't be used until after the data step procedure is completed. So, that means, &file. can't be resolved and can't be used in filename.

Is there a way to for resolve the macro variable?

Thanks!

Answer 1

If you have many customer files the use of pipe to grep can be an expensive operating system action, and on SAS servers potentially disallowed (pipe, x, system, etc...)

You can read all pattern-named files in a single data step using the wildcard feature of infile and the filename= option to capture the active file being read from.

Sample:

%let sandbox_path = %sysfunc(pathname(WORK));

* create 99 customer files, each with 20 customers;

data _null_;
  length outfile $125;
  do index = 1 to 99;
    outfile = "&sandbox_path./" || 'cust_' || put(index,z2.) || '.txt';
    file huzzah filevar=outfile;
    putlog outfile=;

    do _n_ = 1 to 20;
      custid+1;
      put custid=;
      put "firstname=Joe" custid;
      put "lastname=Schmoe" custid;
      put "street=";
      put "city=";
      put "zip=";
      put "----------";
    end;
  end;
run;

* read all the customer files in the path;
* scan each line for 'landmarks' -- either 'lastname' or 'firstname';    

data want;
  length from_whence source $128;
  infile "&sandbox_path./cust_*.txt" filename=from_whence ;
  source = from_whence;
  input;

  select;
    when (index(_infile_,"firstname")) topic="firstname";
    when (index(_infile_,"lastname")) topic="lastname";
    otherwise;
  end;

  if not missing(topic);

  line_read = _infile_;
run;