How do I read a text file with variable hexadecimal value columns in matlab?

Question

I have a rather large text file (over 16,000 lines) that has the following format:

#ID     #Line Num       #Var Col Length Values (HEX):
45      00001           FFFF FFFF 0000 0000
45      00002           0000 0000 FFFF FFFF
47      00003           AAAA 1111 AAAA 1111 AAAA 1111
49      00004           BBBB 2222 

Note: This is obviously made up data, as there are more HEX values in the actual file.

In Matlab I tried using the textscan command of a single line:

fp = fopen(filePath, 'rt');
readLine = fgetl(fp);
[ignored, pos] = textscan(readLine, '%d');
values = textscan(readLine(pos+1:end), '%x');

I get an error of a badly formatted string. I'm assuming that textscan doesn't support conversion of hex values. I've also tried the solution found here:

Problem (bug?) loading hexadecimal data into MATLAB

but that also doesn't seem to work as well. I'm trying to avoid converting each Hex Value individually (somewhat the solution I have implemented now) as this takes a very long time to do. How do I scan/parse variable column width hexadecimal values from a text file?

Answer 1

You can use the following approach read the text file into a cell array of strings and split it into separate values using regexp:

fp = fopen(filePath, 'rt');
C = textscan(fp, '%s', 'CommentStyle', '#', 'Delimiter', '');
C = regexp(C{:}, '\w+', 'match');
fclose(fp);

This should yield a cell array of cell arrays that, for your example, would look like this:

C =
    {'45'    '00001'    'FFFF'    'FFFF'    '0000'    '0000'}
    {'45'    '00002'    '0000'    '0000'    'FFFF'    'FFFF'}
    {'47'    '00003'    'AAAA'    '1111'    'AAAA'    '1111'    'AAAA'    '1111'}
    {'49'    '00004'    'BBBB'    '2222'}

You can manipulate the resulting cell array to your liking. For instance, discard the first two columns in each row, and convert everything to decimal:

result = cellfun(@(x)hex2dec(x(3:end)), C, 'UniformOutput', false)