Trouble in reading values from parsed XML data using XML::Simple

Question

I am half way through writing a script using XML::Simple. I have read that is not so "simple", and even its own documentation discourages its use in new code, but I have no other choice as this script will be an extension to existing code.

What I am doing is this

1. Get XML by reading from a URL
2. Parse it using XML::Simple
3. Read the required elements from the data
4. Run different checks on these required elements

I could parse and do some checks on a few of the elements, but while reading elements that are in array, I am getting undef.

This is my code:

#!/usr/bin/perl

use strict;
use warnings;

use LWP::UserAgent;
use LWP::Simple;
use XML::Simple;
use DBI;

use Data::Dumper;

my $str = "<Actual_URL>";

my $ua = LWP::UserAgent->new;
$ua->timeout( 180 );
$ua->agent( "$0/0.1 " . $ua->agent );

my $req = HTTP::Request->new( GET => $str );

my $buffer;
$req->content_type( 'text/xml' );
$req->content( $buffer );

my $response = $ua->request( $req );

my $xml = $response->content();
print "Value of \$xml is:\n";
print $xml;

my $filename = 'record.txt';
open( my $fh, '>', $filename ) or die "Could not open file '$filename' $!";
print $fh $xml;
close $fh;

my $number_of_lines = `wc -l record.txt | cut -d' ' -f1`;
print "Number of lines in $filename are: $number_of_lines\n";
if ( $number_of_lines >= 50 ) {
    print "TEST_1 SUCCESS\n";
}

my $mysql_dbh;
my $test_id;

my $xst;
my %cmts_Pre_EQ_tags;

if ( ( not defined $xml ) or ( $xml =~ m/read\stimeout/i ) ) {
    &printXMLErr( 'DRUM request timed out' );
}
else {
    my $xs = XML::Simple->new();
    $xst = eval { $xs->XMLin( $xml, KeyAttr => 1 ) };
    &printXMLErr( $@ ) if ( $@ );
    print "Value of \$xst inside is:\n";
    print Dumper( $xst );
}

$cmts_Pre_EQ_tags{'$cmts_Pre_EQ_groupDelayMag'} =
    $xst->{cmts}->{Pre_EQ}->{groupDelayMag}->{content};

#More elements like this are checked here
$cmts_Pre_EQ_tags{'$cmts_Pre_EQ_ICFR'} =
    $xst->{cmts}->{Pre_EQ}->{ICFR}->{content};

my $decision1 = 1;
print "\%cmts_Pre_EQ_tags:\n";
foreach ( sort keys %cmts_Pre_EQ_tags ) {
    print "$_ : $cmts_Pre_EQ_tags{$_}\n";
    if ( $cmts_Pre_EQ_tags{$_} eq '' ) {
        print "$_ is empty!\n";
        $decision1 = 0;
    }
}
print "\n";

if ( $decision1 == 0 ) {
    print "TEST_2_1 FAIL\n";
}
else {
    print "TEST_2_1 SUCCESS\n";
}

my $cpeIP4 = $xst->{cmts}->{cpeIP4}->{content};
print "The cpe IP is: $cpeIP4\n";

if ( $cpeIP4 ne '' ) {
    print "TEST_2_2 SUCCESS\n";
}
else {
    print "TEST_2_2 FAIL\n";
}

# Working fine until here, but following 2 print are showing undef
print Dumper ( $xst->{cmts}{STBDSG}{dsg}[0]{dsgIfStdTunnelFilterTunnelId} );
print Dumper ( $xst->{cmts}{STBDSG}{dsg}[0]{dsgIfStdTunnelFilterClientIdType} );
print "After\n";

Output of last three print statements is:

$VAR1 = undef;
$VAR1 = undef;
After

I can't provide the entire XML or the output of print Dumper($xst) as it's too big and gets generated dynamically, but I'll provide a sample of it.

The part of the XML that is causing trouble is

<cmts>
  <STBDSG>
    <dsg>
      <dsgIfStdTunnelFilterTunnelId>1</dsgIfStdTunnelFilterTunnelId>
      <dsgIfStdTunnelFilterClientIdType>caSystemId</dsgIfStdTunnelFilterClientIdType>
    </dsg>
    <dsg>
      <dsgIfStdTunnelFilterTunnelId>2</dsgIfStdTunnelFilterTunnelId>
      <dsgIfStdTunnelFilterClientIdType>gaSystemId</dsgIfStdTunnelFilterClientIdType>
    </dsg>
  </STBDSG>
</cmts>

And when this part is parsed, then its corresponding output in $xst is

$VAR1 = {
    'cmts' => {
            'STBDSG' => {
                'dsg' => [
                         {
                           'dsgIfStdTunnelFilterTunnelId' => '1',
                           'dsgIfStdTunnelFilterClientIdType' => 'caSystemId',
                         },
                         {
                           'dsgIfStdTunnelFilterTunnelId' => '2',
                           'dsgIfStdTunnelFilterClientIdType' => 'gaSystemId',
                         }
                         ]
                     },
    },
};

The XML part where after parsing the values are fetched fine is like this

<cmts>
    <name field_name="Name">cts01nsocmo</name>
    <object field_name="Nemos Object">888</object>
    <vendor field_name="Vendor">xyz</vendor>
</cmts>

Which was converted as:

    $VAR1 = {
      'cmts' => {
        'name' => {
                    'content' => 'cts01nsocmo',
                    'field_name' => 'Name'
                  },
        'object' => {
                      'content' => '888',
                      'field_name' => 'Nemos Object'
                    },
        'vendor' => {
                      'content' => 'xyz',
                      'field_name' => 'Vendor'
                    }
         },
};

So basically when there is no array in parsed content, the values are being fetched correctly in variables.

It seems that the reason why this

print Dumper ( $xst->{cmts}{STBDSG}{dsg}[0]{dsgIfStdTunnelFilterTunnelId} );
print Dumper ( $xst->{cmts}{STBDSG}{dsg}[0]{dsgIfStdTunnelFilterClientIdType} );

is getting undef is related to setting correct values to either KeyAttr or ForceArray. I am trying to find it by reading XML::Simple, but I wanted to see if there's something distinct that I am missing here.

Answer 1

It's worth considering the use of XML::Twig, regardless of what the rest of your project does

In particular, XML::Twig::Elt objects -- the module's implementation of XML elements -- have a simplify method, whose documentation says this

Return a data structure suspiciously similar to XML::Simple's. Options are identical to XMLin options

So you can use XML::Twig for its precision and convenience, and apply the simplify method if you need to pass on any data that looks like an XML::Simple data structure