I am trying to find all <script>
elements in a series of HTML-documents by using html-agility-pack (plus LINQ and XPath).
The documents have script-elements placed in header, and a Google Analytics in footer. First I am trying to target the header scripts and remove them. My Notepad++ shows me that I have 719 script-elements present, but my console appliaction only finds 55 of them.
I need som help to target them properly, so I can remove them from the document.
Source document (structure of head),
<!doctype html system "html.dtd">
<html>
<head>
<link rel="stylesheet" href="../IRstyle.css" type="text/css">
<title>Non-hierarchic document clustering</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="keywords" content="">
<meta name="VW96.objecttype" content="Document">
<script language="JavaScript" type="text/JavaScript">
//Javascript-code goes here
</script>
</head>
<body>
<!--Body contents goes here-->
<!-- in footer -->
<script src="http://www.google-analytics.com/urchin.js" type="text/javascript">
</script>
<script type="text/javascript">
_uacct = "UA-67XXXX-X";
urchinTracker();
</script>
</body>
</html>
So far I've tried targeting the 'Language' type with JavaScript, but only get a few hits when parsing html/head. My method takes filenames from a list. For now, method prints out number of scripts collected in list, this will change to 'Scripts.Remove();' once I get the search string right.
private static void FindTagsToRemove(IEnumerable<string> files)
{
var doc = new HtmlDocument();
List<string> scripts = new List<string>();
List<string> errors = new List<string>();
try
{
foreach (var file in files)
{
doc.Load(@file);
var head = doc.DocumentNode.SelectSingleNode("html/head");
var nodes = new List<HtmlNode>();
bool isScript = false;
foreach (var node in head.ChildNodes.ToList())
{
if (node.NodeType == HtmlNodeType.Element && node.Name.Contains("script"))
{
isScript = !isScript;
scripts.Add(node.OuterHtml);
Console.WriteLine(node.OuterHtml);
}
else if (isScript)
{
nodes.Add(node);
node.Remove();
}
}
}
int nr_scripts = scripts.Count();
Console.WriteLine("Number of scripts in collection: {0}", nr_scripts);
}
catch (Exception Ex)
{
Console.WriteLine(Ex.Message);
}
}
If someone have a better way of targeting the JavaScripts in the head-node, it will be much appreciated. ANY help is appreciated! :)
If you only need <script>
element nodes, use descendant-or-self (//
). Sample HTML:
var html =
@"<!doctype html system 'html.dtd'>
<html>
<head>
<link rel='stylesheet' href='../IRstyle.css' type='text/css'>
<title>Non-hierarchic document clustering</title>
<meta http-equiv='Content-Type' content='text/html; charset=iso-8859-1'>
<meta name='keywords' content=''>
<meta name='VW96.objecttype' content='Document'>
<script language='JavaScript' type='text/JavaScript'>
//Javascript-code goes here
</script>
</head>
<body>
<!--Body contents goes here-->
<!-- in footer -->
<script src='http://www.google-analytics.com/urchin.js' type='text/javascript'>
</script>
<script type='text/javascript'>
_uacct = 'UA-67XXXX-X';
urchinTracker();
</script>
</body>
</html>";
Parse sample:
var document = new HtmlDocument();
document.LoadHtml(html);
// target only <script> in <head>
// var scriptTags = document.DocumentNode.SelectNodes("//head/script");
var scriptTags = document.DocumentNode.SelectNodes("//script");
foreach (var script in scriptTags) script.Remove();
document.Save(OUTPUT);
Output:
<!doctype html system 'html.dtd'>
<html>
<head>
<link rel='stylesheet' href='../IRstyle.css' type='text/css'>
<title>Non-hierarchic document clustering</title>
<meta http-equiv='Content-Type' content='text/html; charset=iso-8859-1'>
<meta name='keywords' content=''>
<meta name='VW96.objecttype' content='Document'>
</head>
<body>
<!--Body contents goes here-->
<!-- in footer -->
</body>
</html>