Search code examples
c#web-scrapinghtml-agility-pack

How to use HtmlAgilityPack to receive JSON data from script


My question is about the following html: https://pastebin.com/qT97gBh5

I download the HTML Site by using

            var url = "https://www.twitch.tv/monstercat";
            var web = new HtmlWeb();
            var doc = web.Load(url);

Now the only thing that interests me is the JSON Data off the following section.

<script type="application/ld+json">[{"@context":"http://schema.org","@type":"VideoObject","thumbnailUrl":["https://static-cdn.jtvnw.net/previews-ttv/live_user_monstercat-{width}x{height}.jpg"],"embedUrl":"https://player.twitch.tv/?channel=monstercat&player=facebook&autoplay=true","name":"Monstercat - Twitch","description":"Non Stop Music - Monstercat Radio 🎶","videoQuality":"1080p","publication":{"@type":"BroadcastEvent","isLiveBroadcast":true,"startDate":"03/29/2020 19:04:06"},"author":{"@type":"Person","name":"Monstercat","url":"https://www.twitch.tv/monstercat"},"uploadDate":"03/29/2020 19:04:06"}]</script>

How would i use HtmlAgilityPack to use the xpath /html/head/script[2] to receive the JSON Data?


Solution

  • you can search about agility pack in c# and after that use agility to get this X path

    /html/head/script[2]
    

    now you have a json string. convert json to property with newtonsoft(add this package with Nuget).congratulation so you can read description in your class.