How to collect data from website or (web scraping) via flutter?

Question

I need to web scraping and collect data from the website https://ipcost.com and use that collected data (in this case, my IP details) in my Flutter app.

My question is how to get this information and display it in the UI.

Answer 1

I tried the web_scraper package and it solved the problem.

Step 1 : add web_scraper to dependencies .

Step 2 : Checking the website and getting the pattern of placing the desired information on the website. in my case for "Titles" , pattern is :

'div.e > small'

for "Subtitle" , pattern is :

'div.e > strong'

Step 3 : I created an object of the Webscraper class and defined it by setting the value of the website URL.

Step 4-1 : In my case , I don't need urlPath and set it empty .

  final String urlPath = '';

Step 4-2 : In my case , Attributes always null and I don't need them.

  final List<String> attributeList = [''];

Step 5 : I define one method and call them in initState , for get values from website .

  void fetchData() async {
    if (await webScraper.loadWebPage(urlPath)) {
      setState(() {
        webScrapeResultTitle =
            webScraper.getElement(innerAddressTitle, attributeList);
        webScrapeResultDescription =
            webScraper.getElement(innerAddressDescription, attributeList);


        ///TODO:  By review this prints  , I detect structure of recievied data .


        // print("webScrapeResultTitle::::${webScrapeResultTitle!.toList()}");
        // print(
        //     "webScrapeResultDescription::::${webScrapeResultDescription!.toList()}");
      });
    }
  }

Step 6 : set build method like below for show results .

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: webScrapeResultTitle == null || webScrapeResultDescription == null
          ? Center(
        child: LoadingWidget(),
      )
          : ListView.builder(
          itemCount: webScrapeResultDescription!.length,
          itemBuilder: (BuildContext context, int index) {
            return IpInfoItem(
                titleResult: webScrapeResultTitle![index][titleResultKey],
                descriptionResult: webScrapeResultDescription![index]
                [descriptionResultKey]);
          }),
    );
  }
}

See my full code from here

Gist full Snippets: Gist.Github.com

Repository : Github.com