I'm building web scraper. I tried to improve this code:
for (int i = 1; i < 6; i++) {
Elements siteElements = document.select("div.grid__col.grid__col--20-80-80.b-products-wrap > ul > li:nth-child(" + i + ")");
System.out.println(siteElements.select(" > div > div.b-products-list__desc-wrap > div > div.b-products-list__main-content > div.b-products-list__desc-prime > div.b-products-list__manufacturer-holder").select("a").first().text());
System.out.println(siteElements.select(" > div > div.b-products-list__desc-wrap > div > div.b-products-list__main-content > div.b-products-list__desc-prime > div.b-products-list__title-holder > a").first().text());
System.out.println(siteElements.select(" div.b-products-list__price-holder > a").first().text());
System.out.println(siteElements.first().attr("data-ppc-id"));
}
To this code (dont mind about last line, I know it is wrong after improval). So I took the part that is shared between 3 sys.out
> div > div.b-products-list__desc-wrap > div > div.b-products-list__main-content >
and put it in siteElements
variable (btw. is it good name for this variable?)
for (int i = 1; i < 6; i++) {
Elements siteElements = document.select("div.grid__col.grid__col--20-80-80.b-products-wrap > ul > li:nth-child(" + i + ") > div > div.b-products-list__desc-wrap > div > div.b-products-list__main-content >");
System.out.println(siteElements.select(" div.b-products-list__desc-prime > div.b-products-list__manufacturer-holder").select("a").first().text());
System.out.println(siteElements.select(" div.b-products-list__desc-prime > div.b-products-list__title-holder > a").first().text());
System.out.println(siteElements.select(" div.b-products-list__price-holder > a").first().text());
//System.out.println(siteElements.first().attr("data-ppc-id"));
}
but then I got exception:
Exception in thread "main" org.jsoup.select.Selector$SelectorParseException: Could not parse query '': unexpected token at ''
at org.jsoup.select.QueryParser.findElements(QueryParser.java:206)
at org.jsoup.select.QueryParser.parse(QueryParser.java:59)
at org.jsoup.select.QueryParser.parse(QueryParser.java:42)
at org.jsoup.select.QueryParser.combinator(QueryParser.java:87)
at org.jsoup.select.QueryParser.parse(QueryParser.java:67)
at org.jsoup.select.QueryParser.parse(QueryParser.java:42)
at org.jsoup.select.Selector.select(Selector.java:91)
at org.jsoup.nodes.Element.select(Element.java:363)
at Main.main(Main.java:23)
What am I making wrong? The site from which I scrape data: https://merlin.pl/bestseller/?option_80=10349074
You can not end a selector with >
, because this requires another selector to be valid. Just remove that or use > *
or something similar and you won't get this exception. Maybe you have to adjust the selector a bit further to get elements you want.