I'm trying to work with a book API from openlibrary. so, my question is, how can I scrape the data? what I need is to scrape the title, author, publisher and the publish date.
JSON:
{
"ISBN:0789721813": {
"bib_key": "ISBN:0789721813",
"preview": "noview",
"preview_url": "https://openlibrary.org/books/OL18096553M/Red_Hat_Linux",
"info_url": "https://openlibrary.org/books/OL18096553M/Red_Hat_Linux",
"details": {
"number_of_pages": 757,
"subtitle": "installation and configuration handbook",
"latest_revision": 3,
"contributions": [
"Hellums, Duane"
],
"title": "Red Hat Linux",
"languages": [
{
"key": "/languages/eng"
}
],
"subjects": [
"Linux",
"Operating systems (Computers)"
],
"publish_country": "inu",
"by_statement": "Duane Hellums, et al",
"type": {
"key": "/type/edition"
},
"revision": 3,
"other_titles": [
"Red Hat Linux version 6.0"
],
"publishers": [
"Que"
],
"last_modified": {
"type": "/type/datetime",
"value": "2010-08-18T08:53:00.844526"
},
"key": "/books/OL18096553M",
"publish_places": [
"Indianapolis, Ind"
],
"pagination": "xix, 757 p. :",
"created": {
"type": "/type/datetime",
"value": "2008-10-10T19:27:28.086386"
},
"lccn": [
"99063852"
],
"notes": {
"type": "/type/text",
"value": "\"Red Hat Linux version 6.0.\"--Cover\n\nIncludes index"
},
"identifiers": {
"librarything": [
"261776"
],
"goodreads": [
"3382689"
]
},
"isbn_10": [
"0789721813"
],
"publish_date": "2000"
}
}
}
and here is my code:
class JsonClass {
public static void main(String[] args) throws IOException {
org.jsoup.nodes.Document docKb = Jsoup
.connect("https://openlibrary.org/api/books?bibkeys=ISBN:0789721813&jscmd=details&format=json")
.ignoreContentType(true).get();
String json = docKb.body().text();
String titulo;
Container fullJsonObject = new Gson().fromJson(json, Container.class);
for (Details i : fullJsonObject.details) {
System.out.println("Author: " + i.by_statement);
System.out.println("Title: " + i.title);
System.out.println("Editora: " + i.type.publishers[0]);
System.out.println("Ano de publicação: " + i.type.notes);
}
}
private class Container {
Details[] details;
}
private class Details {
String title;
String by_statement;
Type type;
}
private class Type {
String publishers[];
Notes notes;
}
private class Notes {
String publish_date;
}
}
I tried and it just gives me a java.lang.NullPointerException at this line:
for (Details i : fullJsonObject.details) {
I'm pretty numb, so any answer could help, thx.
Your problem is that you are trying to parse one Container
having an array of Detail
s but the response is actually a Map
having Container.bib_key
as a key and Container
itself as a value so Map<String, Container>
. It seems that this API method is ready to return more than one Container
at a time
Also the Container.details
is not an object but a single value. So changing Container
to:
private class Container {
Details details;
}
and parsing a right kind of an object might give you better results, something like:
java.lang.reflect.Type type =
new TypeToken<Map<String, Container>>(){}.getType();
Map<String, Container> fullJsonObject = new Gson().fromJson(json, type);