I am using the below code to receive Tweets from Twitter4j Search API in the form of JSON response. I am receiving the result in the form of List as specified in Twitter4j search API in the line
List<Status> tweets = result.getTweets();
The problem is that the tweets returned as List where one Status entry is having non-empty and non-null GeoLocation whereas another Status entry is having a null or empty GeoLocation. Since to retrieve the relevant fields from each Status entry (i.e. Tweet), I iterate over the List and call getters which is throwing me null for the Status entries where the GeoLocation field is null.
The approach I tried to follow:
I created a POJO TweetJSON_2 (defined at the bottom of the post) with the relevant fields and their getters and setters. I am using Jackson ObjectMapper to handle null values like below:
JsonGenerator generator = new JsonFactory().createGenerator(os);
generator.setPrettyPrinter(new DefaultPrettyPrinter());
TweetJSON_2 rawJSON;
ObjectMapper mapper = new ObjectMapper();
//mapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY);
mapper.configure(SerializationFeature.WRITE_NULL_MAP_VALUES, false);
mapper.setSerializationInclusion(Include.NON_NULL);
// ... rawJSON is populated ...
mapper.writeValue(generator, rawJSON);
However, when I am trying to get the geoLocation field from Status, using the below line which is marked with **
List<Status> tweets = result.getTweets();
I am getting the Java NullPointerException as follows:
[Mon Apr 20 11:32:47 IST 2015]{"statuses":[{"retweeted_status":{"contributors":null,"text":"<my text>",**"geo":null**,"retweeted":false,"in_reply_to_screen_name":null,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[],"hashtags": ... &include_entities=1","since_id_str":"0","completed_in":0.029}}
**Exception in thread "main" java.lang.NullPointerException
at analytics.search.twitter.SearchFieldsTweetsJSON_2.main(SearchFieldsTweetsJSON_2.java:78)**
String s = "{\"first\": 123, \"second\": [{\"second_first\":null, \"second_second\":null}, {\"second_third\":null}, null], \"third\": 789, \"fourth\":null}";
"{\"first\": 123, \"third\": 789}";
I tried the Object Model parsing mechanism which is a javax.json.stream.JsonParser.Event based method but would need multiple times of access and object replacement on the JSON String depending on at what level the null is nested making this approach very complicated. At the same time if I use Tree Model mechanism, the entire JSON response would have to be stored as a Tree which may overflow my JVM heap memory because the JSON size can be pretty large based on my query parameters. I need to find a workable solution to overcome this problem. Any suggestions on solving the above discussed problem will be highly appreciated.
The code is as follows:
public class SearchFieldsTweetsJSON_2 {
/* Searches specific fields from Tweets in JSON format */
public static void main(String[] args) throws FileNotFoundException, IOException {
if (args.length < 2) {
System.out.println("java twitter4j.examples.search.SearchTweets [query][outputJSONFile]");
System.exit(-1);
}
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("XXXXXXXXXXXXXXXXXXXXXXXXXX")
.setOAuthConsumerSecret("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
.setOAuthAccessToken("NNNNNNNNN-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
.setOAuthAccessTokenSecret("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
.setJSONStoreEnabled(true);
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
try {
Query query = new Query(args[0]);
QueryResult result;
File jsonFile = new File(args[1]);
System.out.println("File Path : " + jsonFile.getAbsolutePath());
OutputStreamWriter os = new OutputStreamWriter(new FileOutputStream(jsonFile));
JsonGenerator generator = new JsonFactory().createGenerator(os);
generator.setPrettyPrinter(new DefaultPrettyPrinter());
TweetJSON_2 rawJSON;
ObjectMapper mapper = new ObjectMapper();
//mapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY);
mapper.configure(SerializationFeature.WRITE_NULL_MAP_VALUES, false);
mapper.setSerializationInclusion(Include.NON_NULL);
do {
result = twitter.search(query);
List<Status> tweets = result.getTweets();
for (Status tweet : tweets) {
rawJSON = new TweetJSON_2();
rawJSON.setStatusId(Long.toString(tweet.getId()));
rawJSON.setUserId(Long.toString(tweet.getUser().getId()));
rawJSON.setUserName(tweet.getUser().getScreenName());
rawJSON.setStatusText(tweet.getText());
rawJSON.setGeoLocation(tweet.getGeoLocation().toString()); **<< Giving error at tweet.getGeoLocation() since GeoLocation is null**
mapper.writeValue(generator, rawJSON);
System.out.println(rawJSON.toString());
}
} while ((query = result.nextQuery()) != null);
generator.close();
System.out.println(os.toString());
} catch (TwitterException te) {
te.printStackTrace();
System.out.println("Failed to search tweets : " + te.getMessage());
System.exit(-1);
}
}
}
I have defined my TweetJSON_2 Java object as follows:
public class TweetJSON_2 {
public String statusId;
public String statusText;
public String userId;
public String userName;
public String geoLocation;
public String getStatusId() {
return statusId;
}
public void setStatusId(String statusId) {
this.statusId = statusId;
}
public String getStatusText() {
return statusText;
}
public void setStatusText(String statusText) {
this.statusText = statusText;
}
public String getUserId() {
return userId;
}
public void setUserId(String userId) {
this.userId = userId;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public String getGeoLocation() {
return geoLocation;
}
public void setGeoLocation(String geoLocation) {
this.geoLocation = geoLocation;
}
@Override
public String toString() {
return "TweetJSON_2 [ statusId = " + statusId + ", statusText = " + statusText + "]";
}
}
I have tried with reconfiguring my POJO in the below way and it successfully replaced all the nulls as specified in the setter methods. Didn't need to follow either Tree or Event-based model parsing of JSON string. HTH
The modified TweetJSON_2 POJO:
public class TweetJSON_2 {
public Long statusId = null;
public String statusText = null;
public Long userId = null;
public String userName = null;
public GeoLocation geoLocation = null;
public Long getStatusId() {
if (this.statusId==null)
return new Long(0L);
return statusId;
}
public void setStatusId(Long statusId) {
if (statusId==null)
this.statusId = new Long(0L);
else
this.statusId = statusId;
}
public String getStatusText() {
if (this.statusText==null)
return new String("");
return statusText;
}
public void setStatusText(String statusText) {
if (statusText==null)
this.statusText = new String("");
else
this.statusText = statusText;
}
public Long getUserId() {
if (this.userId==null)
return new Long(0L);
return userId;
}
public void setUserId(Long userId) {
if (userId==null)
this.userId = new Long(0L);
else
this.userId = userId;
}
public String getUserName() {
if (this.userName==null)
return new String("");
return userName;
}
public void setUserName(String userName) {
if (userName==null)
this.userName = new String("");
else
this.userName = userName;
}
public GeoLocation getGeoLocation() {
if (this.geoLocation==null)
return new GeoLocation(0.0,0.0);
return geoLocation;
}
public void setGeoLocation(GeoLocation geoLocation) {
if (geoLocation==null)
this.geoLocation = new GeoLocation(0.0,0.0);
else
this.geoLocation = geoLocation;
}
@Override
public String toString() {
return "TweetJSON_2 [ statusId = " + statusId + ", statusText = " + statusText + "]";
}
}