I have a Java desktop application that downloads 700K entries in batches of 50K from a service and stores them in a Derby base
The problem is that this works only if i have set -Xmx1024 otherwise the application crashes with a heap error message. Although the final size of the database is about 100M
Could you please suggest a way to optimize the following code to use less memory
the code for doing the download is pretty much like this
public static final void getItems() {
ItemsRequest req = new ItemsRequest();
Gson g = new Gson();
int next_index_to_read = 0;
int max_read_entry_count = 50000;
req.setLimit(max_read_entry_count);
ItemsResponse resp = null;
do{
resp = g.fromJson(postRequest(g.toJson(req)), ItemsResponse.class);
resp.saveItems();
next_index_to_read += max_read_entry_count;
req.setFrom(next_index_to_read);
}while(resp.getTotalItemsEntryCount()>next_index_to_read);
}
and the code responsible for saving data is
public class ItemsResponse
public void saveItems() {
PersistenceManagerFactory pmf = PMF.getPmf();
PersistenceManager pm = pmf.getPersistenceManager();
Transaction tx = pm.currentTransaction();
try {
tx.begin();
if (data != null) {
for (Item item : data) {
Item item = null;
try {
item = pm.getObjectById(Item.class, item.getItemId());
} catch (Exception e) {
continue;
}
pm.makePersistent(item);
}
}
tx.commit();
} catch (Exception e) {
Logger.getLogger("com.example").error("ItemResponse.saveData", e);
} finally {
if (tx.isActive()) {
tx.rollback();
}
}
pm.close();
}