When i get large JSON from REST service i receive java.lang.OutOfMemoryError: Java heap space...
This is occurs after ~1000 calls the REST service
HttpEntity<ProductRequest> requestUpdate = new HttpEntity<>(request, httpHeaders);
ResponseEntity<String> messageEntity = restTemplate.exchange(getEndPointAddress(), HttpMethod.POST, requestUpdate, String.class);
Map<String, Object> map = gson.fromJson(Helper.cleanJson(messageEntity.getBody()), Map.class);
<pre>
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:2694) ~[?:1.7.0_181]
at java.lang.String.<init>(String.java:203) ~[?:1.7.0_181]
at java.lang.StringBuilder.toString(StringBuilder.java:405) ~[?:1.7.0_181]
at org.springframework.util.StreamUtils.copyToString(StreamUtils.java:80) ~[stormjar.jar:4.0.3]
at org.springframework.http.converter.StringHttpMessageConverter.readInternal(StringHttpMessageConverter.java:86) ~[stonjar.jar:4.0.3]
at org.springframework.http.converter.StringHttpMessageConverter.readInternal(StringHttpMessageConverter.java:41) ~[stonjar.jar:4.0.3]
at org.springframework.http.converter.AbstractHttpMessageConverter.read(AbstractHttpMessageConverter.java:193) ~[stonjar.jar:4.0.3]
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:104) ~[stonjar.jar:4.0.3]
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:884) ~[stonjar.jar:4.0.3]
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:868) ~[stonjar.jar:4.0.3]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:622) ~[stonjar.jar:4.0.3]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580) ~[stonjar.jar:4.0.3]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498) ~[stonjar.jar:4.0.3]
at com.myorg.wcf.PMiddle.retrieveP(PMiddle.java:95) ~[ssar.jar:4.0.3]
at com.myorg.RetrieveP.getPJsonFromService(RetrieveP.java:188) ~[ssar.jar:4.0.3]
at com.myorg.RetrieveP.execute(RetrieveP.java:107) ~[ssar.jar:4.0.3]
at org.apache.storm.daemon.executor$fn__5044$tuple_action_fn__5046.invoke(executor.clj:727) ~[storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.daemon.executor$mk_task_receiver$fn__4965.invoke(executor.clj:459) ~[storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.disruptor$clojure_handler$reify__4480.onEvent(disruptor.clj:40) ~[ss-core-1.1.0.jar:1.1.0]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) ~[ss-core-1.1.0.jar:1.1.0]
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) ~[ss-core-1.1.0.jar:1.1.0]
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) ~[ss-core-1.1.0.jar:1.1.0]
at org.apache.storm.daemon.executor$fn__5044$fn__5057$fn__5110.invoke(executor.clj:846) ~[ss-core-1.1.0.jar:1.1.0]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) [ss-core-1.1.0.jar:1.1.0]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.7.0_181]</pre>
RestTemplate and underlying Spring beans are not designed for stream processing, they build the entire ResponsEntity in the memory. Based on your example you are not using RestTemplate as intended. You are reading String instead of delegating response object Map<String, Object> deserialization to underlying converter bean.
If the JSON response is really too big to fit into your heap you can replace RestTemplate with URLConnection or any other HTTP client library that will let you access the response InputStream directly. This will allow you to manually read and parse the response e.g. by using com.google.gson.stream.JsonReader:
URLConnection conn = // create and open
JsonReader reader = new JsonReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
// read manually using reader
You will be able to save memory that way by avoiding the one big String object that holds the entire JSON body.