Search code examples
javamemoryhashmap

Preserving memory with two HashMaps that contain similar values


I am loading 2 large datasets into two separate HashMaps, sequentially. (The datasets are serialized into many Record objects, depicted below). The HashMaps are represented like so, with the key as the id of the Record:

Map<Long, Record> recordMapA = new HashMap<>();
Map<Long, Record> recordMapB = new HashMap<>();

The Record object looks like so:

class Record {
  Long id; 
  Long timestamp; 
  String category;
  String location;  
}    

In many cases, the records are the same between the two datasets, except that the timestamp field differs. For my use case, any two Record objects are equal if all field values except for the timestamp field are the same.

// These two records are the same because only the timestamp differs
Record recordA = new Record(54321, 1615270861975L, "foo", "USA"); 
Record recordB = new Record(54321, 1615357219994L, "foo", "USA"); 

To preserve memory, is there a way to make it so that if two Record objects are "equal", both of those map entry values in maps A and B would refer to the same Record object in memory? I've overridden the equals and hashCode methods for the Record object to ignore timestamp, then checked if RecordMapA already contains the same record. If so, I put the record from RecordMapA into RecordMapB instead of putting the new Record that has been serialized from dataset B into Map B. However the impact on memory seems negligible so far.

One side note is that I need to retain both maps (instead of merging them into one) for purposes of comparison later.


Solution

  • If the records are 'small enough' then I would not bother trying anything fancy. For large records, the easiest way seems to be to do what you're doing.

    void addToMap(Long key, Record rec, Map<Long,Record> map, 
                  Map<Long,Record> otherMap) {
        Record existing = otherMap.get(key);
        map.put(key, existing != null ? existing : rec);
    ]
    

    Assumes that if the key is present then the record located by the key must be the same. If not the case, you'll need to check.

    void addToMap(Long key, Record rec, Map<Long,Record> map, 
                  Map<Long,Record> otherMap) {
        Record existing = otherMap.get(key);
        if (existing != null && existing.equals(rec))
             map.put(key, existing);
        else
             map.put(key, rec);
    ]