The situation:
A (huge) array of elements requires performant transformation into a List, extended by one value. To solve the problem, a reduced implementation of AbstractList was devised and used in a dedicated method, which performs very well.
The Problem:
Simply applying Arrays.asList(String element, String[] elements)
will not work (which would be very fast). Trying any other, seemingly straight-forward approaches will fail, due to serious performance problems.
Consider following examples, of which none will perform well:
protected static <T> List<T> slowAsList(final T last, final T... elements) {
//1
return IntStream.rangeClosed(0, length)
.mapToObj(i -> i < length ? elements[i] : last)
.collect(Collectors.toList());
//2
return Stream.concat(Stream.of(first), Arrays.stream(elements))
.parallel()
.collect(Collectors.toList());
//3
return new ArrayList() {{ for (Object o : elements) add(o); add(last); }};
//4
final ArrayList arrayList = new ArrayList();
arrayList.add(last);
for (Object o : elements) arrayList.add(o);
return arrayList;
//5
final int arrayLen = elements.length + 1;
final Object[] array = Arrays.copyOf(elements, arrayLen);
for (int i = 1; i < arrayLen; i++) {
array[i] = elements[i - 1];
}
array[arrayLen - 1] = last;
return Arrays.asList(array);
}
The method will stall in a JUnit test and not finish in a feasible amount of time:
@Timeout(15)
@Test
public void fastAsListTest() {
final Object[] testStrings1MwithNull = IntStream.rangeClosed(0, 1000000)
.mapToObj(i -> i == 1000000 ? null : "TEST" + i)
.toArray();
slowAsList(testStrings1MwithNull);
}
When replacing the implementation in the method with a variation of the AbstractList
implementation, the test will finish within a few milliseconds, instead:
protected static <T> List<T> fastAsList(final T last, final T... elements) {
if (elements == null) {
return null;
}
return new AbstractList<T>() {
@Override
public int size() {
return elements.length;
}
@Override
public T get(int index) {
return index < 0 ? null : elements[index];
}
};
}
EDIT:
This was the original code, with first
as additional element
protected static List<Object> asList(final Object first, final Object[] elements) {
return new AbstractList<Object>() {
@Override
public int size() {
return elements.length + 1;
// return elements != null ? elements.length + 1 : 0;
}
@Override
public Object get(int index) {
return (index == 0) ? first : elements[index - 1];
// return (index == 0) ? first : elements!= null ? elements[index - 1] : null;
}
};
The commented-out code shows the added null
checks, for nulls causing com.sun.jdi.InvocationException
with elements null
.
EDIT2:
Jet another attempt, this time trying to apply the fast code as inline Collector
in a Stream
approach (with same results):
final int length = elements.length;
return IntStream.rangeClosed(0, length)
.mapToObj(i -> i < length ? elements[i] : null)
.collect(() -> new AbstractList<T>() {
@Override
public int size() {
return elements == null ? null : elements.length;
}
@Override
public T get(int index) {
return index < 0 ? null : elements[index];
}
},
(a,b) -> {return;},
(a,b) -> {return;}
);
What could be causing this issue, might a lack of memory be the problem here?
I've ramped-up Xms/Xmx to 2G/6G for the test and have an overall memory consumption of ~83% (of 16G in total) in Windows. Lowering the values hasn't changed anything, though.
EDIT3:
The only option that seems to provide comparable performance so far - working only, if no additional value is required - is a plain wrapped Arrays.asList()
:
protected static <T> List<T> fastAsList(final T... elements) {
return elements == null ? null : Arrays.asList(elements);
}
My guess is: The return object of AbstractList is a simple Reference instance. The field of return object is your huge array.
And in other words, there is not real copy.