For a java simulation project I have a log file that keeps track of everything that happens during a single run. I do this by adding objects of a custom TimeLog.class to a List. When the simulation is complete I loop through this list and write an ArrayList of strings for each entry, followed by writing it to a csv file using the following "CSVHelper" class which I found online (credits to Keke Chen).
Creating ArrayList of strings:
public static void saveData(List<TimeLog> data, String fileName, Simulator simulator,
ArrayList<String> header) throws Exception {
File csvFile = new File(simulator.getOutputFolder()+fileName);
FileOutputStream fos = new FileOutputStream(csvFile);
Writer fw = new OutputStreamWriter(fos, "UTF-8");
CSVHelper.writeLine(fw, header);
for (TimeLog entry : data) {
ArrayList<String> row = new ArrayList<>();
row.add(String.valueOf(entry.getTime())); //time of entry, double
row.add(String.valueOf(entry.getMessageType())); //type of message (SETUP, DEBUG, EVENT, ERROR)
if (entry.getEvent() == null) {
row.add("");
} else {
row.add(entry.getEvent().getTypeName()); //event type if event
}
row.add(entry.getLog()); //message
CSVHelper.writeLine(fw, row);
}
fw.flush();
fw.close();
}
CSVHelper.writeLine :
/** Class for processing csv file.
* created by Keke Chen (keke.chen@wright.edu)
* For Cloud Computing Labs
* Feb. 2014
*/
public static void writeLine(Writer w, ArrayList<String> values) throws Exception {
boolean firstVal = true;
for (String val : values) {
if (!firstVal) {
w.write(",");
}
w.write("\"");
for (int i=0; i<val.length(); i++) {
char ch = val.charAt(i);
if (ch=='\"') {
w.write("\""); //extra quote
}
w.write(ch);
}
w.write("\"");
firstVal = false;
}
w.write("\n");
}
For debugging purposes, I catch exceptions thrown by the simulator, add information about them to my logging list, write the log file, and then throw the exception again. This way, when a problem occurs I can look in the log file what happened.
Until recently this worked just fine, but now I get a NullPointerException during the 2nd for-loop in the writing process. For some reason the line
for (int i=0; i<val.length(); i++)
throws an exception halfway during a string. If I open the log file that is created, the last entry has a half finished message or timestamp as well (for example: "tool X was res" instead of "tool X was reserved for item Y" or "212" instead of "212312.16").
Is there any limitation to writing csv files, keeping lists in java, using for-loops on characters of strings, or anything else that I am not aware of? This is giving me quiet the headache.
Ciao, Robin
EDIT: As requested, an example of the list entries:
//initiating:
private List<TimeLog> logging = new ArrayList<>();
//message:
public void message(MessageType mt, String s, Event e) {
if (saveLog) {
logFile.add(new TimeLog(now(), mt, s, e)); //now() gets the current time of the simulator
}
}
//example:
Event e = new ReservationEvent(simulator.now(), X, Y);
simulator.message(MessageType.EVENT, "Tool "+X.getId()+" was reserved for item "+Y+getId(), e);
this is the stack trace:
java.lang.NullPointerException
at model.simulator.transport.input.CSVHelper.writeLine(CSVHelper.java:27)
at model.simulator.TimeLog.saveData(TimeLog.java:73)
at model.simulator.Simulator.endSimulation(Simulator.java:98)
at model.simulator.Simulator.runSimulation(Simulator.java:64)
at test.simulator.compare.TestFullFab.testRunning(TestFullFab.java:147)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:539)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:761)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:461)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:207)
Something else that I noticed due to Emax's comment. I added
System.out.println("0:"+row.get(0)+", 1:"+row.get(1)+", 2:"+row.get(2)+", 3:"+row.get(3));
in the saveData method, just before CSVHelper.writeLine(fw,row), to see the list of strings before they get written. This resulted in the full list of logs in the console, so also the ones that did not (or partly) get written to the csv-file. Somehow, the TimeLog for-loop continues while the writeLine is throwing exceptions.
EDIT2: structure of TimeLog:
public class TimeLog {
private double time;
private MessageType mt;
private String log;
private Event e;
public TimeLog(double time, MessageType mt, String log, Event e) {
this.time = time;
this.mt = mt;
this.log = log;
this.e = e;
}
//see full method above
public static void saveData(List<TimeLog> data, String fileName, Simulator simulator,
ArrayList<String> header) throws Exception { ...
//further more, there are some getters
}
An example of the output in the console as a result of the System.out.println() line:
0:27131.490112666303, 1:EVENT, 2:TOOLEVENT, 3:Tool input event, item 3998 has gone into tool QSD301
keep in mind that the 0:,1:,2:, and 3: are not in the original list but added in the println() method.
UPDATE: Using Emax's idea of printing the values where it returns null revealed the problem. The value of val indeed becomes null, and it happens at the NullPointerException (which has no message), so e.getMessage() returns null. Strangly enough, the Writer simply does not write all the values in the csv file, but seems to have some kind of delay (maybe it first collects a default number of characters before actually writing it to the file or something?). Because of this, I thought the NullPointer happened during a loop through val, while it had already finished. Thanks for the help!
The only possible reason that cause a NullPointerException on that line is that val is null since val is taken from a list, the problem bubble up to:
row.add(String.valueOf(entry.getTime()));
row.add(String.valueOf(entry.getMessageType()));
row.add(entry.getEvent().getTypeName());
row.add(entry.getLog());
One of this add is adding a null to the list, you should check for one of this. However post the full stracktrace including (if present) the "Caused By"
Try to modify writeLine like this:
[...]
if (!firstVal) {
w.write(",");
}
w.write("\"");
// EDIT HERE
if (val == null) {
System.out.println("This val is null: " + values);
}
// .....
for (int i = 0; i < val.length(); i++) {
[...]
Post the output with the case that caused the NullPointerException