I wrote a custom serializing/de-serializing logic for persisting some of the data as Java default serialization turned out to be both time and memory expensive. For this purpose I wrote readObject(ObjectInput in) and writeObject(ObjectOutput out) methods for the class(es) that needs persisting. However I noticed that if I do not use any out.writeObject(obj) in writeObject(ObjectOutput out) method then it always throws EOFException.
Consider the following example:
Data.java
public class Data implements BaseData {
private String messageUID;
private String rawData;
private String data;
private Long type;
private Boolean processed = false;
private String processedMessage;
private String processedDetaildMessage;
// getter setter
public void readObject(ObjectInput in) throws IOException, ClassNotFoundException {
messageUID = in.readUTF();
rawData = in.readUTF();
data = in.readUTF();
type = in.readLong();
processed = in.readBoolean();
if (processed) {
processedMessage = in.readUTF();
processedDetaildMessage = in.readUTF();
}
}
public void writeObject(ObjectOutput out) throws IOException {
out.writeUTF(messageUID);
out.writeUTF(rawData);
out.writeUTF(data);
out.writeLong(type);
out.writeBoolean(processed);
if (processed) {
out.writeUTF(processedMessage);
String tempDetailsMessage[] = processedDetaildMessage.split(" more");
out.writeUTF(tempDetailsMessage[tempDetailsMessage.length - 1]);
}
}
However whenever I use above code the out stream is always missing some information at the end (from processedDetaildMessage field) and I get EOFException while reading it form in, stacktrace below (Data.java line 216 is processedDetaildMessage = in.readUTF());
java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2766)
at java.io.ObjectInputStream$BlockDataInputStream.readUTFChar(ObjectInputStream.java:3158)
at java.io.ObjectInputStream$BlockDataInputStream.readUTFBody(ObjectInputStream.java:3055)
at java.io.ObjectInputStream$BlockDataInputStream.readUTF(ObjectInputStream.java:2864)
at java.io.ObjectInputStream.readUTF(ObjectInputStream.java:1072)
at com.smartstream.common.Data.readObject(Data.java:216)
at com.smartstream.common.PerformanceTest.getObjectFromBytes(PerformanceTest.java:168)
at com.smartstream.common.PerformanceTest.access$0(PerformanceTest.java:160)
at com.smartstream.common.PerformanceTest$1.mapRow(PerformanceTest.java:119)
at com.smartstream.common.PerformanceTest$1.mapRow(PerformanceTest.java:1)
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92)
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60)
at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:651)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:639)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:668)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:676)
at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:731)
at com.smartstream.common.PerformanceTest.readFromDb(PerformanceTest.java:109)
at com.smartstream.common.PerformanceTest.main(PerformanceTest.java:66)
so I though I would put some extra byte/s of information at the end after writing all required fields and will not read them so that I don't reach end of file while reading. I tried all of these out.writeByte(-1), out.writeInt(-1), out.writeLong(2342343l), out.writeUTF("END_OF_STREAM") but those make no difference. finally I did this out.writeObject(new String("END_OF_STREAM")) and it works fine. Can someone please explain as to why outputstream misses some information if none of the information is written using writeObject() method. Below is how I read and write to/from streams;
private byte[] getObjectAsBytes(Data data) {
byte[] byteArray = null;
ByteArrayOutputStream bos = null;
ObjectOutputStream oos = null;
try {
bos = new ByteArrayOutputStream();
oos = new ObjectOutputStream(bos);
// Use this for java default serialization
// oos.writeObject(data);
data.writeObject(oos);
byteArray = bos.toByteArray();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (oos != null) {
try {
oos.flush();
oos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return byteArray;
}
private Data getObjectFromBytes(byte[] byteArray) {
Data data = new Data();
ByteArrayInputStream bais = null;
ObjectInputStream ois = null;
try {
bais = new ByteArrayInputStream(byteArray);
ois = new ObjectInputStream(bais);
// Use this for java default serialization
// data = (Data) ois.readObject();
data.readObject(ois);
} catch (IOException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
} finally {
if (ois != null) {
try {
ois.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return data;
}
If anyone is interested below is what is written in the streams;
persisted data with original code (throws EOFException and missing information) (don't confuse the stacktrace with original issue this stacktrace is persisted as field processedDetailedMessage)
¬í---z-------3507319347632941385----FEEDER-----1437052314954 ---This is a random string---N---þ%J---!this is message of processed dataÛ
Caused by: java.sql.SQLException: ORA-01691: unable to extend lob segment TLM_DBO.SYS_LOB0000076335C00008$$ by 8192 in tablespace WIN_SL_TABLE
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:205)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1008)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3449)
at oracle.jdbc.driver.OraclePre
persisted data after writing extra string at the end using writeObject method
¬í---z-------3507319347632941385----FEEDER-----1437052314954 ---This is a random string---N---þ%J---!this is message of processed dataÛ
Caused by: java.sql.SQLException: ORA-01691: unable to extend lob segment TLM_DBO.SYS_LOB0000076335C00008$$ by 8192 in tablespace WIN_SL_TABLE
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:205)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1008)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3449)
at oracle.jdbc.driver.OraclePrz-----NeparedStatement.execute(OraclePreparedStatement.java:3550)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1374)
at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecute(WSJdbcPreparedStatement.java:975)
at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.execute(WSJdbcPreparedStatement.java:642)
at com.smartstream.control.engine.config.dao.jdbc.ProcessExecutionAuditDetailDao$1.doInPreparedStatement(ProcessExecutionAuditDetailDao.java:115)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:586)
... 23t
END_OF_STREAM
PS ---- represents unreadable bytes
Your persisted data is incomplete because you are creating your byte array before flushing the ObjectOutputStream. In getObjectAsBytes(Data) move byteArray = bos.toByteArray(); after the finally block to make it work. Alternatively, the method could be written more succinctly as follows (requires Java 7+):
private byte[] getObjectAsBytes(Data data) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try (ObjectOutputStream oos = new ObjectOutputStream(bos)) {
data.writeObject(oos);
} catch (IOException e) {
e.printStackTrace();
}
return bos.toByteArray();
}
I tested both ways in my own program and they both prevent the EOFException from being thrown.
As far as why having a writeObject was working, that's because the underlying writeObject implementation toggles block data mode at the beginning and ending of the method, and changing the block data mode performs a drain which writes all data to the underlying OutputStream, which for a ByteArrayOutputStream is effectively the same as a flush.