I have an open question I would like to ask before starting development of functionality, only to learn more about the subject afterwards, just to start all over again ;) I tried searching, but am not really sure what to search for. Therefor, I would like to pick your brains a bit here...
Here is the current situation: I have an app that stores activity/event logs. In a simplified example: "Person A filled a glass with 100ml of water"
This is stored as a record in a single table.
Now, for the requirement of the new functionality: I connect an external storage device via USB to my tablet. On this storage device there are several text files. Each file contains parameters of an activity/event, similar to the one described above, but with different data.
I would like to create a function to import these files, read some of their content into the database as new records, and then store the file somewhere on the tablet, with a link to the file as a field in the DB. Ideally I would have liked to store the contents of the entire text file in a field, or a field in a related 1:1 table, but these files could be as big as 1MB (1MB would be very rare, but possible. Avg is around 100KB).
So, my questions are:
Thanks in advance, all replies are much appreciated!
No one wants to take the chance? :)
For future readers with the same problem: It is possible to read a file from a usb connected device. Keep in mind that reading the file directly from the file system is different than reading it as an attachment in a mail for instance. Not much more than a different path though ;)
I ended up reading the file into a DB, splitting the lines across a record. In other words, not adding the whole file (or even a complete line) in a field, but rather parsing the file into a single record, split up into various fields.
Two things I came across that might help you. Reading a file is pretty fast, but do not make the mistake I made by writing these lines to the database one by one. Better to add them all to a transaction, and commit at once. The other, you can use a read buffer to read the file, instead of the default inputstream. This also speeds up reading of large files.