I have data with 5 different fields (combination of ints, strings, and large strings) and I'd like to hold it in some sort of flat file container. I have tens of thousands of such entries but I don't have a need for any sort of database (at all, just need to iterate through data, no need to query). All the formats I've examined (XML, JSON, YAML) all require redundant fields for each entry even though my data is structured and homogenous. Something like CSV would be great except I can't use commas or new lines as delimiters. Are there any formats you'd reccomend?
Example of data format:
id | epoch | short string | url | large description
You could use a JSON array instead of an object. This way you limit the noise to a minimum. It could be a single array or an array of arrays depending on the format of your data.
It would be less verbose than XML. Don't know for YAML.
For example, you could have:
[
[123, 123456789, "short string", "http://url", "large ... description"],
[123, 123456789, "short string", "http://url", "large ... description"],
[123, 123456789, "short string", "http://url", "large ... description"],
[123, 123456789, "short string", "http://url", "large ... description"]
]