currently want to stream realtime IoT data into OpenStack Swift, but it seems there is no support for doing this - also the API seems to support only atomic pushes.
Does anyone have any other experience or any workaround for that?
Thanks a lot!
Consider Dynamic Large Objects (DLO) and Static Large Object (SLO) manifests. You can use a number of smaller objects to construct a large object via the concept of segmentation. From OpenStack Large Object Support, “Segments of the larger object are uploaded and a special manifest file is created that, when downloaded, sends all the segments concatenated as a single object.” OpenStack Swift Object Storage does not support appending to a object per se, however it does provide facility for concatenation of existing objects dynamically and statically.
For example, you can define a filename pattern rawdata_ and establish a DLO manifest within your Object Storage container. Whenever you issue a get for this manifest, OpenStack swift will automatically concatenate all objects matching the defined filename pattern and return it as a single result. This manifest object can also be loaded via the Bluemix Spark service notebooks (e.g. swift://...) to facilitate analytics on the data.
While manifests are primarily used to support > 5 Gb objects, I've found them quite useful to slice and dice objects to construct different amounts of raw data or to handle batch processes that continually add raw data to a given container.
I've written a blog post on Object Storage manifests for the Bluemix service @ http://blog.ibmjstart.net/2016/04/14/e-pluribus-unum-creating-openstack-manifest-objects-in-ibm-bluemix-object-storage/ that may also help shed more light on their usefulness.
Hope this helps and good luck.