How do I create a connection to Object Storage on Bluemix in a Data Science Experience Project?

Question

I'm attempting to set up a connection to a Bluemix Object Storage for a project that is different than the default one the project created. This is a problem because:

1) When I go to add a new connection, the Object Storage instance I want to use is not under data service.

2) When I go to add a Softlayer Object Storage, the credentials I'm asked for are (Login URL, Access Key, and Secret Key), but the credentials I have for my instance are ("auth_url":"project":"projectId":"region":"userId":"username":"password":"domainId":"domainName":"role")

3) I have a good interface to a placeholder object storage, but I would like to replace it with the other instance.

Please help me access the data in a different Bluemix Object Storage instance other than the one attached to the project by default.

Answer 1

In addition to what @Sumit Goyal Answered. You need to download the file in local gpfs in order to use apis or libraries that don't support reading from swift object storage or in other words only support reading from local storage/file system.

objStorCred = { "auth_url": "https://identity.open.softlayer.com", "project": "object_storage_XXXXX", "projectId": "XXXXX5a3", "region": "dallas", "userId": "XXXXXX98a15e0", "username": "admin_fXXXXX9", "password": "XXXXX", "domainId": "aXXXX5a", "domainName": "XXXX", "role": "admin" }

from io import StringIO import requests import json import pandas as pd

# @hidden_cell

# This function accesses a file in your Object Storage. The definition contains your credentials.

# You might want to remove those credentials before you share your notebook.

def get_object_storage_file(container, filename):

"""This functions returns a StringIO object containing the file content from Bluemix Object Storage."""

url1 = ''.join(['https://identity.open.softlayer.com', '/v3/auth/tokens'])
data = {'auth': {'identity': {'methods': ['password'],
        'password': {'user': {'name': objStorCred['username'],'domain': {'id': objStorCred['domainId']},
        'password': objStorCred['password']}}}}}
headers1 = {'Content-Type': 'application/json'}
resp1 = requests.post(url=url1, data=json.dumps(data), headers=headers1)
resp1_body = resp1.json()
for e1 in resp1_body['token']['catalog']:
    if(e1['type']=='object-store'):
        for e2 in e1['endpoints']:
                    if(e2['interface']=='public'and e2['region']=='dallas'):
                        url2 = ''.join([e2['url'],'/', container, '/', filename])
s_subject_token = resp1.headers['x-subject-token']
headers2 = {'X-Auth-Token': s_subject_token, 'accept': 'application/json'}
resp2 = requests.get(url=url2, headers=headers2)
return resp2

Note the instead of getting a stringIO object, we get the response object.

Now you can use intermediate local storage to store the .mat file.

Then call this function.

r = get_object_storage_file("containerr1", "example.mat")

with open('example.mat', 'wb') as file:  
file.write(r.content)

Now read the file using h5py. You may need to install h5py using pip install h5py.

import h5py

f = h5py.File('example.mat') f.keys()

Thanks, Charles.