Search code examples
pythonazuredatasetdatastoreazure-machine-learning-service

How do i extract saved numpy array from my Azure DataStore?


I have a numpy array which is saved in my azure datastore. I have all the necessary credentials. However, when it comes to extracting numpy arrays, Azure has not provided any methods for their retrieval. I have tried this code but it is unable to reach the correct location.

If i go to the correct location, the data is shown stored in an https format which doesnt make sense. Please advise

path=str(self.datastore)+str(self.train_data_outputs_folder) + '/' + 'w2v_emb'+ '_' + str(self.date_range) + '.npy'
        print("The new path is:",path)
        ds=np.DataSource()
        # f=ds.open(path=(self.datastore,
        #           self.train_data_outputs_folder + '/' + 'w2v_emb'+ '_' + self.date_range + '.npy'))
        f=ds.open(path)
        dat=np.fromfile(f)


Solution

  • You can download a numpy array from_files method of dataset similar to shown below

     def file_download(path):
                dataset = Dataset.File.from_files(path=(self.datastore,self.train_data_outputs_folder+"/"+ path))
                datafolder = os.path.join(os.getcwd(), 'numpy_file')
                dataset.download(datafolder, overwrite=True)
    

    the essence is that you got to download the files to your local repo. you can use that using the download method.