Search code examples
rjupyter-notebookdata-science-experiencernotebook

Loading .RData file into Data Science Experience

I am trying to load a .RData file into my R Notebook in DSX. I have followed the instructions in this notebook (https://apsportal.ibm.com/exchange/public/entry/view/90a34943032a7fde0ced0530d976ca82) but am still unable to load my data. So far, I have been successful in the following steps:

  1. I have loaded my dataset into object storage.
  2. I inserted my credentials using the Insert to code -> Insert Credentials button. This seemed to work as expected.
  3. In the next cell, I chose the Insert to code -> Insert textConnection object option. This seemed to work as expected also.
  4. The output of step # 3 was as follows:

Your data file was loaded into a textConnection object and you can process the data with your package of choice.

data.1 <- getObjectStorageFileWithCredentials_xxxxxxxxxx("projectname", "file.RData")

  1. After this, since my file is a .RData file, I typed the following command:

data <- load("file.RDA")

When I ran this cell, I got the following output:

Warning message in readChar(con, 5L, useBytes = TRUE): “cannot open compressed file 'file.RDA', probable reason 'No such file or directory'”

Error in readChar(con, 5L, useBytes = TRUE): cannot open the connection Traceback:

  1. load("file.RDA")

  2. readChar(con, 5L, useBytes = TRUE)

  3. When I type in the following command to print the dataset:

data

I get the following output:

X.html..h1.Forbidden..h1..p.Access.was.denied.to.this.resource...p...html.

Please can someone help?

Thanks, Venky

Solution

  • Here is a workaround given that load can't read from a response object since to read objects from Object storage, only way is the REST api.

    I tried to use rawConnection instead of textConnection but it seems to be not helping.

    So instead of passing the read object from OS directly to load or readRDS function.You can write it to GPFS of spark service attached and read it from there same as reading from local.

    Change this lines from generated code:-

        rawdata <- content(httr::GET(url = access_url, add_headers ("Content-Type" = "application/json", "X-Auth-Token" = x_subject_token)), as="raw")
rawdata

    Basically instead of returning text , return raw object and then write that as binary object to local GPFS.

    data.3 <- getObjectStorageFileWithCredentials_216c032f3f574763ae975c6a83a0d523("testObjectStorage", "sample.rdata")


writeBin(data.3,"sample.rdata")

    Now read it back using readRDS or load.

    load("sample.rdata")

    To see loaded dataframe. ls()

    I hope it helps.

    Thanks, Charles.