The sqldf::read.csv.sql()
function has been useful retrieving only a small portion of a large CSV.
However, the connection remains open and eventually produces the following warnings (after running it a few times):
Warning messages:
closing unused connection 11 (C:\Users\wibeasley\AppData\Local\Temp\asfasdf\fileasdfasdfasdf)
Four years ago, it was recommended to issue
base::closeAllConnections()
.
Is there a newer way to selectively close only the connection created by sqldf::read.csv.sql()
?
path <- tempfile()
write.csv(mtcars, file=path, row.names=F)
# read.csv(path)
ds <- sqldf::read.csv.sql(path, "SELECT * FROM file", eol="\n")
base::closeAllConnections() # I'd like to be more selective than 'All'.
unlink(path)
The real code is the middle two lines. The first three lines set up the pretend file. The final base::unlink()
deletes the temp CSV.
My attempts to pass an existing file connection (so I can later explicitly close it) apparently still leave the connection open when I run it several times:
Warning messages:
1: In .Internal(sys.call(which)) : closing unused connection 13 ()
path <- tempfile()
write.csv(mtcars, file=path, row.names=F)
ff <- base::file(path) # Create an explicit connection.
ds <- sqldf::read.csv.sql(sql="SELECT * FROM ff")
base::close(ff)
unlink(path)
Both of my OP's snippets still produce warnings about the connections. This way avoids them.
path_db <- tempfile(fileext = ".sqlite3")
path_csv <- tempfile(fileext = ".csv")
write.csv(mtcars, file=path_csv, row.names = F)
# read.csv(path_csv) # Peek at the results.
db <- DBI::dbConnect(RSQLite::SQLite(), dbname = path_db)
# DBI::dbExecute(db, "DROP TABLE if exists car;") # If desired
RSQLite::dbWriteTable(db, name = "car", value = path_csv)
ds <- DBI::dbGetQuery(db, "SELECT * FROM car")
str(ds)
#> 'data.frame': 32 obs. of 11 variables:
#> $ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
#> $ cyl : int 6 6 4 6 8 6 8 4 4 6 ...
#> $ disp: num 160 160 108 258 360 ...
#> $ hp : int 110 110 93 110 175 105 245 62 95 123 ...
#> $ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
#> $ wt : num 2.62 2.88 2.32 3.21 3.44 ...
#> $ qsec: num 16.5 17 18.6 19.4 17 ...
#> $ vs : int 0 0 1 1 0 1 0 1 1 1 ...
#> $ am : int 1 1 1 0 0 0 0 0 0 0 ...
#> $ gear: int 4 4 4 3 3 3 3 4 4 4 ...
#> $ carb: int 4 4 1 1 2 1 4 2 2 4 ...
DBI::dbDisconnect(db)
unlink(path_db)
unlink(path_csv)
Created on 2022-03-23 by the reprex package (v2.0.1)