Search code examples
rmapshaper

ms_simplify from rmapshaper creates corrupted geojson file

I am trying to use rmapshaper ms_simplify function, using the system version as I am working with a large shapefile. The first steps of the simplification seem to work well but then I get an error that a temporary file created by ms_simplify is corrupted.

> simplif_coo2 <- ms_simplify(raw, keep = 0.5, keep_shapes = TRUE, sys=T, sys_mem=6, no_repair=T)
Allocating 6 GB of heap memory
[i] Snapped 200 points
[o] Wrote C:\TEMP\RtmpkFv92L\file25f818895f48.geojson
Error: Cannot open "C:\TEMP\RtmpkFv92L\file25f818895f48.geojson"; The source could be corrupt or not supported. See `st_drivers()` for a list of supported formats.

Indeed I cannot open the file saved in this directory. I tried changing temporary directory and it had no effect. I have the feeling that the problem comes from the function (since it's a file created by the function that cannot open) but maybe I forgot to install a package or something else? Any idea? If you need to reproduce the error, the shapefile I'm trying to simplify is stored here: https://github.com/victorcazalis/RedList_countries/releases/tag/v2022.1

More info on my session (following Andy's questions). I created raw as a shapefile (I also tried as geojson with sf_geojson(raw) but this was not any better): raw<-st_read("RedList_Countries_V2022.1.shp")

sessionInfo() gives:

R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.utf8  LC_CTYPE=French_France.utf8    LC_MONETARY=French_France.utf8 LC_NUMERIC=C                  
[5] LC_TIME=French_France.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] dplyr_1.1.0      sf_1.0-9         rmapshaper_0.4.6

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.10        rstudioapi_0.14    magrittr_2.0.3     units_0.8-1        tidyselect_1.2.0   jsonvalidate_1.3.2
 [7] R6_2.5.1           rlang_1.0.6        fansi_1.0.4        tools_4.2.2        grid_4.2.2         utf8_1.2.2        
[13] KernSmooth_2.23-20 cli_3.6.0          e1071_1.7-12       DBI_1.1.3          class_7.3-20       tibble_3.1.8      
[19] lifecycle_1.0.3    httpcode_0.3.0     vctrs_0.5.2        curl_5.0.0         crul_1.3           glue_1.6.2        
[25] V8_4.2.2           proxy_0.4-27       pillar_1.8.1       compiler_4.2.2     generics_0.1.3     classInt_0.4-8    
[31] geojsonlint_0.4.0  jsonlite_1.8.4     pkgconfig_2.0.3

I installed mapshaper from the console with npm and check_sys_mapshaper() gives:

mapshaper version 0.6.24 is installed and on your PATH
                                            mapshaper-xl 
"C:\\Users\\Victor\\AppData\\Roaming\\npm\\MAPSHA~3.CMD"
Solution

  • The problem actually came from a limit of size set to geoJSON files. By default files of >200MB are not read and the error "the file is corrupted" is returned. The solution was thus to increase that limit by using:

    Sys.setenv(OGR_GEOJSON_MAX_OBJ_SIZE=900)

    It may be worth adding a note in the help section? Thanks for your help Andy!