When I tried to open and read a certain geo tiff image with gdal and rasterio, I can read and do things like img.meta and img.descriptions. But when I tried to do img.read() with rasterio or img.GetRasterBand(1).ReadAsArray() with gdal the kernel always died after a certain runtime. It's not happening to all geo tiff images but some. Could anyone help me? Thanks!
Python version: 3.9
System: Mac Big Sur Version 11.3.1
File size: 400 MB
Band number: 3
Coordinate reference system: EPSG:26917
Metadata: {'driver': 'GTiff', 'dtype': 'uint8', 'nodata': 255.0, 'width': 580655, 'height': 444631, 'count': 3, 'crs': CRS.from_epsg(26917), 'transform': Affine(0.08000000000000004, 0.0, 607455.9245999996, 0.0, -0.080000000000001, 4859850.802499999)}
Raster description: (None, None, None)
Geotransform : | 0.08, 0.00, 607455.92| | 0.00,-0.08, 4859850.80| | 0.00, 0.00, 1.00|
# with rasterio
img = rasterio.open('certain_tiff_file.tif')
metadata = img.meta
print('Metadata: {metadata}\n'.format(metadata=metadata))
# kernel died if run the line below
full_img = img.read()
# with gdal
img = gdal.Open('certain_tiff_file.tif')
img_band1 = img.GetRasterBand(1)
img_band2 = img.GetRasterBand(2)
img_band3 = img.GetRasterBand(3)
# kernel died if run the line below
array = img_band1.ReadAsArray()
I had the same problem when reading large .tiff files.
Following what @Val said in the comments I checked for how much free RAM memory I had as described here:
import psutil
psutil.virtual_memory()
And indeed my issue was that I was running out of RAM. You may try to use del arr once you're done with some array and you don't need to use that anymore to clean a bit of memory. Might be worth looking into gc.collect as well.