Search code examples
pythonclassificationrandom-forestimage-classification

Random forest for classifying, cannot reshape array of size 16884608 into shape (4221152,3)


I am applying one script using random forest for object detection on optical image. I use tif as image format. I use this script, below, but I have always te same error, and I don't know as resolved it. The original script is here link.

from __future__ import print_function, division
from osgeo import gdal, gdal_array
import numpy as np
import matplotlib.pyplot as plt
gdal.UseExceptions()
gdal.AllRegister()

# Read in our image and ROI image
img_RS = 'image_satellite.TIF'
roi_ds = gdal.Open('landuse_roi.tif')

# load image data
img_ds = gdal.Open(img_RS, gdal.GA_ReadOnly)
img = np.zeros((img_ds.RasterYSize, img_ds.RasterXSize, img_ds.RasterCount),
               gdal_array.GDALTypeCodeToNumericTypeCode(img_ds.GetRasterBand(1).DataType))
for b in range(img.shape[2]):
    img[:, :, b] = img_ds.GetRasterBand(b + 1).ReadAsArray()
    
roi = roi_ds.GetRasterBand(1).ReadAsArray().astype(np.uint8)

The raster dataset features by 0 and 1, whereas 0 identifies no data and 1 correct data. Before the raster was one shapefile where 1 identifies polygons that they have to classified. And here it is not clear, why there are 3 labels, infact the result is 0, 1 and 3, when I have only two classes.

labels = np.unique(roi[roi > 0])

We will need a "X" matrix containing our features, and a "y" array containing our labels These will have n_samples rows. In other languages would need to allocate these and them loop to fill them, but NumPy can be faster.

X = img[roi >= 0, :]  # include 8th band, which is Fmask, for now
y = roi[roi >= 0]
# Mask out clouds, cloud shadows, and snow using Fmask
clear = X[:, 1] <= 4
X = X[clear, :4]  # we can ditch the Fmask band now
y = y[clear]
from sklearn.ensemble import RandomForestClassifier
rf = RandomForestClassifier(n_estimators=100, oob_score=True, max_leaf_nodes=20, verbose=1, n_jobs=-1)
X = np.nan_to_num(X)
rf2 = rf.fit(X, y)
df = pd.DataFrame()
df['truth'] = y
df['predict'] = rf.predict(X)

Take our full image, ignore the Fmask band, and reshape into long 2d array (nrow * ncol, nband) for classification

new_shape = (img.shape[0] * img.shape[1], img.shape[2] - 1)
img_as_array = img[:, :, :np.int_(img.shape[2])].reshape(new_shape)

Here I have this error: ValueError: cannot reshape array of size 16884608 into shape (4221152,3)


Solution

  • Reading through the docs of np.reshape(), it says that

    the shape should be compatible with the original shape.

    As I understand it, it means that if you have an array of 34 (12 values) the new shape should also have 9 values, so you can respahe a 34 to a 121 or a 43 or a 62, but you cannot reshape into a 33 (it has 9 values, you'd lose info).

    In your case, the error says

    cannot reshape array of size 16884608 into shape (4221152,3)

    If we do the math: 1688468 is not equal in number to 4221152*3 = 12663456

    In the docs it says that one shape dimension can be -1 (and the function will fill that up with what it needs to be)

    One shape dimension can be -1. In this case, the value is inferred from the length of the array and remaining dimensions.

    Maybe you can use that to solve the problem