Search code examples
pythoncsvnormalization

How Normalize Data Mining in Python with library


How Normalize Data Mining MinMax from csv in Python 3 with library this is example of my data

RT      NK    NB    SU    SK    P    TNI IK   IB     TARGET
84876   902  1192  2098  3623  169   39  133  1063   94095
79194   902  1050  2109  3606  153   39  133   806   87992
75836   902  1060  1905  3166  161   39  133   785   83987
75571   902   112  1878  3190  158   39  133   635   82618
83797  1156   134  1900  3518  218   39  133   709   91604
91648  1291   127  2225  3596  249   39  133   659   99967
79063  1346   107  1844  3428  247   39  133   591   86798
84357  1018   122  2152  3456  168   39  133   628   92073
90045   954   110  2044  3638  174   39  133   734   97871
83318   885   198  1872  3691  173   39  133   778   91087
93300  1044   181  2077  4014  216   39  133   635  101639
88370  1831   415  2074  4323  301   39  133   502   97988
91560  1955   377  2015  4153  349   39  223   686  101357
85746  1791   314  1931  3878  297   39  215   449   94660
93855  1891   344  2064  3947  287   39  162   869  103458
97403  1946   382  1937  4029  289   39  122  1164  107311

the formula MinMax is

= (data-min)/(max-min)*0.8+0.1

i got the code but the normalize data is not each column

I know how to count it like this

(first data of RT - min column RT data) / (max column RT- min column RT) * 0.8 + 0.1, etc

so does the next column

(first data of NK - min column NK data) / (max column NK- min column NK) * 0.8 + 0.1

like this please help me

this is my code, but i don't understand

from sklearn.preprocessing import Normalizer
from pandas import read_csv
from numpy import set_printoptions
import pandas as pd

#df1=pd.read_csv("dataset.csv")
#print(df1)

namaFile = 'dataset.csv'
nama = ['rt', 'niagak', 'niagab', 'sosum', 'soskhus', 'p', 'tni', 'ik', 'ib', 'TARGET']
dataFrame = read_csv(namaFile, names=nama)
array = dataFrame.values

#membagi array
X = array[:,0:10]
Y = array[:,9]

skala = Normalizer().fit(X)
normalisasiX = skala.transform(X)

#data hasil
print('Normalisasi Data')
set_printoptions(precision = 3)
print(normalisasiX[0:5,:])

the results of manual counting with code are very different


Solution

  • we can use pandas python library.

    import pandas as pd
    
    df = pd.read_csv("filename")
    
    norm = (df - df.min()) / (df.max() - df.min() )*0.8 + 0.1
    

    norm will have the normalised dataframe