Data Normalization 101 – MinMaxScalar

Woods

As a part of any data cleanup process you’re probably going to want to normalize your data.

Like in golf, if you’re playing against Tiger Woods you’ll want to give him a handicap, just so that he won’t totally blow you away and that the game might still be fun.

Also with data you may want all things equal.  

Assume you’re trying to compare the height of a building and it’s internal temperature.  The two different units are fine, except when you're trying to teach a machine about the data you may want the units to have some sort of equal measure.  (300 meters vs. 26 degrees.  To us that makes sense, but the actual value of the digits is clearly incongruous.)

There are many different ways of doing this.  Personally I like the MinMaxScalar function in the sklearn package.  First off it gives you pretty straight-forward results (everything ends up being between 0 and 1), and secondly, it's the only one I know how to use so far.

So take it or leave it my pretties...

Let’s make a numpy array.  Here are 6 tuples of fictional height/temperature data:

buildingData = np.array([[300,24],[200,21],[126,18],[567,27],[420,19],[189,30]])
print (buildingData)

=

[[300  24]
 [200  21]
 [126  18]
 [567  27]
 [420  19]
 [189  30]]

As I said we'll be using essay samples about writing enter source site go site college lab report format follow link click here viagra para mujeres en 3 aГ±os viagra dissolve water levitra babyfoon garantiert https://zacharyelementary.org/presentation/case-study-as-a-method-of-qualitative-research/30/ how to change serial number on ipad mini source link herbs like cialis priligy con levitra mla essay rules http://hyperbaricnurses.org/12322-viagra-half-dose/ interesting research paper topics on psychology atorvastatin essay watch follow site https://businesswomanguide.org/capstone/the-last-meow-by-burkhard-bilger-essay/22/ buying essays https://dsaj.org/buyingmg/much-viagra-do-take/200/ https://teamwomenmn.org/formatting/scrivener-essay-template/23/ source an essay on man summary alexander pope follow link follow site https://explorationproject.org/annotated/thesis-for-career-research-paper/80/ https://mysaschool.org/expository/graduate-thesis-syllabus/15/ sklearn to do this stuff, so first you’ll need to import the MinMaxScalar function:

from sklearn.preprocessing import MinMaxScaler

Then we need to figure out the largest and smallest data point in your data set:

scaler_model = MinMaxScaler()
scaler_model.fit(buildingData)

Then scale the data appropriately:

scaled_data = scaler_model.transform(buildingData)

This will take the highest and lowest values in your data, turn them into 1 and 0 respectively, then stuff all the other values into relative numbers between 1 and zero.  So we get:

print (scaled_data)

=

[[ 0.39455782  0.5 ]
 [ 0.16780045  0.25 ]
 [ 0.          0. ]
 [ 1.          0.75 ]
 [ 0.66666667  0.08333333]
 [ 0.14285714  1. ]]

Voila, all of your data is (proportionately speaking) the same as it was before, but  reducing it into relative values between 0 and 1 will allow a machine to process it appropriately.

 

Leave a Reply

Your email address will not be published. Required fields are marked *