My question is how to change the size of font in seaborn using correlation matrix I don't know why somehow the font is too large for me
Regrettably I don't think that's configurable, but what I would recommend is just to make the figure larger e.g.
f, ax = plt.subplots(figsize=(10, 10))
sns.corrplot(df, ax=ax)
If that's not an option and you're primarily interested in the heatmap (not the numerical values), you could do
sns.corrplot(df, annot=False, sig_stars=False, diag_names=False)