I'm trying to understand why the category of sets is defined the way it is, with singleton sets as terminal objects. If the "Set" category contains all of the possible sets, and all of the possible morphisms between those sets, why wouldn't there be injective, non-surjective morphisms from the singleton sets to all other sets with infinite cardinality? In this case, there wouldn't be any terminal objects.
So what is the rule that leads it to being defined the way it is defined, rather than being defined with infinite sets and morphisms. I guess that it has something to do with it being a "concrete" category. But I don't see how it's so obvious.
A terminal object in a category by definition is an object T with the property that for any object C in the category, there is exactly one arrow C -> T. In the category of sets, a singleton object, for example {1} has this property:
For any set S there is a function f:S -> {1} defined by setting f(s)=1 for any element s of S. There can't be another function g:S -> {1}, because for some element s of S, g(s) would have to be an element t of {1} other than 1, but there is no such element t. So there is exactly one function from S to {1} for each object S of the category.
Your remark about there being many functions from {1} to a set S is irrelevant; they go in the wrong direction to be relevant to the definition of terminal object.