Can I set it so that pull always pulls from upstream and push always pushes to origin unless specified otherwise.
P.S. - I don't want a different push and pull (set-url) for a specific remote but rather different default remotes.
Edit - Check Postscript before reporting it duplicate.
UPDATE - just realized you said default remote
You can set remote.pushDefault
to the remote you want to push to. Branch tracking settings would still determine the default behavior of pull I believe.
Original answer which works per-branch
You can set the config option branch.<name>.remote
to the remote you want to pull from, and set branch.<name>.pushRemote
to the remote you want to push to.