On a new repository I very often find myself having made changes to the master branch, where of course I should have started with the develop branch.
Of course I should create a feature branch before I start making changes, in which case I'd notice that I'm on master, not develop.
However is there also a setting in git to change the default clone/checkout branch to develop?
The default branch git checks out is determined by the remote repo's HEAD, which is almost always pointed at master. I'm not aware of a local setting that would change clone's default. (You can approximate that behavior with caveats; see below.) Because git can't generally assume a branch of any given name (including master) would exist in an arbitrary remote, I wouldn't expect it to have such a feature; it relies on the remote to tell it what the default branch is (again, via the remote's HEAD).
Checking out master by default is not the most convenient thing for developers in a lot of branching models, but it is a pretty deeply ingrained convention; so my recommendation is to adapt your workflow habits to that, instead of trying to do the opposite.
But what are some options?
For a given clone command, you can use the --branch option to select what is iitially checked out
git clone --branch develop http://some.server/repo.git
Building on that, you could define an alias to clone with the --branch option. (But the alias will fail if used on a repo that doesn't have a branch with your chosen name.)
For repos you control, you could change the HEAD to point to the branch of your choice, but I really recommend you don't for several reasons. First, it may cause confusion if the repo is shared. (Similarly it could complicate the setup of build tools, though that really isn't hard to deal with.) But the biggest reason I wouldn't is, then you're building habits when using your own repos, and those habits will not translate to using others' repos.