I am using Rank SVM [https://en.wikipedia.org/wiki/Ranking_SVM , http://research.microsoft.com/en-us/people/hangli/l2r.pdf ] which is based on pairwise approach i.e. the model learns by comparing between samples with different grades WITHIN the same query. So, while preparing the train data set which one would be the better approach? -- 200 queries with each one having 10 graded documents or 100 queries with 20 graded documents in each? In both the approaches the number of grading is same.
There is no definite answer for such question as it is heavily data dependent factor. In other words, the only general answer is "you have to check both and select the best". You could for example gather smaller sample and test your hypothesis there.
The only theoretical aspect that could guide you is the fact that ranking SVM is actually transforming your problem into quadratic (in terms of number of relations) size of binary classification problems. So if by "10" or "20 graded documents" you mean that for each query you get 10 or 20 ordered results, then there is more information in 100 queries with 20 documents, as it gives you 100 * (20^2) = 40,000 training samples, while the second approach only 200 * (10^2) = 20,000. However, in practise these 40,000 samples can be less informative than 20,000 (as they cover only half of the documents space, furthermore ordering of 20 documents can be simply useless, as most of them would be "equally bad").