Wouldn't it be closer to:
n * (n - 1) / 2
The above formula is the answer to this middle school Math-team problem:
"You have n people in a room and they all shake hands with everyone else. How many handshakes took place?"
Wouldn't this also apply to the number of people communicating within a software project?
Disclaimer
I haven't read the book (yet), but I've seen the n^2 formula referenced elsewhere.
If you had read the book, you wouldn't have asked the question. Here's what it actually says:
If there are n workers on a project, there are (n^2-n)/2 interfaces across which there may be communication, and there are potentially almost 2^n teams within which coordination must occur.
For those playing the home game, this is in Chapter 7, under the heading Organization in the Large Programming Project.
So the answer is that you are right, but that's what the book says too.