Suppose you have two Docker containers (two microservices respectively), each 10 GB RAM, each withstand the load of 1000 rps. At some point you have 1900 rps on the first container (microservice), and 100 rps on the second microservice (container).
Is Docker or Docker Swarm clever enough to not waste 10 GB of RAM allocated for the second microservice (Docker container)?
With no constraints set, Docker simply uses what memory is available to the host. That would satisfy your use case above, with the tradeoff that if one microservice suddenly used 15GB of memory of a total 16GB of memory, Docker would be ok with it. That could cause problems.
To mitigate an OOM situation, you can set hard limits per running container with a --memory flag, or soft limits with a --memory-reservation flag.
See the resource constraints documentation for the full breakdown.