I recently learned that GPU can do operations as CPU do, although there is tradeoff in the operation latency. Regardless the latency, can a GPU run an OS by its own?
Imagine a computer with no persistent storage, no user input, no networking and no sense of time/date. For this hypothetical computer, there's no way for anything in the real world (e.g. a user) to influence its behavior, and no way to tell if it has an OS or is merely showing a predetermined slide show.
In theory, if you can write an OS for this hypothetical computer, you wouldn't because there's no possible reason to bother. This describes a typical "GPU + video card memory" scenario (where a host OS running on the CPU is the only means of influencing the work done by the GPU).
Now imagine a "GPU" with various devices attached to it (some persistent storage, some networking, something for user input, a graphics card, ...) so that the "GPU" is able to manage (and abstract, and maybe even allow multiple programs to securely share) these devices. In this case, the "GPU" has become a central unit that does processing (a CPU!). It'll probably be a relatively awful CPU (with poor flow control and over-powered SIMD), but it's no longer a GPU.
Can we run OS in a gpu only device?
I don't know. If a tree falls in the woods, but nobody is around to hear it, does it make a sound?