I'm trying to draw opengl 3d terrain , however I started to wonder if there is a huge cpu headache if I have a lot of vertices without drawing any triangles with them.
There can be some overhead, but it should not be huge. A lot of this is highly platform dependent.
GPUs mostly use address spaces that are different from what the CPU uses. To make memory pages accessible to the GPU, the pages have to be mapped into the GPU address space. There is some per-page overhead to create these mappings. Memory pages accessed by the GPU may also have to be pinned/wired to prevent them from being paged off while the GPU is accessing them. Again, there can be some per-page overhead to wire the pages.
As long as the buffer remains mapped, you only pay the price for these operations once, and not for each frame. But if resource limits are reached, either by your application, or in combination with other applications that are also using the GPU, your buffers may be unmapped, and the overhead can become repeated.
If you have enormous buffers, and are typically only using a very small part of them, it may be beneficial to split your geometry into multiple smaller buffers. Of course that's only practical if you can group your vertices so that you will mostly use the vertices from only a small number of buffers for any given frame. There is also overhead for binding each buffer, so having too many buffers is definitely not desirable either.
If the vertices you use for a draw call are in a limited index range, you can also look into using glDrawRangeElements() for drawing. With this call, you provide an index range that can be used by the draw call, which gives the driver a chance to map only part of the buffer instead of the entire buffer.