I have learnt how to implement shared memory as a communication method between two user mode processes. But I am curious about how it can be done for kernel-user mode communication.
I am wondering if the communication between kernel mode and user mode via using shared memory (without using IOCTL at all!!!) can be implemented without creating a system thread.
Update: For example, user mode program or kernel mode driver allocates a space in the memory, and both use this space to communicate with the help of pointers.
One of the problems with using shared memory for communication is that there's no cooperation with the scheduler. Specifically; there's no way for a task to block (so it uses no CPU time) until data arrives/changes and then unblock when data does arrive/change.
For communication between user-space and kernel you'd have the same problem. E.g. user-space code modifies data in the "shared with kernel" memory, and the kernel doesn't know if/when that happened; so (to avoid the kernel wasting CPU time constantly polling the shared memory) user-space code has to use a normal kernel call to say "Hey, look at the shared memory now!", but if you're using a normal kernel call anyway then you could just pass a pointer to the data without using shared memory.
Another problem with using shared memory for communication is security risks. Specifically, a task can see that data arrived/changed, validate the new data to make sure it's acceptable, then malicious attackers can change the data after it was validated, then the task can act on the "validated" data. A very simple example would be something like "if(new_value_in_shared_memory < MAX_VALUE) { myPrivateArray[new_value_in_shared_memory]++;" (where the malicious attacker changes new_value_in_shared_memory after it was checked, tricking the task into modifying something that it shouldn't). For tasks that trust each other (e.g. a process communicating with an fork() of itself) this isn't a problem at all; and when participants don't trust each other (e.g. kernel doesn't trust user-space code) it's major pain (extremely easy to make a mistake and get pawned). The easiest way to guard against this kind of attack is to copy the data from the shared memory into a private buffer, then validate it (knowing that its impossible for attacker to modify the copy), then act on the data in the copy. This copying adds overhead and mostly ruins any performance advantages of shared memory.
For the "user-space communicating with kernel" case there are a few possible alternatives - the kernel can suspend all threads that can access the shared memory during the "validate then use" phase (which would be a performance disaster, especially for multi-CPU systems); and the kernel could do virtual memory management tricks (e.g. set the underlying pages to a "page fault if user-space tries to modify it" state) during the "validate then use" phase (which would be a performance disaster, especially for multi-CPU systems).
Note that the same kind of "modified after validated" security risk occurs for "kernel call accepts pointer to data from user-space" and for "kernel call relies on data from user-space task's stack". However; for both of these cases (which don't involve shared memory but do involve "kernel accesses task's normal memory") typically the kernel doesn't actually access the data and only transfers it. For example, for a write() the kernel might forward the data to file system code (without touching the data itself), which might forward the data to a storage device driver (without touching the data itself), which might transfer the data to a hard drive (without touching the data itself).