When I ssh into cell_z1. then I can see these routing tables.
$ sudo iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
w--prerouting all -- anywhere anywhere
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
w--prerouting all -- anywhere anywhere
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
w--postrouting all -- anywhere anywhere
Chain w--instance-coiaggg2s3f (1 references)
target prot opt source destination
DNAT tcp -- anywhere cell-z1-0.node.dc1.cf.internal tcp dpt:60036 /* ac4154dd-a2bd-41d8-46bb-c5dfa3c8bfb2 */ to:10.254.0.6:8080
DNAT tcp -- anywhere cell-z1-0.node.dc1.cf.internal tcp dpt:60037 /* ac4154dd-a2bd-41d8-46bb-c5dfa3c8bfb2 */ to:10.254.0.6:2222
Chain w--instance-coiaggg2s3l (1 references)
target prot opt source destination
DNAT tcp -- anywhere cell-z1-0.node.dc1.cf.internal tcp dpt:60040 /* 74ab1082-7eca-4a09-7364-b266a23a7fdf */ to:10.254.0.2:8080
DNAT tcp -- anywhere cell-z1-0.node.dc1.cf.internal tcp dpt:60041 /* 74ab1082-7eca-4a09-7364-b266a23a7fdf */ to:10.254.0.2:2222
Chain w--postrouting (1 references)
target prot opt source destination
MASQUERADE all -- 10.254.0.0/30 !10.254.0.0/30 /* executor-healthcheck-8946f5d6-063c-4bae-474d-0032f72b8fcb */
MASQUERADE all -- 10.254.0.4/30 !10.254.0.4/30 /* ef658bba-214d-4eef-5228-410e8e8aeb69 */
MASQUERADE all -- 10.254.0.8/30 !10.254.0.8/30 /* 3cb958eb-409a-4aa9-48f1-41bb6573ebc6 */
MASQUERADE all -- 10.254.0.12/30 !10.254.0.12/30 /* 9600ee8c-9e63-4682-bed3-b14767ea46d3 */
MASQUERADE all -- 10.254.0.16/30 !10.254.0.16/30 /* executor-healthcheck-eda5cee2-81be-4890-6d67-2a9f108d6dda */
Chain w--prerouting (2 references)
target prot opt source destination
w--instance-coiaggg2s3f all -- anywhere anywhere /* ac4154dd-a2bd-41d8-46bb-c5dfa3c8bfb2 */
w--instance-coiaggg2s3l all -- anywhere anywhere /* 74ab1082-7eca-4a09-7364-b266a23a7fdf */
Question is: What is the perpose of these destinations? When I curl in cell_z1, it returns 301 error. So, I think it's removed.
But, it causes router returns 502 error in some pushed application when router-emitter mapped that application port to 60036, 60037, 60040, 60041
My environment: Host OS : Ubuntu Server 16.10 VirtualBox : 5.0.32
$ bosh -e bosh-lite releases
Using environment '192.168.50.4' as client 'admin'
Name Version Commit Hash
cf 254+dev.1* 80a8305a+
cf-mysql 34.2.0+dev.1* b8dcbe32
cf-rabbitmq 222.15.0+dev.1* 377afa0a+
cf-rabbitmq-test 0.1.7 98720fb8
cflinuxfs2-rootfs 1.60.0* 0b44b228+
diego 1.11.0+dev.1* 4ee830c6
garden-runc 1.4.0* 60f9e9dd
routing 0.147.0 255f268f
~ 0.136.0 d29132da+
UPDATED 4/11/2017
I have found that this information comes from the Kawasaki (Guardian's Network Library). I see route table below. But Unlike 10.254.0.6, The route table does not have virtual NIC and route for 10.254.0.2 (10.254.0.0/30)
$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.244.16.1 0.0.0.0 UG 0 0 0 wcl8gbnff7q4-1
10.244.16.0 * 255.255.255.0 U 0 0 0 wcl8gbnff7q4-1
10.254.0.4 * 255.255.255.252 U 0 0 0 wbrdg-0afe0004
When you deploy an application to CF, it runs in a container on one of your Diego Cells. The container is given an internal port, at the moment this will always be 8080 with Diego, and the Cell publishes an external port (to the GoRouters). The external port is mapped to the internal port with an iptables rule on the Cell. I believe that is what you're seeing / asking about.
In summary, traffic takes a path like this from your browser to the app in the container:
Browser -> HTTP(S) -> Load Balancer -> HTTP(S) -> GoRouter -> (HTTP) -> External Port on Cell -> iptables -> Internal Port in Container -> Application
You might also be wondering about port 2222, this is similar but the port is used for cf ssh traffic into the container.
A layperson should never manually remove or adjust any of the iptables rules on a Diego Cells.