New NIC with public IP has no internet connection, while existing NIC works fine

Question

I'm experiencing an issue with a newly added network interface (NIC) on my Azure VM. While the existing NIC works perfectly, the new one fails to connect to the internet. Here are the details:

Current Setup:

• VM created with 'Basic SKU dynamic' network configuration

• Existing NIC has multiple public IPs (Basic SKU, dynamic) and works fine

• Can successfully use curl --interface <existing interface ipv4 address> http://example.com

Problem:

1. Added a new NIC with a new public IP to the VM

   (eth1 is a newly added nic.)
   azureuser@instanceXX:~$ ip addr show
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
       inet 127.0.0.1/8 scope host lo
          valid_lft forever preferred_lft forever
       inet6 ::1/128 scope host
          valid_lft forever preferred_lft forever
   2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
       link/ether 60:45:bd:48:e7:f1 brd ff:ff:ff:ff:ff:ff
       inet 10.0.0.5/24 brd 10.0.0.255 scope global eth0
          valid_lft forever preferred_lft forever
    # ~~~~skipping the middle~~~~ 
       inet6 fe80::6245:bdff:fe48:e7f1/64 scope link
          valid_lft forever preferred_lft forever
   3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
       link/ether 7c:1e:52:2b:a9:ff brd ff:ff:ff:ff:ff:ff
       inet 10.0.0.104/24 metric 200 brd 10.0.0.255 scope global eth1
          valid_lft forever preferred_lft forever
       inet6 fe80::7e1e:52ff:fe2b:a9ff/64 scope link
          valid_lft forever preferred_lft forever
   4: enP12745s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP group default qlen 1000
       link/ether 60:45:bd:48:e7:f1 brd ff:ff:ff:ff:ff:ff
       altname enP12745p0s2
       inet6 fe80::6245:bdff:fe48:e7f1/64 scope link
          valid_lft forever preferred_lft forever
   

   Looking at # route -n I get this output:

   Kernel IP routing table
   Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
   0.0.0.0         10.0.0.1        0.0.0.0         UG    100    0        0 eth0
   10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
   10.0.0.0        0.0.0.0         255.255.255.0   U     200    0        0 eth1
   10.0.0.1        0.0.0.0         255.255.255.255 UH    100    0        0 eth0
   168.63.129.16   10.0.0.1        255.255.255.255 UGH   100    0        0 eth0
   168.63.129.16   0.0.0.0         255.255.255.255 UH    200    0        0 eth1
   169.254.169.254 10.0.0.1        255.255.255.255 UGH   100    0        0 eth0
   

2. Both Primary and Secondary type IPs on the new NIC fail to connect:

   azureuser@instanceXX:~$ curl --interface 10.0.0.104 http://example.com
   curl: (28) Failed to connect to example.com port 80 after 134224 ms: Connection timed out
   
   

3. Existing NIC and its IPs still work correctly

4. Both NICs share the same Network Security Group (NSG), which hasn't been modified since VM creation

Expected Behavior: Two NICs, each with 225 public IPs (Basic SKU, dynamic), all functioning normally.

Additional Notes:

• Not using Standard SKU due to cost constraints

• NSG settings are default from initial VM creation

Has anyone encountered a similar issue or can suggest a solution to get the new NIC working with internet connectivity? Any insights would be greatly appreciated...!!!

Attempted Solutions:

1. Used Azure's Connection troubleshoot and Support + troubleshooting tools (no resolution)

2. Tried creating IPs with different SKUs, but VM failed to boot due to SKU mismatch

3. Created a new VM and replicated the setup, but encountered the same issue

Answer 1

To address the issue of a newly added network interface on your Azure VM not being able to connect to the internet while the existing NIC works fine, follow these steps-

Create a Virtual Network and Subnet

az network vnet create \
  --resource-group arkorg \
  --name myVNet \
  --address-prefix 10.0.0.0/16 \
  --subnet-name mySubnet \
  --subnet-prefix 10.0.0.0/24

Create a Network Security Group (NSG)

az network nsg create \
  --resource-group arkorg \
  --name myNSG

Add Inbound Rules to the NSG for HTTP and SSH

az network nsg rule create \
  --resource-group arkorg \
  --nsg-name myNSG \
  --name AllowInternetInBound \
  --priority 1000 \
  --direction Inbound \
  --access Allow \
  --protocol Tcp \
  --destination-port-range 80 \
  --source-address-prefix Internet \
  --destination-address-prefix '*'

az network nsg rule create \
  --resource-group arkorg \
  --nsg-name myNSG \
  --name AllowSSH \
  --priority 1100 \
  --direction Inbound \
  --access Allow \
  --protocol Tcp \
  --destination-port-range 22 \
  --source-address-prefix Internet \
  --destination-address-prefix '*'

Create Public IP Addresses for the NICs

az network public-ip create \
  --resource-group arkorg \
  --name myExistingPublicIP \
  --sku Basic \
  --allocation-method Dynamic

az network public-ip create \
  --resource-group arkorg \
  --name myNewPublicIP \
  --sku Basic \
  --allocation-method Dynamic

Create Network Interfaces and Associate Them with Public IPs

az network nic create \
  --resource-group arkorg \
  --name myExistingNIC \
  --vnet-name myVNet \
  --subnet mySubnet \
  --network-security-group myNSG \
  --public-ip-address myExistingPublicIP

az network nic create \
  --resource-group arkorg \
  --name myNewNIC \
  --vnet-name myVNet \
  --subnet mySubnet \
  --network-security-group myNSG \
  --public-ip-address myNewPublicIP

Create a VM with your existing NIC

az vm create \
  --resource-group arkorg \
  --name myVM \
  --nics myExistingNIC \
  --image Ubuntu2204 \
  --admin-username azureuser \
  --generate-ssh-keys

Now comes your main problem that is update the VM with new IP and it should be able to connect to the net.

So first deallocate the old one

az vm deallocate \
  --resource-group arkorg \
  --name myVM

Followed by adding of your new NIC and restarting the VM

az vm nic add \
  --resource-group arkorg \
  --vm-name myVM \
  --nics myNewNIC

az vm start \
  --resource-group arkorg \
  --name myVM

If done till here, then you're sorted. Now you just have to SSH into the VM using the public IP of the existing NIC

ssh azureuser@<existing-public-ip>

Run the following commands to set up source-based routing:

sudo su
echo "200 eth0" >> /etc/iproute2/rt_tables
echo "201 eth1" >> /etc/iproute2/rt_tables
ip rule add from 10.0.0.4/32 table eth0
ip rule add from 10.0.0.5/32 table eth1
ip route add 10.0.0.0/24 dev eth0 src 10.0.0.4 table eth0
ip route add default via 10.0.0.1 dev eth0 table eth0
ip route add 10.0.0.0/24 dev eth1 src 10.0.0.5 table eth1
ip route add default via 10.0.0.1 dev eth1 table eth1

Verify the routing rules

ip rule show
ip route show table eth0
ip route show table eth1

Test Connectivity

curl --interface 10.0.0.4 http://example.com
curl --interface 10.0.0.5 http://example.com