[ upstream commit 6ce55626e436c2831892941801143cda03c6e754 ]

Currently the tunnel device MTU will not be set. If the underlying device
is running some other mtu size this creates a case where mtu on tunnel
device is not in-sync. For example on system with 9000 mtu,

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:01:02:03:04:05 brd ff:ff:ff:ff:ff:ff
7: cilium_net@cilium_host: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 9001 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 56:92:81:65:04:b6 brd ff:ff:ff:ff:ff:ff
8: cilium_host@cilium_net: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 9001 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether f6:9f:7e:fd:be:4a brd ff:ff:ff:ff:ff:ff
9: cilium_vxlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether de:d3:72:2a:69:8a brd ff:ff:ff:ff:ff:ff

Or going the other way with network facing interface <1500 my GKE setup
is the following,

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 42:01:0a:8a:00:19 brd ff:ff:ff:ff:ff:ff
13: cilium_net@cilium_host: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1460 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 82:80:8c:7a:36:4a brd ff:ff:ff:ff:ff:ff
14: cilium_host@cilium_net: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1460 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether ca:98:07:56:a1:2f brd ff:ff:ff:ff:ff:ff
15: cilium_vxlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 52:a7:67:e4:53:e4 brd ff:ff:ff:ff:ff:ff

Sending 1500 MTU on a 1460 interface doesn't make sense. But the routes
configured in the main table will help,

$ kubectl exec -ti -n cilium cilium-cwcbc -- ip r s
default via 10.138.0.1 dev eth0 proto dhcp src 10.138.0.25 metric 100
10.36.0.0/24 via 10.36.1.3 dev cilium_host src 10.36.1.3 mtu 1333
10.36.1.0/24 via 10.36.1.3 dev cilium_host src 10.36.1.3 mtu 1333
10.36.1.3 dev cilium_host scope link
10.138.0.1 dev eth0 proto dhcp scope link src 10.138.0.25 metric 100
169.254.123.0/24 dev docker0 proto kernel scope link src 169.254.123.1 linkdown

Still it would be best if the tunnel interface uses the same mtu size as
cilium_{host|net}.

Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Maciej Kwiek <maciej@isovalent.com>