VLAN interface stops working when exiting promiscuous mode

[davidebeatrici]

Describe the bug

My ISP provides internet access through IPoE (no PPPoE) on an 802.1q VLAN interface.

It also allows me to establish a BGP session on another VLAN, inside the internet one.

Through SSH, I launched tcpdump on the child interface to monitor the BGP traffic, stopped it and then the router became inaccessible from the outside right after the graceful exit output was printed by the tool.

I quickly found out the internet VLAN was not working anymore, even disabling and enabling it again doesn't have an effect. Rebooting OPNsense appears to be the only fix.

According to dmesg, all interfaces are set to promiscuous mode with the exception of the ISP VLANs, presumably because they're not part of a bridge unlike all the others.

When running tcpdump they're both set to promiscuous mode and still work fine, but as soon as they return to "normal" mode the traffic on them just stops flowing.

To Reproduce

Steps to reproduce the behavior:

1. Create a VLAN interface and force its type to 802.1q (didn't check whether that affects the outcome).
2. Create a VLAN interface with the one created previously as parent.
3. Configure an IP address on the parent VLAN and monitor the traffic from another machine.
4. Run tcpdump on the child interface, exit the tool and the traffic on the parent VLAN should stop flowing immediately.

Environment

OPNsense 24.7.12_4-amd64
FreeBSD 14.1-RELEASE-p6
OpenSSL 3.0.15

Intel Core i5-7500T
Emulex OCE11102

[OPNsense-bot]

Thank you for creating an issue.
Since the ticket doesn't seem to be using one of our templates, we're marking this issue as low priority until further notice.

For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

The easiest option to gain traction is to close this ticket and open a new one using one of our templates.

[fichtner]

Normally this happens when you spoof the MAC of the VLAN, but don't set the parent to promiscuous mode. The other option is to set the MAC address on the parent instead of the VLAN, which sets it for all VLAN children.

Cheers,
Franco

[davidebeatrici]

I actually didn't spoof the MAC address, it's the same as the physical NIC port for both VLAN interfaces.

Also, I just edited the bug description because the child interface appears to be 802.1q as well according to ifconfig:

oce1: flags=1008043<UP,BROADCAST,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: SFP_2 (opt4)
        options=400a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10Gbase-SR <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

vlan0.835: flags=1008043<UP,BROADCAST,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: WAN (opt9)
        options=0
        ether xx:xx:xx:xx:xx:xx
        inet xxx.xxx.xxx.xxx netmask 0xffffffff broadcast xxx.xxx.xxx.xxx
        inet6 fe80::xxxx:xxxx:xxxx:xxxx%vlan0.835 prefixlen 64 scopeid 0xc
        groups: vlan
        vlan: 835 vlanproto: 802.1q vlanpcp: 0 parent interface: oce1
        media: Ethernet autoselect (10Gbase-SR <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

qinq0.835.474: flags=1008043<UP,BROADCAST,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 1496
        description: BGP (opt10)
        options=0
        ether xx:xx:xx:xx:xx:xx
        inet6 fe80::xxxx:xxxx:xxxx:xxxx%qinq0.835.474 prefixlen 64 scopeid 0xd
        inet6 fd00:cafe:cfcf:cfcf::2 prefixlen 64
        groups: vlan
        vlan: 474 vlanproto: 802.1q vlanpcp: 0 parent interface: vlan0.835
        media: Ethernet autoselect (10Gbase-SR <full-duplex>)
        status: active
        nd6 options=121<PERFORMNUD,AUTO_LINKLOCAL,NO_DAD>

[fichtner]

I haven't seen a lot of oce(4) usage so this could be a driver bug. The workaround is to enable promiscuous mode on the parent then anyway. That's all I can really offer from here.

Cheers,
Franco

[davidebeatrici]

No problem, I'll try and let you know. Thanks!

[davidebeatrici]

I can confirm the proposed workaround works.

I'm closing this since we're probably dealing with an upstream (FreeBSD) bug.