realtek: l3: rtl839x: route with the egress device's MAC

On RTL839x every routed frame leaves with the switch's own MAC,
whatever device the route goes out of, and frames to a device with a
MAC of its own are not routed: every ROUTING_SA_CTRL entry the driver
writes (0 to 14) holds the switch's MAC, and every ROUTING row names
entry 0 (#25768, items 3 and 4).

The 16 entries are the source MACs a ROUTING row names (SW_MAC_IDX,
Realtek GPL SDK, dal_cypress_l3_routeEntry_set()) and, as measured,
also the MACs whose frames the switch routes. Entry 0 stays the
switch's MAC (dal_cypress_l3_init()). Any other MAC that a route goes
out of a device with takes an entry, per MAC and held by the routes
through it, the on-link prefix included, so a device routes on its MAC
while it has an address, if an entry was free when it got it. A
ROUTING row is now keyed on the gateway's MAC and that entry. Unused
entries hold FF:FF:FF:FF:FF:FE, as the SDK leaves them.

What the hardware cannot do traps. A route through a gateway that
finds no entry free traps, and asks again on the gateway's next
neighbour update. Two devices with different MACs on one VLAN reach a
gateway through one L2 entry, which can name only one of their rows:
the routes already forwarding keep it. A device that takes a new MAC
has its entry rewritten when only its routes hold it and no other
entry has the new MAC; otherwise its routes move to an entry for the
new MAC, those through a gateway trapping until a resolve gives them
their new row, and the old entry, still routed, is given up only with
the rows that name it.

Measured on an HPE 1920-48G (RTL8393), on main f29dfe25ba with the
previous patch and bench debugfs files, before the rebase onto
e5ec5e199f; 8 echo requests per case, forwarding off, switch.3 and
switch.7 with MACs of their own (M3, M7), gateway G in VLAN 3 and, with
the same MAC, in VLAN 7, sending now and then as a live gateway does:
- from VLAN 7 to M7, route out of switch.3: 8/8 at G, TTL 63, source
  M3 (on the #25808 image: 0/8, all to the CPU)
- routes via G out of switch.3 and switch.7: two ROUTING rows, 8/8
  with M3 and 8/8 with M7
- switch.3 down: its entry freed, the switch.7 route still 8/8 with
  M7; up again: M3 back, 8/8
- switch.7 to a new MAC: its entry rewritten in place, 8/8 with the
  new MAC, the switch.3 route unchanged
- switch.7 to M3, which switch.3's entry holds: its routes moved to
  that entry, its old one freed, both routes 8/8 with M3
- with one entry held, 16 more VLAN devices with MACs of their own:
  the entries ran out after 14, and a gateway route out of a device
  without one trapped; once the others were gone it took an entry on
  its next neighbour update (debugfs and register reads, no traffic;
  G was never on those VLANs, so before e5ec5e199f its routes there
  got an invented L2 entry, and on main they trap for want of one:
  only the switch MAC entries were checked)
- rerun from #25805 and #25808: two prefixes in both orders, one
  gateway for two routes, neighbour loss: same results

Other RTL839x boards: routes keep entry 0, as before, while every
device they go out of has the MAC eth0 had at probe. A device with
another MAC now takes an entry, and frames to it are routed by the
switch instead of the CPU.

RTL930x sets no set_switch_mac: its routes take no entry, and its
netdev notifier path is unchanged. Compile-tested on rtl930x. RTL838x
and RTL931x have no setup, so their FIB and neighbour events are ignored
and none of this runs there.

Not tested: two devices with different MACs on one VLAN, a move to a
new entry, a device going back to the switch's MAC, a failed ROUTING
row write.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Gennaro Cimmino <gcimmino@rayonra.net>
Link: https://github.com/openwrt/openwrt/pull/25822
Signed-off-by: Markus Stockhausen <markus.stockhausen@gmx.de>

target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/l2.c

@@ -306,9 +306,9 @@ int otto_l2_nexthop_add(struct rtl838x_switch_priv *priv, struct otto_l3_nexthop
 	nh->rvid = e.rvid;
 	nh->dev_id = e.stack_dev;
 	/* If the entry is already a valid next hop entry, don't change it, but
-	 * for the DMAC entry it names, one per gateway MAC: every route through
+	 * for the DMAC entry it names: every route through the entry holds that
-	 * the entry holds that one, and the entry may still name one given up
+	 * one, and the entry may still name one given up while it stayed a next
-	 * while it stayed a next hop.
+	 * hop, or, on RTL839x, the row of the device's old MAC.
 	 */
 	if (e.next_hop) {
 		if (e.nh_route_id == nh->dmac_id)

target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/l3.c

@@ -47,7 +47,11 @@
  * holds an L2 entry for its MAC. A trapping rule only takes frames addressed
  * to one of the switch's MACs, so a frame for another station goes its way. A
  * destination no rule matches needs no catch-all: its frames are addressed to
- * the switch, which traps them.
+ * the switch, which traps them. The switch's MACs (ROUTING_SA_CTRL) are both
+ * the source MACs a routed frame leaves with and the MACs whose frames are
+ * routed: entry 0 is the switch's own, and a device with a MAC of its own
+ * takes another for as long as a route out of it holds one, its on-link
+ * prefix included.
  *
  * State per family:
  * - RTL930x: IPv4 and IPv6 unicast routes through a gateway are forwarded.
@@ -63,7 +67,9 @@
  *   interfaces share one 1536 byte MTU. The programmed state is shown in
  *   debugfs, under realtek_otto_l3.
  * - RTL839x: IPv4 unicast routes through a gateway are forwarded, through the
- *   first 512 PIE rules. A route counts its packets from when its gateway
+ *   first 512 PIE rules, with the MAC of the device they go out of. Up to 15
+ *   MACs besides the switch's own are taken; a route out of a device whose
+ *   MAC finds no entry traps. A route counts its packets from when its gateway
  *   resolves, while a counter is free. Trapped to the CPU: the same IPv4
  *   routes as on RTL930x. No IPv6, and an event the driver could not take is
  *   not replayed.
@@ -186,9 +192,9 @@ static void otto_l3_839x_nexthop_read(struct otto_l3_ctrl *ctrl, int idx, struct
 	rt->nh.gw = v >> 16;
 }
 
-/* A ROUTING row holds a gateway's MAC and the index of the source MAC to
+/* A ROUTING row holds a gateway's MAC and the switch MAC entry to route with
- * route with (SW_MAC_IDX). The routes through one gateway share its row, the
+ * (SW_MAC_IDX). The routes through one gateway out of one device share a row,
- * way RTL930x routes share a DMAC entry: the L2 entry of the gateway names
+ * the way RTL930x routes share a DMAC entry: the L2 entry of the gateway names
  * one row, whichever route claimed it.
  */
 static int otto_l3_839x_set_dmac(struct otto_l3_ctrl *ctrl, u32 idx,
@@ -199,6 +205,12 @@ static int otto_l3_839x_set_dmac(struct otto_l3_ctrl *ctrl, u32 idx,
 	return otto_table_write(RTL8390_TBL_ROUTING, idx, &data);
 }
 
+static void otto_l3_839x_set_switch_mac(struct otto_l3_ctrl *ctrl, int idx, u64 mac)
+{
+	sw_w32(mac >> 32, RTL839X_ROUTING_SA_CTRL + idx * 8);
+	sw_w32(mac, RTL839X_ROUTING_SA_CTRL + idx * 8 + 4);
+}
+
 /* The prefix rows of an RTL839x are the PIE rules L3 keeps for itself. A rule
  * matches the destination prefix. One that forwards points at the gateway's
  * L2 entry, which names the ROUTING row holding the gateway's MAC. One that
@@ -280,10 +292,13 @@ static void otto_l3_839x_setup_port_macs(struct otto_l3_ctrl *ctrl)
 	dev = priv->ports[priv->r->cpu_port].dp->user;
 	mac = ether_addr_to_u64(dev->dev_addr);
 
-	for (int i = 0; i < 15; i++) {
+	/* Entry 0 is the switch's own MAC, the others wait for a device with a
-		sw_w32(mac >> 32, RTL839X_ROUTING_SA_CTRL + i * 8);
+	 * MAC of its own (otto_l3_switch_mac_get())
-		sw_w32(mac, RTL839X_ROUTING_SA_CTRL + i * 8 + 4);
+	 */
-	}
+	ctrl->switch_macs[0] = mac;
+	otto_l3_839x_set_switch_mac(ctrl, 0, mac);
+	for (int i = 1; i < MAX_SWITCH_MACS; i++)
+		otto_l3_839x_set_switch_mac(ctrl, i, SWITCH_MAC_FREE);
 }
 
 static int otto_l3_839x_setup(struct otto_l3_ctrl *ctrl)
@@ -1687,12 +1702,12 @@ static bool otto_l3_dmac_equal(const struct otto_l3_dmac *a, const struct otto_l
 }
 
 /* Routes through one gateway share its L2 entry, and that names a single
- * DMAC entry, whichever route claimed it first. So the entry is held per MAC
+ * DMAC entry, whichever route claimed it first. So the entry is held per MAC,
- * by every route through it, the way the SDK holds it
+ * and on RTL839x per switch MAC entry too, by every route through it, the way
- * (_dal_longan_l3_dmacEntry_alloc()), not per route: a route that goes leaves
+ * the SDK holds it (_dal_longan_l3_dmacEntry_alloc()), not per route: a route
- * it to the others. A route holds one at most, and there are no more routes
+ * that goes leaves it to the others. A route holds one at most, and there are
- * than entries. Entry 0 stays unused, as it did while route 0, which is never
+ * no more routes than entries. Entry 0 stays unused, as it did while route 0,
- * allocated, named it.
+ * which is never allocated, named it.
  */
 static int otto_l3_dmac_get(struct otto_l3_ctrl *ctrl, const struct otto_l3_dmac *dmac)
 {
@@ -1740,6 +1755,89 @@ static void otto_l3_dmac_put(struct otto_l3_ctrl *ctrl, int idx)
 	mutex_unlock(ctrl->lock);
 }
 
+/* On RTL839x the source MACs a routed frame leaves with are also the MACs
+ * whose frames are routed. Entry 0 is the switch's own; each other MAC a route
+ * goes out of a device with, its on-link prefix and its addresses included,
+ * takes an entry while a route holds it, so a device with a MAC of its own
+ * routes what arrives on it while it has an address. Devices with one MAC
+ * share its entry. An entry no route holds any more goes back to the unused
+ * value, after the ROUTING rows that named it, which the routes held too.
+ */
+static int otto_l3_switch_mac_get(struct otto_l3_ctrl *ctrl, u64 mac)
+{
+	int i, unused = -1;
+
+	if (mac == ctrl->switch_macs[0])
+		return 0;
+
+	mutex_lock(ctrl->lock);
+
+	for (i = 1; i < MAX_SWITCH_MACS; i++) {
+		if (ctrl->switch_mac_refs[i] && ctrl->switch_macs[i] == mac)
+			break;
+		if (unused < 0 && !ctrl->switch_mac_refs[i])
+			unused = i;
+	}
+
+	if (i == MAX_SWITCH_MACS) {
+		i = unused;
+		if (i < 0) {
+			mutex_unlock(ctrl->lock);
+			return -ENOSPC;
+		}
+
+		ctrl->switch_macs[i] = mac;
+		ctrl->cfg->set_switch_mac(ctrl, i, mac);
+	}
+
+	ctrl->switch_mac_refs[i]++;
+	mutex_unlock(ctrl->lock);
+
+	return i;
+}
+
+static void otto_l3_switch_mac_put(struct otto_l3_ctrl *ctrl, int idx)
+{
+	if (idx <= 0)
+		return;
+
+	mutex_lock(ctrl->lock);
+	if (!--ctrl->switch_mac_refs[idx])
+		ctrl->cfg->set_switch_mac(ctrl, idx, SWITCH_MAC_FREE);
+	mutex_unlock(ctrl->lock);
+}
+
+/* A route the switch cannot route keeps to the CPU, its next hop as it was */
+static void otto_l3_route_trap_sw(struct otto_l3_ctrl *ctrl, struct otto_l3_route *r)
+{
+	if (r->attr.action == ROUTE_ACT_TRAP2CPU)
+		return;
+
+	r->attr.action = ROUTE_ACT_TRAP2CPU;
+	r->attr.ttl_dec = false;
+	r->attr.ttl_check = false;
+	otto_l3_route_rewrite(ctrl, r);
+}
+
+/* A gateway's L2 entry names one ROUTING row, which on RTL839x also names the
+ * switch MAC entry. Two devices with different MACs on one VLAN reach a
+ * gateway through the same L2 entry, but each with a row of its own: the
+ * routes forwarding through the entry keep it.
+ */
+static bool otto_l3_l2_entry_taken(struct otto_l3_ctrl *ctrl, struct otto_l3_route *r,
+				   const struct otto_l3_dmac *key)
+{
+	struct otto_l3_route *q;
+
+	list_for_each_entry(q, &ctrl->routes_list, list)
+		if (q != r && q->attr.action == ROUTE_ACT_FORWARD && q->nh.mac == key->mac &&
+		    q->nh.rvid == r->nh.rvid && q->nh.dmac_id >= 0 &&
+		    !otto_l3_dmac_equal(&ctrl->dmacs[q->nh.dmac_id], key))
+			return true;
+
+	return false;
+}
+
 static void otto_l3_route_update_hw(struct otto_l3_ctrl *ctrl, struct otto_l3_route *r,
 				    u64 mac)
 {
@@ -1750,6 +1848,8 @@ static void otto_l3_route_update_hw(struct otto_l3_ctrl *ctrl, struct otto_l3_ro
 	struct otto_l3_dmac key = { .mac = mac };
 	bool first = !r->nh.mac;
 	bool no_port, no_l2, trap;
+	int old_sa = -1, sa;
+	bool new_sa = false;
 	int dmac = r->id;
 
 	if (otto_l3_row_frozen(ctrl, r))
@@ -1772,14 +1872,47 @@ static void otto_l3_route_update_hw(struct otto_l3_ctrl *ctrl, struct otto_l3_ro
 		otto_l3_route_rewrite(ctrl, r);
 	}
 
+	/* An RTL839x route routes with the switch MAC entry of its device's
+	 * MAC. One without, or whose device took another MAC, takes one here,
+	 * and gives the old one up only with the ROUTING row that names it.
+	 * With none free it keeps to the CPU, its next hop untouched.
+	 */
+	if (ctrl->cfg->set_switch_mac &&
+	    (r->switch_mac_id < 0 || ctrl->switch_macs[r->switch_mac_id] != r->switch_mac)) {
+		sa = otto_l3_switch_mac_get(ctrl, r->switch_mac);
+		if (sa < 0) {
+			otto_l3_route_trap_sw(ctrl, r);
+			return;
+		}
+		old_sa = r->switch_mac_id;
+		r->switch_mac_id = sa;
+		new_sa = true;
+	}
+
+	/* An RTL839x ROUTING row also names the switch MAC entry */
+	if (r->switch_mac_id > 0)
+		key.smac_id = r->switch_mac_id;
+
 	if (ctrl->cfg->set_dmac) {
 		dmac = otto_l3_dmac_get(ctrl, &key);
 		if (dmac < 0) {
 			dev_err(ctrl->dev, "no DMAC entry for %016llx: %d\n", mac, dmac);
-			return;
+			goto undo_sa;
 		}
 	}
 
+	if (ctrl->cfg->set_switch_mac && otto_l3_l2_entry_taken(ctrl, r, &key)) {
+		if (!r->l2_taken)
+			dev_info(ctrl->dev, "gateway %pI4 also reached from another MAC on VLAN %d, routing %s in software\n",
+				 &r->gw_ip.s6_addr32[3], r->nh.rvid,
+				 otto_l3_route_dst(r, dst, sizeof(dst)));
+		r->l2_taken = true;
+		otto_l3_dmac_put(ctrl, dmac);
+		otto_l3_route_trap_sw(ctrl, r);
+		goto undo_sa;
+	}
+
+	r->l2_taken = false;
 	r->nh.mac = r->nh.gw = mac;
 	r->nh.port = priv->r->port_ignore;
 	r->nh.id = r->id;
@@ -1790,8 +1923,11 @@ static void otto_l3_route_update_hw(struct otto_l3_ctrl *ctrl, struct otto_l3_ro
 	if (!no_l2)
 		r->nh.l2_installed = true;
 
-	/* The route names its gateway's entry now, through its L2 entry */
+	/* The route names its gateway's entry now, through its L2 entry, and
+	 * the old row gave up the switch MAC entry it named
+	 */
 	otto_l3_dmac_put(ctrl, old_dmac);
+	otto_l3_switch_mac_put(ctrl, old_sa);
 
 	/* A next hop with no port delivers the frame twice, so where the
 	 * route entry can trap, let the CPU route it alone until an update
@@ -1837,6 +1973,13 @@ static void otto_l3_route_update_hw(struct otto_l3_ctrl *ctrl, struct otto_l3_ro
 	}
 
 	otto_l3_route_install(ctrl, r);
+	return;
+
+undo_sa:
+	if (new_sa) {
+		otto_l3_switch_mac_put(ctrl, r->switch_mac_id);
+		r->switch_mac_id = old_sa;
+	}
 }
 
 /* The hardware would go on forwarding to the address the gateway last
@@ -2068,6 +2211,81 @@ struct otto_l3_egress_work {
 	int ifindex;
 };
 
+/* Whether an entry other than @skip holds @mac */
+static bool otto_l3_switch_mac_held(struct otto_l3_ctrl *ctrl, u64 mac, int skip)
+{
+	if (mac == ctrl->switch_macs[0])
+		return true;
+
+	for (int i = 1; i < MAX_SWITCH_MACS; i++)
+		if (i != skip && ctrl->switch_mac_refs[i] && ctrl->switch_macs[i] == mac)
+			return true;
+
+	return false;
+}
+
+/* An RTL839x device that took a new MAC routes with it. A switch MAC entry
+ * that only routes out of the device hold takes the new MAC where it is, and
+ * the ROUTING rows naming it follow. Routes on any other entry move: one
+ * without a gateway names no row and moves now; one through a gateway traps
+ * and moves in otto_l3_route_update_hw(), with its row, once the resolve
+ * asked for here brings its gateway back.
+ */
+static void otto_l3_switch_mac_sync(struct otto_l3_ctrl *ctrl, struct net_device *dev, u64 mac)
+{
+	struct otto_l3_route *r;
+	bool none = false;
+	int old;
+
+	/* An entry keeps to one MAC, so it is only rewritten where no other
+	 * entry has the new one
+	 */
+	for (int k = 1; k < MAX_SWITCH_MACS; k++) {
+		unsigned int n = 0;
+
+		list_for_each_entry(r, &ctrl->routes_list, list)
+			if (r->gw_ifindex == dev->ifindex && r->switch_mac_id == k)
+				n++;
+
+		if (n && n == ctrl->switch_mac_refs[k] && !otto_l3_switch_mac_held(ctrl, mac, k)) {
+			mutex_lock(ctrl->lock);
+			ctrl->switch_macs[k] = mac;
+			ctrl->cfg->set_switch_mac(ctrl, k, mac);
+			mutex_unlock(ctrl->lock);
+		}
+	}
+
+	list_for_each_entry(r, &ctrl->routes_list, list) {
+		if (r->gw_ifindex != dev->ifindex)
+			continue;
+
+		r->switch_mac = mac;
+		old = r->switch_mac_id;
+		if (old >= 0 && ctrl->switch_macs[old] == mac)
+			continue;
+
+		if (r->gw_ip.s6_addr32[3]) {
+			otto_l3_route_trap_sw(ctrl, r);
+			continue;
+		}
+
+		otto_l3_switch_mac_put(ctrl, old);
+		r->switch_mac_id = otto_l3_switch_mac_get(ctrl, mac);
+		none |= r->switch_mac_id < 0;
+	}
+
+	if (none)
+		dev_err(ctrl->dev, "no switch MAC entry free for %pM, routes out of %s stay in software\n",
+			dev->dev_addr, dev->name);
+
+	/* One resolve brings back every route through a gateway */
+	list_for_each_entry(r, &ctrl->routes_list, list)
+		if (r->gw_ifindex == dev->ifindex && r->attr.type == ROUTE_TYPE_IP4UC &&
+		    r->gw_ip.s6_addr32[3] && r->attr.action == ROUTE_ACT_TRAP2CPU &&
+		    !otto_l3_gw_seen(ctrl, r, NULL))
+			otto_l3_port_gw_resolve(ctrl, dev, &arp_tbl, &r->gw_ip);
+}
+
 /* Whether every hold on an egress interface is a route out of this device */
 static bool otto_l3_intf_only_dev(struct otto_l3_ctrl *ctrl, int if_id, int ifindex)
 {
@@ -2105,6 +2323,11 @@ static void otto_l3_egress_sync(struct work_struct *work)
 
 	mac = ether_addr_to_u64(dev->dev_addr);
 
+	if (ctrl->cfg->set_switch_mac) {
+		otto_l3_switch_mac_sync(ctrl, dev, mac);
+		goto out;
+	}
+
 	list_for_each_entry(r, &ctrl->routes_list, list) {
 		if (r->gw_ifindex != dev->ifindex || r->nh.if_id < 0)
 			continue;
@@ -2166,6 +2389,13 @@ static int otto_l3_netdev_notifier(struct notifier_block *nb, unsigned long even
 	if (!net_eq(dev_net(netdev_notifier_info_to_dev(ptr)), &init_net))
 		return NOTIFY_DONE;
 
+	/* RTL839x keeps no router MAC table: the routes hold its switch MACs */
+	if (!ctrl->cfg->set_router_mac) {
+		if (event == NETDEV_CHANGEADDR)
+			otto_l3_egress_sync_request(ctrl, netdev_notifier_info_to_dev(ptr));
+		return NOTIFY_DONE;
+	}
+
 	switch (event) {
 	case NETDEV_CHANGEUPPER:
 		/* A port joins or leaves its master. One joining a bridge stops
@@ -2297,6 +2527,7 @@ static void otto_l3_route_teardown(struct otto_l3_ctrl *ctrl, struct otto_l3_rou
 	if (r->is_host_route || !ctrl->prefix_rows_stale) {
 		otto_l3_route_put_intf(ctrl, r);
 		otto_l3_dmac_put(ctrl, r->nh.dmac_id);
+		otto_l3_switch_mac_put(ctrl, r->switch_mac_id);
 	}
 
 	otto_l3_route_free(ctrl, r);
@@ -2337,6 +2568,7 @@ static struct otto_l3_route *otto_l3_route_alloc(struct otto_l3_ctrl *ctrl,
 	r->pr.packet_cntr = -1;
 	r->nh.if_id = -1;
 	r->nh.dmac_id = -1;
+	r->switch_mac_id = -1;
 	r->is_host_route = host;
 	INIT_LIST_HEAD(&r->srcs);
 
@@ -2534,6 +2766,14 @@ static int otto_l3_fib_add_v4(struct otto_l3_ctrl *ctrl, struct fib_entry_notifi
 	route->nh.rvid = vlan;
 	route->gw_ifindex = ndev->ifindex;
 
+	if (ctrl->cfg->set_switch_mac) {
+		route->switch_mac = ether_addr_to_u64(ndev->dev_addr);
+		route->switch_mac_id = otto_l3_switch_mac_get(ctrl, route->switch_mac);
+		if (route->switch_mac_id < 0)
+			dev_err(ctrl->dev, "no switch MAC entry free for %pM, %pI4/%d stays in software\n",
+				ndev->dev_addr, &info->dst, info->dst_len);
+	}
+
 	if (ctrl->cfg->set_router_mac) {
 		u64 mac = ether_addr_to_u64(ndev->dev_addr);
 
@@ -2578,6 +2818,7 @@ static int otto_l3_fib_add_v4(struct otto_l3_ctrl *ctrl, struct fib_entry_notifi
 out_free_rmac:
 out_free_rt:
 	otto_l3_route_put_intf(ctrl, route);
+	otto_l3_switch_mac_put(ctrl, route->switch_mac_id);
 	otto_l3_route_free(ctrl, route);
 	return 0;
 }
@@ -4169,6 +4410,7 @@ const struct otto_l3_config otto_l3_838x_cfg = {
 
 const struct otto_l3_config otto_l3_839x_cfg = {
 	.set_dmac = otto_l3_839x_set_dmac,
+	.set_switch_mac = otto_l3_839x_set_switch_mac,
 	.route_rows_move = otto_l3_839x_route_rows_move,
 	.route_write = otto_l3_839x_route_write,
 	.setup = otto_l3_839x_setup,
@@ -4382,7 +4624,7 @@ int otto_l3_probe(struct device *dev, struct rtl838x_switch_priv *priv)
 		return dev_err_probe(dev, err, "Failed to register netevent notifier\n");
 	}
 
-	if (ctrl->cfg->set_router_mac) {
+	if (ctrl->cfg->set_router_mac || ctrl->cfg->set_switch_mac) {
 		ctrl->nd_nb.notifier_call = otto_l3_netdev_notifier;
 		err = register_netdevice_notifier(&ctrl->nd_nb);
 		if (err) {

target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/l3.h

@@ -15,6 +15,9 @@ struct fib6_info;
 #define MAX_SMACS 64
 #define MAX_DMACS 2048
 #define MAX_ROUTER_MACS 64
+#define MAX_SWITCH_MACS 16
+/* What an unused switch MAC entry holds (Realtek GPL SDK, dal_cypress_l3_init()) */
+#define SWITCH_MAC_FREE 0xfffffffffffeULL
 #define MAX_HOST_ROUTES		1536
 #define MAX_ROUTES		512
 
@@ -97,7 +100,9 @@ struct otto_l3_route {
 	bool srcs_incomplete;		/* a source could not be tracked */
 	struct list_head list;		/* all routes, for every lookup */
 	u32 tb_id;			/* routing table the route came from */
-	u16 switch_mac_id;		/* Index into switch's own MACs, RTL839X only */
+	int switch_mac_id;		/* Switch MAC entry it routes with, RTL839x */
+	u64 switch_mac;			/* MAC of its device, which that entry holds */
+	bool l2_taken;			/* its gateway's L2 entry names another row */
 	struct otto_l3_nexthop nh;
 	struct pie_rule pr;
 	struct otto_l3_route_attr attr;
@@ -131,6 +136,10 @@ struct otto_l3_config {
 	void (*set_nexthop)(struct otto_l3_ctrl *ctrl, int idx, u16 dmac_id, u16 interface);
 	int (*route_lookup_hw)(struct otto_l3_ctrl *ctrl, struct otto_l3_route *rt);
 	int (*route_rows_move)(struct otto_l3_ctrl *ctrl, int dst, int src, int count);
+	/* RTL839x: one entry of the switch's own MACs, the source MACs it
+	 * routes with and the MACs it routes what arrives on
+	 */
+	void (*set_switch_mac)(struct otto_l3_ctrl *ctrl, int idx, u64 mac);
 	void (*route_read)(struct otto_l3_ctrl *ctrl, int idx, struct otto_l3_route *rt);
 	void (*route_write)(struct otto_l3_ctrl *ctrl, int idx, struct otto_l3_route *rt);
 	int (*setup)(struct otto_l3_ctrl *ctrl);
@@ -155,6 +164,8 @@ struct otto_l3_ctrl {
 	unsigned int intf_refs[MAX_SMACS];	/* routes holding each */
 	struct otto_l3_dmac *dmacs;		/* what each DMAC entry holds */
 	unsigned int *dmac_refs;		/* routes holding each */
+	u64 switch_macs[MAX_SWITCH_MACS];	/* MAC of each switch MAC entry */
+	unsigned int switch_mac_refs[MAX_SWITCH_MACS];	/* routes holding each */
 	DECLARE_BITMAP(router_mac_bm, MAX_ROUTER_MACS);	/* entries written here */
 	bool prefix_rows_stale;	/* a move failed, the rows are not where we say */
 	bool v4_fwd_off;	/* policy rules keep IPv4 forwarding in software */