From_a_motel_6

title: From A Motel 6 date: 2006-05-09 05:51:11

Came across a weird problem on the firewall at work last week. It's running 4-STABLE, and was last updated about a month ago. It's got fxp0 for an outside interface, and em0 plus a bunch of vlan devices for inside interfaces.

When I added either of these two rules:

ipfw allow tcp from 192.168.16.34 to 192.168.19.33 1230,1236 keep-state via vlan19
ipfw allow tcp from 192.168.19.33 to 192.168.16.34 1230,1237 keep-state via vlan19

then suddenly DNS queries from inside our main LAN (192.168.0.0/24 on em0) to outside servers -- say, our main inside nameserver doing recursive queries for A records for Google -- stopped working: queries would pass through natd and go out with the source address changed, but the reply from the server would be accepted by the firewall box, rather than passed to natd and then back inside to the machine that'd made the query. Since the firewall box hadn't made the request, it would send back an ICMP port-unreachable packet to the outside nameserver. In other words:

1. 192.168.0.2 -> ns.google.com: www.google.com A?
2. 192.168.0.1 (firewall box) passes that to natd
3. natd changes packet to...
4. firewall outside IP -> ns.google.com: www.google.com A?
5. ns.google.com -> firewall outside IP: www.google.com A 1.2.3.4
6. firewall accepts that packet...
7. ...but realizes it doesn't have anything listening for a UDP packet from ns.google.com...
8. ...and rejects it:
9. firewall outside IP -> ns.google.com: ICMP port-unreachable

Took me most of the day to figure this out, because I found a separate problem and was convinced that these rules had nothing to do with it. And they don't, really -- wrong protocol, wrong interface, wrong addresses -- but remove the rules and everything's fine. Freakin' bizarre.

I spent a lot of time checking out state rules and such, and I'm pretty certain that's not it. At this point, I'm assuming that it's either a bug in ipfw (possibly related to this PR, or my upgrade from 4.8 to 4-STABLE did not go as cleanly as I thought. I'm going to try installing FreeBSD here and see if I can duplicate this...maybe get another one-character patch into FreeBSD. Woot!