pf temporary dropouts

Septic · #1 **(View Single Post)** 27th March 2010

Hi all, I'm hoping the wealth of experience here will be able to solve my problem!

I've recently setup an OpenBSD 4.6 box to use as a dedicated firewall, got it configured, and all seems well - except that the packet filter is causing connection dropouts every few minutes or so.

e.g. If I try to download a large file, it will download for anywhere between 1 - 9 minutes, then simply hang, until I manually cancel. My connection to IRC is also dropped out constantly too.

I also bought 2 dedicated NICs (Dlink DGE-528T, supported Chipset) to use instead of the onboard gigabit NICs to see if they could be at fault, but the same thing occurs.
This is definitely a pf issue, as if I do a pfctl -d everything works fine from that point on.

If anyone has any idea as to what could be at fault in my configuration I'd be most grateful

- after what little troubleshooting I can do at the moment, being somewhat of a BSD noob, all I can think of is forcing the adapters to 100/Mb & 1000/Mb.

Below are my ifconfig & pf.conf, and I also attached an image with systat output, with pf enabled on the left, and with it disabled on the right (my IRC connection got reset twice in the 6 minute period it was enabled). If any extra info. is needed I'd be happy to supply.

Thanks

Simple Network Layout

Code:

	INTERNET
	   |
	   |
       [ Router ]
	   |
	   |
	 (re0)
      [ Firewall ]
	 (re1)
	   |
	   |
       [ Switch ]
	   |
	   |
	  LAN

relevant ifconfig output

Code:

re0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:26:5a:e3:53:cc
        priority: 0
        groups: egress
        media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause)
        status: active
        inet6 fe80::226:5aff:fee3:53cc%re0 prefixlen 64 scopeid 0x2
        inet 192.168.134.2 netmask 0xffffff00 broadcast 192.168.134.255
re1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:26:5a:e3:52:8d
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
        status: active
        inet6 fe80::226:5aff:fee3:528d%re1 prefixlen 64 scopeid 0x3
bridge0: flags=41<UP,RUNNING> mtu 1500
        priority: 0
        groups: bridge

pf.conf

Code:

# Ignore the loopback interface
set skip on lo0

# Defines - NIC for external/internal network routes and other addresses
nic_ext = "re0"
nic_int = "re1"
myaddr = "192.168.134.2"
mysub = "192.168.134.0/24"
myweb = "192.168.134.250"


# Allow everything through the secondary (Firewall->Switch) interface
set skip on $nic_int

# Implicit deny through the primary (Firewall->Router) interface
block in on $nic_ext all
block out on $nic_ext all

# Anti-spoof
antispoof quick for $nic_ext


# Now setup the standard rules


# SSH from internal network, else block & log
pass in quick proto tcp from $mysub to $myaddr port 22
block in log quick on $nic_ext proto tcp from any port 22

# Allow DNS requests, as we have a DNS server behind us (and in front)
pass out quick proto udp from $mysub port 53

# Hard-Coded blocks - todo: move these into a dedicated file & table
# Do not block: 208.100.20.98 - proxyscan.rizon.net
# 66.102.9.0/24 - google.com subnet, mostly for ad-blocking
# 63.88.212.91 - webtrends
blacklist = "{ 208.201.239.101, 222.208.183.218, 125.230.150.249, 66.102.9.0/24, 63.88.212.91 }"
block in quick on $nic_ext from $blacklist

# Allow inbound port 8080 traffic to the webserver only
pass in quick on $nic_ext proto tcp from any to $myweb port 8080

# Allow outbound traffic from internal network
pass out quick on $nic_ext proto tcp all from $mysub flags S/SA
pass out quick on $nic_ext proto udp all from $mysub

# NAT rule
nat on $nic_ext from !($nic_ext) to any -> ($nic_ext)

J65nko · #2 **(View Single Post)** 28th March 2010

The ruleset allows out requests to your blacklisted IP addresses, but block
the incoming replies.
Try this (untested) version which does not allow outgoing requests to your blacklist.

Code:

# Defines - NIC for external/internal network routes and other addresses
nic_ext = "fxp0"
nic_int = "lo1"

myaddr = "192.168.134.2"
mysub = "192.168.134.0/24"
myweb = "192.168.134.250"

# Hard-Coded blocks 
# Do not block: 208.100.20.98 - proxyscan.rizon.net
# 66.102.9.0/24 - google.com subnet, mostly for ad-blocking
# 63.88.212.91 - webtrends

table <blacklist>  { 
    208.201.239.101
    222.208.183.218
    125.230.150.249
    66.102.9.0/24
    63.88.212.91
}

# Allow everything through the secondary (Firewall->Switch) interface
# and the loopback interface
set skip on { $nic_int lo0 }

# --- NAT rule
#nat on $nic_ext from !($nic_ext) to any -> ($nic_ext)

# --- Default policy
block log all

# -- External interface

antispoof quick for $nic_ext

# Allow outbound traffic from internal network      
pass out quick on $nic_ext inet proto tcp from $mysub to !<blacklist> port != ssh  
pass out quick on $nic_ext inet proto udp from $mysub to !<blacklist>

# Allow inbound port 8080 traffic to the webserver only
pass in quick on $nic_ext inet proto tcp from any to $myweb port 8080

You can see I elminated your DNS rule. If you pass out all UDP traffic you already allow DNS on port 53

If this doesn't work as expected, you can visualize which packets are being blocked with running tcpdump on the pflog0 interface.

Code:

# tcpdump -eni pflog0

BTW why doesn't your internal interface re1 have any IPv4 address? And what is that bridge0 interface supposed to do?

Septic · #3 **(View Single Post)** 28th March 2010

My apologies - my desired setup was to have a transparent Firewall, so all that would be needed was a straight bridged connection between the two adapters. I think I mis-read part of the documentation, but as it was working (~ish) I hadn't noticed!

Something along the lines of:

Code:

INTERNET -> [ Router ] -> (re0 - 1st NIC) [ Firewall ]
					| (Bridge)
			  [ Firewall ] (re1 - 2nd NIC) -> [ Switch ] -----> LAN Client	<--- Internet Access
									|-> LAN Client

I tried changing some of the settings as you had stated, but it didn't affect the dropouts - 20.8mb of a file & 9 minutes for IRC - viewing the # tcpdump -eni pflog0, nothing came up at all, even when I tried to trigger it.
Ctrl+C'ing out of it provided the stats (and yes, pf was enabled):

Code:

0 packets received by filter
0 packets dropped by kernel

As this particular bridge shouldn't have an IP address, I decided to rebuild the configuration from scratch (didn't reinstall).

Here is what I did:

Code:

# nano /etc/hostname.re0
up

# nano /etc/hostname.re1
up

# nano /etc/sysctl.conf
net.inet.ip.forwarding=1

# nano /etc/resolv.conf
nameserver 192.168.134.1
lookup file bind

# nano /etc/mygate
192.168.134.1

# nano /etc/bridgename.bridge0
add re0
add re1
up

# nano /etc/rc.conf
pf=YES

# pfctl -nf /etc/pf.conf

# reboot

# ifconfig
re0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> MTU 1500
        lladdr 00:26:5a:e3:53:cc
        priority: 0
        media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause)
        status: active
        inet6 fe80::226:5aff:fee3:53cc%re0 prefixlen 64 scopeid 0x2
re1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:26:5a:e3:52:8d
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
        status: active
        inet6 fe80::226:5aff:fee3:528d%re1 prefixlen 64 scopeid 0x3
bridge0: flags=41<UP,RUNNING> mtu 1500
        priority: 0
        groups: bridge

After the routing tables updated, I've been able to connect up fine, and in the current 20 minutes it's been setup, I haven't had a dropout yet

No idea what I'd set the first time round that caused it to go awry..

I've updated the pf.conf to handle some of the new changes now too (such as the onboard NIC just for SSH), and so far all is running smoothly.

I had originally setup the blacklist purely for hosts that flagged in my webservers apache log, hence only the inbound block - thanks for pointing out it needed updating!

Many thanks for all your help, I would've spent many more hours banging my head against the table in frustration otherwise

I have only one remaining question - when I try to browse to a site that has a blacklisted entry, the page will continue to be in a loading state until the TCP connection times out - will set block-policy return solve this by simply resetting it immediately?

Thanks again

J65nko · #4 **(View Single Post)** 28th March 2010

The ifconfig output from your first post shows 192.168.134.2 as the IP address of re0, and no IP for re1.
In your revised configuration, both re0 and re1 don't have an IP address. That could be a difference.

Re: set block-policy return

Just try it, it won't make your firewall less secure.

From the pf.conf man page, the block policy section

Code:

return           This causes a TCP RST to be returned for TCP pack-
                 ets and an ICMP UNREACHABLE for other types of
                 packets.

Septic · #5 **(View Single Post)** 29th March 2010

Yeah, that works exactly as the man page describes.

Have found some interesting side effects from blacklisting the 3 main Google subnets I have seen so far (good luck trying to use youtube, for example), but browsing the web on the whole is quicker now

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode