pf, hfsc and load balancing

[hamba]

Hi

I have a bit of a problem getting hfsc to work properly in pf with load balancing.
For some reason ssh_login and ssh_bulk doesn't work.

Here are the rules from my pf.conf

# Quees for upload bandwidth
altq on { $ext_if1, $ext_if2 } bandwidth 550Kb hfsc queue { ack, dns, ssh, bulk }
    queue ack        bandwidth 80% priority 7 qlimit 500 hfsc (realtime 50%)
    queue dns        bandwidth  7% priority 6 qlimit 500 hfsc (realtime  5%)
    queue ssh        bandwidth 10% priority 5 qlimit 500 hfsc (realtime 10%) {ssh_login, ssh_bulk}
      queue ssh_login bandwidth 90% priority 5 qlimit 500 hfsc
      queue ssh_bulk  bandwidth 10% priority 4 qlimit 500 hfsc
    queue bulk       bandwidth  1% priority 4 qlimit 500 hfsc (realtime 5% default)

# SSH OUT
pass in quick on $int_if  route-to { ( $ext_if2 $ext_gw2 ) } proto tcp from $lan_net to any port $ssh_ports queue (ssh_bulk, ssh_login)

#  load balance outgoing tcp traffic from internal network.
pass in on $int_if route-to { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin proto tcp from $lan_net to any queue (bulk, ack)
pass in on $int_if route-to { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin proto tcp from $lan_net to any port $ssh_ports queue (ssh_bulk, ssh_login)

#  general "pass out" rules for external interfaces
pass out on $ext_if1 proto tcp from any flags S/SA modulate state queue (bulk, ack)
pass out on $ext_if1 proto tcp from any port $ssh_ports flags S/SA modulate state queue (ssh_bulk, ssh_login)
pass out on $ext_if2 proto tcp from any flags S/SA modulate state queue (bulk, ack)
pass out on $ext_if2 proto tcp from any port $ssh_ports flags S/SA modulate state queue (ssh_bulk, ssh_login)

And this is what I see when running pfctl

# pfctl -vs queue
queue root_ng0 on ng0 bandwidth 550Kb priority 0 {ack, dns, ssh, bulk}
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/ 50 ]
queue  ack on ng0 bandwidth 440Kb priority 7 qlimit 500 hfsc( realtime 275Kb )
  [ pkts:     685566  bytes:   29700254  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue  dns on ng0 bandwidth 38.50Kb priority 6 qlimit 500 hfsc( realtime 27.50Kb )
  [ pkts:       7907  bytes:     586194  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue  ssh on ng0 bandwidth 55Kb priority 5 qlimit 500 hfsc( realtime 55Kb ) {ssh_login, ssh_bulk}
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue   ssh_login on ng0 bandwidth 49.50Kb priority 5 qlimit 500
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue   ssh_bulk on ng0 bandwidth 5.50Kb priority 4 qlimit 500
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue  bulk on ng0 bandwidth 5.50Kb priority 4 qlimit 500 hfsc( default realtime 27.50Kb )
  [ pkts:     273706  bytes:   77178876  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue root_ng1 on ng1 bandwidth 550Kb priority 0 {ack, dns, ssh, bulk}
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/ 50 ]
queue  ack on ng1 bandwidth 440Kb priority 7 qlimit 500 hfsc( realtime 275Kb )
  [ pkts:     649871  bytes:   28008679  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue  dns on ng1 bandwidth 38.50Kb priority 6 qlimit 500 hfsc( realtime 27.50Kb )
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue  ssh on ng1 bandwidth 55Kb priority 5 qlimit 500 hfsc( realtime 55Kb ) {ssh_login, ssh_bulk}
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue   ssh_login on ng1 bandwidth 49.50Kb priority 5 qlimit 500
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue   ssh_bulk on ng1 bandwidth 5.50Kb priority 4 qlimit 500
  [ pkts:          0  bytes:          0  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]
queue  bulk on ng1 bandwidth 5.50Kb priority 4 qlimit 500 hfsc( default realtime 27.50Kb )
  [ pkts:     848882  bytes:  379008486  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/500 ]

Or from pftop

QUEUE                             BW SCH  PRIO     PKTS    BYTES   DROP_P   DROP_B QLEN BORROW SUSPEN     P/S     B/S
root_ng0                        550K hfsc    0        0        0        0        0    0                     0       0
root_ng1                        550K hfsc    0        0        0        0        0    0                     0       0
 ack                            440K hfsc    7   416042 18048590        0        0    0                    76    3303
 ack                            440K hfsc    7   402913 17342565        0        0    0                    67    2863
 dns                           38500 hfsc    6     5461   404573        0        0    0                     1     103
 dns                           38500 hfsc    6        0        0        0        0    0                     0       0
 ssh                           55000 hfsc    5        0        0        0        0    0                     0       0
 ssh                           55000 hfsc    5        0        0        0        0    0                     0       0
  ssh_login                    49500 hfsc    5        0        0        0        0    0                     0       0
  ssh_login                    49500 hfsc    5        0        0        0        0    0                     0       0
  ssh_bulk                      5500 hfsc    4        0        0        0        0    0                     0       0
  ssh_bulk                      5500 hfsc    4        0        0        0        0    0                     0       0
 bulk                           5500 hfsc    4   123264 46077552        0        0    0                    37   16096
 bulk                           5500 hfsc    4   595013  262099K        0        0   37                   219   65886

As you can see ack and bulk are working fine as well as dns but ssh see no traffic at all.
Does anyone have an idea as to why this is happening and maybe can offer a possible solution.

Thanks
hamba