tech-net: Re: IFQ_MAXLEN: How large can it be?
Subject: Re: IFQ_MAXLEN: How large can it be?
To: None
<tech-net@netbsd.org>
From: Christoph Kaegi
<kgc@zhwin.ch>
List: tech-net
Date: 11/16/2006 08:55:35
On 15.11-14:10, Greg Troxel wrote:
>
> So I bumped this number on our quite busy firewall up from 256
> to 1024 and later to 4096, but I still get 1'026'678 dropped
> packets during 8 days uptime.
>
> I think this is unwise and will just result in more memory stress. If
> 4096 doesn't help, you're not keeping up often, or there's something
> else going on. I'd be nervous putting this above 256 - people usually
> don't and then you'll be stressing the mbuf system more than others
> have stressed it and fixed it.
>
> Where are the packets being dropped? Normally received packets get
> put on the (single, system-wide) IP input queue, and then a soft
> interrupt causes them to be processed and placed on output queues.
> Can you post your statistics that point at this? "netstat -s" is
> very useful if you haven't run that, as is "netstat -i".
>
After what I can see, they're dropped from the IP input queue
(see netstat -q output at the end).
# netstat -s
---------------------------- 8< ----------------------------
ip:
2848237254 total packets received
2 bad header checksums
0 with size smaller than minimum
18 with data size < data length
0 with length > max ip packet size
0 with header length < data size
0 with data length < header length
0 with bad options
0 with incorrect version number
0 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped (out of ipqent)
0 malformed fragments dropped
0 fragments dropped after timeout
0 packets reassembled ok
395403 packets for this host
0 packets for unknown/unsupported protocol
2801729041 packets forwarded (0 packets fast forwarded)
124599 packets not forwardable
1635 redirects sent
0 packets no matching gif found
2184588 packets sent from this host
0 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
0 output packets discarded due to no route
0 output datagrams fragmented
0 fragments created
0 datagrams that can't be fragmented
99 datagrams with bad address in header
icmp:
1482423 calls to icmp_error
8793 errors not generated because old message was icmp
Output histogram:
echo reply: 320
destination unreachable: 60035
routing redirect: 1635
time exceeded: 1298856
13 messages with bad code fields
0 messages < minimum length
0 bad checksums
0 messages with bad length
Input histogram:
destination unreachable: 13
echo: 320
320 message responses generated
0 path MTU changes
igmp:
0 messages received
0 messages received with too few bytes
0 messages received with bad checksum
0 membership queries received
0 membership queries received with invalid field(s)
0 membership reports received
0 membership reports received with invalid field(s)
0 membership reports received for groups to which we belong
0 membership reports sent
tcp:
743848 packets sent
736473 data packets (87991304 bytes)
1119 data packets (994232 bytes) retransmitted
6181 ack-only packets (11003 delayed)
0 URG only packets
0 window probe packets
0 window update packets
81 control packets
0 send attempts resulted in self-quench
392068 packets received
375345 acks (for 87986634 bytes)
8819 duplicate acks
0 acks for unsent data
14376 packets (805497 bytes) received in-sequence
18 completely duplicate packets (52 bytes)
0 old duplicate packets
0 packets with some dup. data (0 bytes duped)
10 out-of-order packets (0 bytes)
0 packets (0 bytes) of data after window
0 window probes
503 window update packets
10 packets received after close
0 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
20 connection requests
43 connection accepts
63 connections established (including accepts)
159080 connections closed (including 7 drops)
0 embryonic connections dropped
0 delayed frees of tcpcb
374663 segments updated rtt (of 91312 attempts)
530 retransmit timeouts
0 connections dropped by rexmit timeout
0 persist timeouts (resulting in 0 dropped connections)
103 keepalive timeouts
97 keepalive probes sent
6 connections dropped by keepalive
4490 correct ACK header predictions
6627 correct data packet header predictions
100 PCB hash misses
10 dropped due to no socket
0 connections drained due to memory shortage
0 PMTUD blackholes detected
0 bad connection attempts
43 SYN cache entries added
0 hash collisions
43 completed
0 aborted (no space to build PCB)
0 timed out
0 dropped due to overflow
0 dropped due to bucket overflow
0 dropped due to RST
0 dropped due to ICMP unreachable
0 delayed free of SYN cache entries
0 SYN,ACKs retransmitted
0 duplicate SYNs received for entries already in the cache
4 SYNs dropped (no route or no space)
0 packets with bad signature
0 packets with good signature
udp:
3029 datagrams received
0 with incomplete header
0 with bad data length field
0 with bad checksum
14 dropped due to no socket
81 broadcast/multicast datagrams dropped due to no socket
0 dropped due to full socket buffers
2934 delivered
2449 PCB hash misses
79825 datagrams output
arp:
52026 packets sent
21730 reply packets
30296 request packets
486913 packets received
21310 reply packets
173743 valid request packets
447459 broadcast/multicast packets
0 packets with unknown protocol type
0 packets with bad (short) length
291860 packets with null target IP address
0 packets with null source IP address
0 could not be mapped to an interface
0 packets sourced from a local hardware address
0 packets with a broadcast source hardware address
0 duplicates for a local IP address
0 attempts to overwrite a static entry
0 packets received on wrong interface
0 entrys overwritten
0 changes in hardware address length
23096 packets deferred pending ARP resolution
4809 sent
13537 dropped
0 failures to allocate llinfo
---------------------------- 8< ----------------------------
netstat -i
---------------------------- 8< ----------------------------
(network addresses anonymized)
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Colls
wm0 1500 <Link> 00:30:48:dd:9c:d4 870327391 0 1099730201 0 0
wm0 1500 120.81.1/24 gw1.zhwin.ch 870327391 0 1099730201 0 0
wm1 1500 <Link> 00:30:48:dd:9c:d5 789681410 0 656853416 0 0
wm1 1500 120.81.3/24 gw2.zhwin.ch 789681410 0 656853416 0 0
wm2 1500 <Link> 00:04:23:dd:b9:cc 1072700285 0 874684304 0 0
wm2 1500 120.81.5/24 gw3.zhwin.c 1072700285 0 874684304 0 0
wm3 1500 <Link> 00:04:23:dd:b9:cd 118809847 0 175760316 0 0
wm3 1500 120.81.8/24 gw4.zhwin.ch 118809847 0 175760316 0 0
wm4* 1500 <Link> 00:04:23:dd:f5:c2 0 0 0 0 0
wm5* 1500 <Link> 00:04:23:dd:f5:c3 0 0 0 0 0
lo0 33192 <Link> 0 0 0 0 0
lo0 33192 loopback/8 localhost 0 0 0 0 0
---------------------------- 8< ----------------------------
# netstat -q
---------------------------- 8< ----------------------------
arpintrq:
queue length: 0
maximum queue length: 50
packets dropped: 4419
ipintrq:
queue length: 0
maximum queue length: 4096
packets dropped: 1102533
---------------------------- 8< ----------------------------
--
----------------------------------------------------------------------
Christoph Kaegi kgc@zhwin.ch
----------------------------------------------------------------------