Squid

From MediaWiki

Jump to: navigation, search

Contents

Enabling SNMP in Squid

Overview

Squid version 2.2 and later is SNMP capable. By using an SNMP monitoring program like MRTG, it is possible to draw interesting graphs and gather statistics of the Squid cache server.

Requirements

  • Squid > v2.2 compiled with SNMP support
  • MRTG

RPM Installation

The Squid RPMS shipped with RHL > 7.1 has SNMP support compiled in. Install squid and mrtg RPMS. 

# rpm -ivh squid-2.3.STABLE4-10.i386.rpm
# rpm -ivh mrtg-2.9.6-2.i386.rpm

I assume that Squid has been configured to work in your environment and you just want to add in SNMP monitoring.

Configuration

The configuration part is to be done in 2 phases -

  1. Enable SNMP on Squid
  2. Configure MRTG to monitor Squid using SNMP

Configuring Squid 2.2

Edit the squid config file (squid.conf) and change the SNMP ACL lines to look like this:

-- squid.conf -- 

snmp_port 3401
acl snmppublic snmp_community public
snmp_access allow snmppublic localhost
snmp_access deny all

-- squid.conf --

Restart squid 

# service squid restart

Check the squid cache.log (/var/log/squid/cache) for any SNMP related messages. You should see this: 

2001/08/24 10:46:02| Starting Squid Cache version 2.3.STABLE4 for i386-redhat-linux-gnu...
2001/08/24 10:46:03| Accepting HTTP connections at 0.0.0.0, port 8000, FD 13.
2001/08/24 10:46:03| Accepting ICP messages at 0.0.0.0, port 3130, FD 14.
2001/08/24 10:46:03| Accepting SNMP messages on port 3401, FD 15.

Resources

  1. Squid FAQ 18.2: Enabling SNMP in Squid

http://www.squid-cache.org/Doc/FAQ/FAQ-18.html#ss18.2

Testing

  1. Query Squid using "snmpwalk"

If SNMP has been enabled on Squid and the ACL have been defined correctly, the following command will return a similar output - 

# snmpwalk -p 3401 localhost public .1.3.6.1.4.1.3495.1.1
If it gives output like:
enterprises.3495.1.1.1.0 = 120
enterprises.3495.1.1.2.0 = 82692
enterprises.3495.1.1.3.0 = Timeticks: (99027) 0:16:30.27

Use this command if you have a very recent version of net-snmp tools: 

[godzilla] ~# snmpwalk -V
NET-SNMP version: 5.0.8
[godzilla] ~# snmpwalk -v1 -c public localhost:3401 .1.3.6.1.4.1.3495.1.1
SNMPv2-SMI::enterprises.3495.1.1.1.0 = INTEGER: 84
SNMPv2-SMI::enterprises.3495.1.1.2.0 = INTEGER: 810
SNMPv2-SMI::enterprises.3495.1.1.3.0 = Timeticks: (149104) 0:24:51.04

then it is working ok, and you should be able to make nice statistics out of it.

Resources

  1. Squid FAQ 18.5: How can I query the Squid SNMP Agent?

http://www.squid-cache.org/Doc/FAQ/FAQ-18.html#ss18.5

Now that we have Squid + SNMP working, we can procced to configure MRTG.

Configuring MRTG to use Squid SNMP

mrtg.cfg

Edit mrtg.cfg (/etc/mrtg/mrtg.cfg) and add the following lines in it: 

########################################################
# Squid related sections                               #
########################################################

# Change this to the correct location of squid/mib.txt
LoadMIBS: /etc/squid/mib.txt

Target[proxy-hit]: cacheHttpHits&cacheServerRequests:public@localhost:3401
MaxBytes[proxy-hit]: 10000
Title[proxy-hit]: HTTP Hits
PageTop[proxy-hit]: <H2>Proxy Cache Statistics: HTTP Hits / Requests</H2>
  <TABLE>
<TR><TD>System:</TD> <TD>proxy.domain.com</TD></TR>
<TR><TD>Maintainer:</TD> <TD>Squid Maintainer</TD></TR>
<TR><TD>Description:</TD><TD>Squid Proxy server</TD></TR>
  </TABLE>
Suppress[proxy-hit]: y
LegendI[proxy-hit]: HTTP hits
LegendO[proxy-hit]: HTTP requests
Legend1[proxy-hit]: HTTP hits
Legend2[proxy-hit]: HTTP requests
YLegend[proxy-hit]: perminute
ShortLegend[proxy-hit]: req/min
Options[proxy-hit]: nopercent, perminute, dorelpercent, unknaszero

Target[proxy-srvkbinout]: cacheServerInKb&cacheServerOutKb:public@localhost:3401
MaxBytes[proxy-srvkbinout]: 10000
Title[proxy-srvkbinout]: Cache Server Traffic In / Out
PageTop[proxy-srvkbinout]: <H2>Cache Statistics: Server Traffic Volume (In/Out) </H2>
  <TABLE>
<TR><TD>System:</TD> <TD>proxy.mydomain.com</TD></TR>
<TR><TD>Maintainer:</TD> <TD>Squid Maintainer</TD></TR>
<TR><TD>Description:</TD><TD>Squid Proxy server</TD></TR>
  </TABLE>
Suppress[proxy-srvkbinout]: y
LegendI[proxy-srvkbinout]: Traffic In
LegendO[proxy-srvkbinout]: Traffic Out
Legend1[proxy-srvkbinout]: Traffic In
Legend2[proxy-srvkbinout]: Traffic Out
YLegend[proxy-srvkbinout]: per minute
ShortLegend[proxy-srvkbinout]: b/min
kMG[proxy-srvkbinout]: k,M,G,T
kilo[proxy-srvkbinout]: 1024
Options[proxy-srvkbinout]: nopercent, perminute, unknaszero

-- mrtg.cfg --

Run "mrtg /etc/mrtg/mrtg.cfg" thrice. Ignore the warnings. 

# mrtg /etc/mrtg/mrtg.cfg 
Rateup WARNING: /usr/bin/rateup could not read the primary log file for proxy-hit
Rateup WARNING: /usr/bin/rateup The backup log file for proxy-hit was invalid as well
Rateup WARNING: /usr/bin/rateup Can't remove proxy-hit.old updating log file
Rateup WARNING: /usr/bin/rateup Can't rename proxy-hit.log to proxy-hit.old updating log file
Rateup WARNING: /usr/bin/rateup could not read the primary log file for proxy-srvkbinout
Rateup WARNING: /usr/bin/rateup The backup log file for proxy-srvkbinout was invalid as well
Rateup WARNING: /usr/bin/rateup Can't remove proxy-srvkbinout.old updating log file
Rateup WARNING: /usr/bin/rateup Can't rename proxy-srvkbinout.log to proxy-srvkbinout.old updating log file
 
# mrtg /etc/mrtg/mrtg.cf
Rateup WARNING: /usr/bin/rateup Can't remove proxy-hit.old updating log file
Rateup WARNING: /usr/bin/rateup Can't remove proxy-srvkbinout.old updating log file

# mrtg /etc/mrtg/mrtg.cf

The third time there wont be any errors.

Testing

Point your browser to the following URLs to see the graphs:

Files

  1. /etc/squid/squid.conf
  2. /etc/mrtg/mrtg.cfg
  • RPMS
  1. squid-2.3.STABLE4-10.i386.rpm
  2. mrtg-2.9.6-2.i386.rpm


  • There are lots of scatterred docs on Squid + MRTG floating around. I will try and add them as i dig em up.
  1. Squid FAQ 18.2: Enabling SNMP in Squid http://www.squid-cache.org/Doc/FAQ/FAQ-18.html#ss18.2
  2. Squid FAQ 18.5: How can I query the Squid SNMP Agent? http://www.squid-cache.org/Doc/FAQ/FAQ-18.html#ss18.5
  3. MRTG For Squid. Includes nice mrtg.cfg files http://www.arnes.si/~matija/

TODO

Curently we monitor only the following SNMP attributes:

  • cacheServerInKb
  • cacheServerOutKb
  • cacheHttpHits
  • cacheServerRequests

This is only the tip of the iceberg. There are many many more attributes which one can monitor using SNMP. See the squid/mib.txt for a complete list of MIBs.

DelayPools

Overview (from webmin help)

Delay pools provides a means a restrict bandwidth usage of clients. With delay pools, clients can be restricted at the individual level, or in groups. ACLs are used to apply bandwidth limits to users.

The following recipe implements a network wide limit on the bandwidth usage to 64kbps with an overall strict limit also set to 64kbps. The Webmin Squid configuration module is very useful for configuring delay pools if this functionality has to be provided in the hands of an end user.

squid.conf 

#
# Class 1 delay pool
#
acl lansrc 192.168.1.0/255.255.255.0
acl intranet dst 192.168.1.0/255.255.255.0

delay_pools 1
delay_class 1 1
delay_parameters 1 8000/8000
delay_access 1 allow lan !intranet
delay_access 1 deny all
delay_initial_bucket_level 50

"delay_access 1 allow lan !intranet" is to ensure that only traffic originating from the Internet is subjet to rate limiting and not the local Intranet traffic. A download from http://downloads.com/ will be rate limited to 8Kb/s while a download from http://192.168.1.x/ will not. 

#
# Class 2 delay pool
#
# I use this at work, every one is limited to an individual bucket 7.5Kb and
# the aggregate bucket is set to 11Kb. Keeps interactive ssh sessions snappy on
# your 16Kb DSL link
#
delay_pools 1

# class 2, individual limit + aggregate limit
delay_class 1 2
delay_parameters 1 10000/11000 7500/7500

delay_access 1 allow lan !local-intranet
delay_access 1 deny all
delay_initial_bucket_level 0

The delay pool counter can be viewed using the cachemgr.cgi script via the browser.

http://proxy.mydomain.com/cgi-bin/cachemgr.cgi?host=localhost&port=3128&user_name=&operation=delay&auth=

Notes

  • Squid must be compiled with --enable-delay-pools
  • The delay pool counters are not measurable via SNMP, AFAIK.
  • "delay_initial_bucket_level" is not very clear to me. I have set it to zero as I feel it improves interactivity.


ToDo

  • Can the queing schedular be changed?
  • Can you do fairness queing so that no one person can hog the entire bandwidth? I think I will have to resort to class 2 delay pools for that. But even then, I dont think I can give equal costs to every host to divide the bandwidth proportionally to every host currenlty "seen" by the cache.
Retrieved from "http://shankerbalan.net/wiki/Squid"
Personal tools