Monitor Memcached status

It is quite a important task to manage and monitor the memcache caching system. There are quite a lot of options to carry out this operation.

1. Using Telnet Command Line interface

List of supported commands are available at

http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt



telnet localhost 11211



List of keys used

stats items


STAT items:5:number 2
STAT items:5:age 47
STAT items:5:evicted 0
STAT items:5:evicted_nonzero 0
STAT items:5:evicted_time 0
STAT items:5:outofmemory 0
STAT items:5:tailrepairs 0
END



2. Using memcached-tool

The memcached tool is available in the source directory .

/usr/src/memcached-1.4.4/scripts/memcached-tool

Stats

/usr/src/memcached-1.4.4/scripts/memcached-tool 127.0.0.1:11211 stats

Show slab information

/usr/src/memcached-1.4.4/scripts/memcached-tool 127.0.0.1:11211 display

Show keys and values

/usr/src/memcached-1.4.4/scripts/memcached-tool 127.0.0.1:11211 dump


3. Using the damemtop

A flexible 'top' like utility for viewing memcached clusters. You can use it to manage a interactive cluster status.

The utility is available in the source directory. /usr/src/memcached-1.4.4/scripts/damemtop


It require you to install the following perl modules

cpan AnyEvent Term::ReadKey YAML


Now you need the /etc/damemtop.yaml file with the following contents.


delay: 3
mode: t
top_mode:
sort_column: "hostname"
sort_order: "asc"
columns:
- hostname
- all_version
- all_fill_rate
- hit_rate
- evictions
- bytes_written
- "2:get_hits"
servers:
- 127.0.01:11211


In a distributed environment you can add all the servers in the cluster to the file and issue the following command.


/usr/src/memcached-1.4.4/scripts/damemtop


That would give you a ideal monitoring setup..

Install Memcached caching sytem in centos from source

memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

It is mainly used in MYSQL + PHP or MYSQL + JAVA applications which are quite database intensive.

Install LIBEVENT


yum -y install libevent libevent-devel


Download Source

wget http://memcached.googlecode.com/files/memcached-1.4.4.tar.gz
tar zxf memcached-1.4.4.tar.gz
cd memcached-1.4.4
./configure && make && make install



Test the installation


memcached -m 256 -u nobody -vv
slab class 1: chunk size 80 perslab 13107
slab class 2: chunk size 104 perslab 10082
<29 send buffer was 110592, now 268435456
<29 server listening (udp)
<29 server listening (udp)



Start the Daemon


memcached -d -m 256 -u nobody -p 11211 -l 192.168.1.33


The above command will start the memcached daemon in port 11211

Ensure that it is working.

netstat -plan|grep 11211
tcp 0 0 192.168.1.33:11211 0.0.0.0:* LISTEN 7049/memcached




Thats it.. Enjoy the power of caching system..

Check Ubuntu 8.04 server security using OpenVAS

Install openvas server from source in ubunutu 8.04


apt-get install build-essential libgtk2.0-dev libglib2.0-dev libssl-dev htmldoc libgnutls-dev libpcap0.8-dev bison libgpgme11-dev libsmbclient-dev snmp pnscan


add the following line to /etc/apt/sources.list


deb http://ftp.de.debian.org/debian sid main


Install the server


apt-get install openvas-server openvas-server-dev openvas-plugins-dfsg openvas-plugins-base libopenvas2 libopenvas2-dev libopenvasnasl2 libopenvasnasl2-dev


Add your user


openvas-adduser


Update the feeds


openvas-nvt-sync



start the server


/etc/init.d/openvas-server start


Install the client

It would ask you to upgrade php etc and hence I compiled the agent from source. The trunk is incompatible with the openvas-server provided by debian repository.


svn co https://svn.wald.intevation.org/svn/openvas/branches/openvas-client-2-0/ openvas-client



cd openvas-client

./configure && make && make install


Running the agent in batch mode:

You can run the client to produce beautiful html with graphs using the following command line.

create a file named target with the host you want to check

cat >target
localhost
ctrl+c


OpenVAS-Client -q localhost 9390 ovasuser mypasswd target localhost-result -T html_graph -x


The run would take around 15 minutes and you will get the results in the localhost-result directory. View the HTML files using browser.

Minimize Downtime(DNS propagation) using tcp packet forwarding using TCPPROXY

A TCP proxy is a server that acts as an intermediary between a client and another server, called the destination server. Clients establish connections to the TCP proxy server, which then establishes a connection to the destination server. The proxy server sends data received from the client to the destination server and forwards data received from the destination server to the client.

It can be used to forward all the tcp connections to another ports in another server. This can be very effective during server migration time when we may face downtime due to DNS propagation delay. The DNS update in the nameservers will sometimes take more than 24 hours to fully propagate in the internet and as a result all the requests from the nameservers that are not updated will be accessing the old server. The old server might be either not accessible or showing old contents or we may be updating the old server with new data. This is especically critical when it comes to emails. We may sent emails to the wrong server which may become unavailable.

In these kind of circumstances we can initiate a forwarding setting which will forward all the requests that were sent to the old server to be forwarded to the new server. This can be accomplished by many free softwares licensed by GPL. In my case I used TCPPROXY a C++ application simple to install and setup as a proxy.

Installation and setup of tcpproxy for redirecting all traffic to another server

wget http://www.quietsche-entchen.de/download/tcpproxy-2.0.0-beta15.tar.gz

tar zxf tcpproxy-2.0.0-beta15.tar.gz

cd tcpproxy-2.0.0-beta15

make all


Setup tcpproxy as a proxy:

Suppose I have a old server 192.168.1.12 and new server 192.168.15.20 then I will have to forward all http requests to port 80 in old server to port 80 in new server. This can be accomplished using the command in the old server.

tcpproxy -b 80 192.168.15.20:80


General syntax:

Whatever maybe the service that you are using the general syntax would be

tcpproxy -b source port destination server:destination port

Suppose I want to use SMTP then the syntax would be

tcpproxy -b 25 192.168.15.20:25

Service not supported:

* FTP is not supported because it uses a second TCP connections for data transmission

Xen with grub2

This is small article to configure xen boot in grub2. First install xen in the machine using yum. I used centos and hence used yum to install it.
-----------------------------------------

yum install xen virt-manager kernel-xen
------------------------------------------

Now xen and xen kernel is installed in the machine. Now you have to configure grub2 to recognize xen.

In my case I installed xen in another partition other than boot partition. My centos was in /dev/sda1 and boot partition was /dev/sda4

I had to mount that partition and configure the grub configuration to recognize XEN.

mount /dev/sda4 /test
vi /test/boot/grub/grub.cfg

Add the following lines to the end of the configuration.
------------------------------------------------------
menuentry "CentOS (2.6.18-128.1.10.el5xen)"{
set root=(hd0,1)
multiboot /xen.gz-2.6.18-128.1.10.el5
module /vmlinuz-2.6.18-128.1.10.el5xen ro root=LABEL=/ rhgb quiet
module /initrd-2.6.18-128.1.10.el5xen.img
--------------------------------------------------------


Now the xen kernel will be available in the grub 2 menu.

Dual Boot vyatta VC5 with centos, ubuntu, debian,

I came across some issue in dual boot of vyatta VC5 with centos. There has been some modification in the VC5 grub they are using grub2 which is still in testing phase.

The dual boot centos,Ubuntu... in vyatta can be performed by adding the following entries in /boot/grub/grub.cfg . The default grub configuration in grub2 is /boot/grub/grub.cfg

---------------
menuentry " Centos"{
set root=(hd0,1)
linux /vmlinuz-2.6.18-92.el5 ro root=LABEL=/ rhgb quiet
initrd /initrd-2.6.18-92.el5.img
}
-----------------

Vyatta VC5 uses grub2 and it's format is completely different from legacy grub.

In GRUB2 the root system starts from "1" instead of "0". Hence hda1 partition will point to root(hd0,1) instead of legacy root(hd0,0)

Enjoy dual boot in latest GRUB.

You can get new command list from http://grub.enbug.org/CommandList

Block unwanted scrapers and bots using .htaccess and robots.txt

You can block unwanted bots by adding a robots.txt file with the following entries.
-------------------------
User-agent: *
Disallow: /
-------------------------


A list of bad bots that can be denied access using .htaccess. Add the following entries in .htaccess the list of scrapers.
-----------------------------
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]
---------------------------------------------