Set Up Metric Collection Using Graphite and Statsd on Centos 6.3 x86_64

Visualizing metrics is as important as collecting them. We are using the awesome open source tool, graphite to help us visualize our application and system metrics. We also use the fantastic work done by Etsy and use statsd,  their simple node.js daemon that collects and aggregates metrics via UDP and flushes them to graphite.

Notes:

  • The install process assumes that you are running statsd and carbon/graphite-web on the same server
  • The installation configuration for graphite currently has hardcoded paths to /opt/graphite. It is possible to change this but would require modification to the graphite installation scripts.

The following gist has all the steps to install and configure graphite and statsd.

cd /usr/local/src
git clone git://github.com/joyent/node.git
cd node
./configure --prefix=/opt
make
make install
ln -s /opt/node/bin/node /usr/local/bin
ln -s /opt/node/bin/npm /usr/local/bin

cd /opt
git clone git://github.com/etsy/statsd.git

yum install python-devel memcached python-setuptools mysql-devel 
yum install cairo-devel
yum install pycairo-devel
yum install bitmap-fonts

pip install django==1.3
pip install django-tagging
pip install python-memcached
pip install carbon
pip install whisper
pip install graphite-web

pip install twisted

This takes care of the all the package install. Now next is config and running
#
# Configure carbon
#
cd /opt/graphite/conf/
cp carbon.conf.example carbon.conf
# Create storage schema and copy it over
# Using the sample as provided in the statsd README
cat >> /tmp/storage-schemas.conf << EOF
# Schema definitions for Whisper files. Entries are scanned in order,
# and first match wins. This file is scanned for changes every 60 seconds.
#
# [name]
# pattern = regex
# retentions = timePerPoint:timeToStore, timePerPoint:timeToStore, …
[stats]
priority = 110
pattern = ^stats\..*
retentions = 10s:6h,1m:7d,10m:1y
EOF
cp /tmp/storage-schemas.conf storage-schemas.conf
# Make sure log dir exists for webapp
sudo mkdir -p /opt/graphite/storage/log/webapp
# Copy over the local settings file and initialize database
cd /opt/graphite/webapp/graphite/
cp local_settings.py.example local_settings.py
python manage.py syncdb
# Follow the prompts, creating a superuser is optional
# StatsD configuration
cat >> /tmp/localConfig.js << EOF
{
graphitePort: 2003
, graphiteHost: “127.0.0.1”
, port: 8125
}
sudo cp /tmp/localConfig.js /opt/statsd/localConfig.js

To test your installation, do the following:

Run carbon-cache:

cd /opt/graphite && sudo ./bin/carbon-cache.py –debug start

Run graphite-web:

cd /opt/graphite && sudo ./bin/run-graphite-devel-server.py .

Run statsd:

cd /opt/statsd && node ./stats.js ./localConfig.js

Run the example client (any one will suffice, python client shown here):

cd /opt/statsd/examples && python ./python_example.py

Point your web browser at http://127.0.0.1:8080/ to bring up the graphite webapp and start creating graphs!

Advertisements

One thought on “Set Up Metric Collection Using Graphite and Statsd on Centos 6.3 x86_64

  1. [root@otrs node]# ./configure ‘–prefix=/opt’
    File “./configure”, line 435
    arm_float_abi = ‘hard’ if is_arm_hard_float_abi() else ‘default’
    ^
    SyntaxError: invalid syntax

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s