Install Graphite under pyenv virtualenv on Ubuntu

It took me one and an half day to get the Graphite working…

1. Make sure you have pyenv and pyenv-virtualenv installed.

 

2. Install the following packages.

sudo apt-get install python-dev pkg-config libcairo2-dev memcached

 

3. Install Python 2.7 with enable-shared.

env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 2.7.8

 

4. Create a new virtualenv. I name it graphite-poc in this example.

pyenv virtualenv 2.7.8 graphite-poc

 

5. Activate the graphite-poc.

pyenv activate graphite-poc

 

6. Prepare the following shell script to install py2cairo. Edit the SOURCE_VERSION and PYENV_VERSION which should be the same as your virtualenv setup.
install-py2cairo.sh

#!/bin/sh

set -e
PYENV_ROOT="${PYENV_ROOT:-${HOME}/.pyenv}"
SOURCE_VERSION="2.7.8" # the source version of virtualenv
export PYENV_VERSION="graphite-poc" # the virtualenv
export PYTHON_CONFIG="${PYENV_ROOT}/versions/${SOURCE_VERSION}/bin/python-config"

test -f py2cairo-1.10.0.tar.bz2 || wget http://cairographics.org/releases/py2cairo-1.10.0.tar.bz2
rm -rf py2cairo-1.10.0
tar xvf py2cairo-1.10.0.tar.bz2
cd py2cairo-1.10.0
./waf --version
(
  cd .waf-*
  wget -O- "https://gist.github.com/yyuu/5566264/raw/612da9ff5b215cda42eb62fc3d332f3d2ae172db/python.py.patch" | patch -p0
)
./waf --prefix="${PYENV_ROOT}/versions/${PYENV_VERSION}" --includedir="/usr/include" configure
./waf build
./waf install

 

7. Make it executable and run the script.

# Change the file permission
chmod 744 install-py2cairo.sh
# Execute install-py2cairo.sh
./install-py2cairo.sh

 

8. Install other pre-requisites. (django 1.6 will cause graphite-web error.)

pip install whisper django==1.5.8 django-tagging daemonize pytz pyparsing tagging zope.interface python-memcached twisted txamqp

 

9. Install Carbon at /opt/graphite.

pip install carbon

OR if you want to install Graphite at a custom location.

pip install carbon --install-option="--prefix=/<custom-path>/graphite" --install-option="--install-lib=/<custom-path>/graphite/lib"

 

10. Prepare the Carbon configuration files under <graphite-root>/conf.

cp carbon.conf.example carbon.conf
cp storage-schemas.conf.example storage-schemas.conf

 

11. Install graphite-web at /opt/graphite.

pip install graphite-web

OR if you want to install Graphite at a custom location and it should be the same where you install the graphite-web.

pip install graphite-web --install-option="--prefix=/<custom-path>/graphite" --install-option="--install-lib=/<custom-path>/graphite/webapp"

 

12. Copy <graphite-root>/webapp/graphite/local_settings.py.example to <graphite-root>/webapp/graphite/local_settings.py. Edit the SQLite DB file path if you have a custom Graphite installation path.

DATABASES = {
  'default': {
    'NAME': '/opt/graphite/storage/graphite.db',
    'ENGINE': 'django.db.backends.sqlite3',
    'USER': '',
    'PASSWORD': '',
    'HOST': '',
    'PORT': ''
  }
}

 

13. Also generate a long string for the SECRET_KEY in local_settings.py too. (Django Secret Key Generator)

#####################################
# General Configuration #
#####################################
# Set this to a long, random unique string to use as a secret key for this
# install. This key is used for salting of hashes used in auth tokens,
# CRSF middleware, cookie storage, etc. This should be set identically among
# instances if used behind a load balancer.
#SECRET_KEY = 'UNSAFE_DEFAULT'
SECRET_KEY = '<your secret key>'

 

14. Enable the logs or otherwise the Apache error log will throw errors. (No handlers could be found for logger “cache”)

# Logging
LOG_RENDERING_PERFORMANCE = True
LOG_CACHE_PERFORMANCE = True
LOG_METRIC_ACCESS = True

 

15. Initialize the SQLite DB.

cd /opt/graphite/webapp/graphite
sudo python manage.py syncdb

 

16. Make sure the Graphite root folder is accessible and the db file should be writable by Apache.

chown -R <user>:www-data /opt/graphite
chmod -R 775 /opt/graphite/storage
chmod 660 /opt/graphite/storage/graphite.db

 

17. Install Apache to for hosting the graphite-web.

sudo apt-get install --assume-yes apache2 apache2-mpm-worker apache2-utils apache2.2-bin libapache2-mod-wsgi libapache2-mod-python

 

18. Copy <graphite-root>/conf/graphite.wsgi.example to <graphite-root>/conf/graphite.wsgi and edit the sys.path if necessary and append your virtualenv to sys.path too.

import os, sys
sys.path.append('/opt/graphite/webapp')
sys.path.append('/home/<user>/.pyenv/versions/graphite-poc/lib/python2.7/site-packages')
os.environ['DJANGO_SETTINGS_MODULE'] = 'graphite.settings'

import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()

# READ THIS
# Initializing the search index can be very expensive, please include
# the WSGIImportScript directive pointing to this script in your vhost
# config to ensure the index is preloaded before any requests are handed
# to the process.
from graphite.logger import log
log.info("graphite.wsgi - pid %d - reloading search index" % os.getpid())
import graphite.metrics.search

 

19. Make sure you have mod_wsgi.c enabled.

20. Create a virtualhost for the graphite-web by copying the <graphite-root>/examples/example-graphite-vhost.conf. Update the WSGISocketPrefix, ServerName, Graphite installation path, @DJANGO_ROOT@ and the graphite.wsgi access rules accordingly.

#WSGISocketPrefix run/wsgi
WSGISocketPrefix /var/run/apache2/wsgi

<VirtualHost *:80>
  ServerName sandbox.hatuey.com
  DocumentRoot "/opt/graphite/webapp"
  ErrorLog /opt/graphite/storage/log/webapp/error.log
  CustomLog /opt/graphite/storage/log/webapp/access.log common

  # I've found that an equal number of processes & threads tends
  # to show the best performance for Graphite (ymmv).
  WSGIDaemonProcess graphite processes=5 threads=5 display-name='%{GROUP}' inactivity-timeout=120
  WSGIProcessGroup graphite
  WSGIApplicationGroup %{GLOBAL}
  WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL}

  WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi 

  Alias /content/ /opt/graphite/webapp/content/
  <Location "/content/">
    SetHandler None
    Options FollowSymlinks
    AllowOverride none
    Require all granted
  </Location>

  # @DJANGO_ROOT should located inside your pyenv
  # In this example it would be /home/<user>/.pyenv/versions/graphite-poc/lib/python2.7/site-packages/django
  Alias /media/ "@DJANGO_ROOT@/contrib/admin/media/"
  <Location "/media/">
    SetHandler None
  </Location>

  # The graphite.wsgi file has to be accessible by apache. It won't
  # be visible to clients because of the DocumentRoot though.
  <Directory /opt/graphite/conf/>
    Options All
    AllowOverride All
    Require all granted
  </Directory>

</VirtualHost>

 

21. Restart Apache.

sudo /etc/init.d/apache restart

 

22. Verify the graphite-web setup.
graphite-installation-1
 

23. Edit the /opt/graphite/lib/carbon/util.py to fix the daemonize path.

#from twisted.scripts._twistd_unix import daemonize
import daemonize

 

24. Start the Carbon cache daemon.

cd /opt/graphite/bin
python carbon-cache.py start

 

25. After a while, you should be able to find the carbon.agent data on the graphite-web portal.
graphite-installation-2
 

26. Start the example-client.py to send some data to Graphite.

cd /opt/graphite/examples
python example-client.py

 

27. See if you could get the system.loadavg data.
graphite-installation-3
 

Done =)

References:

Advertisements

One thought on “Install Graphite under pyenv virtualenv on Ubuntu”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s