Graphite Feeder
Introduction
Feed metrics from several sources to graphite.
Installation
No node package deployed to npm yet. Please see section about development to build app yourself.
Configuration
node-config is used for managing configurations.
Configurations are stored in configuration files within your application, and can be overridden and extended by environment variables, command line parameters, or external sources.
Development
Setup docker container running graphite docker-graphite-statsd
sudo docker run -d \ --name graphite \ -p 80:80 \ -p 2003:2003 \ -p 8125:8125/udp \ hopsoft/graphite-statsd
If you are developing on MacOS you need to have boot2docker installed and running to be able to use docker.
To access the graphite web UI go to http://localhost/
On MacOS you need to use the proper IP of the boot2docker container (boot2docker ip
) instead.
Clone repo and change to directory
git clone git@github.com:commercetools/graphite-feeder.git graphite-feeder && cd $_
Create a configuration file config/local.yaml
and adjust missing values.
# configuration for graphite/carbon graphite: host: localhost # use boot2docker IP on MacOS port: 2003 # carbon port # configuration for Salesforce feeder salesforce: username: # required when feeding from Salesforce password: # required when feeding from Salesforce token: # required when feeding from Salesforce
Resolve and install required dependencies.
npm install
Build and run salesforce feeder (using DEBUG level for logging)
grunt && DEBUG=* ./bin/feeder-salesforce -d
Release npm package using the predefined grunt task.
grunt release
Docker
There is also a docker container for easy setup/execution of the CLI.
Run docker container:
docker run -v /path/to/config/:/config -e NODE_CONFIG_DIR=/config/ sphereio/graphite-feeder
Set an alias for repeated calls:
alias feeder='docker run -v /path/to/config/:/config -e NODE_CONFIG_DIR=/config/ sphereio/graphite-feeder'
After setting the alias you can simply use:
feeder
Feeders
Feeders are used for quering data from different sources and feeding the data into graphite. The following section describes the available feeders.
Salesforce
The Salesforce feeder queries data using Salesforce API and feeds the metric to graphite.
Sample configuration to push a metric to graphite. The configuration defines for each data you want to query a SOQL (Salesforce Object Query Language) as well as the mapping to resulting graphite metric.
# configuration for Salesforce feeder salesforce: prefix: salesforce # prefix for graphite path datasources: # sources revenue: # query invoiced opportunities query: SELECT Account.name, Amount, CurrencyIsoCode,Type FROM Opportunity WHERE FiscalYear=2015 AND (StageName='Invoice' OR StageName='Receipt of payment') mapping: # configure mapping salesforce result -> graphite metric path: de.commercetools.opportunities.#{Account.Name}.#{Type}.amount.#{CurrencyIsoCode} # define path (field names from result allowed) value: Amount # value field in result timestamp: CloseDate # date field in result
You can define multiple data sources if you like to query and push different metrics from Salesforce to graphite.
# configuration for Salesforce feeder salesforce: prefix: salesforce # prefix for graphite path datasources: # <key>: ... <key>: ...