prometheus-kafka-connect
Connector to prometheus client which only includes sink. This will be useful to stream data from Kafka to Prometheus and subsequently to Grafana. This connector depends on, kafka-connect and node-sinek as the consumer. Please read up those dependencies to make things clearer.
You can use javascript consumer or native which depends on librdkafka. The example can be found in example
directory, which there is a comment to specify where you need to do the transformation. Basically, you need to do that in ETL function, that needs to be passed to the connector. Or just like the following.
const runSinkConnector ConverterFactory = ;const config = ; console; const etl = { // Do the transformation here let record; try record = metric: messagename value: 1 type: "counter" // or "gauge" ; catcherr // Do nothing if record && recordmetric && recordvalue // Continue with the transformed record return ; // Continue without throwing error return ; } const converter = ConverterFactory; ;
Configuration
- For the native and non-native consumer please take a look at node-sinek project.
- There is an example of doing local setup complete with kafka, zookeeper, prometheus, and grafana in
local-setup
directory - Please take a look at
example/config.js
file, this is an excerpt from there.
... connector: options: job: "promclient_job" additionalLabels: "method" ...
job
This is to mark the job that needed to be consumed by prometheus, the configuration from prometheus is defined in the prometheus.yml
, take a look at local-setup
directory
additionalLabels
The additionalLabels that are needed for the metric, the default is label, but you can add as many as you want in the array format.