I use Grafana at work to great effect. After doing TONS of review on other graphite front ends and not being satisfied with the usability and flexibility of any of them I was close to giving up and was considering building my own solution. Then Grafana appeared from the mist. It has come a long way but from day one it was very functional. I use the templating system inherited from Kibana(which Grafana was original based on) to great effect. It allows me to create drill down and overview dashboards to zoom in on particular sites, clusters, or servers. Integrating is a snap by linking in with query parameters from other tools which get fed into the template system. I actually can't imagine a time without Grafana and would encourage anyone using graphite(or one of the supported backends like influxDB) to give it a spin.
I'm working at integrating my product with it (prob influxdb if it can handle the load). I'm worried its going to be one of those products that ends up everywhere then i won't look quite as magical. It really is that good.
Can anybody speak to the difference between Graphite+Grafana and the ELK stack? Are there any fundamental differences in analysis capabilities between these?
We are currently using ELK for doing some lightweight analysis of user behavior (we are using Elasticsearch anyway, so this was the easiest way to get up and running), but as far as I can tell there are some limits to the Elasticsearch/Kibana combination. For example, there seems to be no way to correlate separate events, e.g. how many users that performed action A then performed action B within some time window (or even as the next action). Is this possible with Graphite?
For correlation between distinct events Graphite or any Time series database is not the right tool.
Time series are usually more about being able to collect huge amount of metrics. Metrics that can then be combined, averaged, filtered, put through a processing pipeline (analytical functions), summarized by different intervals. All in order to visualize (usually through graphs) recent live trends or long term trends and statistics.
Grafana is all about maximizing the power and ease of use of the underlying time series store so the user can focus om on building informative and nice looking dashboards. It is also about letting users define generic dashboards through variables that can be used in metric queries, this allows users to reuse the same dashboard for different servers, apps or experiments as long as the metric naming follows a consistent pattern.
Grafana also uses Elasticsearch but not for log analytics, but for annotating graphs with event/log information.
At some point in in the coming 1-3 years log analytics and metric analytics & visualization is going to converge and be solved/addressed by the same piece of software. But that is tricky right now without sacrificing either domain.
Thanks for the insight, that's kind of what I expected. Do you know of any tools/products that can perform this kind of analysis? I looked at the whole range of web analytics tools/platforms (Google Analytics, Mixpanel, Piwiki and others), but these are very much focused on web analytics, not so much user behavior inside a (web) application and offer very limited customization. And they usually do not work offline anyway.
Any idea on how to deal with downsampling from graphite? IE don't send such high resolution to the browser because it can quickly cripple older machines. Now I know that with graphite I can specify different granularity but I don't know if there is an easy way to switch between them when looking at a day vs a week of events.
Is there a reason why the InfluxDB backed graphs, in edit mode, don't allow to conveniently select series with dropdown menu's like it does for Graphite backed ones?
That's the one thing holding me back from giving Grafana a whirl (my younger self promised me to never touch graphite again).
Its because InfluxDB does not have the same API for exploring the metric name hierarchy. But that will change with InfluxDB 0.9 where they will introduce measurements and tags and a lot of inspection queries that will enable a much better query building experience.
yes, but only for the import process that will read them from influxdb and save them into the grafana database (either embedded sqlite3 or external mysql, postgres).
InfluxDB is not really suitable as a general purpose data source (storing dashboards as influxdb metric series was more a temporary solution so influxdb users did not need to install elasticsearch, and influxdb dashboard storage did not support tag filtering).
With Grafana 2.0 needs to store a lot more, users, accounts, user starred dashboards & preferences. So a more general purpose database was needed.
Great news about the backend rendering to PNG. We can't run the current version on our monitoring screen, because it's powered by a raspberry pi, which takes about 90 seconds to render all the svg stuff (down to 30seconds with a Pi v2).
This is probably because you are running Graphite 0.9.12 which does not have the maxDataPoints feature for the json api, with 0.9.13 it work a lot better. I know of some who run it on raspberry with decent performance, not great but far from 90 seconds.
How have people found wrapping other non carbon back ends (even own RDBMS) in graphite-compatible API - in other words I I am collecting time series somewhere, how simple is it to get grafana as a front end?
There are two datasource plugins here that might help you get a picture how to add new types of data sources to grafana. There is basically no documentation for how to write data sources so it requires some reverse enginering (trying to understand the existing datasources), but there is an interface, and custom data sources can provide their own UI editors. https://github.com/grafana/grafana-plugins
We are using Grafana with OpenTSDB and loving it -- really looking forward to some of the 2.0 features and about to buy a bunch more TVs for big shiny Grafana dashboards in our NOC. Way to go guys!
Grafana is not a time series store or metric fetching agent. It s dashboard and graph composer that currently support Graphite, InfluxDB and OpenTSDB (and KariosDB via plugin).
So if you have metrics in one of those time series stores then Grafana is a really awesome tool for visualizing those metrics.
Thank you! Means a lot, been working on this in the dark for a very long time (I originally though it might be paid version of Grafana). Really great to have the code public and show of some of the great things the backend will enable.
Yes, everything will be open source. Thanks to Grafana sponsors and me being part of the Raintank company that will provide SaaS and support services around Grafana and other existing open source monitoring/metrics projects as well as new components that we are working on (all open source!)
