Jump to content

Data Platform/Systems/EventLogging/Monitoring

From Wikitech

< Data Platform | Systems | EventLogging

The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Grafana

Here's some really cool Grafana dashboards that display our Graphite metrics:

EventLogging Dashboard, shows hitrates of all events by schema, status of Kafka brokers, error counts and more.
EventLogging-schema Dashboard, focus on an individual schema. Useful for embedding and inter-dashboard links from graphs showing the actual EventLogging data. Example: performance-metrics.

Graphite

Raw metrics can be browsed at https://graphite.wikimedia.org/ (under Metrics -> eventlogging).

We publish 4 types of counts to graphite:

Overall counts
Per schema counts
Server side counts
Client side counts

Within the overall counts, there are 4 submetrics:

Raw counts: Number of all events that reach the system.
Valid counts: Number of events that pass validation.
insertAttempted: Number of events that get queued up for insertion to MySQL database.
inserted: Number of events that get actually inserted into MySQL database.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Data_Platform/Systems/EventLogging/Monitoring&oldid=2201885"