VictoriaMetrics static scraper

I’ve got an Electric Imp Environment Tail in my office. It monitors the temperature, humidity and pressure. Currently, to display a graph, it’s using flot.js and some shonky Javascript that I wrote. It remembers samples from the last 48 hours.

Rather than write more shonky Javascript, or post it to a third-party metrics service, I’m just going to add it to my cluster’s VictoriaMetrics+Grafana setup.

Exposing Metrics

The first thing to do is to expose the most-recent readings in Prometheus-compatible format. I’ve updated the agent source code to include the following:

app.get("/metrics", function(context) {
    // Unix epoch, seconds.
    local t = time();
    // Multiplying by 1000 overflows, so just jam some zeroes on the end in the string format.
    context.send(200, format("temperature %f %d000\nhumidity %f %d000\npressure %f %d000\n",
        LATEST.tempHumid.temperature, t,
        LATEST.tempHumid.humidity, t,
        LATEST.pressure.pressure, t));

Scraping metrics

To scrape those metrics, we need a VMStaticScrape resource:

kind: VMStaticScrape
  name: imp-environment-tail
  namespace: monitoring-system
  jobName: "imp-environment-tail-office"
    - targets: [""]
        env: office
      scheme: "https"
      path: "/agent-id-goes-here/metrics"

The annoying part here is that it won’t take a URL; you need to specify the scheme and path separately from the targets.

The labels are to make it easier to find later in Grafana.


To actually run the scraper, we need a VMAgent resource:

kind: VMAgent
  name: agent
  namespace: monitoring-system
  staticScrapeSelector: {}
  staticScrapeNamespaceSelector: {}
    - url: "http://vmsingle-vm-database.monitoring-system.svc.cluster.local:8429/api/v1/write"

Note the <service>.<namespace>.svc.cluster.local bit. I think the selectors are required as well, otherwise it doesn’t bother scraping anything.

Agent status

You can check the status of the VM agent with the following command and a browser:

kubectl --namespace monitoring-system port-forward \
  --address \
  service/vmagent-agent 8429:8429


Once that was all working, I quickly cobbled together a dashboard: