Real-Time Infrastructure Monitoring with Amazon Echo

Years ago, managing your infrastructure through voice was a science-fiction movie, but thanks to virtual assistants like Alexa it becomes a reality. In this post, I will show you how I was able to monitor my infrastructure on AWS using a simple Alexa Skill.

At a high level, the architecture of the skill is as follows:

I installed data collector agent (Telegraf) in each EC2 instance to collect metrics about system usage (disk, memory, cpu ..) and send them to a time-series database (InfluxDB)

Once my database is populated with metrics, Amazon echo will transform my voice commands to intents that will trigger a Lambda function, which will use the InfluxDB REST API to query the database.

Enough with talking, lets build this skill from scratch, clone the following GitHub repository:

1	git clone https://github.com/mlabouardy/alexa-monitor

Create a simple fleet of EC2 instances using Terraform. Install the AWS provider:

1	terraform init

Set your own AWS credentials in variables.tfvars. Create an execution plan:

1	terraform plan --var-file=variables.tfvars

Provision the infrastructure:

1	terraform apply --var-file=variables.tfvars

You should see the IP address for each machine:

To install Telegraf on each machine, I used Ansible, update the ansible/inventory with your nodes IP addresses as follows:

[nodes]
35.178.10.226 hostname=Rabat
35.177.164.157 hostname=Paris
52.56.126.138 hostname=London

[nodes:vars]
remote_user=ec2-user
influxdb_ip=35.177.123.180

Execute the playbook:

1	ansible-playbook -i inventory playbook.yml --private-key=key.pem

If you connect via SSH to one of the server, you should see the Telegraf agent is running as Docker container:

In few seconds the InfluxDB database will be populated with some metrics:

Create an invocation name – aws – This is the word that will trigger the Skill.

In the Intent Schema box, paste the following JSON code:

{
  "intents" : [
    {
      "intent" : "GetSystemUsage",
      "slots" : [
        {
          "name" : "Metric",
          "type" : "TYPE_OF_METRICS"
        },
        {
          "name" : "City",
          "type" : "LIST_OF_CITIES"
        }
      ]
    }
  ]
}

Create a new slot types to store the type of metric and machine hostname:

TYPE_OF_METRICS:

CPU
DISK
MEMORY

LIST_OF_CITIES:

Paris
Rabat
London

Under Uterrances, enter all the phrases that you think you might say to interact with the skill

1	GetSystemUsage {Metric} usage of machine in {City}

Click on “Next” and you will move onto a page that allows us to use an ARN (Amazon Resource Name) to link to AWS Lambda.

Before that, let’s create our lambda function, go to AWS Management Console and create a new lambda function from scratch:

Note: Select US East (N.Virginia), which is a supported region for Alexa Skill Kit.

Make sure the trigger is set to Alexa Skills Kit, then select Next.

The code provided uses the InfluxDB client to fetch metrics from database.

function MetricsDB(){
  this.influx = new Influx.InfluxDB({
    host: process.env.INFLUXDB_HOST,
    database: process.env.INFLUXDB_DATABASE
  })
}

MetricsDB.prototype.getCPU = function(machine, callback){
  this.influx.query(`
      SELECT last(usage_system) AS system, last(usage_user) AS "user"
      FROM cpu
      WHERE time > now() - 5m AND host='${machine}'
    `).then(result => {
      var system_usage = result[0].system.toFixed(2)
      var user_usage = result[0].user.toFixed(2)
      callback(`System usage is ${system_usage} percent & user usage is ${user_usage} percent`)
  }).catch(err => {
      callback(`Cannot get cpu usage values`)
  })
}

Specify the .zip file name as your deployment package at the time you create the Lambda function. Don’t forget to set the InfluxDB Hostname & Database name as an environment variables: