Centos Lab10 : Telemetry – Kilo

In this lab you will set up the core OpenStack Telemetry (Ceilometer) module on your AIO node and then enable a set of Ceilometer agents on both your control and compute nodes so that we can capture some intial metrics from both nodes. While Ceilometer was initially intended to support the concept of metering as a pre-cursor input to a billing system, it has been leveraged to provide a level of monitoring and even analytics for event triggering capabilities (as with HEAT). We will enable the basic data capture in this configuration and collect a simple metric.

Telemetry Module Installation on AIO Node

Step 1: Log into the AIO node and install the core Ceilometer components

If you have not already, SSH to your AIO node and run:

ssh centos@aio110
sudo su -
source ~/openrc.sh

Step 2: First you will install the core Ceilometer components:

yum install openstack-ceilometer-api openstack-ceilometer-collector openstack-ceilometer-notification openstack-ceilometer-central openstack-ceilometer-alarm python-ceilometerclient -y

Now you’ve installed:

• ceilometer-api – service to query and view data recorded by collector
• ceilometer-collector – daemon designed to gather and record event and metering data created by notification and polling agents.
• ceilometer-notification – daemon designed to listen to notifications on message queue and convert them to Events and Samples.
• ceilometer-central – polls the public REST APIs of other OpenStack services such as nova and glance, in order to keep tabs on resource existence.
• ceilometer-alarm – daemons to evaluate and notify based on defined alarming rules.
• python-ceilometerclient – the python CLI and SDK components for Ceilometer

We will want to gather data on VMs running on our compute nodes using the ceilometer-compute agent. This agent polls the local libvirt daemon to gather performance data for the instances on a given node and emits this information as AMQP notifications. Since you have installed nova-compute on aio110 and are using this node to provide compute services, you will also want to install ceilometer-compute agent and related python packages to enable compute process monitoring on that node:

yum install openstack-ceilometer-compute -y

Create Database for Telemetry Service

Step 3: The Telemetry service uses a database to store information. Unlike other services installed so far, ceilometer uses MongoDB as default. Mongo is a “noSQL” database that provides much greater capacity and preformnce when compared to mySQL/MariaDB for this class of use (data collection) and was chosen due to the massive amount of data that could potentially be gathered in an even moderately sized OpenStack deployment. Install and specify the location of the database in the configuration file.

In our lab we will deploy MongoDB on the AOIO node:

yum install mongodb-server mongodb -y

Which installs the server code and the client tools and libraries.

Edit the MongoDB configuartion file /etc/mongod.conf to create smaller default files (useful in this environment where the system automatically creates databases per service), and to bind the database server to the management/public IP address of our AIO node:

openstack-config --set /etc/mongod.conf '' smallfiles true
openstack-config --set /etc/mongod.conf '' bind_ip 10.1.64.110

Enable and start the mongoDB service:

systemctl enable mongod.service
systemctl start mongod.service
systemctl status mongod.service

Now that we have installed and configured the MongoDB service, we can create the actual Database we will use with Ceilometer.

mongo --host aio110 --eval ' db = db.getSiblingDB("ceilometer"); db.createUser({user: "ceilometer", pwd: "pass", roles: [ "readWrite", "dbAdmin" ]})'

Example output:

MongoDB shell version: 2.6.9
connecting to: aio110:27017/test
WARNING: The 'addUser' shell helper is DEPRECATED. Please use 'createUser' instead
Successfully added user: { "user" : "ceilometer", "roles" : [ "readWrite", "dbAdmin" ] }

Create User and User-Roles

Step 4: Create a ceilometer user that Ceilometer uses to authenticate with the Keystone.

As with all our services, we once again create a service specific user and associate them with the service tenant and provide an admin role. First create the user:

openstack user create ceilometer --password pass --email [email protected]

And then associate the user with the tenant and associate the role:

openstack role add --project service --user ceilometer admin

Define services and service endpoints

Step 5: Register the Ceilometer with Keystone so that other OpenStack services, clients and SDKs can locate it.

Register the service name:

openstack service create --name ceilometer --description "Telemetry" metering

Create the service end point for the service.

openstack endpoint create --publicurl http://aio110:8777 --internalurl http://aio110:8777 --adminurl http://aio110:8777 --region RegionOne metering

Step 6: Configure the Ceilometer Service

We’ll use the openstack-config tool to update the specific configuration options in the /etc/ceilometer/ceilometer.conf file. As before, we’ll configure the rabbit and keystone services. Since we’re not associating a mysql database, we instead point to the mogodb database we created instead.

In the Ceilometer case, while we configure the keystone admin configuration as normal, we also configure an OS ‘user’ set of credentials. In our environment this is the same user, but it will be used for agents to communicate back to Ceilometer, in which case, we are acting in a fashion more similar to a user connecting to Ceilometer.

We also create a metering secret for services to pass information into the metering sub-system directly via the AMQP service as opposed to an agent talking in a “CLI-like” fashion.

openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rpc_backend rabbit
openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/ceilometer/ceilometer.conf database connection mongodb://ceilometer:pass@aio110:27017/ceilometer
openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_uri http://aio110:5000/v2.0
openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken identity_uri http://aio110:35357
openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_tenant_name service
openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_user ceilometer
openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_password pass
openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_auth_url http://aio110:5000/v2.0
openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_username ceilometer
openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_tenant_name service
openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_password pass
openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_endpoint_type internalURL
openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_region_name regionOne
openstack-config --set /etc/ceilometer/ceilometer.conf publisher telemetry_secret  pass
openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_host aio110
openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_userid test
openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_password test

Configure Compute agent for Telemetry

Ceilometer provides an API service that provides a collector (AMQP based) and agent (CLI-like) for most of the OpenStack services. We will now configure the Nova compute agent and collector:

Step 7: Update the /etc/nova/nova.conf file with the openstack-config tool. Here we define some default collection parameters for the agent to consume.

openstack-config --set /etc/nova/nova.conf DEFAULT instance_usage_audit True
openstack-config --set /etc/nova/nova.conf DEFAULT instance_usage_audit_period hour
openstack-config --set /etc/nova/nova.conf DEFAULT notify_on_state_change vm_and_task_state
openstack-config --set /etc/nova/nova.conf DEFAULT notification_driver messagingv2

Configure the Image Service for Telemetry

Step 8: We will also configure the Glance Agent by updating the /etc/glance/glance-api.conf and /etc/glance/glance-registry.conf files. We just need to configure the notification driver.

openstack-config --set /etc/glance/glance-api.conf DEFAULT notification_driver messagingv2

Edit /etc/glance/glance-registry.conf

openstack-config --set /etc/glance/glance-registry.conf DEFAULT notification_driver messagingv2

Add Block Storage service agent for Telemetry

Step 9: To retrieve data related to Cinder volumes, you must configure the Block Storage service to send notifications to the bus. Edit /etc/cinder/cinder.conf

openstack-config --set /etc/cinder/cinder.conf DEFAULT notification_driver messagingv2
openstack-config --set /etc/cinder/cinder.conf DEFAULT control_exchange cinder

Step 10: Enable and Start the Ceilometer services

First we can enable all the core Ceilometer services:

systemctl enable openstack-ceilometer-api.service openstack-ceilometer-notification.service openstack-ceilometer-central.service openstack-ceilometer-collector.service openstack-ceilometer-alarm-evaluator.service openstack-ceilometer-alarm-notifier.service
systemctl start openstack-ceilometer-api.service openstack-ceilometer-notification.service openstack-ceilometer-central.service openstack-ceilometer-collector.service openstack-ceilometer-alarm-evaluator.service openstack-ceilometer-alarm-notifier.service
systemctl status openstack-ceilometer-api.service openstack-ceilometer-notification.service openstack-ceilometer-central.service openstack-ceilometer-collector.service openstack-ceilometer-alarm-evaluator.service openstack-ceilometer-alarm-notifier.service

We’ll also start the compute specific Ceilometer service:

systemctl enable openstack-ceilometer-compute.service
systemctl start openstack-ceilometer-compute.service
systemctl status openstack-ceilometer-compute.service

The we’ll restart the other openstack services that we modifed to enable their embedded Ceilometer notification services:

systemctl restart openstack-glance-registry.service openstack-glance-api.service openstack-cinder-api.service openstack-cinder-scheduler.service openstack-cinder-volume.service openstack-nova-compute.service

Step 11: Validate that Ceilomter is enabled, and that an inital set of metering elements are enabled.
Use the ceilometer meter-list command to test the access to ceilometer:

ceilometer meter-list

Example output:

+----------------+-------+---------+--------------------------------------+----------------------------------+-------------+
| Name           | Type  | Unit    | Resource ID                          | User ID                          | Project ID                       |
+----------------+-------+---------+--------------------------------------+----------------------------------+-------------+
| image          | gauge | image   | 3470f821-8483-4dd6-a55f-d0c102348c7e | None                             | e0eb7387... |
| image          | gauge | image   | 80ef3d94-c959-4ea4-af59-57933fed4bf5 | None                             | e0eb7387... |
| image.size     | gauge | B       | 3470f821-8483-4dd6-a55f-d0c102348c7e | None                             | e0eb7387... |
| image.size     | gauge | B       | 80ef3d94-c959-4ea4-af59-57933fed4bf5 | None                             | e0eb7387... |
| network        | gauge | network | c2e66fbf-672b-4cf6-8b5a-395c3b776c0f | dea011bd9e8449099707b7d18048f795 | e0eb7387... |
| network.create | delta | network | c2e66fbf-672b-4cf6-8b5a-395c3b776c0f | dea011bd9e8449099707b7d18048f795 | e0eb7387... |
+----------------+-------+---------+--------------------------------------+----------------------------------+-------------+