UCHIWA – Awesome Dashboard for Sensu

It’s been more than a year since i’ve started playing with Sensu. It was one of coolest monitoring projects that i’ve ever worked with. Perfect for Cloud infrastructure and backed by a cool community. Though Sensu is still in Juvenile (v 0.12) state, its mature enough to tackle majority of the monitoring issues. Recently i started migrating all our monitoring from traditional Nagios to Sensu and the pretty cool thing with sensu is, we can directly use the Nagios plugins with Sensu. That’s an easy task for migration. We don’t have to rebuild all the check’s to make with Sensu. But most of the people outside was pointing to the sensu’s default dashboard. Though the dashboard doesn’t looks pretty fancy, it can do all the functions. But having a good dashboard which can display the current status is always a time saver.

So i was being searching for a good dashboard and the first choice on the google search was Sensu-Admin. A Rails project, which needs a backend DB. But still i was not satisfied with it, and i started looking out for something different. The second choice was sabisu. Sabisu uses Cloudant’s hosted Couchdb with Lucene. We just need to store all the events in a Redis List and a custom script which reads the data from the Redis List and pushes it to the Cloudant’s CouchDB. So we basically need a Cloudant account and the Webapp makes Lucene queries to the Cloudant DB and displays the results on the Sabisu dashboard. Though i tried to rebuild the same setup locally, like running a CouchDB+Lucene locally and sending the same data to the local couchdb. With some codehack’s i was able to make the webapp talk to my local CouchDB and display the results on the dashboard.

But then, I found a super cool dashboard project called UCHIWA which was started in Github a few days back by Simon Plourde. Uchiwa is simple dashboard built with NodeJS and uses SocketIO for real time updates. The screenshot’s looks super cool and i decided to give it a try. It has only one dependency, NodeJS, no backend DB’s required as it talks to the Sensu’s API in realtime.

Setting Up NodeJS

For Ubuntu, we can use the chris-lea’s PPA.

apt-get install python-software-properties        # required for "apt-add-repository" binary

apt-add-repository ppa:chris-lea/node.js

apt-get update

apt-get install nodejs

now we have the latest NodeJS on our system, we can start setting up Uchiwa.

Setting Up Uchiwa Dashboard

Uchiwa’s source is available in Github.

git clone https://github.com/palourde/uchiwa.git

Once cloned, the repository contains the “package.json” file which contains the list of necessary dependencies. We can use “npm” (node package manager) to install all these.

cd uchiwa

npm install

Now we need to create a config file for the app. There is a sample config file available in the repo. So we need to mention the Sensu’s API IP and Port number and also the auth credentials if any. Plus auth credentials for accessing Uchiwa Dashboard page.

once all these are set, we are done. We just need to start the service.

node app.js 

We can access the page via http://localhost:3000/, or we can proxy pass from the webserver. Instructions for Nginx is available on the Readme of the project. Now we need to keep this app running all the time. So it’s better to create a init/upstart process for the same, so that the process will start automatically when the system reboots. There is a cool Node project called forever which is a simple CLI tool for ensuring that a given script runs continuously.

I’ve created an upstart script for Uchiwa, bu putting a conf file “uchiwa.conf” in the “/etc/init” directory. Below is the content for the conf file. Once the file is in place, we have to do a reload of the upstart configuration. initctl reload-configuration will do the trick.

description "uchiwa - dashboard for sensu"
env APP_PATH = "/usr/local/uchiwa/"

start on startup
stop on shutdown

script
  cd $APP_PATH
  exec forever start app.js
end script

Uchiwa looks pretty cool and neat and it has the stash support also. There are couple of addons required like “Downtime”, but Uchiwa is a pretty new project and i’m sure that this project is gonna grow soon. It has already received 99 stars on the Github. Kudos to Simon Plourde for Open Sourcing this awesome project.

Phone Call Notification for Sensu Using Plivo

It’s almost 2 weeks since i’ve joined Plivo’s DevOps family. I was spending a lot of time on understanding their API’s as i’m new to telephony. Plivo’s API made telephony so easy that even a person with basic programming language can built powerfull telephony apps to suit to their infrastructure. So when i started playing around with the API, i decided to take it to a different level. I’m strong lover of Sensu Monitoring tool, so i decided to integrate Plivo with Sensu to built a new Notification system using the Phone call. Many people rely on Pagerduty for the same feature. But that part is completly managed by PagerDuty, where in for the alerts which we sent to PagerDuty, they will notify us via phone call to the phone numbers mentioned on Pager Duty. So i decided to built a similar Handler to Sensu, so that we can have a similar feature on Sensu. In this blog i will explain how i achieved the same with Plivo Framework.

Plivo provides Application Programming Interfaces (APIs) to make and receive calls, send SMS, make a conference call, and more. These APIs are used in conjunction with XML responses to control the flow of a call or a message. Developers can create Session Initiation Protocol (SIP) endpoints to perform the telephony operations and the APIs are platform independent and can be used in any programming environment such as PHP, Ruby, Python, etc. It also provides helper libraries for these programming languages.

Here the Sensu will be initiating outbound calls using Plivo’s Call API, to the required numbers and will read out the Alert using Plivo’s Text to Speech Engine. First we need an account on Plivo Cloud, we can go to SignUP page, and need to create an account. By using the default, we can make test calls. But for longer usage, i will suggest to buy some credits for using the service extensively. Once we login with credentials, at the dashboard we can see Plivo AuthID and Plivo AuthToken, which is required to access Plivo’s API. Now for making out bound calls, we need to use the Call API. We also need to provide an ”answer_url” which contains XML instructions to direct the progress of the call. Plivo will fetch the ”answer_url” and executes te XML instructions. So here i will be using Sinatra to create a web app that will returns the XML. The main challenge is the text to be mentioned in the XML need to retrieved from the alert. So we cannot predefine the text as well as the request url, because it will be in dynamic in nature.

Solution :- Here i’m going to use a publically accessible Redis server which is protected with a password. When Sensu handler receives the alert, it will create a hash from the alert received with a random UUID as the name of the hash. And the same UUID will be used as the request path for the answer_url. And when sinatra recieves a request, by default the requested path wont be existing in the sinatra’s config file, as it’s a dynamically generated. So by default Sinatra will return a 404 response. But in Sinatra, there is an option called ”not_found”, where we can customize the response instead of returning 404 directly. So i will be using the “not_found” option and instead of directly returing a 404, i will make sinatra to talk to my redis server. Since the UUID can be fetched from the request URL, and the same is used in the redis as the name of the Hash, Sinatra can get the details of the alert from the Redis. These details are then used to create the XML and will be returned as the response to Plivo. It will be better to go through Plivo’s API documentation to understand more about the XML responses and outbound calls.

For those who don’t have a dedicated server to host the Sinatra app and Redis server, heroku can be used to host the Sinatra app. We can also use the redis addon availabe at the Heroku for our usage.

Sinatra App

Below is the code for the Sinatra app. The ”not_found” option is the one which performs the connection with the Redis.

require 'plivo'
require 'sinatra'
require 'redis'
require 'rest_client'

get '/' do
  play_loop = 1
  lang = "en-US"
  voice = "WOMAN"
  text = "Congratulations! You just made a text to speech app on Plivo cloud!"
  @path = request.path
  puts @path
  speak_params = {
                     'loop' => play_loop,
                     'language' => lang,
                     'voice' => voice,
                     }

  plivo = Plivo::Speak.new(text, speak_params)

  response = Plivo::Response.new()
  response.add(p)
  return response.to_xml
end

not_found do
  @path = request.path
  keyword = @path[1..-1]
  response = Redis.new(:host => "<redis_host_name>", :port => <redis_port>, :password => "<redis_pass>")
  data_red = response.get("#{keyword}")
  if data_red == nil
         return "404 Not Found"
  else
         data = JSON.parse(response.get("#{keyword}"))
         text = data["text"]
         play_loop = 1
         lang = "en-US"
         voice = "WOMAN"
         speak_params = {
                        'loop' => play_loop,
                        'language' => lang,
                        'voice' => voice,
                        }

         plivo = Plivo::Speak.new(text, speak_params)

         response = Plivo::Response.new()
         response.add(p)
         return r.to_xml
   end
end

The above app will redirect all the non found requests and make a requests to the Redis server using the request.path as the name of the hash. If there is a valid hash table, it will generate a Plivo XML else it will return “404 Not Found”.

Plivo Handler

Below is the Plivo Handler code. this is the Initial code, need to be tweaked to suit to the Sensu’s coding standard. Also the settings option need to be added so that all the parameters can be provided in the JSON file of the Handler. But as of now, this handler has been tested with Sensu and it is working perfectly fine.

require 'rubygems'
require 'sensu-handler'
require 'timeout'
require 'securerandom'
require 'plivo'
require 'redis'
require 'timeout'


class PLIVO < Sensu::Handler

  def short_name
    @event['client']['name'] + '/' + @event['check']['name']
  end

  def action_to_string
    @event['action'].eql?('resolve') ? "RESOLVED" : "ALERT"
  end

  def handle
    plivo_auth_id = "XXXXXXXXXXXXXXXX"          # => available at the plivo dashboard
    plivo_auth_token = "XXXXXXXXXXXXXXXX"       # => available at the plivo dashboard
    body = <<-BODY.gsub(/^ {14}/, '')
            #{@event['check']['output']}
            Host: #{@event['client']['name']}
           BODY
    uuid = SecureRandom.hex
    r = Redis.new(:host => "<redis_host_name>", :port => <redis_port>, :password => "<redis_pass>")
    temp = {
            'text' => "#{body}"
            }
    plivo_number = "<YOUR PLIVO NUMBER"
    to_number = "<DESTINATION NUMBER>"
    answer_url = "<YOUR HEROKU APP URL>/#{uuid}"
    call_params = {
                     'from' => plivo_number,
                     'to' => to_number,
                     'answer_url' => answer_url,
                     'answer_method' => 'GET'
                     }
         r.set "#{uuid}", temp.to_json
         r.expire "#{uuid}", <EXPIRY TTL>
         sleep 5
     begin
       timeout 10 do
         puts "Connecting to Plivo Cloud.."
         plivo = Plivo::RestAPI.new(plivo_auth_id, plivo_auth_token)
     details = plivo.make_call(call_params)
       end
       rescue Timeout::Error
       puts "timed out while attempting to contact Plivo Cloud"
         end
  end
end

The above handler will make an outbound call to the Destination number and plivo’s text to speech engine will read out the Alert summary on the call. Now If you want to call multiple user’s, simply create an array or hash with the contact numbers and iterate over it.

This handler will give us the same feature of Phone Call notification similar to Pager Duty. We just need to pay for the phone call usage. Dedicated to all those who wants an affordable yet a good notification system with Sensu. I will be merging out my code soon to the sensu-community repository.

Sensu Admin – a GUI for Sensu API

In my previous post’s, i’ve explained on How to setup Sensu server and setting up check’s and handler’s. The default dashboard is very simple with limited options, but for those who wants a full fledged dashboard, there is a Rails project in Github Sensu-Admin. So let’s try setting it up.

First clone the repository from Github.

$ git clone https://github.com/sensu/sensu-admin.git

Now go to sensu-admin folder, and run bundle install to install all the dependency gems. Now go inside the ”config” folder, edit the ”database.yml” and fill in the database details. I’m going to use mysql, below is my database config.

development:
   adapter: mysql2
   database: sensudb
   username: sensu
   password: secreto
   host: localhost
production:
   adapter: mysql2
   database: sensudb
   username: sensu
   password: secreto
   host: localhost

Now run rake db:migrate and then rake db:seed. The seed file creates auser account named ”admin@example.com” with password ”secret”.

We can start the Rails app by running “rails s”, this will start the app using the thin webserver at port 3000. Access the dashboard using the url ”http://server_ip:3000” Login to the dashboard with the admin@example.com and go to the “*Account” tab and modify the default user name and password. Now we go through tabs and check if it displays the checks, clients, events etc properly. This is a screenshot of the SensuAdmin dashboard.

Sensu – Adding Check’s and Handler’s

In my previous post, i’ve explained on how to setup Sensu Server and Client. Now i’m going to explain how to setup Check’s and Handler’s in Sensu. There is a very good collection of sensu-community-plugins.

Setting up Check’s

On the Sensu Client Node,

First clone the plugins repository on the client node. Now install the ”sensu-plugin” gem on the client node. And then copy the required plugins to /etc/sensu/plugins/ folder.

On the Sensu Server,

We need to define the check first. Create a json config file for the check in /etc/sensu/conf.d. Following is a sample check config,

     {
    "checks": {
         "snmp_check": {
         "handlers": ["default"],
         "command": "/etc/sensu/plugins/check-snmp.rb -w 10 -c 20",
         "interval": 30,
         "subscribers": [ "snmp" ]
          }
      }
   }

The above check will be applied to all clients subscribed to ”snmp” exchange. Based on the interval, Server will publish this check request, which will reach all the clients subscribed to the ”snmp” exchange using an arbitrary queue. The client will run the command mentioned in the command part, and then it will publish the result back to th server through Result queue. The check_snmp is a small plugin written by me. If we check the sensu-server log, we can see the result coming from the client machine. Below one is a similar log output in my sensu-server log.

{"timestamp":1366968018},"check":{"handlers":["default","mailer"],"command":"/etc/sensu/plugins/check-snmp.rb -w 1 -c 3","interval":100,"subscribers":["snmp"],"name":"snmp_check","issued":1366968407,"executed":1366968028,"output":"CheckSNMP WARNING: Warning state detected\n","status":1,"duration":0.526,"history":["0","0","1"]},"occurrences":1,"action":"create"},"handler":{"type":"pipe","command":"true","name":"default"}}

The above log line shows us what are handler’s enabled for this check, what is the executed command, subcribers, name of the check, timestamp at the time when the command was issued, timestamp of the time when the server has received the result, Output of the check command etc. If there is any while executing th check command, we can see the errors popping in the log’s soon after this line in the server log.

Setting up Handler’s

Sensu has got a very good collection Handler’s, available at the sensu-community-plugin repo in github. For example there is a hanlder called ”show”, available at the debug section in Handler’s, which will display a more debug report about the Event as well as the Sensu server’s settings. This is the output which i got after applying ”show” handler in my serverlog. But it’s not possible to go check the log’s continously, so there another plugin called “mailer”, which can send email alerts like how nagios does.

So first get the “mailer” plugin files from the sensu-community-plugin repo in github.

wget -O /etc/sensu/handlers/mailer.rb https://raw.github.com/sensu/sensu-community-plugins/master/handlers/notification/mailer.rb
wget -O /etc/sensu/conf.d/mailer.json https://raw.github.com/sensu/sensu-community-plugins/master/handlers/notification/mailer.json

Now edit the mailer.json, and change the settings to fit to our environment. We need to define a new pipe handler for this new handler. So create a file /etc/sensu/conf.d/handler_mailer.json, and add the below lines to it.

        {
    "handlers": {
        "mailer": {
        "type": "pipe",
        "command": "/etc/sensu/handlers/mailer.rb"
        }
          }
      }

Now go to the one of the check config files, where we want to apply this new “mailer” handler.

           {
    "checks": {
         "snmp_check": {
         "handlers": ["default", "mailer"],         
         "command": "/etc/sensu/plugins/check-snmp.rb -w 10 -c 20",
         "interval": 30,
         "subscribers": [ "snmp" ]
          }
      }
   }

Now restart the sensu-server to make the new changes to come into effect. If everything goes fine, when the sensu detects a state change it will execute this mailer handler, we can also see the below lines in server log.

"action":"create"},"handler":{"type":"pipe","command":"/etc/sensu/handlers/mailer.rb","name":"mailer"

Sensu is executing the mailer script, and if there is any problem, we will see the corresponding error following the above line, or we will receive the email alert to email id mentioned in the “mailer.json” file. But in my case, i was getting an error, when the sensu invoked the “mailer” handler.

{"timestamp":"2013-04-25T15:03:32.002132+0530","level":"info","message":"/etc/sensu/handlers/mailer.rb:28:in `handle': undefined method `[]' for nil:NilClass (NoMethodError)"}
{"timestamp":"2013-04-25T15:03:32.002308+0530","level":"info","message":"\tfrom /var/lib/gems/1.9.1/gems/sensu-plugin-0.1.7/lib/sensu-handler.rb:41:in `block in <class:Handler>'"}

After playing for some time, i came to know that, it was not parsing the options from the mailer.json file, so i manually added the smtp and email settings directly in mailer.rb file. Then it started working fine. I’m writing a small script which will be using the basic ‘net/smtp’ library to send out mails. There are many other cool Handler’s like sending matrices to Graphite, Logstash, Graylog, sending notifcations to irc,xmpp,campfire etc. Compare to traditional monitoring tools, Sensu is an Amazing tool, we can use any check script’s, whether it’s ruby or perl or bash, doesn’t matter. The one common thing which i heard about other people, was the lack of proper dashboard like the traditional monitoring tools. Though Sensu dashboard is a simple one, i’m sure it will improve a lot in future.

Since I’m a CLI Junky, I dont care much about the dashboard thing, apart from that i have many good and interesting stuffs to hang around with Sensu. Cheers to portertech and sonian for open sourcing such an amazing tool.

Sensu – Cloud Monitoring Tool

Monitoring always plays an important role, especially for sysadmins. There are a lot of Monitoring tools available, like Nagios, Zenoss, Icinga etc. Sensu is a new generation Cloud monitoring tool designed by Sonian. Sensu is bascially written in Ruby, uses RabbitMQ Server as the Message Broker for Message transactions, and Redis for storing the data’s.

Sensu has 3 operation Mode.

1) Request-Reply Mode, where the server will send a check request to the clients through the RabbitMQ and the clients will reply back the results.

2) Standalone Mode, where the server will not send any check request, instead the client itself will run the checks according to interval mentioned, and sends the results to the sensu master through the Result queue in RabbitMQ.

3) Push Mode, where the client will send out results to a specific handler.

So now we can start installing the dependencies for sensu, ie, RabbitMQ and Redis.

Setting up RabbitMQ

Let’s add the RabbitMQ official APT repo.

$ echo "deb http://www.rabbitmq.com/debian/ testing main" >/etc/apt/sources.list.d/rabbitmq.list

$ curl -L -o ~/rabbitmq-signing-key-public.asc http://www.rabbitmq.com/rabbitmq-signing-key-public.asc

$ apt-key add ~/rabbitmq-signing-key-public.asc && apt-get update

Now we can install RabbitMQ

$ apt-get install rabbitmq-server erlang-nox

Now we need to generate SSL certificates for RabbitMQ and the sensu clients. We can use RabbitMQ with out ssl also, but it will more secure with SSL, @joemiller has wrote a script to generate the SSL certificates. It’s avaliable in his GitHub repo. Clone the repo and modify the “openssl.cnf” according to our need and then we can go ahead with generating the certificates.

$ git clone git://github.com/joemiller/joemiller.me-intro-to-sensu.git

$ cd joemiller.me-intro-to-sensu/

$ ./ssl_certs.sh clean && /ssl_certs.sh generate

Now copy the server key and cert files to the RabbitMQ folder in “/etc/rabbitmq/”

$ mkdir /etc/rabbitmq/ssl

$ cp server_key.pem /etc/rabbitmq/ssl/

$ cp server_cert.pem /etc/rabbitmq/ssl/

$ cp testca/cacert.pem /etc/rabbitmq/ssl/

Now create the RabbitMQ config file, “/etc/rabbitmq/rabbitmq.config”, and add the following lines in it.

[
  {rabbit, [
      {ssl_listeners, [5671]},
      {ssl_options, [{cacertfile,"/etc/rabbitmq/ssl/cacert.pem"},
               {certfile,"/etc/rabbitmq/ssl/server_cert.pem"},
               {keyfile,"/etc/rabbitmq/ssl/server_key.pem"},
               {verify,verify_peer},
               {fail_if_no_peer_cert,true}]}
    ]}
].

Once the config file is created, restart the RabbitmQ server. Now RabbitMQ has a cool management console, we can enable this by running ”rabbitmq-plugins enable rabbitmq_management” in console. Once the Management console is enabled, we can access it RabbitMQ Web UI: Username is “guest”, password is “guest” – http://SENSU-SERVER:55672. Protocol amqp should be bound to port 5672 and amqp/ssl on port 5671.

Now let’s create a vhost and user for Sensu in RabbitMQ.

 $ rabbitmqctl add_vhost /sensu

 $ rabbitmqctl add_user sensu mypass

 $ rabbitmqctl set_permissions -p /sensu sensu ".*" ".*" ".*"

Setting up Redis Server

Now we can set up Redis server. This will used by Sensu for stroring data’s. Ubuntu’s Apt repo ships with latest Redis server, so we can directly install it.

$ apt-get install redis-server

Installing Sensu Server

Sensu has a public repository which can be used to install the necessary sensu packages. First we need to add the repository public key.

$ wget -q http://repos.sensuapp.org/apt/pubkey.gpg -O- | sudo apt-key add -

Now add the repo sources in APT

$ echo " deb     http://repos.sensuapp.org/apt sensu main" >> /etc/apt/sources.list && apt-get update

$ apt-get install sensu

Enable the sensu services to start automatically during system startup.

$ update-rc.d sensu-server defaults

$ update-rc.d sensu-api defaults

$ update-rc.d sensu-client defaults

$ update-rc.d sensu-dashboard defaults

Copy the client ssl cert and key to /etc/sensu folder, say to a subfolder ssl.

$ cp client_key.pem client_cert.pem  /etc/sensu/ssl/

Now we need setup the sensu master, create a file ”/etc/sensu/config.json” and add the below lines.

         {
        "rabbitmq": {
          "ssl": {
            "private_key_file": "/etc/sensu/ssl/client_key.pem",
            "cert_chain_file": "/etc/sensu/ssl/client_cert.pem"
          },
          "port": 5671,
          "host": "localhost",
          "user": "sensu",
          "password": "mypass",
          "vhost": "/sensu"
        },
        "redis": {
          "host": "localhost",
          "port": 6379
        },
        "api": {
          "host": "localhost",
          "port": 4567
        },
        "dashboard": {
          "host": "localhost",
          "port": 8080,
          "user": "admin",
          "password": "sensu@123"
        },
        "handlers": {
          "default": {
            "type": "pipe",
            "command": "true"
          }
        }
      }

By default sensu package comes with all sensu-server,sensu-client,sensu-api and sensu-dashboard., If we dont want to use the current machine as a client, we can stop the sensu-client from running, and do not create the client config. But for testing purpose, i’m going to add the current machine as client also. Create a file ”/etc/sensu/conf.d/client.json” and add the client configuration in JSON format.

        {
          "client": {
          "name": "sensu.test.com",
          "address": "192.168.1.108",
          "subscriptions": [ "vmmaster" ]
         }
       }

Now restart the sensu-client to affect the changes. The logs are recorded at ”/var/log/sensu/sensu-client.log” file. We can access the sensu-dashboard from “http://SENSU SERVER:8080”, with the username and password mentioned in the config.json file.

Setting up a Separate Sensu-Client Node

If we want to setup sensu-client on a separate node, just dd the Sensu apt repo, and install the sensu package. After that just enable only the sensu-client service and remove all other sesnu-services. Then create a config.json file and add only the rabbitmq server details in it. Now generate a separate SSL certificate for the new client and use that in the config file.

       {
      "rabbitmq": {
        "ssl": {
          "private_key_file": "/etc/sensu/ssl/client1_key.pem",
          "cert_chain_file": "/etc/sensu/ssl/client1_cert.pem"
        },
        "port": 5671,
        "host": "192.168.1.108",
        "user": "sensu",
        "password": "mypass",
        "vhost": "/sensu"
      }
    }

Now create the “client.json” in the “/etc/sensu/conf.d/” folder.

        {
              "client": {
              "name": "client1.test.com",
              "address": "192.168.1.212",
              "subscriptions": [ "vmmaster" ]
             }
           }

Restart the the sensu-clinet, and check the “/var/log/sensu/sensu-client.log”, if things goes fine, we can see client connecting to the RabbitMQ server also we can see the config is getting applied.

{"timestamp":"2013-04-23T22:53:27.870728+0530","level":"warn","message":"config file applied changes","config_file":"/etc/sensu/conf.d/client.json","changes":{"client":[null,{"name":"client1.test.        com","address":"192.168.1.212","subscriptions":["vmmaster"]}]}}
{"timestamp":"2013-04-23T22:53:27.879671+0530","level":"info","message":"loaded extension","type":"mutator","name":"only_check_output","description":"returns check output"}
{"timestamp":"2013-04-23T22:53:27.883504+0530","level":"info","message":"loaded extension","type":"handler","name":"debug","description":"outputs json event data"}

Once the Sensu Server and Client are configured successfully, then we can go ahead adding the check’s. One of the best thing of sensu, all the config’s are written in JSON format, which very easy for us to create as well as to understand things. In the next blog, i will explain on how to create the check’s and how to add these check’s to various clients, and how to add handler’s like Email alerts, Sending Metrics to graphite.