[![Sensu Bonsai Asset](https://img.shields.io/badge/Bonsai-Download%20Me-brightgreen.svg?colorB=89C967&logo=sensu)](https://bonsai.sensu.io/assets/sensu-plugins/sensu-plugins-disk-checks)
[![Build Status](https://travis-ci.org/sensu-plugins/sensu-plugins-disk-checks.svg?branch=master)](https://travis-ci.org/sensu-plugins/sensu-plugins-disk-checks)
[![Gem Version](https://badge.fury.io/rb/sensu-plugins-disk-checks.svg)](http://badge.fury.io/rb/sensu-plugins-disk-checks)
[![Code Climate](https://codeclimate.com/github/sensu-plugins/sensu-plugins-disk-checks/badges/gpa.svg)](https://codeclimate.com/github/sensu-plugins/sensu-plugins-disk-checks)
[![Test Coverage](https://codeclimate.com/github/sensu-plugins/sensu-plugins-disk-checks/badges/coverage.svg)](https://codeclimate.com/github/sensu-plugins/sensu-plugins-disk-checks)
[![Dependency Status](https://gemnasium.com/sensu-plugins/sensu-plugins-disk-checks.svg)](https://gemnasium.com/sensu-plugins/sensu-plugins-disk-checks)

## Sensu Disk Checks Plugin
- [Overview](#overview)
- [Usage examples](#usage-examples)
- [Configuration](#configuration)
  - [Sensu Go](#sensu-go)
    - [Asset definition](#asset-definition)
    - [Check definition](#check-definition)
  - [Sensu Core](#sensu-core)
    - [Check definition](#check-definition)
- [Functionality](#functionality)
- [Additional information](#additional-information)
- [Installation from source and contributing](#installation-from-source-and-contributing)

### Overview

This plugin provides native disk instrumentation for monitoring and metrics collection, including: health, usage, and various metrics.

The Sensu assets packaged from this repository are built against the Sensu ruby runtime environment. When using these assets as part of a Sensu Go resource (check, mutator or handler), make sure you include the corresponding Sensu ruby runtime asset in the list of assets needed by the resource.  The current ruby-runtime assets can be found [here](https://bonsai.sensu.io/assets/sensu/sensu-ruby-runtime) in the [Bonsai Asset Index](bonsai.sensu.io)

#### Files
 * bin/check-disk-usage.rb
 * bin/check-fstab-mounts.rb
 * bin/check-smart-status.rb
 * bin/check-smart.rb
 * bin/check-smart-tests.rb
 * bin/metrics-disk.rb
 * bin/metrics-disk-capacity.rb
 * bin/metrics-disk-usage.rb

### Usage examples

#### Help

**check-disk-usage.rb**
```
Usage: check-disk-usage.rb (options)
    -c PERCENT                       Critical if PERCENT or more of disk full
    -w PERCENT                       Warn if PERCENT or more of disk full
    -t TYPE[,TYPE]                   Only check fs type(s)
    -K PERCENT                       Critical if PERCENT or more of inodes used
    -i MNT[,MNT]                     Ignore mount point(s)
    -o TYPE[.TYPE]                   Ignore option(s)
    -p PATHRE                        Ignore mount point(s) matching regular expression
    -x TYPE[,TYPE]                   Ignore fs type(s)
    -I MNT[,MNT]                     Include only mount point(s)
    -W PERCENT                       Warn if PERCENT or more of inodes used
    -m MAGIC                         Magic factor to adjust warn/crit thresholds. Example: .9
    -l MINIMUM                       Minimum size to adjust (in GB)
    -n NORMAL                        Levels are not adapted for filesystems of exactly this size, where levels are reduced for smaller filesystems and raised for larger filesystems.
```

**metrics-disk-usage.rb**
```
Usage: metrics-disk-usage.rb (options)
    -B, --block-size BLOCK_SIZE      Set block size for sizes printed
    -f, --flatten                    Output mounts with underscore rather than dot
    -i, --ignore-mount MNT[,MNT]     Ignore mounts matching pattern(s)
    -I, --include-mount MNT[,MNT]    Include only mounts matching pattern(s)
    -l, --local                      Only check local filesystems (df -l option)
        --scheme SCHEME              Metric naming scheme, text to prepend to .$parent.$child
```


### Configuration
#### Sensu Go
##### Asset registration

Assets are the best way to make use of this plugin. If you're not using an asset, please consider doing so! If you're using sensuctl 5.13 or later, you can use the following command to add the asset: 

`sensuctl asset add sensu-plugins/sensu-plugins-disk-checks`

If you're using an earlier version of sensuctl, you can download the asset definition from [this project's Bonsai Asset Index page](https://bonsai.sensu.io/assets/sensu-plugins/sensu-plugins-disk-checks).

##### Asset definition

```yaml
---
type: Asset
api_version: core/v2
metadata:
  name: sensu-plugins-disk-checks
spec:
  url: https://assets.bonsai.sensu.io/73a6f8b6f56672630d83ec21676f9a6251094475/sensu-plugins-disk-checks_5.0.0_centos_linux_amd64.tar.gz
  sha512: 0ce9d52b270b77f4cab754e55732ae002228201d0bd01a89b954a0655b88c1ee6546e2f82cfd1eec04689af90ad940cde128e8867912d9e415f4a58d7fdcdadf
```

##### Check definition

```yaml
---
type: CheckConfig
spec:
  command: "metrics-disk-usage.rb"
  handlers: []
  high_flap_threshold: 0
  interval: 10
  low_flap_threshold: 0
  publish: true
  runtime_assets:
  - sensu-plugins/sensu-plugins-disk-checks
  - sensu/sensu-ruby-runtime
  subscriptions:
  - linux
  output_metric_format: graphite_plaintext
  output_metric_handlers:
  - influx-db
```
#### Sensu Core
##### Check definition
```json
{
  "checks": {
    "metrics-disk-usage": {
      "command": "metric-disk-usage.rb",
      "subscribers": ["linux"],
      "interval": 10,
      "refresh": 10,
      "handlers": ["influxdb"]
    }
  }
}

```

### Functionality

**check-disk-usage**

Check disk capacity and inodes based upon the gem sys-filesystem.

Can adjust thresholds for larger filesystems by providing a 'magic factor'
(`-m`).  The default, `1.0`, will not adapt threshold percentages for volumes.

The `-l` option can be used in combination with the 'magic factor' to specify
the minimum size volume to adjust the thresholds for.

By default all mounted filesystems are checked.

The `-x` option can be used to exclude one or more filesystem types. e.g.

    check-disk-usage.rb -x debugfs,tracefs

The `-p` option can be used to exlucde specific mount points. e.g.

    check-disk-usage.rb -p /run/lxcfs

It's also possible to use regular expressions with the `-x` or `-p` option

    check-disk-usage.rb -p '(\/var|\/run|\/sys|\/snap)'

Refer to [check_mk's](https://mathias-kettner.de/checkmk_filesystems.html)
documentation on adaptive thresholds.

You can also visualize the adjustment using
[WolframAlpha]([https://www.wolframalpha.com/input/) with the following:

    y = 100 - (100-P)*(N^(1-m))/(x^(1-m)), y = P for x in 0 to 1024

Where P = base percentage, N = normalize factor, and m = magic factor

**check-fstab-mounts**

Check the mount points in */etc/fstab* to ensure they are all accounted for.

**metrics-disk-capacity**

Acquire disk capacity metrics from `df` and convert them to a form usable by graphite

**metrics-disk**

Read */proc/iostats* for disk metrics and put them in a form usable by Graphite.  See [iostats.txt](http://www.kernel.org/doc/Documentation/iostats.txt) for more details.

**metrics-disk-usage**

Based on disk-capacity-metrics.rb by bhenerey and nstielau. The difference here being how the key is defined in graphite and the size we emit to graphite(now using megabytes), inode info has also been dropped.

**check-smart-status**

Check the SMART status of hardrives and alert based upon a given set of thresholds

**check-smart**

Check the health of a disk using `smartctl`

**check-smart-tests**

Check the status of SMART offline tests and optionally check if tests were executed in a specified interval

### Additional information
The `check-smart.rb` and `check-smart-status.rb` scripts can make use of a json file living on disk in `/etc/sensu/conf.d`with the name `smart.json`. You can see a sample input file used by these scripts below. Please refer to the individual scripts for further details.
```json
{
  "smart": {
    "attributes": [
      { "id": 1, "name": "Raw_read_Error_Rate", "read": "left16bit" },
      { "id": 5, "name": "Reallocated_Sector_Ct" },
      { "id": 9, "name": "Power_On_Hours", "read": "right16bit", "warn_max": 10000, "crit_max": 15000 },
      { "id": 10 , "name": "Spin_Retry_Count" },
      { "id": 184, "name": "End-to-End_Error" },
      { "id": 187, "name": "Reported_Uncorrect" },
      { "id": 188, "name": "Command_Timeout" },
      { "id": 193, "name": "Load_Cycle_Count", "warn_max": 300000, "crit_max": 600000 },
      { "id": 194, "name": "Temperature_Celsius", "read": "right16bit", "crit_min": 20, "warn_min": 10, "warn_max": 40, "crit_max": 50 },
      { "id": 196, "name": "Reallocated_Event_Count" },
      { "id": 197, "name": "Current_Pending_Sector" },
      { "id": 198, "name": "Offline_Uncorrectable" },
      { "id": 199, "name": "UDMA_CRC_Error_Count" },
      { "id": 201, "name": "Unc_Soft_read_Err_Rate", "read": "left16bit" },
      { "id": 230, "name": "Life_Curve_Status", "crit_min": 100, "warn_min": 100, "warn_max": 100, "crit_max": 100 }
    ]
  },
  "hardware": {
    "devices": [
	  { "path": "sda", "ignore" : [ 187 ] },
	  { "path": "sdb", "override": "/dev/twa0 -d 3ware,0" }
	]
  }
}
```

### Installation

### Sensu Go

See the instructions above for [asset registration](#asset-registration)

### Sensu Core
Install and setup plugins on [Sensu Core](https://docs.sensu.io/sensu-core/latest/installation/installing-plugins/)


### Certification Verification

If you are verifying certificates in the gem install you will need the certificate for the `sys-filesystem` gem loaded in the gem certificate store. That cert can be found [here](https://raw.githubusercontent.com/djberg96/sys-filesystem/ffi/certs/djberg96_pub.pem).