A plugin that collects metrics from Hadoop Nodes version 2.7.0 and later.
You may also refer to this video for a recorded walk through of the install and configuration process.
Prerequisites
- Supported OS: Linux
- TrueSight Pulse meter 4.5 or later
- Hadoop 2.7.0 or later
To install new meter go to 'Settings-> Installation' or see instructions.
To upgrade the meter to the latest version see instructions.
To upgrade the meter to the latest version see instructions.
Metrics Collected
- HADOOP_NAMENODE_FS_CAPACITY_REMAINING
- HADOOP_NAMENODE_FS_CORRUPTED_BLOCKS
- HADOOP_NAMENODE_FS_MISSINGED_BLOCKS
- HADOOP_NAMENODE_FS_BLOCKSED_TOTAL
- HADOOP_NAMENODE_FS_FILES_TOTAL
- HADOOP_NAMENODE_FS_BLOCKED_CAPACITY
- HADOOP_NAMENODE_FS_UNDER_REPLICATED_BLOCKS
- HADOOP_NAMENODE_FS_CAPACITYED_USED
- HADOOP_NAMENODE_FS_TOTAL_LOAD
- HADOOP_NAMENODE_FSSTATE_VOLUME_FAILURES_TOTAL
- HADOOP_NAMENODE_FSSTATE_NUMBER_OF_LIVE_DATA_NODES
- HADOOP_NAMENODE_FSSTATE_NUMBER_OF_DEAD_DATA_NODES
- HADOOP_NAMENODE_FSSTATE_NUMBER_OF_STALE_DATA_NODES
- HADOOP_MAP_REDUCED_PROCESS_CPU_TIME
- HADOOP_MAP_REDUCED_PROCESS_CPU_LOAD
- HADOOP_MAP_REDUCED_SYSTEM_CPU_LOAD
- HADOOP_MAP_REDUCED_AVAILABLE_PROCESSORS
- HADOOP_MAP_REDUCED_TOTAL_SWAP_SPACE_SIZE
- HADOOP_MAP_REDUCED_FREE_SWAP_SPACE_SIZE
- HADOOP_MAP_REDUCED_FREE_PHYSICAL_MEMORY_SIZE
- HADOOP_MAP_REDUCED_TOTAL_PHYSICAL_MEMORY_SIZE
- HADOOP_YARN_APPLICATION_RUNNING
- HADOOP_YARN_APPLICATION_FAILED
- HADOOP_YARN_APPLICATION_KILLED
- HADOOP_YARN_APPLICATION_PENDING
- HADOOP_YARN_AVAILABLE_MEMORY
- HADOOP_YARN_AVAILABLE_VCORES
- HADOOP_YARN_NUMMBER_OF_UNHEALTHY_NODES
- HADOOP_YARN_NUMMBER_OF_ACTIVE_NODES
- HADOOP_YARN_NUMMBER_OF_LOST_NODES
- HADOOP_YARN_USED_MEMORY
- HADOOP_YARN_NUMBER_OF_CONTAINERS
- HADOOP_DATANODE_HEAP_MEMORY_USED
- HADOOP_DATANODE_HEAP_MEMORY_MAX
- HADOOP_DATANODE_GC_COUNT
- HADOOP_DATANODE_GC_TIME_MILLIS
- HADOOP_DATANODE_GC_NUMBER_OF_WARN_THREADSHOLD_EXCEEDED
- HADOOP_DATANODE_GC_NUMBER_OF_INFO_THREADSHOLD_EXCEEDED
- HADOOP_DATANODE_GC_TOTAL_EXTRA_SLEEP_TIME
- HADOOP_DATANODE_BLOCKS_READ
- HADOOP_DATANODE_BLOCKS_WRITTEN
- HADOOP_DATANODE_VOLUME_FAILURES
- HADOOP_DATANODE_NUMBER_OF_FAILED_STORAGE_VOLUMES
- HADOOP_DATANODE_REMAINING_CAPACITY
1) Log into the Pulse.
2) Go to Settings > Plugins
3) Click on 'Get Plugins'.
4) Scroll to 'hadoop' and click 'Install'.
5) Pulse provides a list of meters to associate to the plugin. Please select a meter. Now an "Add a plugin..." drop down box will display. Click 'hadoop'.
6) You will be prompted to "Add a hadoop".
7) You must enter details as per your hadoop configuration.
| Field | Description |
| Host | The hadoop host name |
| Type | Type of hadoop node (single node or multi node) to monitor |
| RM Port | Resource manager hadoop port number |
| DataNode Port | DataNode hadoop port number |
| NameNode Port | NameNode hadoop port number |
| PollInterval | The frequency (in milliseconds) used to poll for metrics. Default value is 1000 ms, we do not recommend a value smaller than the default value |
8) Click 'OK'
9) You can add multiple Hadoop nodes as required.
10) Click 'Save'
10) Click 'Save'
11) The following dashboards will be created:
- Hadoop NameNode
- Hadoop CPU
- Hadoop YARN
- Hadoop DataNode
References