Posts

How to avoid host key checking during ansible script execution

During execution of ansible playbook or ansible script on new remote host (which you have not connected via ssh so far) the script might fail with below error

UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ----}
To avoid this you can set the environment variable (ANSIBLE_HOST_KEY_CHECKING) to disable host check before ssh during ansible script execution.

export ANSIBLE_HOST_KEY_CHECKING=False
Now you can execute the ansible script.

HMaster not starting up, error "java.io.IOException: Filesystem closed"

HMaster fails to start with below error

util.FSHDFSUtils: attempt=0 on file=hdfs://krbhdfs/apps/hbase/data/MasterProcWALs/state-00000000000000000001.log after 1ms java.io.IOException: Filesystem closed.
There could be multiple causes of the error. 

If you are willing to loose the data, remove manually the logs under "apps/hbase/MasterProcWALs" and  "apps/hbase/WALs" from HDFS. After doing this start the HMaster service.

RegionServer crashing with error "Direct buffer memory"

Region server might crash with below error

INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=ip-100-122-218-159.us-east-1.ec2.aws.symcpe.net:2181,ip-******.us-east-1.ec2.aws.net:2181,ip-*****.us-east-1.ec2.aws.net:2181 sessionTimeout=180000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@253c1256 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2666) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.a…

HMaster error during startup

HMaster throws error as below during startup

FATAL [ip-:16000.activeMasterManager] master.HMaster: The coprocessor org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw java.lang.RuntimeException: java.io.FileNotFoundException: /etc/hbase/2.5.0.55-1/0/xasecure-audit.xml (No such file or directory)
This is happening because the ranger co-processor is defined in xml config to load but the requested plugin is not installed. Remove below attributes from hbase-site.xml

hbase.coprocessor.master.classeshbase.coprocessor.region.classeshbase.coprocessor.regionserver.classes Now restart the HMaster process.

Unable to start namenode. Error "couldn't find resource file location"

Sometimes starting the namenode might throw below error.
ERROR config.RangerConfiguration (RangerConfiguration.java:addResourceIfReadable(110)) - addResourceIfReadable(ranger-hdfs-security.xml): couldn't find resource file location INFO provider.AuditProviderFactory (AuditProviderFactory.java:(77)) - AuditProviderFactory: creating.. FATAL conf.Configuration (Configuration.java:loadResource(2672)) - error parsing conf file:/etc/hadoop/2.5.0.55-1/0/xasecure-audit.xml java.io.FileNotFoundException: /etc/hadoop/2.5.0.55-1/0/xasecure-audit.xml (No such file or directory)
This is causing because, ranger plugin is not installed and authorization provider class is registered for ranger. Your config (hdfs-site.xml) will have below xml attribute with the value
ATTRIBUTE : dfs.namenode.inode.attributes.provider.class VALUE: org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer
Now search for above attribute in hdfs-site.xml and remove attribute.Now you are good to start the se…

Install PIP on CentOS 7

Python is widely used programing language and pip is the tool need to install different python packages.

Before you go ahead check if you have pip installed by using command below.
$ pip --version pip 9.0.1 from /Library/Python/2.7/site-packages (python 2.7)
You can install pip in multiple ways on CentOS. I will discuss here two ways to install pip. 

Install PIP using yup

Update the packages first, just to be updated.
$ yum -y update Once update is complete, you can run below command to install the pip. 

$ yum -y install python-pip If you get error, did not found anything to install python-pip then you need to enable Extra Packages for Enterprise Linux (EPEL) repository, as python-pip is part of EPEL repo. If you don't want to enable EPEL repo then you can use other installation method.

Install PIP using curl

First step is to download the python script to install pip. 
$ curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py" Once the script is downloaded you can run…

Setup python 2.7 on CentOS 7

Python is most commonly used development and scripting tool. You can setup Python on CentOS by multiple ways.

Setup Python using Yum

This is simplest method to install development tools on CentOS. Run below commands to setup the python using yum.
[vagrant@localhost ~]$ sudo yum -y update [vagrant@localhost ~]$ sudo yum groupinstall -y development Or
[vagrant@localhost ~]$ sudo yum groupinstall -y 'development tools'
Setup Python using source code

To setup python using source code, you would first download the source code for the version you want to install. Then build the source code and configure.

You can use "wget" command to download the compressed source code. If you don't have wget installed, use yum to install wget as below.
[vagrant@localhost ~]$ sudo yum install wget You would need "xz" tool to decode the source code.
[vagrant@localhost ~]$ sudo yum install xz-libs Now you are ready to download the source code. 
Download the source code for python 2.7.6 usin…