Lowe's - Senior Hadoop Administrator - Linux Cluster (4-13 yrs)
Role and Responsibilities :
- Maintain hadoop infrastructure, monitor various services, manage configurations, operating system patching and updates
- Work with various internal engineering teams & cloudera Support in infrastructure design and operational issues.
- Plan and implement Cloudera hadoop Software Upgrades
- Ensure availability and validity of system backups and disaster recovery preparedness.
- Monitor performance, collect and present usage metrics
- Develop and implement policies and procedures to manage all aspects of Hadoop administration, including security standards, workload management, availability monitoring and alerting.
- Proactively identify potential issues and bottlenecks across hardware, Software, batch and user processes
- Support data engineering solutions team and analytic users on various project needs and issues
- Contribute to Hadoop cluster capacity planning and roadmap
- 4-6 years of Infra Admin experience with a 3-4 years of experience in administering Hadoop clusters
- 2-3 years of experience in managing large Linux clusters.
- 2-3 years of exposure to data processing applications using HDFS, Java Map Reduce, Hive/Tez, HBase, Pig, Sqoop, Falcon, Oozie and Flume.
- 2-3 years of scripting (bash, Perl, Python etc.) and devops experience
- Thorough understanding of Cloudera Hadoop architecture, and services such as HDFS, Sentry, HBase, Impala, Hue, Spark, Hive, Kafka, YARN, and Zookeeper.
- Experience in designing large scale hadoop clusters, adding/removing nodes, tuning memory /CPU and workload management parameters & software components installation
- Experience in use of administrative tools like Ambari and troubleshooting various service logs
- Good understanding of Linux administration in terms of access set up, security, storage etc.
- Strong performance monitoring and performance tuning skills.
- Experience in managing Cloud Iaas and PaaS hadoop environments in GCP or Azure will be an added advantage
- Bachelor's degree in Computer Science, CIS, or related field and 6+ years experience