Big Data Jobs

Job Title - Big Data Engineer

Location - Malvern PA/ Charlotte NC

Relevant Experience - 6+ Years

Technical/Functional Skills

AWS, Big Data, Scala, Python, SPARK, Hive

Technology requirements

• Support production environment
• Participating in requirement analysis and planning the development
• Must have experience in CI/CD, AWS S3, AWS EC2, AWS IAM, AWS Data Lake, SQL, Hive, Shell Scripting
• Min 6 yrs hands on experience in Spark, Python, Scala, Hadoop, Java
• Must have hands on experience in PySpark on Jupyter notebook
• Must have hands on experience in AWS ETL Glue, Lambda, DynamoDB
• Reviewing the code, design and providing expertise in the development and integration of systems
• Building and managing public and private cloud infrastructure with AWS, EC2 and S3 resources
• Designing automation processes as per the operational needs of an organization

Roles & Responsibilities

• Work in a Support team to analyze, triage and debug the incidents and failures occurred
• Work in a team of Data Engineer for Data ingestion process from Legacy System to AWS
• Participating in requirement analysis and planning the development
• Building and managing public and private cloud infrastructure with AWS, EC2 and S3 resources
• Support production environment
• Designing automation processes as per the operational needs of an organization
• Monitoring the cluster jobs in a daily basis