Senior Lead Data Engineer
 Location: Remote/ Bangalore
Experience: 11+ years
About the Role:
Are you passionate about building high-performance data systems? Do you thrive at the intersection of Data Engineering, Cloud, and Big Data technologies?
We’re looking for a Senior Lead Data Engineer to join our growing team! If you have deep expertise in ETL/ELT pipelines, Data Warehousing, Cloud platforms (GCP, AWS), and Hadoop ecosystems, we want to hear from you!
ETL Tools: primarily use Pentaho Data Integration (PDI) for designing and managing ETL workflows, including data extraction, transformation, and loading from various sources into target systems.
 Databases:
  • MySQL for transactional data handling and intermediate data staging.
  • Amazon Redshift as the primary data warehouse for analytics and reporting workloads.
 Workflow Monitoring & Orchestration:
  • Azkaban is used for monitoring and managing data pipeline executions and ensuring task dependencies are handled correctly.
 Job Scheduling & Scripting:
  • Crontab and Unix shell scripting are utilized for scheduling recurring jobs and automating data processes on a daily or hourly basis.
 Cloud Services (AWS):
  • Hands-on experience with:
  • RDS for managing relational database services.
  • Amazon Redshift for handling analytical queries and data warehousing.
  • EC2 for running compute workloads.
  • Basic IAM (Identity and Access Management) for managing user roles and access permissions.
 CI/CD & Deployment:
  • Jenkins is used for automating the deployment of ETL scripts and scheduling code pushes to production environments.
What You'll Do:
  • Architect, design, and build robust ETL/ELT pipelines using Informatica PowerCenter and cloud-native solutions.
  • Work hands-on with Data Warehousing concepts and Data Modeling to power analytics and insights.
  • Build large-scale Hadoop workflows with tools like Hive, Pig, Spark, and HDFS.
  • Collaborate closely with business and technical teams to translate requirements into scalable solutions.
  • Produce high-quality technical documents: design docs, mapping specs, and unit test plans.
  • Optimize big data processing and cloud infrastructure for performance, scalability, and cost.
  • Mentor junior engineers and contribute to a culture of technical excellence.
Tech Stack You'll Work With:
  • Informatica PowerCenter
  • GCP (BigQuery, Dataflow, Pub/Sub)
  • AWS (Redshift, S3, Glue, Lambda)
  • Hadoop (Hive, Pig, Spark, HDFS)
  • SQL, Python (bonus!)
  • DevOps, CI/CD, Docker/Kubernetes (nice to have!)
What You Bring:
  • 11+ years of IT experience with a strong focus on Data Engineering.
  • Solid foundation in Data Warehouse architecture and Data Modeling principles.
  • Deep understanding of Cloud and Big Data technologies.
  • Excellent communication, documentation, and troubleshooting skills.
  • Ability to thrive in a fast-paced, agile environment.
Your Background:
  • Bachelor's or Master’s in Computer Science, Information Technology, or related field.