| Job Description: |
Required Skills: · Extensive hands-on experience with Databricks and Apache Spark (PySpark, Scala, or SQL). · Proficiency in at least one programming language (Python, Scala, or Java). · Strong understanding of cloud data architectures (Azure Data Lake, AWS S3, GCP Storage) and related ecosystem. · Experience with CI/CD tools and DevOps practices for data engineering. · Familiarity with data warehousing concepts and tools (Delta Lake, SQL, etc.). · Solid grasp of distributed computing, performance tuning, and big data best practices. · Excellent communication and problem-solving skills. · Desirable skills: · Iceberg table Implementation · Databricks Automation Bundles and DQX framework. · Exposure and understanding of GenAI and its opportunities in data engineering.
Roles & Responsibilities
· Architect and develop scalable data pipelines using Databricks, Apache Spark, and related
technologies.
· Collaborate with data scientists, analysts, and business stakeholders to understand
requirements and deliver robust solutions.
· Optimize ETL workflows for performance, reliability, and cost efficiency on cloud platforms
(Azure, AWS, or GCP).
· Implement data governance, security, and compliance best practices within the Databricks
environment.
· Mentor junior engineers and contribute to code reviews, architectural decisions, and platform
enhancements.
· Develop and maintain documentation for data pipelines, architecture, and operational
procedures.
· Troubleshoot and resolve complex technical issues related to data ingestion, transformation,
and storage.
Generic Managerial Skills, If any
We are seeking a highly skilled Databricks Developer/Architect with deep hands-on coding experience. The
ideal candidate will design, implement, and optimize big data solutions on the Databricks platform,
leveraging advanced data processing frameworks, cloud architectures, and automation strategies.
Role Descriptions: Azure Architect- with Databricks Automation Bundles and DQX framework| understanding of GEN AI and ICEBERG table implementation
Essential Skills: Azure Architect- with Databricks Automation Bundles and DQX framework| understanding of GEN AI and ICEBERG table implementation
Desirable Skills:
Keyword:
Skills: Digital : Databricks
Experience Required: 8-10 |
