| Job Description: |
Role Descriptions:
ep understanding of Databricks Lakehouse architecture, including Delta Lake, Unity Catalog, and
MLflow for data governance and machine learning lifecycle management.
Hands-on experience designing solutions using Workflows (Jobs), Delta Lake, Delta Live Tables (DLT), Unity Catalog,
and MLflow and medallion architecture for building robust pipelines
Proficiency and Hands on experience in Databricks Notebooks, cluster configuration, and performance tuning
for large-scale workloads
Expertise in integrating Databricks with AWS, Azure (e.g. AWS S3, Azure Data Factory, S3, Synapse Analytics)
Advanced skills in Apache Spark (PySpark, Spark SQL) for batch and streaming data processing
Strong command of Python and SQL for ETL, data modeling, and analytics workflows
Experience working with Unity Catalog, RBAC, tokenization, and data classification frameworks
Experience in integrating Databricks with CI/CD pipelines using tools such as GitHub Actions.
Working knowledge of Git, CI/CD, VS Code
Working Knowledge of AI/ML concepts
Must Have - Databricks Certified Data Engineer Associate/Professional
Must Have - Cloud Platform Certifications (AWS, Azure)
Optional - Databricks Certified Machine Learning Associate/Professional
Roles & Responsibilities
Work closely with the Client Lead Architect to evaluate solution options and support hands‑on execution of proof‑of‑concepts (POCs)
to validate technical feasibility and scalability.
Lead Databricks delivery on client projects by driving solution implementation, ensuring designs are executed efficiently and aligned
with architectural standards.
Provide technical leadership in building and optimizing end‑to‑end data pipelines leveraging Databricks and integrated cloud services.
Oversee the design, development, and maintenance of large‑scale data processing solutions on the Databricks Lakehouse Platform to
meet business requirements.
Guide the implementation of multi‑layered data models following Bronze (raw), Silver (cleansed), and Gold (curated) patterns across
multiple data domains.
Drive adoption of Delta Lake, Unity Catalog, and Databricks advanced capabilities to enable governed data management, versioning,
and controlled data sharing.
Ensure data solutions support both batch and near‑real‑time processing patterns for analytics and downstream consumption. Apply best practices for data storage, processing, and optimization using Databricks Delta Lake. Enforce secure implementations using Unity Catalog, role‑based access controls (RBAC), encryption, token‑based authentication, and data lineage in line with enterprise compliance policies. Lead implementation of data governance practices, including data quality checks, SLA adherence, metadata management, and data observability metrics. Collaborate with business stakeholders to translate functional and reporting requirements into scalable Databricks solutions. Participate in solution design discussions, architecture walkthroughs, and working sessions with enterprise IT, data, and business teams. Stay current with emerging capabilities in Databricks Lakehouse, cloud-native services, and Generative AI, and assess their applicability for existing platforms. Support evaluation and controlled rollout of new Databricks features and partner integrations to improve platform efficiency and delivery outcomes. Mentor engineers and provide technical guidance, code reviews, and best‑practice recommendations across Databricks implementations.
Generic Managerial Skills, If any
Clear Communication Skills to Interact with Technical & Business Stakeholders
Leadership in mentoring teams and enforcing best practices across data engineering and AI/ML workflows
Proven experience leading or mentoring global, cross-functional teams across multiple time zones and engagements.
Ability to work independently in agile or hybrid delivery models, while guiding junior engineers and ensuring solution quality
Desirable Skills:
Keyword:
Skills: Digital : Amazon Web Service(AWS) Cloud Computing~Digital : Databricks~PL/SQL
Experience Required: 6-8 |
