| Job Description: |
Descriptions: Data Engineer – Data bricks
**Must be local to Boston only as we have in person evaluation **
Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL
Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
Must have 8+ Years of hands-on development experience in working on Azure Cloud.
Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
Must have experience and a good understanding on the below topics
Databricks Delta Lake storage
Unity Catalog
Autoloader
Lakehouse Architecture
Medallion Architecture
Databricks Serverless Options.
Dataset and Data frame
Databricks Clusters Configuration and Sizing
Databricks Job Orchestration
Spark Structured Streaming
Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services
The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
Skills: Digital : Python~Digital : Databricks
Experience Required: 8-10 |
