| Job Description: |
Descriptions:
About the Role
We’re seeking a Data Engineer with deep Microsoft SQL Server expertise to design, build, and optimize high‑quality data pipelines and platforms. You’ll own data ingestion, transformation, modeling, and performance tuning across OLTP and OLAP workloads, enabling analytics, reporting, and downstream applications.
Key Responsibilities
Pipeline Development: Design and implement robust ETL/ELT workflows that move data from diverse sources (APIs, flat files, cloud stores, operational DBs) into SQL Server data marts and data warehouse layers (staging, core, semantic).
SQL Engineering: Write high‑performance T‑SQL (stored procedures, functions, views, window functions, CTEs) and optimize queries using execution plans, indexing strategies, partitioning, and statistics maintenance.
Data Modeling: Build dimensional models (star/snowflake), conformed dimensions, and fact tables; enforce data standards, naming conventions, and SCD strategies (Type 1/2).
Performance & Reliability: Implement job orchestration, monitoring, alerting, and data quality checks (validations, reconciliations, SLAs) for production pipelines.
Automation & CI/CD: Use Git and pipeline tools (e.g., Azure DevOps/Jenkins/GitHub Actions) to version, test, and deploy database and data pipeline changes.
Cloud & Storage (Nice to have or Required—your choice): Integrate SQL Server with Azure (ADF, Databricks, Synapse, ADLS) or AWS (S3, Glue, Redshift/SQL Server on EC2/RDS); work with files in Parquet/CSV/JSON.
Collaboration: Partner with analysts, data scientists, and application teams to translate requirements into scalable data solutions; contribute to sprint planning and documentation.
Security & Governance: Implement role‑based access, encryption at rest/in transit, and follow governance practices (data catalogs, lineage, metadata).
Minimum Qualifications
3–7+ years of professional data engineering experience (adjust range as needed).
Advanced T‑SQL skills: complex joins, window functions, dynamic SQL, error handling, and transaction control.
Strong hands‑on with SQL Server engine internals: indexes, partitioning, execution plans, statistics, tempdb usage, isolation levels, and deadlock analysis.
Proven experience building ETL/ELT with one or more: SSIS, Azure Data Factory, Informatica, Talend, PySpark, or Airflow.
Solid understanding of data warehousing concepts (Kimball/Dimensional modeling), SCDs, and data quality practices.
Proficiency with Git and at least one CI/CD tool; experience scripting in Python or PowerShell.
Experience operating in Windows/Linux server environments.
Preferred Qualifications
Experience with Azure Data Platform (ADF, Databricks, Synapse, ADLS, Key Vault) or AWS equivalents.
Exposure to NoSQL (Cosmos DB, MongoDB) and event/streaming (Kafka/Event Hubs).
BI exposure (Power BI/Tableau) and DAX/semantic modeling understanding.
Knowledge of Terraform/ARM/Bicep for infra‑as‑code and Kubernetes/Docker for containerized workloads.
Performance tuning at scale: workload management, columnstore indexes, table distribution strategies, and querying external data (PolyBase/Linked Servers where applicable).
Bachelor’s/Master’s in Computer Science, Information Systems, Engineering, or related field.
Success Metrics (First 90–180 Days)
Deliver production‑ready pipelines with >99% success rate and documented SLAs.
Reduce key report or model refresh time by 30%+ via indexing/partitioning/tuning.
Implement a DQ framework (profiling, thresholds, alerts) across critical datasets.
Establish versioned, automated deployments for DB objects and ETL/ELT code.
Tools & Technologies (customize to your stack)
Databases: SQL Server (2016+), Azure SQL, SQL Server on‑prem/VM/RDS
ETL/Orchestration: SSIS, Azure Data Factory, Databricks, Airflow
Languages: T‑SQL, Python, PowerShell, SQL
Cloud/Storage: Azure (ADLS, Synapse), AWS (S3, Glue), Files (Parquet/CSV/JSON)
DevOps: Git, Azure DevOps/Jenkins/GitHub Actions, Automated testing
Monitoring: SQL Server Agent, Alerts, Query Store, CloudWatch/Azure Monitor, ELK/Splunk (optional)
BI (nice to have): Power BI, Tableau
Essential Skills: Data Engineer and Technical Project manager
Desirable Skills:
Keyword:
Skills: Microsoft SQL Server 2016~Project Management
Experience Required: 10 & Above
|
