Data Engineer (Azure / Databricks / PySpark)
Description:
About Sharp Infotech Inc
Sharp Infotech Inc is a technology consulting and digital solutions company that partners with organizations across the United States to deliver enterprise IT platforms, digital transformation initiatives, and scalable business solutions.
Position Overview
Sharp Infotech Inc is seeking Data Engineers to support enterprise data platform and analytics initiatives for our clients. The selected candidates will design and build scalable ETL/ELT pipelines, data lakehouse architectures, and enterprise analytics datasets using technologies such as Azure Data Factory, Databricks, PySpark, Delta Lake, and modern cloud data platforms.
Key Responsibilities
• Build and maintain ETL/ELT pipelines using Azure Data Factory and Databricks
• Develop PySpark notebooks for large-scale data transformation and analytics workloads
• Implement Bronze–Silver–Gold data architecture patterns
• Develop Delta Lake tables with schema evolution and partitioning strategies
• Integrate data from APIs, event streams, and relational databases
• Support streaming and batch ingestion using Azure Event Hubs and ADLS Gen2
• Implement data validation, monitoring, and alerting frameworks
• Optimize Spark jobs and data processing pipelines for performance and cost efficiency
• Collaborate with analysts, architects, and stakeholders to deliver curated datasets
Required Qualifications
• Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field
• Experience with Python and PySpark
• Experience with Azure Data Factory or Databricks
• Experience building ETL/ELT pipelines
• Strong SQL and relational database knowledge
• Experience using Git or version control systems
Preferred Qualifications
• Experience with Delta Lake or lakehouse architectures
• Experience with Azure cloud services
• Experience with streaming data platforms
• Experience supporting enterprise analytics platforms