Senior Data Engineer

• Developing data architecture and relational data models, designing database repositories, capturing data requirements, and preparing data dictionaries.
• Develop and maintain scalable data pipelines using Apache Spark, Python, and related technologies to process large volumes of structured and unstructured data efficiently.
• Design, implement, and optimize ETL processes to extract, transform, and load data from various sources into our data lake or data warehouse.
• Collaborate with cross-functional teams including data scientists, analysts, and software engineers to understand data requirements and deliver robust data solutions.
• Fine-tune Spark jobs and optimize performance by identifying and resolving bottlenecks in data processing pipelines.
• Implement data quality checks and ensure data integrity throughout the data lifecycle.
• Optimizing database throughput by fine-tuning connection pools for varying application needs.
• Participating in all aspects of agile software development, including design, implementation, and deployment.
• Developing Data Models and Data Marts from various source systems, creating and maintaining ETL scripts using data flow design.
• Identifying and leveraging objects and code to reduce development effort and enforce consistent business rules.
• Identifying opportunities for analysis and reporting solutions within the company.
• Participating in the evaluation and selection of Data Modeling, BI, and DW tools.
• Translating business requirements into technical Data Models and Data Marts, understanding key business drivers.
• Clearly documenting all development activities to company standards, communicating task status, progress, and problems effectively to the project lead.

• Successfully completed a degree in BS in Computer Science, Software Engineering, IT, BBIT or related field
• 3-5 years of experience in Data Analytics Field in the following Tool & Technologies:
• Strong analytical and problem-solving skills
• Excellent communication and interpersonal skills

• Proficiency in Azure Data Factory, Databricks, Azure Datawarehouse and SQL
• Expertise in Apache Spark and Python for developing scalable data processing pipelines.
• Familiarity with Azure Synapse Analytics for data warehousing solutions.
• Experience in cloud implementations, with a preference for Microsoft Azure, showcasing practical expertise in leveraging cloud technologies.
• Hands-on experience working with NoSQL databases, highlighting a comprehensive understanding of their structure and functionality.

The Entertainer