Deskripsi pekerjaan Data Engineer Bhumi Varta Technology

• Bachelor’s degree in Computer Science, Software Engineering, or equivalent practical experience.

• 3+ years of experience in data engineering.

• Strong proficiency in building data pipelines using Debezium, Airflow, Spark, Hive Metastore, Trino (Presto).

• Solid understanding of data lake and data mart architectures.

• Experience with ETL processes and data transformation techniques.

• Proficiency in SQL and experience with relational and NoSQL databases.

• Strong knowledge of software design patterns and principles.

• Experience with containerization (Docker) and version control (Git).

• Proven track record of writing clean, maintainable code.

• Proficiency in unit testing and test-driven development.

• Excellent problem-solving, communication, and collaboration abilities.

• Experience in event-driven architecture is a plus.

• Design, develop, and maintain scalable data pipelines.

• Build data ingestion processes using Change Data Capture and ETL / ELT jobs.

• Build data pipeline to cleanse and transform raw data into clean data in the targeted format to be stored in Data lake.

• Build and manage jobs to preprocess data into an optimized and highly structured data format into Data mart.

• Write clean, efficient, and well-tested code for data processing and transformation.

• Optimize data storage and retrieval processes to ensure high performance and scalability.

• Collaborate with cross-functional teams to understand data requirements and deliver solutions.

• Participate in code reviews and contribute to technical discussions.

• Monitor and optimize data pipeline performance.

• Document technical specifications and data pipeline documentation.

• Troubleshoot and resolve complex technical issues related to data processing.

• Preferred