Data Engineer

Data Engineer

Job Duties / Job Description

Role: Data Engineer

As a Data Engineer, you play a critical role in building and maintaining the data infrastructure of our client. You are responsible for the design, construction, installation, and maintenance of large-scale processing systems. Your work ensures that data is accessible and readily available for use in analytics and reporting.

Key Responsibilities:

  1. Data Pipeline Design and Development:
  • Design, implement, and maintain scalable data pipelines to ingest, process, and store data from various sources, including databases, data lakes, APIs, and logs.
  • Develop and manage ETL processes for efficient data movement, transformation, and loading.
  • Develop and implement data transformation processes to cleanse, enrich, and aggregate raw data into usable formats for analytics and reporting.
  • Participate in developing and maintaining information sources and contacts as directed; may attend meetings to collect information related to projects.
  • Create information models using ontologies and taxonomies and other semantic models.
  • Implement information models in relational or graph databases as appropriate.
  • Employ programming languages like SQL, NoSQL, OWL, RDF, SPARQL, R, or Python to create data pipelines, linking data sources and ensuring that data pipelines are reliable and efficient.
  • Manipulate, transform, and organize data to prepare for analysis and visualization. – 50% OF THE WORK.
  1. Data Modeling:
  • Create and maintain data models and schemas to support business requirements while ensuring data accuracy, integrity, and performance.
  • Administer and optimize databases and data warehouses to ensure high availability, reliability, and efficient query performance. Implement robust data integration solutions, allowing different systems to share and sync data efficiently. – 15% OF THE WORK.
  1. Data Quality Assurance:
  • Establish and maintain data quality checks and data monitoring processes to detect and resolve data inconsistencies and anomalies.
  • Optimize data pipelines, databases, and queries to enhance system performance, reduce latency, and lower operational costs. – 10% OF THE WORK.

 

  1. Data Security:
  • Implement data security measures, access controls, and encryption to protect sensitive information.
  • Identify improvements to processes and workflows, related to physical and logical data protection.
  • Collaborate with Information Security, Network Security and compliance teams to validate that Data security design, processes and procedures align with broader agency policies, standards and requirements. – 10% OF THE WORK.
  1. Collaboration:
  • Work closely with data scientists, data analysts, and business stakeholders to understand their data requirements, provide support, and deliver solutions to meet their needs.
  • Collaborate with senior study leaders to develop, construct, test, and maintain the data architecture for research projects or portfolios. – 10% OF THE WORK.
  1. Documentation:
  • Document data engineering processes, configurations, and best practices for knowledge sharing and to maintain an accessible knowledge base. – 5% OF THE WORK.

 

Apply for this position

Allowed Type(s): .pdf, .doc, .docx
Cart

No products in the cart.

Select the fields to be shown. Others will be hidden. Drag and drop to rearrange the order.
  • Image
  • SKU
  • Rating
  • Price
  • Stock
  • Availability
  • Add to cart
  • Description
  • Content
  • Weight
  • Dimensions
  • Additional information
  • Attributes
  • Custom attributes
  • Custom fields
Click outside to hide the comparison bar
Compare