Senior Data Engineer

The Digital, Data and Technology (DDAT) directorate has recently been established in UKEF, drawing together the expertise in digital currently within the organisation to provide representation of digital services, user centred design, analytics and technology at the highest levels. The areas with the directorate work closely with all areas of the business such as the Business Group, Operations and Strategy and Policy.

Job description

UKEF is committed to be a more customer-centric organisation, making it easier for customers to deal with us, offering improved response times, quicker decision-making and improved case-processing. To enable this, UKEF is committed to using digital as a primary means for managing relationships with a wider range of stakeholders. Focusing on developing end-to-end services that meet user need and enable business outcomes, this is an exciting new role showing the growth and drive of UKEFs digital ambition.

This role requires you to develop and maintain Data Integration solutions, identify opportunities for improving data infrastructure and processes, develop and maintain ETL processes and manage the deployment and release of data pipelines and applications.

Main Activities

• Monitor and troubleshoot data processing systems to ensure uptime and reliability.
• Develop and maintain data integration solutions for combining data from various sources.
• Provide quality assurance and code reviews of other developer’s work.
• Work with stakeholders to identify opportunities for improving data infrastructure and processes.
• Implement and maintain data backup and recovery processes to ensure data availability in case of system failures.
• Develop and maintain documentation and training materials to support data infrastructure and processing pipelines.
• Responsible for managing data platforms, such as data warehouses, data lakes, and other data storage systems.
• Responsible for developing and implementing data management processes, including data integration, data migration, and data storage.
• Responsible for ensuring that data is secure and compliant with relevant regulations, such as GDPR. This may involve implementing data access controls, data encryption, and data masking.
• Responsible for creating and maintaining ETL (Extract, Transform, Load) processes for structured and unstructured data.
• Responsible for the technical approach to data integrity and back-out plans for all changes to UKEF systems.
• Manage the deployment and release of data pipelines and applications.
• Contribute to the development and execution of the organization's data strategy and roadmap.
• Responsible for staying up-to-date with industry trends in data management and data engineering, ensuring that the organization is using the latest tools and technologies.

Person specification

Essential

Qualifications

• 5 GCSE passes (including Maths and English) at Grade 4 (C) or above; or relevant professional experience. (A)
• Evidence of continuing professional development. (A)

Knowledge

• Expert knowledge of SQL and T-SQL. (A, I)
• Knowledge of ETL and ELT tools. (A, I)
• Expert knowledge of Data warehouse architectures and Data modelling techniques. (A, I)

Skills / Ability

All skill levels are in relation to Senior Data Engineer - Government Digital and Data Profession Capability Framework (ddat-capability-framework.service.gov.uk)

Communicating between the technical and non-technical:

• You can communicate effectively with technical and non-technical stakeholders.
• You can support and host discussions within a multidisciplinary team, with potentially difficult dynamics.
• You can be an advocate for the team externally and can manage differing perspectives.

(Skill level: working) (A, I)

Data analysis and synthesis:

• You can undertake data profiling and source system analysis.
• You can present clear insights to colleagues to support the end use of the data.

(Skill level: working) (A, I)

Data development process:

• You can design, build and test data products that are complex or large scale.
• You can build teams to complete data integration services.

(Skill level: practitioner) (A, I)

Data innovation:

• You can understand the impact on the organisation of emerging trends in data tools, analysis techniques and data usage

(Skill level: working) (A, I)

Data integration design:

• You can understand the concepts and principles of data modelling.
• You can produce relevant data models across multiple subject areas.
• You can reverse-engineer data models from a live system.
• You can understand industry-recognised data modelling patterns and standards, and when to apply them.
• You can compare and align different data models.

(Skill level: practitioner) (A, I)

Data modelling:

• You can understand the concepts and principles of data modelling and can produce relevant data models.
• You can work across government and industry, recognising opportunities for the reuse and alignment of data models in different organisations.
• You can design the method to categorise data models within an organisation.

(Skill level: expert) (A, I)

Metadata management:

• You can design an appropriate metadata repository and present changes to existing metadata repositories.
• You can understand a range of tools for storing and working with metadata.
• You can provide oversight and advice to more inexperienced members of the team.

(Skill level: practitioner) (A, I)

Problem resolution:

• You can respond to problems in databases, data processes, data products and services as they occur.
• You can initiate actions, monitor services and identify trends to resolve problems.
• You can determine the appropriate remedy and assist with its implementation, and with preventative measures.

(Skill level: working) (A, I)

Programming and build:

• You can use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations.
• You can collaborate with others to review specifications where appropriate.

(Skill level: practitioner) (A, I)

Technical understanding:

• You can understand the core technical concepts related to the role and apply them with guidance.

(Skill level: working) (A, I)

Experience

Experience in the following disciplines (A, I) :

• Experience using Azure Cloud Computing, including Azure Managed Instance, Azure Data Lakes, Azure SQL and Microsoft Fabric.
• SQL Coding in T-SQL.
• Experience using MSSQL stack.
• Knowledge and experience of SQL Server Optimisation techniques.
• Efficient and conformant Database Design skills.
• Experience in Troubleshooting SQL Servers.
• Experience using ETL tools (i.e. Informatica IICS).
• Unit testing.

Desirable

Qualifications
N/A

Knowledge

Expert knowledge of Azure Cloud Computing. (A, I)

• Knowledge of Development Patterns relevant to Web and SOA designs. (A, I)
• Knowledge of Automated Testing and CI/CD techniques. (A, I)
• Knowledge of Informatica Intelligent Cloud Services (IICS). (A, I)

Skills/Ability

Testing:

• You can review requirements and specifications and define test conditions.
• You can identify issues and risks associated with work.
• You can analyse and report test activities and results.

(Skill level: working) (A, I)

Experience

• Experience using Azure DevOps and Github. (A, I)

Behaviours

We'll assess you against these behaviours during the selection process:

• Seeing the Big Picture
• Making Effective Decisions
• Leadership

Technical skills

We'll assess you against these technical skills during the selection process:

• Can you provide an example from your own experience where data engineering played a crucial role in driving business decisions? What impact did it have?
• Could you explain a complex ETL process that you designed and implemented? What tools did you use and why?
• Presentation

We only ask for evidence of these technical skills on your application form:

• Can you provide an example from your own experience where data engineering played a crucial role in driving business decisions? What impact did it have?
• Could you explain a complex ETL process that you designed and implemented? What tools did you use and why?

Apply before 11:55 pm on Monday 3rd March 2025