Data Engineers

Office for National Statistics

Apply before 11:55 pm on Tuesday 15th June 2021


Reference number



£29,540 - £41,890
HEO minimum £29,540 to maximum £34,764 / SEO minimum £36,027 to maximum £41,890


Higher Executive Officer
Senior Executive Officer

Contract type

Fixed Term

Length of employment

Posts available all have differing post end dates these can be found within the job description.

Business area

ONS - Census

Type of role

Operational Delivery

Working pattern

Flexible working, Full-time, Job share, Part-time

Number of posts



Newport, Fareham

About the job


Do you want to help analysts answer the questions of the day by providing them with the data they require? Do you want to play your part in the transformation of how statistics are produced to help Britain make better decisions?

The Office for National Statistics (ONS) is the UK’s largest producer of official statistics, covering a range of key economic, social and demographic topics. These include measuring changes in the value of the UK economy, estimating the size, geographic distribution and characteristics of the population, and providing indicator of price inflation, employment, earnings, crime and migration.

Job description

This is an exciting time to join ONS, we are currently recruiting for a number of Data Engineers across the Higher Executive Officer (HEO) and Senior Executive Officer (SEO) grades to join our growing community within the following business areas:

Post 1 - Census Data Transformation (HEO Post) (Fixed term 1 post, contract available until 31 March 2023)
This team is responsible for the processing of data from the 2021 Census so that outputs can be produced to inform decisions by Government, Local Authorities, and the wider user base. The team are responsible for the end to end data journey through data preparation and statistical processing up to the creation of a complete unit record dataset upon which all outputs are based.

Post 2 – Covid Infection Survey Data Processing - Social Survey Transformation (HEO post) (Fixed term 1 post, contract available until 31 July 2022)
The COVID19 Infection Survey (CIS) within ONS, provides statistical analysis on the COVID19 pandemic for Government and Research purposes. The Data Processing Pipeline (DPP) receives data from multiple sources, processes this data in variety of ways, then provides the cleaned / linked data to several Analysis teams for their investigation. The DPP within CIS is an extremely busy function, working to demanding objectives, with rapid development and very short cycle times.

The role is responsible for the technical development and implementation of the processing pipelines specified and designed in conjunction with the COVID analysis teams. You will be working with a small coding team and a wider analysis team to identify the best ways of ingesting and processing the data received.

Post 3 - Data Operations and Engineering (HEO post) (Fixed term 4 posts and 1 permanent post available, contracts available are: Posts 1 and 2 until 31 March 2022, Posts 3 and 4 until 31 December 2023 and Post 5 is offered on a permanent contract)
This team develops and delivers the programmes to import, validate and structure key administrative data that flow into ONS under the Digital Economy Act (DEA). The role is based in an expanding and busy team that supports key business areas from Social Surveys to the Economic Accounts.

Post 4 - Development, Delivery and Support Division (SEO post) (Fixed term 1 post, contract available until 31 March 2022)
This multi-disciplinary role broadly supports the aim to modernise the way in which economic statistics are developed; providing data science supported evidence and data engineering support to projects. At the outset of each project, staff in this role will work from an initial design or concept phase, to successful implementation of both regularly published statistics and ‘one-off’ experimental sets of statistics.

This involves data interrogation, the coding of functions that conduct statistical or data manipulation tasks, as well as the coding of data processing pipelines. The team are developing better economic statistics using new tools and techniques. That primarily includes a new cloud computing platform (DAP), making use of the Spark libraries to process and analyse large datasets (Python and Spark are the main tools that are used in the team) and secondarily supporting smaller projects with local systems built upon Python or R libraries. With this, we can apply data science and data engineering techniques to produce timelier, more granular and more insightful statistics.

A key aim for staff in this role is to think about how new technology and methods can be applied to the development of economic statistics in the future. Individuals will be able to have the time and space to develop their skills and conduct research, with the aim of getting experimental statistics or analysis out to stakeholders and the wider public sooner.


As one of our HEO Data Engineers, you will carry out a wide range of activities with responsibilities that may include:
• Working with subject matter experts to design, build, test and implement processing solutions that will provide high quality data for analysis;
• Working collaboratively with the relevant experts in ONS to ensure corporate standards are understood and implemented in all development activity;
• Work with Data Architecture to ensure data are fully documented using corporate tools and standards;
• Coordination of activities within the team, internal delivery partners within ONS and end users of the data;
• Possibly carry out the line management of staff, this may involve matrix management;
• Ensure that business continuity is considered within the day to day work;
• Liaise with Data Architecture to ensure corporate data management principles are followed;
• Address blockers actively seeking solutions to remove them and proposing alternative routes to delivery;
• Promote a frequent incremental delivery approach;
• Digest, troubleshoot and understand a range of large administrative and wider non-survey data series working with the Senior lead in the area;
• Providing a core coding resource for the rising demand in delivering the underlying processes and code to validate and build key administrative data series. This will involve specifying history, validating data and building coherent and accessible data structures on a series-by-series basis for distribution to users inside and outside of the ONS;
• Support the administrative data engineering strategy, working closely with managers and other teams in ONS Data Acquisition and Operations Division in implementing and developing standard data validation, structuring and standardisation protocols to non-survey data. This is to enable business-wide roll-out of the data and ensure it is used consistently across ONS;
• To develop, test and document the end-to-end process for developing ETL data pipelines to ingest and transform large administrative and commercial data flows as they come into the ONS;
• Supporting (if experienced users of Cloudera tools), other areas of ONS, working collaboratively with teams in validating and linking key non-survey data sources. The key goal is to contribute to the data’s broader development and alignment by enabling business areas to integrate the data to produce better and more timely statistical outputs.

As a Senior Executive Officer Data Engineer, you will carry out a wide range of activities with responsibilities that include:
• Work with Economic Statistics experts to understand what statistics need to be produced;
• Conduct data analysis and visualization of the key datasets used to produce the required statistics;
• Produce initial design documents and diagrams for data processing systems, considering the statistical or data processing methods that are needed for the task, as well as the design of the overall data model;
• Contribute to Python libraries of statistical or data processing functions, ensuring that any functions are well unit tested;
• Write Python code that will systemize the processing of data, based on initial design documents. This includes writing unit / integration tests and managing versions of code with Git;
• Report on the progress of prototype systems to customers, clearly articulating the impact of various data processing steps on the outputs;
• Maintain and review documentation for statistical and data processing functions, as well as data processing pipelines;
• Manage working relationships with stakeholders across Economic Statistics, keeping people up to date on progress and highlighting any risks with projects;
• Apply agile project management principles to managing projects and tasks on a day-to-day basis.

Person specification
Essential skills:
• Intermediate to advance understanding of SQL particularly in the context of analysing data in Hive and Impala;
• An intermediate or higher understanding of Python;
• Familiarity with data tools such as Spark;
• Demonstrable experience of structure and linking within and across very large administrative or datasets;
• Able to work with a range of stakeholders, and to collaborate across teams to ensure a quality outcome;
• Ensures communication has a clear purpose and takes into account people’s individual needs;
• Knowledgeable in the use of GIT.


We'll assess you against these behaviours during the selection process:

  • Managing a Quality Service
  • Delivering at Pace
  • Making Effective Decisions
  • Communicating and Influencing

Technical skills

We'll assess you against these technical skills during the selection process:

  • Intermediate to advance understanding of SQL particularly in the context of analysing data in Hive and Impala
  • An intermediate or higher understanding of Python
  • Familiarity with data tools such as Spark


The Office for National Statistics is part of the Civil Service, and as such we share a number of key benefits with other departments, whilst also having our own unique offerings to support our 5000+ valued employees across the business.

Whether you are hearing about us for the first time or already know a bit about our organisation, we hope that the benefits pack attached (bottom of page) will give you a great insight into the benefits and facilities available to our employees, and our fantastic working culture.

We are an organisation that takes the well-being of its employees seriously and lives and breathes the desire to modernise the workplace of the future. Everyone, from our office-based staff in Newport, London and Titchfield, to our field interviewers and airports and ports passenger survey staff, are part of a diverse and inclusive family.

Inclusion & Diversity

Inclusion & Accessibility

At ONS we're always looking to attract the very best people from the widest possible talent pool, and we are proud to be an inclusive, equal opportunities employer. As a member of the Business Disability Forum and a Disability Confident Leader we’re committed to ensuring that all candidates are treated fairly throughout the recruitment process.

As part of our application process, you will be prompted to provide details of any reasonable adjustments to our recruitment process that you need. If you would like to discuss any reasonable adjustments before applying, please contact the recruitment team in the first instance.

If you would like an accessible version of any of the attachments or recruitment documents below or linked to in this advert, please contact the recruitment team who will be happy to assist.
If you need help with any part of the application process or completing your CV or personal statement please contact our Recruitment Team at or call us on 01633 455556, one of the team will be happy to help you.

Things you need to know


Successful candidates must pass a disclosure and barring security check.
Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check.

See our vetting charter.
People working with government assets must complete basic personnel security standard checks.

Selection process details

This vacancy is using Success Profiles, and will assess your Behaviours, Experience and Technical skills.
We are currently recruiting for a number of Data Engineers across HEO and SEO grades. At application stage we will assess you on your CV and personal statement. Your CV should be tailored to demonstrate any skills, knowledge and experience that are relevant to the content of the role, specifically the essential criteria under the person specification.

Your personal statement should be no longer than 1250 words and should provide clear examples to answer the below questions only:

1. Please describe when you have utilised Python, SQL or PySpark to conduct manipulation of Data Sets? (600 words)
2. Please provide an example of working collaboratively with a range of different people to achieve a quality outcome (600 words)

All candidates applying will be assessed at application stage based on the lower HEO grade criteria which has been identified and listed on the job advert. As the criteria is scored, we would recommend that you give clear examples for each question including the impact of your actions.

Success Profiles Behaviour examples are not required at the application stage and are assessed at interview.

In cases where there is a high number of applications the sift pass mark may be adjusted and candidates will be invited to interview in merit order, i.e. those scoring the highest.

At interview stage you will be assessed and considered for all advertised grades using various assessment techniques aligned to the new Civil Service Success Profiles. If you are successful you will be offered a role at an appropriate grade based on the evidence that you have provided.

A presentation may be required at interview. Further information will be provided to successful candidates prior to interview.

Candidates appointed to the HEO grade will be offered roles in order of merit, those scoring higher will be offered the longer term contracts

Due to current working practices, interviews will be conducted by Video Conference.

A reserve list may be held for a period up to 12 months from which further appointments may be made.

Sift: 16-18 June 2021
Interviews will take place from week commencing 28 June 2021

For the full terms and conditions of the post, please see attachment at bottom of page.

Feedback will only be provided if you attend an interview or assessment.

Nationality requirements

This job is broadly open to the following groups:

  • UK nationals
  • nationals of Commonwealth countries who have the right to work in the UK
  • nationals of the Republic of Ireland
  • nationals from the EU, EEA or Switzerland with (or eligible for) status under the European Union Settlement Scheme (EUSS)
  • relevant EU, EEA, Swiss or Turkish nationals working in the Civil Service
  • relevant EU, EEA, Swiss or Turkish nationals who have built up the right to work in the Civil Service
  • certain family members of the relevant EU, EEA, Swiss or Turkish nationals
Further information on nationality requirements

Working for the Civil Service

The Civil Service Code sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles.
The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

Apply and further information

Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :
Name :  Steph Tolley
Email :
Recruitment team :
Email :

Further information

If you feel your application has not been treated in accordance with the Recruitment Principles and you wish to make a complaint, in the first instance, you should contact Pablo Perez, Office for National Statistics, Government Buildings, Cardiff Road, Newport NP10 8XG. If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission here

Share this page