CMA1956 Lead Data Engineer

Competition & Markets Authority

Apply before 11:55 pm on Monday 9th October 2023



Reference number



£55,133 - £62,417
a DDAT allowance of up to £6750 may be applicable
A Civil Service Pension with an average employer contribution of 27%

Job grade

Grade 7

Contract type


Business area

Data, Technology and Analytics

Type of role

Architecture and Data

Working pattern

Flexible working, Full-time, Part-time

Number of jobs available



Belfast, Cardiff, Edinburgh, London, Manchester

Job summary

This is an exciting opportunity to build your data engineering expertise and join the Data, Technology and Analytics (DaTA) team at the Competition and Markets Authority (CMA) to play a key role at the forefront of technology, investigating and regulating digital firms and markets to make a positive impact on society.

The CMA is a world-leading competition authority at the forefront of tackling pressing public policy questions, including the regulation of the biggest digital markets. Our mission is to make markets work well in the interests of consumers, businesses, and the economy.

Job description

The Data, Technology and Analytics (DaTA) unit is an established multidisciplinary team that brings together data engineers, data scientists, technologists, behavioural scientists and digital forensics and eDiscovery experts.

As a Lead Data Engineer in the DaTA unit, you will join an established team of around 15 data engineers and data scientists who are critical in designing and managing the CMA’s analytical infrastructure and supporting the delivery of our most technically challenging projects.

Some of our recent work in Data Engineering includes:

  • Designing pipelines featuring parallelization, workflow orchestration, data version control and MLOps to process large, complex data sources
  • Securely translating large corpora of documents submitted by third parties using a containerised architecture on our cloud infrastructure
  • Building end-to-end services, defined as Infrastructure as Code, to run jobs, including web scraping at scale and regular ML prediction on large datasets


Your responsibilities will span the entire tech stack, with an emphasis on backend and infrastructure development, data orchestration and handling. This will include:

  • Providing leadership and engineering expertise on projects
  • Creating and implementing new services, pipelines and data driven tools within our cloud platform
  • Driving the development and maintenance of our cloud-hosted analytics platform (JupyterHub, AWS EC2s) and existing services through Infrastructure as Code (AWS CDK)
  • Promoting and producing high quality code in Python (and other languages, as needed)
  • Building the CMA’s coding and data management standards and helping others to improve their coding and data handling capabilities
  • Identifying opportunities to support CMA cases with data engineering expertise

You will be expected to make architectural decisions and lead projects, along with managing internal stakeholders to make sure we are building the most impactful tools possible for the CMA.

Person specification

  • Significant experience of working as a Data Engineer, Developer, or DevOps specialist, developing in the cloud.
  • Proven experience in leading and delivering data projects and setting technical direction.
  • Significant experience of writing, documenting, testing and managing code, including Python, to work with large and messy datasets, ideally using cloud services such as AWS.
  • A good understanding of service architecture and design best practices.
  • Proven interest in learning about and understanding the latest methods and technologies within machine learning and data engineering, and supporting others with learning in these fields.
  • Experience of managing and working collaboratively with a variety of stakeholders.
  • Experience overseeing and writing both architectural, design and technical documentation, and presentations and reports for non-technical audiences.

To find out more about this opportunity please click on the APPLY button.

CMA – Working for the CMA , Civil Service Careers (

Or our LinkedIn page:


We'll assess you against these behaviours during the selection process:

  • Making Effective Decisions
  • Communicating and Influencing
  • Working Together
  • Delivering at Pace

Technical skills

We'll assess you against these technical skills during the selection process:

  • Data Engineering knowledge
  • Coding skills
Alongside your salary of £55,133, Competition & Markets Authority contributes £14,886 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

As well as the interesting and varied work you’ll do to contribute to effective competition in the UK, and to ensure consumers get a good deal when buying goods and services, you’ll also benefit from: 

  • An exceptionally generous pension scheme with a significant employer contribution and range of options, including a tax-free lump sum at the point of retirement and injury, ill-health and life assurance benefits
  • 25 days leave (increasing to 30 days over five years), plus 8 public holidays and an additional day off for the King’s birthday. In addition, you’ll be able to access a wide range of other types of leave as and when you need it, including generous maternity, paternity, shared parental leave and adoption options, as well as paid special leave for volunteering 
  • Season ticket loans, cycle to work scheme, flu vaccinations and eye tests
  • Access to the Civil Service Sports & Leisure, giving discounted gym membership, high street discounts, free access to UK wide attractions and a free Tastecard
  • A range of wellbeing benefits, including an employee assistance programme, flexible working options and family friendly policies, regular networking events and professional learning opportunities at work  

Any move to the CMA from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax-Free Childcare. Determine your eligibility at 
Like many other organisations, we operate a hybrid working model with a mix of office and home working. We believe this supports us to deliver our best work while allowing the flexibility to maintain balanced and healthy lives. We work at least 40% of our time from our office locations across the UK. 

If you’re already a civil servant being appointed on level transfer, you’ll retain your existing terms, including annual leave. If you are being promoted you’ll receive either a 10% pay increase or the new grade minimum, whichever is the greater. Applicants new to the civil service can be considered for appointment at a salary higher than the starting point indicated above, but not more than the advertised mid-point, depending on skills, knowledge, experience and qualifications. 

We are fully committed to being an inclusive employer and ensuring equal opportunities. We want a diverse workforce that reflects the consumers we serve. We welcome applications from under-represented groups, including ethnically diverse people, people with a disability, people with diverse gender identities and from people of different ages. 

We are an accredited Disability Confident Employer and disabled applicants meeting the minimum criteria for the job will be given the opportunity to demonstrate their abilities at an interview.  

Our Values 

The CMA delivers impactful outcomes for people, business and the UK economy.​ We are Ambitious and Evidence-based, and always strive for Excellence.​ We treat everyone with Respect, and are Collaborative and Inclusive.​  

Everything we do is underpinned by the Civil Service values: Honesty, Integrity, Impartiality and Objectivity. 

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

As part of the application process you will be asked to complete a CV, personal statement and an online application form. Further details about application requirements are listed on the application form.

The selection process will comprise of the sift (CV & application review), an assessment and an interview. Arrangements for the interview will be enclosed in the invitation. Interviews will be either conducted either virtually, by Microsoft Teams, or in-person at a CMA office. 

At the sift all elements of the essential criteria listed under Person Specification (above and in the attached role profile) will be reviewed.

Candidates that pass the initial application sift will be invited to do a technical test – a 3-hour coding and data engineering test.

Candidates that pass the technical test will then be invited to interview – a 45-60 minute interview which will include some questions related to your response to the technical test.

The sift will take place between 10 and 13 October 2023.

The coding test will take place week commencing 23 October 2023.

Interviews will be held from week commencing 06 November 2023.

Reasonable adjustments

We want to make sure no one is at a disadvantage during our recruitment process because of a disability, condition or impairment. To assist you with this, we will reduce or remove any barriers where possible and provide additional support where appropriate. 
If you need a change to be made so that you can make your application, you should complete the “Assistance required” section in the “Additional requirements” page of your application form to tell us what changes or help you might need during the recruitment process.  

For example, you may need wheelchair access at interview, or if you are deaf, a Language Service Professional. You might also require additional time to complete a timed assessment or a sign language interpreter to support with the relaying of information. 

Feedback will only be provided if you attend an interview or assessment.


Successful candidates must undergo a criminal record check.
People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

  • UK nationals
  • nationals of the Republic of Ireland
  • nationals of Commonwealth countries who have the right to work in the UK
  • nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
  • nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
  • individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
  • Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service
Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.
The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.
Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

  • Name : Serena Connor
  • Email :

Recruitment team

  • Email :

Further information

If your application has not been treated in accordance with the Recruitment Principles and you wish to make a complaint, you should contact in the first instance. If you are not satisfied with the response you receive you can contact the Civil Service Commission

Share this page