Senior Research Scientist for Foundational AI Safety - AI Safety Institute
Department for Science, Innovation & Technology
Apply before 11:55 pm on Wednesday 15th May 2024
Details
Reference number
Salary
Job grade
Contract type
Loan
Secondment
Length of employment
Business area
Type of role
Working pattern
Number of jobs available
Contents
Location
About the job
Job summary
The AI Safety Institute is the first state-backed organisation focused on advanced AI safety for the public interest. We launched at the AI Safety Summit because we believe taking responsible action on this extraordinary technology requires a capable and empowered group of technical experts within government.
Our staff includes senior alumni from OpenAI, Google DeepMind, start-ups and the UK government, and ML professors from Oxford and Cambridge. We are now calling on the world’s top technical talent to build the institute from the ground up. This is a truly unique opportunity to help shape AI safety at an international level.
We have ambitious goals and need to move fast.
Develop and conduct evaluations on advanced AI systems
We will characterise safety-relevant capabilities, understand the safety and security of systems, and assess their societal impacts.
Drive foundational AI safety research
We will launch moonshot research projects and convene world-class external researchers.
Facilitate information exchange
We will establish clear information-sharing channels between the Institute and other national and international actors. These include stakeholders such as policymakers and international partners.
Our Inclusive Environment
We are building an inclusive culture to make the Department a brilliant place to work where our people feel valued, have a voice and can be their authentic selves. We value difference and diversity, not only because we believe it is the right thing to do, but because it will help us be more innovative and make better decisions.
What we Value:
- Diverse Perspectives: We believe that a range of experiences and backgrounds is essential to our success. We welcome individuals from underrepresented groups to join us in this crucial mission.
- Collaborative Spirit: We thrive on teamwork and open collaboration, valuing every contribution, big or small.
- Innovation and Impact: We are dedicated to making a real-world difference in the field of frontier AI safety and capability, and we encourage innovative thinking and bold ideas.
Job description
About the Role
As a Senior Research Scientist at AISI, you will lead Research Scientists and Research Engineers in planning and executing on research directions in foundational AI safety. You’ll lead and contribute to projects aimed at improving the fundamental safety of advanced AI systems, making novel research contributions that are aimed at informing governance with technical tools on safe AI.
We draw on a wide range of disciplines, and value a diversity of research expertise across our four workstreams. You’ll be primarily associated with one of our research workstreams (please specify in your application which you’re most interested in, details below), but sometimes your work will intersect multiple workstreams:
- Capabilities elicitation & jailbreaking. This workstream consists of research to ensure that AISI can get closer to the “capabilities ceiling” of the models we work with, including development of novel jailbreaks and methods such as “next-gen chain-of-thought" to increase model performance on our evals.
- AI model explainability & accountability. This workstream focuses on techniques for improving the transparency of model behaviour via e.g. casual reasoning traces.
- Interventions on model behaviour. This workstream will investigate ways of intervening on model behaviour pre- and post-deployment to solve safety issues beyond anecdotal patching, i.e. in ways that will be robust to improved prompting techniques. E.g. via data attribution techniques.
- Rethinking alignment. This workstream consists of developing methods for alignment that go beyond existing state-of-the-art, for example by reliance on formal languages or probabilistic approaches.
You will work under Prof. Yarin Gal, and work closely with your team comprising of multiple Research Scientists, Research Engineers, and Software Engineers, as well as benefit from support from our cross-functional Platform Engineering team. You’ll also collaborate with external topic-level experts, contractors, partner organisations and policy makers to coordinate and build on external research.
Day-to-day, you may:
- Conduct novel research
- Supervise Research Scientists in your team and mentor them
- Design experiments to validate hypotheses about models’ behaviour
- Work with experts to assemble relevant literature or datasets
- Prepare papers for internal or external publication
- Manage external research collaborations
- Prepare a write-up of your research to brief policy makers
The ideal candidate will have the skills and experience that allow them to additionally:
- Design experiments to conduct rigorous and impactful machine learning research
- Analyse and interpret data in ways that meaningfully contribute to research at AISI
- Understand the capabilities and potential impact of advanced AI systems, including large language models
- Keep track of and respond to the fast-moving literature, ensuring that our research is additional to that being conducted elsewhere
- Coordinate closely with research and engineering partners in your team and stakeholders across AISI to design new methods
- Clearly communicate insights from studies to influence research decisions across stakeholders throughout AISI and DSIT
There will be significant scope to contribute to the strategy of your workstream team and to design experiments with set-ups of increasing complexity.
Existing Civil Servants and applicants from accredited NDPBs are eligible to apply, but will only be considered on loan basis (Civil Servants) or secondment (accredited NDPBs). Prior agreement to be released on a loan basis must be obtained before commencing the application process. In the case of Civil Servants, the terms of the loan will be agreed between the home and host department and the Civil Servant. This includes grade on return.
Person specification
For the senior research scientist role, you’ll have conducted ML research, research in a domain relevant to your primary workstream, or research at the intersection of your domain and frontier AI systems.
We expect experts in both ML and a specific domain relevant to one of our workstreams to be rare, so we encourage you to apply no matter which research expertise you’re excited to bring to the institute.
We look for the following skills, experience and attitudes:
- Significant track record of conducting and disseminating rigorous and impactful research
- Significant experience leading work in AI/ML research
- PhD or equivalent research experience in a field related to your workstream, or in machine learning. If you have more work experience, you might be assigned a more senior role in the team
- Experience with large language models, potentially related to prompt engineering, tooling, or fine tuning
- Possess a strong curiosity in technical understanding of AI systems and studying their limitations
- Motivated to conduct research that is not only curiosity driven but also solves concrete open questions in AI safety and has a strong grounding for how research may impact governance and policy making
- Experience of mentoring others in a research setting
- Statistics expertise (e.g., data analysis)
- Good technical and coding skills, e.g. Python
- Strong project management skills and demonstrable experience of teamwork, including building rapport with people from diverse backgrounds
Given the changing nature of the field, it’s most of all important to us to build a team with strong problem-solving skills and a preparedness to acquire any missing knowledge necessary to get the job done.
Owing to the rapid development of advanced AI systems, you will likely be up to date with the latest advancements.
Core requirements
- You should be able to spend at least 4 days per week on working with us
- You should be able to join us for at least 12 months
- You should be able work from our office in London (Whitehall) for parts of the week, but we provide flexibility for remote work
Benefits
The Department for Science, Innovation and Technology offers a competitive mix of benefits including:
- A culture of flexible working, such as job sharing, homeworking and compressed hours.
- Automatic enrolment into the Civil Service Pension Scheme, with an average employer contribution of 27% of the base salary.
- A minimum of 25 days of paid annual leave, increasing by 1 day per year up to a maximum of 30.
- An extensive range of learning & professional development opportunities, which all staff are actively encouraged to pursue.
- Access to a range of retail, travel and lifestyle employee discounts.
- The Department operates a discretionary hybrid working policy, which provides for a combination of working hours from your place of work and from your home in the UK. The current expectation for staff is to attend the office or non-home based location for 40-60% of the time over the accounting period.
Things you need to know
Selection process details
Application Process
To apply for this post, you will need to complete the online application process, this should be completed no later than Wednesday 15th May 2024 at 23:55 and will involve completing the sections in the online application outlined in this pack.
We reserve the right to close this vacancy with a two week notice period if we have received a sufficient number of applications. Please submit your application as early as possible to ensure that you are considered for the role.
To apply for this post, you will be required to provide your CV and a personal statement (500 word limit) setting out your answers to the questions below:-
- Why are you interested in joining the AISI right now?
- What are the most important technical projects UK government needs to undertake within the next few months to advance AI Safety?
- What do you want to be doing to drive those projects forward?
- Is there anything else about your skills, experience or background you’d like us to be aware of?
Sift and interview process
Candidates should expect to go through some or all of the following stages once an application has been submitted:-
Coding Test - To be completed within 72 hours
Initial Interview
Technical take home test
Formal interview and review of take home test
Third stage interview
Final Stage interview
Sift and interview dates
Candidates will be sifted on an ad-hoc basis once your application has been submitted. If successful in first sift, we will contact you to arrange initial interviews or coding tests to be taken.
Reasonable Adjustment
We are proud to be a disability confident leader and we welcome applications from disabled candidates and candidates with long-term conditions.
We fully support adjustments throughout our recruitment process and we encourage candidates to discuss their adjustment needs by emailing the job contact which can be found under the contact point for applicants section.
We do not have an exhaustive list of adjustments that we support but just some examples include additional time to complete your application form, behaviour questions up to 72 hours prior to interview and having extra time at interview.
If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'Contact point for applicants' section.
If successful and transferring from another Government Department a criminal record check may be carried out.
New entrants are expected to join on the minimum of the pay band.
A location based reserve list of successful candidates will be kept for 12 months. Should another role become available within that period you may be offered this position.
Please note terms and conditions are attached. Please take time to read the document to determine how these may affect you.
Any move to the Department for Science, Innovation and Technology from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility https://www.childcarechoices.gov.uk
DSIT does not normally offer full home working (i.e. working at home); but we do offer a variety of flexible working options (including occasionally working from home).
In order to process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service on your behalf.
However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading.
Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment.
A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government.
Nationality Requirements
If you do not meet the standard nationality requirements as detailed below, then we would still encourage you to submit an application. We may be able to explore other options such as seconding you in from either your current employer or a third party organisation.
Not meeting the nationality rules set out below should not be seen as an automatic barrier to applying for this role.
Feedback
Feedback will only be provided if you attend an interview or assessment.
Security
Nationality requirements
Working for the Civil Service
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
Diversity and Inclusion
Apply and further information
Contact point for applicants
Job contact :
- Name : active.campaigns@dsit.gov.uk
- Email : active.campaigns@dsit.gov.uk
Recruitment team
- Email : dsitresourcing.grs@cabinetoffice.gov.uk
Further information
