Demand management
Demand management is not being applied to this funding opportunity.
Aim
The overall aim of this funding opportunity is to fund projects that:
- will make a step change in how researchers can discover, access and use social science data
- advance the transition to technology-enabled data pipelines that support data management, curation, discovery and access
- in turn, will develop and upskill the data service infrastructure workforce to support the research community in the future
This funding opportunity arises out of the Future Data Services (FDS) Review which investigated the challenges for accessing and using data and the opportunities for future data infrastructure provision. This funding opportunity will therefore support the delivery of the FDS recommendations and its vision for the future of data services:
“A seamless, connected, user-centred and federated data service landscape, driven by curiosity, collaboration and coordination, and making use of the new technologies that present boundless possibilities for researchers to use data for public good.”
In line with the FDS review this funding opportunity is agnostic of data types. Applications should have considered delivery of discovery and access to all data types, including quantitative, qualitative, longitudinal and cross-section survey data, smart data, administrative data, consumer loyalty data and other types of data generated and used in the social sciences.
Expected outcomes
Through investment in transforming discovery and access to social science data, the following outcomes are expected:
- researchers use transformational technologies to more easily find data and can access consistent harmonised information about data
- data services staff use transformational technologies to more easily manage and curate data
- researchers experience a consistent and efficient journey which is easy to understand when applying to access data
- data owners can more easily deposit their data with data services for onward use by researchers
- data services staff are equipped to deliver transformational services and support researchers, and have the technical skills to develop and implement new technologies and systems
Scope
Your application should be based on the application that you presented at the ‘Future data services sandpit: transforming discovery and access’. It will have evolved since the sandpit, but it should not differ in its project objectives.
This funding opportunity will fund projects with a focus on improving how researchers, users and data depositors engage with data services, improving their experience in discovering, accessing and using data. It will also fund projects with a focus on improving the backend process to transition to technology-enabled data pipelines (particularly those that deploy AI) that support data management, curation, discovery and access.
During the FDS review, the research and infrastructure community highlighted the challenge of proliferation in the landscape. We therefore aim to fund improved data services rather than more data services. As such, this funding opportunity will only consider projects which enhance one or more ESRC social science data service infrastructure investments. All projects developed at the sandpit must have partnered with an existing ESRC social science data service infrastructure investment, though the infrastructure does not have to lead the project. For example, the project lead can come from a different research organisation to the data service and not have previously worked with them. Please see ‘Additional information’ for a list of ESRC social science data service infrastructure investments.
Proposed activity must not duplicate existing data infrastructure services and functionalities but rather should enhance, develop and improve existing infrastructures to resolve challenges experienced by researchers, users and data depositors in their journey. Projects must not propose any new data service infrastructures, any new provision should build on existing services, platforms and approaches, and de-complexify the landscape. Funding will only be available for new tools and services that are supported by plans for adoption by ESRC social science data service infrastructures (including ‘pilot’ projects).
The projects developed at the sandpit should advance at least one, or a combination of the following priority areas identified in the FDS review:
- data discovery
- access to sensitive data
- technology
- upskilling and supporting staff
Projects are not expected to address every objective under the associated priority area; a minimum of one objective must be addressed. Projects must include a work package on upskilling and supporting staff. Activities, timelines and outputs must be feasible within the resource available. Details on each priority area are outlined below.
Data discovery
The FDS review highlighted a clear challenge for users in discovering data and accessing high quality and consistent information about data, due to proliferation of curation standards and discovery search tools. Some tools, and the standards of curation, and available metadata, do not always meet the needs of the research community. Our ambition for this area is that future initiatives will both enable the development of new tools while simultaneously building on existing capability to improve curation and discovery services and capability.
Our objectives for successful awards with a focus on improving the experience of researchers, users and data depositors in discovering data are to:
- improve data discovery and curation practices so that they meet the needs of the research community
- create and develop guidance, resources and support for:
- better data documentation
- better metadata, including support for investments to know the appropriate level of metadata to implement
- curation of data generated by government agencies
- data depositors to deposit data with data services
- making datasets more visible and improved signposting of where to access data
- routine production of synthetic data
- support harmonisation and consistency in approaches to data discovery and curation, such as in metadata, documentation and standards. Supporting interoperability and progression to federation of data discovery
Synthetic data is a growing area for investment, aiding researchers in their understanding of data sources. We have identified specific areas requiring support:
- production of low fidelity synthetic data of non-administrative secure data with associated documentation and privacy assurance
- testing how high-fidelity synthetic data could be produced and utilised outside of Trusted Research Environments
- testing different use cases of synthetic data to establish a sound evidence base for routine synthetic data provision and its utility.
- effectively documenting synthetic data and creating training resources to ensure their effective use
ESRC and ADR UK, with support from the UK Research and Innovation (UKRI) Digital Research Infrastructure Programme, have previously funded projects in this area (please see the FDS report for further details on these projects). Projects focused on synthetic data should review and align to the ADR UK Synthetic Data Working Group guidance on the provision of synthetic forms of secure data.
Access to sensitive data
The FDS review highlighted that users experience uncertainty and delays when trying to access sensitive data for research. Our ambition for this area is that the processes to access data are proportionate, transparent and visible and minimise waiting time for users. Data access forms are easy and unambiguous to understand and complete, and researchers have clarity about their journey to access data.
Our objectives for successful awards with a focus on improving the experiences of researchers and users in accessing sensitive data are to:
- improve access to sensitive data, ensuring processes are proportionate, transparent and visible, and minimise waiting times for users
- create and develop guidance, resources and support for researchers to apply for and access data, such as the provision of templates and examples, training on feasibility assessments and ethical approval, and the publication of the entire data access cycle outlining the researcher journey
- support consistent and harmonised data sharing practices to enable interoperability and progression to federated data access
- ensure data service infrastructure staff are knowledgeable about data, and the use of data for research
Technology
Technology is an underlying theme across FDS, which supports better data discovery and data access, and relies on an upskilled workforce to deliver it. It is therefore an essential element to funding in this area and should be considered for inclusion across all projects.
Our objectives for successful awards with a focus on advancing the transition to a technology-enabled data pipelines that supports data management, curation, discovery and access are to invest in:
- automated processes (including development and adoption of AI) to remove backend data access processing friction and improve data ingest, curation and discovery
- technology (including cloud services, if appropriate) to enhance services or additional tools for the research community
Upskilling and supporting staff
Upskilling staff is a priority area for UKRI. Ensuring staff have the necessary skills to deliver enhanced data services, is essential across all projects. Therefore, each project must include an upskilling staff work package, which address the objectives outlined below and is designed to support the careers of staff working within data service infrastructures. The aim of this is to ensure that researchers and data depositors benefit from interactions with expert service providers, leading to better data services for research. It will also ensure sustainability of the value from this one-off funding.
The FDS review recommended data services produce a People Strategy which outlines proposed roles, functions and approaches to skills assessment and career development to support the data service workforce. With these projects being time bound and to be implemented via existing infrastructures, a full People Strategy at this stage is not required. However, projects are required to demonstrate the training, skills development and learning support they will provide to new and existing staff.
Our objectives for this work package in each successful award will be to:
- invest in teams, resources, training and leadership, across a range of roles, to upskill the data service workforce to deliver better data service provision
- where applicable, provide resources and training to upskill data service workforces to support and adopt new technologies and technical skills
- define career pathways with training and skills matrix which encompasses actual job activities, particularly with reference to non-technical skills such as management and leadership
- enable data service staff to have good familiarity with research, and a thorough understanding of the needs of researchers, for example why they need to discover and access data, to ensure enhancement support researchers in their journey
Data services are made up of diverse roles, with skills and training needs varying. This will need to be considered in project development. Priority skill areas identified (including through the recent UKDS Computational Skills Survey), but not limited to, are:
- understanding social science data and research and in turn why people want to use data
- AI and machine learning, its application and implementation, such as agentic AI
- prompt engineering and use of AI tools
- programming skills and coding
- database and data architecture skills
- workflow automation and reproducibility
- data visualisation and user experience
- leading effective data science and engineering teams working in the social sciences
- creating and curating data
- providing expert support to help researchers make the most of data
- statistical disclosure control and other key skills to support the operation of
- Trusted Research Environments
Across these priority areas, you are encouraged to interact with the existing resources, learning and recommendations developed in this area. Notably, ESRC and UKRI funded a series of pilots as part of the FDS programme focused on:
- federation of data services
- data discovery using machine learning or other AI technologies
- increasing skills capacity for data service professionals
Further details can be found on the FDS webpage. Through this work we hope to adopt and develop the learning and recommendations of previously funded FDS pilots.
Collaboration and engagement
Collaboration refers to the activity which creates, enables and maintains connections between researchers, policymakers, organisations, infrastructures and communities to maximise impact and innovation. Engagement is the mechanisms you use and actions you take to realise and share your intended collaborative outputs and impacts of the project. Collaboration and engagement are essential at all stages of the project.
Collaboration across the data services landscape is key to deliver a seamless, connected, user-centred and federated data service landscape, driven by a ‘whole system’ outlook which enables ESRC’s data infrastructure investments to connect and co-deliver consistent and improved service to help researchers.
Although there are examples of collaboration, the review noted the need for data services to interconnect with each other more often, for example, to overcome the challenges (such as variation in processes, or quality of metadata) that researchers face when they interact with different data services to access different data. This creates a burden on their time and research productivity.
The FDS review highlighted the importance of co-design, whereby data services are designed with researchers by default. Projects therefore must engage with the external user and research community throughout the project to understand and anticipate their needs and ensure enhancements support the intended impact for the community on how they discover, access and use data. Projects should clearly articulate their engagement plans for the development and delivery of projects. We encourage users and data depositors who would benefit from the outputs of this funding opportunity to engage with ESRC’s social science data service infrastructures.
This process is built around fostering collaboration, to co-design, co-create and co-deliver projects that will support a more effective and efficient federated data service landscape, reducing the bureaucracy experienced by researchers when accessing and using data services and ensuring services meet community needs. Supporting collaboration and engagement through this process and the subsequent awards brings us closer to accomplishing our vision of a federated data landscape designed with researchers by default.
This funding opportunity, and planned future FDS opportunities, are being funded as a programme. Successful projects will be expected to engage and collaborate with each other, ESRC-supported data infrastructures and the wider social science community including users where activities and learning may overlap and where this can add value. Applications must outline collaborations and related benefits.
Theory of change
You will outline your project through a theory of change approach, clearly outlining:
- vision, including the problem that is being addressed
- delivery
- outputs
- outcomes
- benefits
Successful projects will be required to submit a final delivery plan within two months of the start of their award. This will be built on the details shared within your application, as part of the application questions, this will include:
- scope and objectives
- how the activity is to be carried out- including work packages
- how it’s going to be implemented and sustained via the existing ESRC Data Service Infrastructures
- risk management: identification and mitigation
- roles and responsibilities: successful applicants must have the capability and resource to deliver projects
- sustainability for the future
- timeline for delivery, including milestones and deliverables
- key performance indicators or critical success factors to monitor outputs and outcomes
- plans for collaboration and engagement throughout the award
- proportionate and effective governance procedures
Full delivery plan requirements will be outlined for the start of the awards.
Investment monitoring
We will set out monitoring and reporting requirements in the terms and conditions of the award. Award holders will be required to produce an updated delivery plan within two months of the start of the award, for subsequent regular discussion with ESRC. This will be signed off by ESRC and the ESRC FDS Programme Board. The ESRC FDS Programme Board will support ESRC and its FDS programme in delivering on its investment and will be made up of experts from across UKRI and external technical and user experts.
The award holders will be expected to provide ESRC with a short, written, quarterly update on activities, including risk, finance, progress, and where applicable impact. More frequent updates will be expected on important activities, risks and major project changes if they present a risk to meeting its objectives. Data will also be collected on research outcomes.
We will assign an investment manager as a lead contact for the award programme.
Contact will include:
- a twice-yearly meeting between ESRC and the project lead, as well as other members of the team where appropriate
- an annual meeting with all successful FDS: transforming discovery and access projects
Applications must include sufficient time for project leads and (where relevant) co-leads to meet these monitoring requirements as well as any other additional governance arrangements they see fit.
For more information on the background of this funding opportunity, go to the ‘Additional information’ section.
Duration
The duration of this award is a minimum of 24 months and a maximum of 48 months.
Projects must start by 1 April 2027 and end by 31 March 20231.
Applications must comply with the ESRC research funding guide.
Funding available
It is expected that around five to eight projects will be funded, sharing up to £21.6 million of total funding at 100% full economic cost (FEC).
ESRC will fund 80% of the FEC.
What we will fund
We will support project proposals that:
- will co-design, co-create, and co-deliver a federated data service landscape
- demonstrate the ability to transform discovery and access to data via existing ESRC social science data services infrastructures, preventing proliferation and duplication in the landscape
- enable the delivery of better research, supporting the needs of the whole community and taking into consideration the views of users
- will make a step change in how researchers can discover, access and use social science data
- advance the transition to technology-enabled data pipelines that support data management, curation, discovery and access
- support the upskilling of the social science data services workforce to ensure that researchers and data depositors benefit from interactions with expert service providers, resulting in better delivered services
What we will not fund
We will not fund:
- the creation of a new infrastructure
- work that duplicates existing data infrastructure services and functionalities
- work conducted in isolation and that does not enhance an existing ESRC social science data service infrastructure
- standard research projects
- writing up previous research
- preparation of books and publications
- literature surveys
- general conference attendance that is not related to conducting the proposed work
- studentships
- new tools and services that are already supported by plans for adoption by ESRC social science data service infrastructures (including ‘pilot’ projects)
Supporting skills and talent
We encourage you to follow the principles of the Concordat to Support the Career Development of Researchers and the Technician Commitment.
Trusted Research and Innovation (TR&I)
UKRI has a long-standing and continuing commitment to supporting and enabling safe and effective collaboration in research and innovation. UKRI is equally committed to ensuring that this takes place with integrity and within strong ethical frameworks.
Our Principles and expectations set out the principles that UKRI applies to TR&I and our general expectations of the research organisations (including businesses, research institutes, and research technical organisations) that we support. Organisations supported by UKRI should be able to provide evidence that their internal controls and processes meet these expectations.
See further guidance and information about TR&I, including where applicants can find additional support.
Data requirements
ESRC recognises the importance of data quality and provenance. Data generated, collected or acquired by ESRC-funded research must be well-managed by the grant holder to enable their data to be exploited to the maximum potential for further research. See our research data policy for details and further information on data requirements. The requirements of the research data policy are a condition of ESRC research funding.
Where relevant, details on data management and sharing should be provided in the ‘Data management’ section. See the importance of managing and sharing data and content for inclusion in a data management plan on the UK Data Service (UKDS) website for further guidance. We expect you to provide a summary of the points provided. The UKDS [email: datasharing@ukdataservice.ac.uk] will be pleased to advise you on the availability of data within the academic community and provide advice on data deposit requirements.
Impact, innovation and interdisciplinarity
We expect you to consider the potential scientific, societal and economic impacts of their research. Outputs, dissemination and impact are a key part of the criteria for most expert review and assessment processes. We also encourage applications that demonstrate innovation and interdisciplinarity (research combining approaches from more than one discipline).
Knowledge exchange and collaboration
We are committed to knowledge exchange and encouraging collaboration between researchers and the private, public and civil society sectors.
Collaborative working benefits both the researchers and the individuals and organisations involved. Through collaboration, partners learn about each other’s expertise, share knowledge and gain an appreciation of different professional cultures. Collaborative activity can therefore lead to a better understanding of the ways that academic research can add value and offer insights to key issues of concern for policy and practice.
Knowledge exchange should not be treated as an ‘add-on’ at the end of a project but considered before the start and built into a project.
Research ethics
We require that the research we support is designed and conducted in such a way that it meets ethical principles and is subject to proper professional and institutional oversight in terms of research governance. We have agreed a Framework for Research Ethics that all submitted proposals must comply with.
Read further details about the Framework for Research Ethics and guidance on compliance.
