Programme 2024
Rapid advancements in LLMs offer exciting new opportunities for social science applications. However, it is crucial to address the limitations of these models, including their propensity to hallucinate, propagate social biases, and occasionally provide false information. The primary goal of the Oxford LLMs workshop is to promote the responsible and effective use of LLMs in social science research.
The workshop is divided into two main parts - the Instruction part happening during the first three days of the workshop and Collaborative Research part, which starts online before the workshop, commences during the last two days of the workshop, and continues afterwards until the projects are finished.
Instruction: Monday 23 Sept - Wednesday 25 Sept
Every day, we start with a technical lecture offering a deep dive into the functioning of modern LLMs, covering both theoretical and practical aspects. After a lunch break, we continue with practical coding sessions: these will provide you with the skills and tools needed to effectively use LLMs in your research. In the afternoon, we conclude with a research talk or panel during which invited social science researchers will share their ongoing projects at the intersections of social science and natural language processing (NLP).
The programme covers:
Introduction to LLMs with deep dives into how everything works under the hood
LLM Evaluation: a brief history of LLM evaluation methods, from Turing test to Chatbot Arena. You will learn about generalisation abilities of LLMs, societal impact metrics and many more!
AI Agents: overviewing the limits of autonomy for the current state of LLMs. Case studies and discussions of existing frameworks and evaluation methods.
LLMs Finetuning and Alignment: supervised finetuning and preference tuning methods (RLHF, DPO, etc). Case studies with respect to social science applications.
Practical tips and tricks with LLMs: advanced prompt engineering, retrieval augmented generation (RAG) and other useful approaches and tools to get the most from LLMs!
Collaborative Research Projects: Thursday 26 Sept - Friday 27 Sept (active phase)
Participants and school conveners will engage in collaborative research projects that aim to result in peer-reviewed publications. These projects will start online slightly before the workshop, intensify during the final two days, and continue afterward.
This is the first time we are running an event with this format. We believe it offers great incentives to work on collaborative projects using the skills learned during the programme and provides a great opportunity to learn from each other. However, we would need to adjust things as we go along.
The focus will be on interdisciplinary projects that lie at the intersection of NLP and social sciences. By the end of the workshop, attendees will be equipped with the knowledge and skills to integrate LLMs responsibly and effectively into their social science research, fostering innovative and ethical applications.
After confirming your participation, you will be offered to join one of the several research projects curated by the workshop organisers. Alternatively, you can propose your own project idea and we will help you develop it further. Before the workshop, we will discuss and refine the ideas and develop research implementation plan and test proof of concepts. During the workshop project sessions on 26-27 September, we will test out and refine the research implementation plans with feedback from our invited speakers and lecturers. After the workshop, we will enter the final implementation and write-up stage that will culminate in sending out final papers to peer-reviewed conferences and journals.
Points of clarification on the Collaborative Research Projects
All people who have in any way contributed to a project will be invited to join a co-authors list, should the project make it through to a paper stage!
We are working on securing funding for API and cloud computing access and will update you soon on that!
You would be able to switch to a different research project during the initial stage but not after the 23 September