SICSS Workshop Data Collection/Annotation & Inferences with LLMs in Social Sciences

This repository contains the code and instructions for the SICSS workshop on data collection/annotation and inference with Large Language Models. In particular, we cover:

1. How LLMs work and how to use them in computational social science research workflows.
2. How to use LLMs for data collection and annotation via API calls to various cloud providers, including best practices and potential pitfalls.
3. How to run LLM inference on sensitive data in SANE, a secure analysis environment.
4. How to use LLM annotations in downstream inferential regression analyses.

A more general version of this repository, which is not specific to the SICSS curriculum and computing infrastructure (e.g., Python 3.8.18, R 4.5.3, SANE), is available here.

Slides for the workshop can be found here.

The materials on this page are CC-BY-4.0 licensed.

Full Workshop Schedule

Time	Title	Notebooks	Recommended Environment
09:30	LLM fundamentals for Social Sciences
11:00	Coffee break
11:20	Data collection/annotation with LLMs	python, R	Google Colab
12:30	Lunch break
13:30	Secure LLM inference in SANE	python, R	SANE only
15:00	Inference with LLM annotations	python, R	Local machines or SANE
16:30	Conclusion & Q&A

Technical details

• No previous experience with LLMs is required.
• R or python programming knowledge is desired but not required.
• In python we will use langchain, in R we will use ellmer to streamline interaction with LLM APIs.

Preparation

Get Your Own API Keys

You will need an API key for the respective LLM provider you plan to use. An API key is a unique identifier that allows you to authenticate and interact with the provider's services.

Follow the instructions below to obtain API keys for the providers we will use in the workshop. More detailed instructions will be provided by the instructor separately during the workshop. In addition, you will be provided with temporary API keys for OpenAI and SURF AI Hub that you can use during the workshop.

• Hugging Face Inference API:

  1. Create an account at https://huggingface.co/.
  2. Go to https://huggingface.co/settings/tokens and create a new access token.

• OpenAI:

  1. Create an account at https://platform.openai.com/.
  2. Create an API key at https://platform.openai.com/api-keys. Note that you need to provide billing information to get access to the API, and that there is no free trial or credit available.

• Groq:

  1. Create an account at https://console.groq.com/.
  2. Create an API key in the Groq console.

• SURF AI Hub:

  1. It is in pilot phase and requires an application and approval process to get access.
  2. Once you have access, you can create an API key at https://willma.surf.nl/.

Save your API keys in a safe place. The notebooks will prompt you to enter the keys at runtime.

Set Up SANE and Replicate SANE Environments Locally

See here for detailed instructions on how to set up your R environment in SANE and reproduce on your own local machine the exact Python/R environment and Ollama models installed in SANE.

Additional Resources

Tutorial Paper

Read and cite our tutorial paper (preprint):

• Fang, Q., Bernardo, J. G., & van Kesteren, E. J. (2026). A Methodological Guide on Using Large Language Models for Reproducible Text Annotation in the Social Sciences and Humanities with Python and R. arXiv preprint arXiv:2604.09638.
• Download from arXiv

Guide to LLM Computing Infrastructure in the Netherlands

• Link

[Optional] Run Locally

Note, if you already have Python, R and Ollama set up on your local machine according to the instructions in the previous section, you can skip this section. That is, you can simply download and open the notebooks in your local Python/R environment and run them directly.

The instructions below are for those who want to set up a Python/R environment that satisfies the minimum requirements to run notebooks in this repository.

With uv and Python

If you plan to run the Python notebooks locally, we recommend using uv to set up a clean Python environment. You can also use uv to launch Jupyter Lab or Notebook.

1. Clone the repository:
   • git clone https://github.com/sodascience/sicss_llm_workshop.git
   • cd sicss_llm_workshop
2. Create and sync the environment:
   • uv venv
   • uv sync
3. Start Jupyter [Optional]:
   • uv run jupyter lab (or uv run jupyter notebook)

If you use a different environment manager, make sure the dependencies in pyproject.toml are installed before running the notebooks.

With renv and R

If you plan to run the R notebooks locally, we recommend using renv to restore the exact package environment used in this workshop.

1. Clone the repository:
   • git clone https://github.com/sodascience/sicss_llm_workshop.git
   • cd sicss_llm_workshop
2. Open the project in RStudio (or start R in the repository root), then restore the environment:

   install.packages("renv")  # skip if renv is already installed
   renv::restore()

3. Open the R notebooks (.qmd for RStudio or .ipynb for Visual Studio Code) and run them as usual.

renv::restore() reads renv.lock and installs the exact package versions recorded there into a project-local library, so your system R installation is not affected.

Contact

This project is developed and maintained by the ODISSEI Social Data Science (SoDa) team.

Do you have questions, suggestions, or remarks? File an issue or feel free to contact Qixiang Fang.