transforEmotion: An Open-Source R Package for Emotion Analysis Using Transformer-Based Generative AI Models

The problem

Communication researchers want to follow emotions across speeches, posts, photos and video. In practice that means one tool for text, another for images, a third for video, and often some Python glue between them. Most tools also come with a fixed list of emotions, such as Ekman’s six basic ones, which may not match the theory a study is testing. And many workflows depend on commercial APIs that change their models, raise their prices or shut down, so the analysis cannot be repeated a year later.

What the package does

transforEmotion puts text, image and video analysis behind one consistent set of R functions. It uses zero-shot classification: you give it any list of emotion words, and the model scores how well each one fits. The default text model is DistilRoBERTa, and any zero-shot text model on Hugging Face can be swapped in. Images and video frames are scored with CLIP. Four CLIP models are built in, including Jina AI’s CLIP-v2, which covers 89 languages.

• 3modalities: text, image, video
• 5core analysis functions
• 4built-in CLIP models
• 0.42zero-shot accuracy, 8 classes

Beyond discrete emotions, vad_scores() gives valence, arousal and dominance on a 0–1 scale, and rag() runs retrieval-augmented generation with small language models such as TinyLLAMA or Gemma 3. With Gemma 3 it can return a label and a confidence score per document as a table. Everything runs locally. The package sets up its own Python environment in the background, so R users do not have to manage Python themselves. An internet connection is needed only to download a model the first time.

A short example

Here we score an excerpt from Martin Luther King’s “I Have a Dream” speech against eight labels of our own choosing:

library(transforEmotion)

emotions <- c("anger", "fear", "joy", "sadness",
              "optimism", "hope", "surprise", "disgust")
results <- transformer_scores(text = text, classes = emotions)

The default model puts “hope” first (0.22), followed by “surprise” and “optimism”. Switching to the larger Facebook BART model takes one argument, and it also ranks “hope” first (0.41). The same pattern works for a photo with image_scores() and for a YouTube link or a local file with video_scores(), as in the figure above.

What the paper shows

This is a software demonstration. Its goal is to make existing models usable for communication research, and it does not propose a new way to detect emotion. We walk through a full workflow on two clips from the MAFW video dataset. For the clip labelled happiness, the video model gave “happy” the highest score (0.48), and so did the text model on its neutral written description (0.78). For the clip labelled anger, the video scores put “angry” narrowly ahead of “contemptuous”, while the text scores were spread across several labels. A RAG step then picked “anger” for that description.

We also run a small, reproducible benchmark on 50 images from FindingEmo, scored with eight scene-level emotion labels and the default CLIP model. Accuracy was about 0.42, with macro precision of 0.45 and macro recall of 0.32. We treat this as a conservative baseline and a template that others can scale up. It is not a full benchmark.

Limitations

The underlying models vary in accuracy across settings, and their validity in specific social science contexts has not been tested systematically. They remain black boxes with their own biases. Non-English and cross-cultural work needs extra validation. Large jobs, such as thousands of videos, are slow on a CPU. Small local language models can hallucinate, and their output depends on retrieval and sampling settings. Any evaluation is only as good as the human labels it is checked against. The package includes evaluate_emotions() and validate_rag_predictions() so that users can test the models on a small annotated sample from their own data before a full analysis.

Who it is for

The package is for researchers in communication, psychology and related fields who work in R and want emotion scores from text, images or video without sending data to an outside service. That matters for sensitive material such as interviews, therapy recordings or health data. Studies so far have used it on therapist and chatbot responses, corporate reports, conversations about art, course evaluations and videos of political leaders.

@article{Aleksandar_Toma_evi__2026, title={transforEmotion: An Open-Source R Package for Emotion Analysis Using Transformer-Based Generative AI Models}, volume={8}, ISSN={2665-9085}, url={http://dx.doi.org/10.5117/CCR2026.2.2.TOMA}, DOI={10.5117/ccr2026.2.2.toma}, number={2}, journal={Computational Communication Research}, publisher={Amsterdam University Press}, author={Aleksandar Tomašević and Hudson Golino and Alexander Christensen}, year={2026}, month=Sept, pages={1} }

Related

• The face of populism: examining differences in facial emotional expressions of political leaders using machine learning (Journal of Computational Social Science, 2025)
• A Latent Oscillator Measurement Model to Simulate Emotional-Expression Score Dynamics in Video (arXiv, 2026)
• AI-Based Emotion Analysis for Text, Image, and Video (Statistical Horizons livestream seminar with Hudson Golino, 4, 6 and 7 November 2026, built around transforEmotion.)
• All publications