Spaces:
Runtime error
Runtime error
Christian Rene Thelen
commited on
Commit
·
6fe86f3
1
Parent(s):
34ab535
Added model card and reference to Model on Huggingface
Browse files
README.md
CHANGED
|
@@ -45,17 +45,88 @@ pip install -r requirements.txt
|
|
| 45 |
|
| 46 |
|
| 47 |
|
| 48 |
-
|
| 49 |
-
im Studiengang Angewandte Mathematik und Informatik (dual) B. Sc. an der Fachhochschule Aachen durchgeführt wurden.
|
| 50 |
|
| 51 |
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
|
|
|
| 54 |
|
| 55 |
-
**
|
|
|
|
| 56 |
|
| 57 |
-
|
| 58 |
|
| 59 |
-
|
| 60 |
|
| 61 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
|
| 47 |
|
| 48 |
+
---
|
|
|
|
| 49 |
|
| 50 |
|
| 51 |
+
# :trophy: Model
|
| 52 |
+
|
| 53 |
+
Model on [Huggingface](https://huggingface.co/cortex359/germeval2025)
|
| 54 |
+
|
| 55 |
+
## Model Details
|
| 56 |
+
|
| 57 |
+
- **Model Type:** Transformer-based encoder (XLM-RoBERTa-Large)
|
| 58 |
+
- **Developed by:** Christian Rene Thelen, Patrick Gustav Blaneck, Tobias Bornheim, Niklas Grieger, Stephan Bialonski (FH Aachen, RWTH Aachen, ORDIX AG, Utrecht University)
|
| 59 |
+
- **Paper:** [AIxcellent Vibes at GermEval 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training](https://arxiv.org/abs/2509.07459v2)
|
| 60 |
+
- **Base Model:** [XLM-RoBERTa-Large](https://huggingface.co/FacebookAI/xlm-roberta-large) (Conneau et al., 2020)
|
| 61 |
+
- **Fine-tuning Objective:** Detection of *candy speech* (positive/supportive language) in German YouTube comments.
|
| 62 |
+
|
| 63 |
+
## Model Description
|
| 64 |
+
|
| 65 |
+
This model is a fine-tuned **XLM-RoBERTa-Large** adapted for the **GermEval 2025 Shared Task on Candy Speech Detection**.
|
| 66 |
+
It was trained to identify *candy speech* at both:
|
| 67 |
+
|
| 68 |
+
- **Binary level:** Classify whether a comment contains candy speech.
|
| 69 |
+
- **Span level:** Detect the exact spans and categories of candy speech within comments, using a BIO tagging scheme across **10 categories** (positive feedback, compliment, affection declaration, encouragement, gratitude, agreement, ambiguous, implicit, group membership, sympathy).
|
| 70 |
+
|
| 71 |
+
The span-level model also proved effective for binary detection by classifying a comment as candy speech if at least one positive span was detected.
|
| 72 |
+
|
| 73 |
+
## Intended Uses
|
| 74 |
+
|
| 75 |
+
- **Research:** Analysis of positive/supportive communication in German social media.
|
| 76 |
+
- **Applications:** Social media analytics, conversational AI safety (mitigating sycophancy), computational social science.
|
| 77 |
+
- **Not for:** Deployments without fairness/robustness testing on out-of-domain data.
|
| 78 |
+
|
| 79 |
+
## Performance
|
| 80 |
+
|
| 81 |
+
- **Dataset:** 46k German YouTube comments, annotated with candy speech spans.
|
| 82 |
+
- **Training Data Split:** 37,057 comments (train), 9,229 (test).
|
| 83 |
+
- **Shared Task Results:**
|
| 84 |
+
|
| 85 |
+
- **Subtask 1 (binary detection):** Positive F1 = **0.891** (ranked 1st)
|
| 86 |
+
- **Subtask 2 (span detection):** Strict F1 = **0.631** (ranked 1st)
|
| 87 |
+
|
| 88 |
+
## Training Procedure
|
| 89 |
+
|
| 90 |
+
- **Architecture:** XLM-RoBERTa-Large + linear classification layer (BIO tagging, 21 labels including “O”).
|
| 91 |
+
- **Optimizer:** AdamW
|
| 92 |
+
- **Learning Rate:** Peak 2e-5 with linear decay and warmup (500 steps).
|
| 93 |
+
- **Epochs:** 20 (with early stopping).
|
| 94 |
+
- **Batch Size:** 32
|
| 95 |
+
- **Regularization:** Dropout (0.1), weight decay (0.01), gradient clipping (L2 norm 1.0).
|
| 96 |
+
- **Postprocessing:** BIO tag correction and subword alignment.
|
| 97 |
+
|
| 98 |
+
## Limitations
|
| 99 |
+
|
| 100 |
+
- **Domain Specificity:** Trained only on German YouTube comments; performance may degrade on other platforms, genres, or languages.
|
| 101 |
+
- **Overlapping Spans:** Cannot handle overlapping spans, as they were rare (<2%) in the training data.
|
| 102 |
+
- **Biases:** May reflect biases present in the dataset (e.g., demographic skews in YouTube communities).
|
| 103 |
+
- **Generalization:** Needs evaluation before deployment in real-world moderation systems.
|
| 104 |
|
| 105 |
+
## Ethical Considerations
|
| 106 |
|
| 107 |
+
- **Positive speech detection** is less studied than toxic speech, but automatic labeling of “supportiveness” may reinforce cultural biases about what counts as “positive.”
|
| 108 |
+
- Must be complemented with **human-in-the-loop moderation** to avoid misuse.
|
| 109 |
|
| 110 |
+
## Citation
|
| 111 |
|
| 112 |
+
If you use this model, please cite:
|
| 113 |
|
| 114 |
+
```
|
| 115 |
+
@inproceedings{thelen-etal-2025-aixcellent,
|
| 116 |
+
title = "{AI}xcellent Vibes at {G}erm{E}val 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training",
|
| 117 |
+
author = "Thelen, Christian Rene and
|
| 118 |
+
Blaneck, Patrick Gustav and
|
| 119 |
+
Bornheim, Tobias and
|
| 120 |
+
Grieger, Niklas and
|
| 121 |
+
Bialonski, Stephan",
|
| 122 |
+
editor = "Wartena, Christian and
|
| 123 |
+
Heid, Ulrich",
|
| 124 |
+
booktitle = "Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops",
|
| 125 |
+
month = sep,
|
| 126 |
+
year = "2025",
|
| 127 |
+
address = "Hannover, Germany",
|
| 128 |
+
publisher = "HsH Applied Academics",
|
| 129 |
+
url = "https://aclanthology.org/2025.konvens-2.33/",
|
| 130 |
+
pages = "398--403"
|
| 131 |
+
}
|
| 132 |
+
```
|