Christian Rene Thelen commited on
Commit
6fe86f3
·
1 Parent(s): 34ab535

Added model card and reference to Model on Huggingface

Browse files
Files changed (1) hide show
  1. README.md +78 -7
README.md CHANGED
@@ -45,17 +45,88 @@ pip install -r requirements.txt
45
 
46
 
47
 
48
- Diese Repository enthält den Code, mit dem die Untersuchungen der Bachelorarbeit **Flauschdetektion (GermEval 2025)**
49
- im Studiengang Angewandte Mathematik und Informatik (dual) B. Sc. an der Fachhochschule Aachen durchgeführt wurden.
50
 
51
 
52
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
 
 
54
 
55
- **Studiengang**
 
56
 
57
- Angewandte Mathematik und Informatik B.Sc. ([AMI](https://www.fh-aachen.de/studium/angewandte-mathematik-und-informatik-bsc)) an der [FH Aachen](https://www.fh-aachen.de/), University of Applied Sciences.
58
 
59
- **Ausbildung mit IHK Abschluss**
60
 
61
- Mathematisch technische/-r Softwareentwickler/-in ([MaTSE](https://www.matse-ausbildung.de/startseite.html)) am Lehr- und Forschungsgebiet Igenieurhydrologie ([LFI](https://lfi.rwth-aachen.de/)) der [RWTH Aachen](https://www.rwth-aachen.de/) University.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
 
47
 
48
+ ---
 
49
 
50
 
51
+ # :trophy: Model
52
+
53
+ Model on [Huggingface](https://huggingface.co/cortex359/germeval2025)
54
+
55
+ ## Model Details
56
+
57
+ - **Model Type:** Transformer-based encoder (XLM-RoBERTa-Large)
58
+ - **Developed by:** Christian Rene Thelen, Patrick Gustav Blaneck, Tobias Bornheim, Niklas Grieger, Stephan Bialonski (FH Aachen, RWTH Aachen, ORDIX AG, Utrecht University)
59
+ - **Paper:** [AIxcellent Vibes at GermEval 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training](https://arxiv.org/abs/2509.07459v2)
60
+ - **Base Model:** [XLM-RoBERTa-Large](https://huggingface.co/FacebookAI/xlm-roberta-large) (Conneau et al., 2020)
61
+ - **Fine-tuning Objective:** Detection of *candy speech* (positive/supportive language) in German YouTube comments.
62
+
63
+ ## Model Description
64
+
65
+ This model is a fine-tuned **XLM-RoBERTa-Large** adapted for the **GermEval 2025 Shared Task on Candy Speech Detection**.
66
+ It was trained to identify *candy speech* at both:
67
+
68
+ - **Binary level:** Classify whether a comment contains candy speech.
69
+ - **Span level:** Detect the exact spans and categories of candy speech within comments, using a BIO tagging scheme across **10 categories** (positive feedback, compliment, affection declaration, encouragement, gratitude, agreement, ambiguous, implicit, group membership, sympathy).
70
+
71
+ The span-level model also proved effective for binary detection by classifying a comment as candy speech if at least one positive span was detected.
72
+
73
+ ## Intended Uses
74
+
75
+ - **Research:** Analysis of positive/supportive communication in German social media.
76
+ - **Applications:** Social media analytics, conversational AI safety (mitigating sycophancy), computational social science.
77
+ - **Not for:** Deployments without fairness/robustness testing on out-of-domain data.
78
+
79
+ ## Performance
80
+
81
+ - **Dataset:** 46k German YouTube comments, annotated with candy speech spans.
82
+ - **Training Data Split:** 37,057 comments (train), 9,229 (test).
83
+ - **Shared Task Results:**
84
+
85
+ - **Subtask 1 (binary detection):** Positive F1 = **0.891** (ranked 1st)
86
+ - **Subtask 2 (span detection):** Strict F1 = **0.631** (ranked 1st)
87
+
88
+ ## Training Procedure
89
+
90
+ - **Architecture:** XLM-RoBERTa-Large + linear classification layer (BIO tagging, 21 labels including “O”).
91
+ - **Optimizer:** AdamW
92
+ - **Learning Rate:** Peak 2e-5 with linear decay and warmup (500 steps).
93
+ - **Epochs:** 20 (with early stopping).
94
+ - **Batch Size:** 32
95
+ - **Regularization:** Dropout (0.1), weight decay (0.01), gradient clipping (L2 norm 1.0).
96
+ - **Postprocessing:** BIO tag correction and subword alignment.
97
+
98
+ ## Limitations
99
+
100
+ - **Domain Specificity:** Trained only on German YouTube comments; performance may degrade on other platforms, genres, or languages.
101
+ - **Overlapping Spans:** Cannot handle overlapping spans, as they were rare (<2%) in the training data.
102
+ - **Biases:** May reflect biases present in the dataset (e.g., demographic skews in YouTube communities).
103
+ - **Generalization:** Needs evaluation before deployment in real-world moderation systems.
104
 
105
+ ## Ethical Considerations
106
 
107
+ - **Positive speech detection** is less studied than toxic speech, but automatic labeling of “supportiveness” may reinforce cultural biases about what counts as “positive.”
108
+ - Must be complemented with **human-in-the-loop moderation** to avoid misuse.
109
 
110
+ ## Citation
111
 
112
+ If you use this model, please cite:
113
 
114
+ ```
115
+ @inproceedings{thelen-etal-2025-aixcellent,
116
+ title = "{AI}xcellent Vibes at {G}erm{E}val 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training",
117
+ author = "Thelen, Christian Rene and
118
+ Blaneck, Patrick Gustav and
119
+ Bornheim, Tobias and
120
+ Grieger, Niklas and
121
+ Bialonski, Stephan",
122
+ editor = "Wartena, Christian and
123
+ Heid, Ulrich",
124
+ booktitle = "Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops",
125
+ month = sep,
126
+ year = "2025",
127
+ address = "Hannover, Germany",
128
+ publisher = "HsH Applied Academics",
129
+ url = "https://aclanthology.org/2025.konvens-2.33/",
130
+ pages = "398--403"
131
+ }
132
+ ```