Update README.md

README.md

@@ -2,35 +2,121 @@
 base_model:
 - Qwen/Qwen2.5-32B-Instruct
 - karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+- NovaSky-AI/Sky-T1-32B-Flash
+- FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview
+- cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
+- TeamDelta/ABEJA-Qwen2.5-32B-base-jp-v0.1
 library_name: transformers
 tags:
 - mergekit
 - merge
-
+license: apache-2.0
+language:
+- en
+- ja
+pipeline_tag: text-generation
 ---
-# SKYDRIVE-32B-v0.1
+## 概要
+このモデルはQwQの長文生成能力とR1の性能を合わせたモデルを作ることを目標にMergekitとFTを用いて製作しました。
+
+
+## How to use
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "DataPilot/SKYDRIVE-32B-v0.1"
 
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+tokenizer_name = ""
 
-## Merge Details
-### Merge Method
+if tokenizer_name == "":
+    tokenizer_name = model_name
 
-This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Qwen/Qwen2.5-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct) as a base.
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)
 
-### Models Merged
+prompt = "メタデータを解析し、自己進化をするAIであるnurture intelligenceが実現した未来の日常生活の姿を教えてください。"
+messages = [
+    {"role": "system", "content": "あなたは優秀な日本語アシスタントであり長考モデルです。問題解決をするための思考をした上で回答を行ってください。"},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
 
-The following models were included in the merge:
-* SKYDRIVE_element_jp_02
-* SKYDRIVE_element_jp_03
-* [karakuri-ai/karakuri-lm-32b-thinking-2501-exp](https://huggingface.co/karakuri-ai/karakuri-lm-32b-thinking-2501-exp)
-* SKYCAVE_element_Sky_jp
-* SKYDRIVE_element_jp_04
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=4096
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
 
-### Configuration
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+
+print(response)
+
+```
 
-The following YAML configuration was used to produce this model:
+## 謝辞
+このモデルの作成者皆様と、計算資源を貸していただいたVOLTMINDに感謝します。 問題解決に協力してくださったhayashiさんにも感謝申し上げます。
+
+## Mergekit config
 
 ```yaml
+merge_method: slerp
+base_model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+models:
+  - model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+  - model: NovaSky-AI/Sky-T1-32B-Flash
+parameters:
+  t: 0.4
+dtype: bfloat16
+name: SKYCAVE_element_Sky_jp
+---
+merge_method: breadcrumbs_ties
+base_model: Qwen/Qwen2.5-32B
+tokenizer_source: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+name: SKYDRIVE_element_jp_01
+models:
+  - model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+    parameters:
+      weight: 1.0
+  - model: FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview
+    parameters:
+      weight: 0.75
+dtype: bfloat16
+---
+merge_method: task_arithmetic
+base_model: Qwen/Qwen2.5-32B
+tokenizer_source: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+name: SKYDRIVE_element_jp_02
+models:
+  - model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+    parameters:
+      weight: 1.0
+  - model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
+    parameters:
+      weight: 0.9
+dtype: bfloat16
+---
+merge_method: slerp
+base_model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+models:
+  - model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
+  - model: TeamDelta/ABEJA-Qwen2.5-32B-base-jp-v0.1
+parameters:
+  t: 0.5
+dtype: bfloat16
+name: SKYDRIVE_element_jp_03
+
+---
 merge_method: model_stock
 
 base_model: Qwen/Qwen2.5-32B-Instruct
@@ -38,9 +124,9 @@ base_model: Qwen/Qwen2.5-32B-Instruct
 models:
   - model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
   - model: SKYCAVE_element_Sky_jp
+  - model: SKYDRIVE_element_jp_01
   - model: SKYDRIVE_element_jp_02
   - model: SKYDRIVE_element_jp_03
-  - model: SKYDRIVE_element_jp_04
   
 dtype: bfloat16
 
@@ -48,4 +134,4 @@ pad_to_multiple_of: 512
 tokenizer_source: base
 
 name: SKYDRIVE-32B-v0.1
-```
+```