Expanding Creative Writing Beyond Simple Stories
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Key Takeaways
- Existing creative writing data is heavily story centric, which limits models when handling diverse creative formats.
- The pipeline uses Reddit r/WritingPrompts as a seed dataset, leveraging GPT-5-mini, GPT-5, Qwen3-235B-A22B-Thinking, and Qwen3-30B-A3B-Instruct.
- Models fine tuned on the Multi-Genre Collection achieve an Arena Hard score of 33.0, WritingBench score of 60.6, and Multi-Genre score of 68.4 when using the Llama model.
- Increasing the number of training genres from 0 to all 13 genres raises the novelty score monotonically from 3.87 to 4.81.
Summary & Methodology Analysis
Language models often struggle with creative writing formats outside of simple storytelling because their training data is heavily story centric. This paper introduces an attribute guided genre expansion pipeline to improve how models handle structural, stylistic, functional, and formatting conventions of diverse creative formats. The methodology begins with source dataset selection using Reddit r/WritingPrompts as a seed dataset, followed by safety filtering and irrelevant content removal via GPT-5-mini. Step one involves thematic seed sampling, where n equals 5 query response pairs are randomly sampled for topical and stylistic breadth. Step two focuses on genre attribute sampling, collecting authoritative genre definitions, extracting structured attribute lists using GPT-5, manually refining them to 15 attributes per genre, and randomly sampling a subset to control instruction specificity. Step three applies attribute guided pair synthesis, prompting GPT-5-mini with a meta prompt that combines the target genre, task instructions, sampled genre attributes, and few shot examples with verbalized sampling to produce queries. Responses are generated using Qwen3-235B-A22B-Thinking, and quality filtered using Qwen3-30B-A3B-Instruct as an LLM as a judge. Fine tuning models on the resulting Multi-Genre Collection yields an Arena Hard score of 33.0, a WritingBench score of 60.6, and a Multi-Genre score of 68.4 when using the Llama model. Furthermore, increasing the number of training genres from 0 to all 13 genres raises the novelty score monotonically from 3.87 to 4.81. The paper does not specify any computational limitations or hardware resource requirements.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the main problem addressed by the paper?
Existing creative writing data for language models remains heavily story-centric, limiting models' ability to follow the structural, stylistic, functional, and formatting conventions of diverse creative formats beyond narrative generation.
Q2. What seed dataset is used for the pipeline?
Reddit's r/WritingPrompts is selected as the seed dataset.
Q3. What happens to the novelty score as training genres increase?
Increasing the number of training genres from 0 to all 13 genres raises the novelty score monotonically from 3.87 to 4.81.
Q4. How many thematic seed pairs are sampled in Step 1?
The paper randomly samples n equals 5 query-response pairs.
Q5. How are genre attributes handled in Step 2?
Authoritative genre definitions are collected, structured attribute lists are extracted using GPT-5, and they are manually reviewed and refined to yield 5 to 15 attributes per genre before randomly sampling a subset.
Q6. Which models are used to generate and filter responses during pair synthesis in Step 3?
Responses are generated using Qwen3-235B-A22B-Thinking and quality-filtered using Qwen3-30B-A3B-Instruct as an LLM-as-a-judge.
Q7. What performance scores do models achieve when fine-tuned on the Multi-Genre Collection using the Llama model?
They achieve an Arena Hard score of 33.0, WritingBench score of 60.6, and Multi-Genre score of 68.4.
Q8. Does the paper report any specific limitations?
The paper does not specify any limitations.
Q9. What tools or models are utilized for safety filtering on the seed dataset?
GPT-5-mini is applied for safety filtering and irrelevant content removal.