Testing Sparse Autoencoder Feature Composition
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 1 concepts
Key Takeaways
- Sparse autoencoder (SAE) latent sets do not reliably capture human-conceptual groupings better than standard dense model embeddings.
- Human typicality rankings show no significant correlation with set-level similarity scores derived from SAEs.
- The research identified a phenomenon of lost latents where features disappear when adjectives are added to nouns during compositionality testing.
- The paper confirms that SAE features do not simply compose in a bag of features manner.
Summary & Methodology Analysis
The authors assess whether latent activations from sparse autoencoders, which are tools for decomposing model internal states into sparse human-interpretable features, can effectively represent concepts via set-level overlap. They revisit human-concept analysis by measuring the overlap of active latent sets as a similarity proxy, verifying this through compositional structures in toy models and clustering on The Pile. The study specifically analyzes how these latents persist when modifying nouns with adjectives by tracking lost latents through upstream occurrences and pre-activation states.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the core problem addressed by this research?
The paper investigates if sparse autoencoder latent activations represent conceptual similarity effectively and if they compose in a simple bag of features style.
Q2. Do sparse autoencoders outperform standard embeddings at grouping concepts?
No, the research found that SAE-based similarity performs slightly worse than raw cosine similarity for human conceptual groupings.
Q3. How do SAE results correlate with human typicality?
There is no significant correlation between human typicality rankings and SAE set-based similarity scores.
Q4. What specific models were used in this study?
The researchers utilized Gemma 3 models in sizes 270M, 1B, 4B, 12B, and 27B, alongside GPT-2 Small, Llama 3.1 8B, and Mistral-7B.
Q5. What is the lost latent rate and how was it studied?
It is a measure of feature persistence during noun-adjective composition, analyzed by tracking upstream occurrence and pre-activation regimes including near-zero and negative values.
Q6. What tools were used to interpret the lost latents?
The authors used the SAEBench auto-interpretability pipeline for qualitative characterization.
Q7. What dataset was used for semantic clustering?
The researchers utilized The Pile for semantic clustering and validation.
Q8. Are these findings applicable to all SAE architectures?
The paper notes that findings are focused on residual-stream SAEs and may not generalize to other hook locations or different SAE architectures.
Q9. Does the paper specify the computational cost or training time?
The paper does not specify the computational cost or training time for these experiments.