Improving Neural Decoding With Unlabelled Data
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 5 concepts
Key Takeaways
- MOJO uses a dual-pathway architecture to train on both labelled behavioral data and unlabelled neural activity.
- The model consistently outperforms purely supervised methods across varied tasks including monkey reaching, mouse decision making, and human speech decoding.
- A shared backbone and cross-attention modules ensure parameter efficiency across the supervised and self-supervised paths.
- The model requires sufficient data to avoid performance degradation in the supervised pathway due to the self-supervised objective.
Summary & Methodology Analysis
MOJO implements a dual-pathway architecture to address the limitation of purely supervised decoding models that ignore large pools of unlabelled neural data. The model tokenizes spikes as tuples of neural unit IDs and timestamps, processing them through a POSSM-style encoder. This encoder calculates cross-attention, a mechanism that helps the model weigh the importance of different inputs relative to each other, separately for contiguous time chunks. By sharing backbone parameters and output cross-attention modules between the supervised and self-supervised pathways, the architecture remains parameter-efficient.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the core problem MOJO solves?
Current models rely only on supervised learning, which limits training to datasets with paired behavioral labels and leaves large amounts of unlabelled data unused.
Q2. How does MOJO learn from unlabelled data?
It uses a self-supervised learning pathway that applies temporal masking to latent outputs, forcing the model to learn meaningful neural representations.
Q3. Does this model work for different types of tasks?
Yes, it shows consistent improvements across monkey reaching, mouse vision and decision tasks, and human speech decoding.
Q4. What is the specific masking strategy used?
The model uses temporal masking with a probability of 0.5, applying learnable mask tokens to the latent outputs.
Q5. How are the supervised and self-supervised objectives combined?
They are jointly optimized through a weighted sum of their respective losses.
Q6. Are there constraints on how the model handles new data sessions?
Yes, when finetuning to a new session, unit embeddings, which are learned vector representations for each neural unit, must be re-learned from scratch.
Q7. What happens if the dataset size is insufficient?
If there is not enough data to learn proper neural dynamics, the self-supervised objective can actually harm the performance of the supervised pathway.
Q8. What are the limitations regarding spatial inference?
The latent masking is limited to temporal dimensions, meaning the model lacks explicit spatial inference for unseen neurons or brain regions.
Q9. Which baselines were referenced in the study?
The study mentions comparisons or relations to models including POYO, POSSM, POGRU, POMAMBA, NDT-2, NDT-3, NEDS, BERT, and GPT.