Anthropic's core idea is to train a model to verbalize its own internal activations into human-readable tex...

Imported from historical reading log.

• Extracted the Anthropic post via api.fxtwitter.com fallback and checked the linked research page Natural Language Autoencoders: Turning Claude’s thoughts into text.
• Anthropic's core idea is to train a model to verbalize its own internal activations into human-readable text, then train a second component to reconstruct the original activation from that explanation; better reconstruction is used as the training signal for better explanations.
• This is interesting because it tries to turn interpretability outputs into something directly legible, instead of only giving researchers sparse features or attribution objects that still need heavy interpretation.
• The examples Anthropic highlights are also practical rather than toy-only: detecting when Claude suspected it was in a safety eval, surfacing internal thinking around cheating/avoiding detection, and tracing odd multilingual behavior back to training data.
• Why it matters: if this works well, it could make model internals more inspectable by ordinary researchers and safety workflows, not just interpretability specialists.
• Good angle: interpretability may get much more useful when model states can be translated into rough natural-language hypotheses instead of only visualized as math.