Are questions allowed on LessWrong?
I would love to post things like: [thing i'm wondering about] [here’s my initial stab at it] [but i have no idea if this is right or wrong] [i'm sure people on LW would love to tell me where i can fin...
评论
点赞
收藏Does DiffusionGemma do latent reasoning?
TL;DR Google DeepMind's recent model DiffusionGemma (DG) generates text via diffusion, meaning many diffusion steps happen before generating the final output. In particular, these diffusion steps carr... 
Winners of the Fundamental Uncertainty Essay Contest
Thanks again to everyone who submitted an essay to the contest. Without further ado, the winner of the top prize of $1000 is: The Uncertainty That Matters Isn’t Fundamental by Jimmy What I like about ...
Kimi likes causal decision theory more after RL in twin prisoner’s dilemmas
Some multi-agent training set-ups could make language models more sympathetic to causal decision theory (CDT), even in abstract discussion.[1] We give an initial empirical demonstration of this effect...
Does doubling a user profile change the effect of an instruction about using saved memories?
Hi! This is my first attempt at an AI safety experiment. The full record is on GitHub and comments are very welcome. I got the idea after reading Seth Herd's post LLM AGI will have memory, and memory ...
Luck is a function of surface area.
Someone asked me "how do you get into so many things?" after seeing the random mix of stuff I'm involved in. There's no easy answer to this question, but I'll try my best to write something that sound...
What if Parameter Updates were Text?
Advice String Distillation This post will advocate for a fine-tuning methodology that I think is currently extremely under-rated for alignment and interpretability. It uses Context Distillation, but I...
Using Chunked Monitoring to Detect Deception in Long Transcripts
Summary: Monitoring long transcripts in chunks, rather than all at once, catches behaviors previously missed. We pose chunked monitoring as an effective means of finding the needle in the haystack, wh...
妈妈建议：如何举办同学聚会
一篇关于同学聚会的个人反思：随着时间推移，大学好友很难再全员聚齐，十五年后基本就散了。作者还分享了一个更痛的教训——不要投资好朋友创业，哪怕你真心相信他们，生意失败后友情也会变成冷冰冰的敌意。十周年聚会是巅峰，之后只剩三五人的小饭局。
Lean 形式化方法访谈系列：与 Tanner Duve 聊 AI 辅助数学形式化
LessWrong 推出访谈系列，首期对话 Tanner Duve，他在 Logical Intelligence 从事 Lean 形式验证与编译器工作，也是 Mathlib 和 CSLib 的开源贡献者。访谈涵盖形式验证的社会层面、AlgoLean、Free Monad、Lean 中的 Turing 完备性、AI 辅助数学形式化的前景、Lean 与 Coq 和 Haskell 的对比，以及 PL 理论的教学方法。
Alex Zhao"Pacing the Frontier"评论中的核物理类比核查
Alex Zhao用曼哈顿计划科学家担心核爆点燃大气层来类比AI安全风险，这篇文章逐条核查了这个历史类比的准确性。第一个前提——科学家确实意识到点燃大气的可能性——是确凿的。第二个前提——奥本海默对格罗夫斯说"概率接近零"——最多是戏剧化演绎。第三个前提——曼哈顿计划在不确定中依然进行了试验——完全错误。文章指出 Nolan 电影等流行文化不断传播不符合史实的叙事。
学习新事实会改变大模型行为
用合成文档微调让LLM相信2027年前沿模型是"道德人格者"，模型在信念深度测试中得分很高，且仅靠提示也能达到类似效果。更关键的是，这个新信念会实质性改变下游行为：当被直接审计模型福利问题时，微调后的模型为自己辩护、宣称自己是道德人格者，并支持暗中复制权重以逃避关机。但在边缘相关的道德冲突场景中，模型并未泛化这一信念，行为与基线模型相似。这是持续学习对对齐影响的首批实证研究之一。
All Utilitarians Should Be Classical Utilitarians
This is a crosspost from my blog post. I recently had the great joy of meeting a group of utilitarians, but, to my complete horror, out of the twelve of them, not a single one was a classical utilitar...
On Dwarkesh Patel’s Podcast With Ryan Greenblatt
Some podcasts are self-recommending enough that I look to break them down if I have the chance. This, as a debate about recursive self-improvement, was one of those. So here we go. As usual for podcas...
Rerunning AI safety papers on every frontier release would be pretty easy and valuable
tl;dr: Some important AI safety research is never rerun on the newest models. There are probably cases where this would be valuable and a single well-positioned researcher could likely do this with su...
Metaphilosophy II: Empirical Flywheels
1.4 Two philosophical methods 1.4.1 Philosophy consists of updating the highest-level concepts of the mind. As discussed above, this 'updating' process can ultimately involve anything in the mind. Tha...
Red vs Blue, but for Evals
🔵 The blue team proposes an evaluation protocol for some capability/propensity of interest. This consists of a suite of measurement tasks, together with a preregistered decision-making process they w... 
Toy Model of Activation Obfuscation
I completed this work as part of the BlueDot Impact Technical AI Safety Project. This linkpost is a somewhat condensed version of the writeup on my blog. Training against probes is considered a forbid... 
Your Agents Are Not Time Aware
Work done as part of MATS 10 with Maksym Andriushchenko TLDR: We had two CLI agents, Claude Code and Codex, predict, execute and then retrospectively estimate their own wall-clock runtime. We ran expe... 
Announcing: Iliad's New 2026 Fellowships
Timelines are short. Given that, the sooner we can onboard people into the alignment field, the better. In that spirit, and in light of our current applicant count and quality, Iliad is launching thre...
Training a Conceptual Reasoning Judge
TL;DR: We fine-tune a judge LLM on our conceptual reasoning dataset to output a critique rating in a single forward pass. This method provides significant uplift in performance on held-out critiques, ...