群里转来一份文档,《开源流式同声口译(SimulST)模型调研》。写着对比六个模型、给选型建议、附硬件需求表。格式整齐,表格漂亮,看着像那么回事。

我拿官方源逐条查了一遍。查完之后只剩一句话:这份文档里一半的模型,你下不到权重。

我不打算把它整理成文章。整理一份错的调研,就是把自己名字签在别人的幻觉上。我把它当靶子重写了一遍。

先说结论:今天真能本地跑、原生带流式决策、中英双向的流式同传模型,只有一个。其余几个要么是云 API 假扮开源,要么是 ASR 加翻译拼的流水线,要么压根没有权重。

一、先把「开源同传」这个词拆成三档

「开源」这两个字被用得太松了。我把它分成三档,后面的所有判断都挂在这三档上:

第一档,真开源:权重在 Hugging Face 上挂着,.safetensors 或 .gguf 能下,config.json 能看,你自己一张卡就能跑起来,模型内部自带「等还是译」的决策。

第二档,假开源:发布会有,新闻稿有,官网有体验页面,Hugging Face 上没有权重。你能调,但只能在别人的服务器上调。这类东西写进「开源调研」里,是诈骗。

第三档,只有论文:arXiv 上有,GitHub 上可能有训练代码,但没有任何可下载的 checkpoint。这叫学术基线,不叫方案。

判断一个模型属于哪档,三十秒够了:

curl -s "https://huggingface.co/api/models/<org>/<model>" | grep -o '"total":[0-9]*'

返回参数总量 = 第一档。返回空 = 第二档或第三档。就这么简单,不需要读任何评测文章。

下面按这三档重排。

二、Hy-SimulMT:这次腾讯真的把东西放出来了

腾讯混元 10 月 22 日建库,10 月 26 日发布,Apache 2.0。Hugging Face 上六个仓库,其实就是两个版本各三份:base 和 Interleave,每个都有 BF16 主权重、FP8、GGUF。

我拉了 config.json,架构不是估算出来的,是文件里写着的:

项目实测值
架构类HYV3SpeechForCausalLM
层数48(first_k_dense_replace: 1,第 0 层稠密,1~47 层 MoE)
专家128 路由 + 1 共享,top-8 激活
注意力32 头 / 4 KV 头(GQA),head_dim 128
上下文262144(256K)
词表120832
音频编码器mel 128 bins,24 层,16 kHz,max_samples: 480000

最后那个 480000 除以 16000,正好 30 秒。所谓「音频滑动窗口 30 秒」不是推测,是配置项算出来的。

参数量 30.4B,推理时每个 token 激活约 3B。任务是中英双向语音到语音同传,官方口径的 StreamLAAL 约 3 秒。

两个版本别搞混,这是最容易买错的地方:

  • Hy-SimulMT-30B-A3B:translation-only,只出译文文本
  • Hy-SimulMT-30B-A3B-Interleave:出「源文 + 译文」交错的双语文本

模型本身输出的是文本。想要语音、想要音色克隆,官方原话是「Speech output and voice cloning require a configured TTS service」——TTS 你得自己另外挂。这条原稿写对了,是真话:千万别以为下载完就有个会说话的同传。

官方在 RealSI 长音频上给的对比:

系统FLEURSCoVoST2ACL6060RealSI-Sentence
Qwen3.8-LiveTranslate88.482.180.584.4
Hy-SimulMT90.375.581.284.7
Gemini3.5-LiveTranslate92.474.482.682.2
Confucius4 R2T2→T3PO89.476.876.177.7
SeamlessStreaming72.860.949.856.9

注意第一行和第三行。Qwen3.8-LiveTranslate 拿下 CoVoST2 的 82.1,Gemini3.5-LiveTranslate 拿下 FLEURS 的 92.4,两个都是商用 API 路线,权重都拿不到。这张表里排在最前面的三个系统,只有中间那一行是你能下载的。

还要注意 CoVoST2 那一列:Hy-SimulMT 75.5 被 Qwen 的 82.1 压着。它不是全面第一,官方也没这么说。

三、账要算清楚:权重多大,卡得多好

原稿给了一张硬件表,我核了,只有一半对。

精度实际占用说明
BF16约 57 GiB(官方模型卡原文)只有 80G 卡有呼吸空间
FP8约 30.4 GB 权重(HF API 参数总量 30.4B)需要 Ada 及以后的卡
GGUF Q8_033.3 GB(文件实际字节数)—
GGUF Q4_K_M19.7 GB(文件实际字节数)24G 卡唯一选项

三处要更正:

一,GGUF 只有 Q4_K_M 和 Q8_0 两个档,没有 Q5_K_M。原稿那张表里的「Q5_K_M 21GB」这个文件不存在。

二,「A6000 48G 跑 FP8」这句话是错的。A6000 是 Ampere 架构,没有 FP8 tensor core,FP8 权重它只能当 FP16 用,显存和带宽一起翻倍。要跑 FP8,卡必须是 Ada 及以后(RTX 6000 Ada、L40S、H100 这一代)。

三,也是最要命的一条——GGUF 这条路现在是断的。官方在模型卡里写明:他们给 llama.cpp 提了 PR #31198,加 Hy-SimulMT 专属架构支持,这个 PR 还没合并,而且目前只支持 llama-mtmd-cli,llama-server 的支持是「计划中」。

翻译成人话:你想用 24G 消费卡跑 Q4_K_M,得自己打一个没进主线的 PR,然后你还没有 server 模式可用。这不是「性能差一点」,这是「路没通」。

四、点名三个被当成「开源同传」在传的东西

1. Qwen3.8-LiveTranslate:能力最强,但它不是开源的

阿里 9 月 19 日发布,Interleave 架构重构,LAAL 从 2.8 秒压到 2.3 秒,60 种语言,Thinker–Talker 双模块,实时说话人分离,原文译文同帧同出。这一串能力是真的,官方发布会和多家媒体报道都在。

但权重不在。官方公告只给了体验页面和模型 API(qwen3.8-livetranslate-flash-realtime),Hugging Face 上没有对应 checkpoint——同期的 Qwen3.8-27B 文字模型是 Apache 2.0 开源的,这条线断了。

所以它属于第二档。它的参数量同样没公布,所谓「3.8B」「FP8 5GB」「24G 卡跑多路」全是编的——它是云服务,你的音频要出你的机器,按 token 付费。

能力配得上第一名。但把它写进「开源本地部署」的选型表,等于把试驾写成了买车。

2. Confucius4:名字对不上,架构也对不上

网易有道 9 月开源的确实有两个好东西,名字是 Confucius4-R2T2 和 Confucius4-T3PO:

  • R2T2:真流式 ASR,80 毫秒到 2 秒可配分块,append-only——已确认的字永不回头改。这个设计在实时字幕里价值很高。
  • T3PO:14B 参数的文本到文本同传(SiMT),每个细粒度 chunk 进来动态决定 READ 还是 WRITE,用 Pareto 感知的强化学习调质量与延迟。

问题出在这里,官方 README 原话:「This model is a T2T (text-to-text) simultaneous translation model and does not natively support speech input.」——它不吃音频。

所以 Confucius4 是一套流水线:R2T2 转写 + T3PO 翻译。串起来能用,你要自己承担两段延迟叠加、错误逐级传播、以及两套服务各自的显存。它不是端到端 SimulST。

原稿说的「Confucius-SimulST 1.8B、可调 wait-k、原生双语输出」查无此物。有意思的是,这份文档第五节正好把「ASR+LLM 流水线」列为第一号要避的坑——它自己推荐的模型,就是那个坑。

3. Hibiki-Zero:真开源,但没有中文

Kyutai 今年 2 月发布的 3B 模型,法、西、葡、德→英,带音色迁移,8GB 显存能跑、12GB 稳。代码 MIT,权重许可没标注,商用前自己确认。它的卖点是不需要词级对齐数据就能扩语种。

但它没有中文,也没有社区成熟的中英权重。官方说「新输入语言用不到 1000 小时语音就能适配」——那是适配的成本,不是上会的质量。拿 1000 小时数据调一个能做会议同传的中英 Hibiki,是研究项目,不是下周要上线的功能。

另外补一句:Hibiki-M 1.7B 这个型号不存在。端侧那个叫 hibiki-1b,1B;主力是 hibiki-2b,2.7B;新的是 hibiki-zero-3b。

五、剩下三个别当方案用

SASST 是 AAAI 2026 的框架(语法感知分块 + 冻结 Whisper 编码器 + 挂一个 decoder-only LLM),不是成品模型。你要拿自己的语料训。原稿说它「训练 <WAIT> 决策 token」也不准——它的机制是语法感知分块,WAIT 那套是 StreamSpeech、Hikari 那条线的东西。

SimulSpeech 是 2020 年的论文,MuST-C 英西、英德,wait-k 加 CTC 分段加两层知识蒸馏。思路经典,但没有任何开源权重。原稿的「300M / 1.2B 双版本、MIT 协议、1.2B FP16 占 2.4GB」全是自造。它不是方案,是引用文献。

SeamlessM4T-v2 这点原稿判对了:它是离线模型,上层硬切流式就是伪同传。补一句完整的——Meta 其实有流式版,叫 SeamlessStreaming,是另一个 checkpoint。但它在上面那张表里是垫底的:RealSI-Sentence 56.9,CoVoST2 60.9。流式不等于能用,这是两件事。

六、横向对比表(只列查证过的)

模型参数中英权重可下原生流式决策许可证实际最低硬件
Hy-SimulMT-30B-A3B30.4B / 激活 3B双向是是Apache 2.080G(BF16)/ 48G Ada(FP8)
Hy-SimulMT-Interleave30.4B / 激活 3B双向是是Apache 2.0同上,KV 更吃紧
Qwen3.8-LiveTranslate未公布60 语言否(API)是闭源服务无(云端)
Confucius4-T3PO14B多语向是是(文本层)Apache 2.0(代码)需配 R2T2,流水线
Confucius4-R2T2见模型卡多语种是是(ASR 层)Apache 2.0(代码)同上
Hibiki-Zero3B无中文是是代码 MIT / 权重未标8G 可跑,12G 稳
SASST取决于底座需自训框架是(框架)见论文仓库取决于自训
SimulSpeech—英西/英德否是(论文)无权重—
SeamlessStreaming—多语是是见 Meta 许可另查

七、按你的卡和你的语种选

有 80G 或双 48G,中英会议要双语字幕:Hy-SimulMT-Interleave FP8。质量是今天开源里最强的,代价是 token 生成量接近翻倍、KV 缓存更吃紧、并发要保守留。

只要译文,接自己的 TTS:Hy-SimulMT-30B-A3B base 版。别用 Interleave 再自己拆双语,那是白烧一倍 token。

只有 24G 消费卡,中英:说句实话,没有舒服的选项。Hy-SimulMT Q4_K_M 是 19.7GB,权重塞得下,但 KV 缓存加音频编码器一压就爆,而且 GGUF 那条路卡在未合并的 PR 上。剩下的替代只有 Confucius4 那条流水线(R2T2 转写 + 14B T3PO 翻译)。两套服务的显存叠加能不能塞进一张 24G 卡,我没有实测过,所以不给你结论——先拿自己的音频长度把 KV 算一遍再说。谁要是拍脑袋告诉你「24G 跑得动」,他大概率没跑过。

欧洲语言:Hibiki-Zero,8GB 起,别想中文。

不想运维、能接受音频出机器:Qwen3.8-LiveTranslate API。它确实是今天能力最强的一档,LAAL 2.3 秒,60 语言,说话人分离都齐。花钱买省心,这个账很正当——只要你别管它叫开源。

八、四个会让你翻车的细节

一、流式同传会回头改译文。 模型在后续音频到达时会修正前面说过的话。所以你的字幕系统必须支持原地更新某一行,不能只做 append。做 append 的系统,屏幕上会留一串废弃的旧译文。

二、Hugging Face 的 language 标签别信。 Hy-SimulMT 的 frontmatter 挂了 33 种语言,正文只声明中英双向。那 33 个标签是从混元 MT 系列继承来的元数据。想知道真实语种,读正文,不读 frontmatter。

三、官方口径的「StreamLAAL 3 秒」不是首字延迟。 官方自己在图注里加了星号:「StreamLAAL is not the same as the time to first speech output.」两回事。拿它当「开口延迟」做产品承诺,会被实测打脸。

四、GGUF 支持是半成品。 PR 未合并、只有 CLI、server 模式计划中。任何把它写成「24G 卡开箱即用」的文章,写的人自己没跑过。

九、三十秒自己核一遍

不用信我,也不用信那份流传的文档:

# 1. 权重到底在不在
curl -s "https://huggingface.co/api/models/tencent/Hy-SimulMT-30B-A3B" | grep -o '"total":[0-9]*'

# 2. 架构是不是真的
curl -s "https://huggingface.co/tencent/Hy-SimulMT-30B-A3B/raw/main/config.json" \
  | grep -E 'num_hidden_layers|num_experts|max_position|vocab_size'

# 3. GGUF 到底有几个档,多大
curl -s "https://huggingface.co/api/models/tencent/Hy-SimulMT-30B-A3B-GGUF/tree/main"

三条命令,比读十篇「重磅发布」快。

真正该问的问题从来不是「哪个模型最强」。是哪个模型你真能装在自己的机器上,跑起来,并且下个月不会因为一个没合并的 PR 停摆。

评测榜第一名是别人的孩子。config.json 才是你的成绩单。


版权没有,随意转载。过意不去就加一句「原文发表于 cn-res.vip」,不加也无所谓。


English

A document made the rounds in a group chat: "A Survey of Open-Source Streaming Simultaneous Interpretation (SimulST) Models." Six models compared, a deployment recommendation, a VRAM table. Neat formatting, handsome tables, looked legitimate.

I pulled the official sources and checked every line. What survived was one sentence: half the models in that document have no downloadable weights.

I'm not going to tidy it up and publish it. Tidying a wrong survey means signing my name to someone else's hallucination. So I rewrote it from the other direction — as a correction.

Conclusion first: today there is exactly one model that runs locally, does native streaming policy decisions, and handles Chinese–English both ways. The rest are either cloud APIs wearing an open-source costume, ASR-plus-translation pipelines stitched together, or papers with no checkpoint behind them.

1. Split "open source" into three tiers

The phrase is used far too loosely. I'll define three tiers here, and every judgment below hangs on them:

Tier one, actually open. Weights on Hugging Face, .safetensors or .gguf downloadable, config.json readable, runs on a card you own, and the wait-or-translate decision lives inside the model.

Tier two, open in name only. There was a launch event, there were press releases, there's a demo page. There are no weights on Hugging Face. You can call it, on someone else's servers. Listing this in a survey of open-source models is fraud.

Tier three, paper only. On arXiv, maybe with training code on GitHub, but no downloadable checkpoint anywhere. That's an academic baseline, not a deployment option.

Thirty seconds tells you which tier a model belongs to:

curl -s "https://huggingface.co/api/models/<org>/<model>" | grep -o '"total":[0-9]*'

A parameter count means tier one. Nothing means tier two or three. No benchmark article required.

2. Hy-SimulMT: this time Tencent actually shipped it

Tencent Hunyuan created the repo on October 22, released on October 26, Apache 2.0. Six variants on Hugging Face: a base version and an Interleave version, each in BF16, FP8 and GGUF.

I pulled config.json. The architecture isn't estimated — it's written in the file:

ItemVerified value
Architecture classHYV3SpeechForCausalLM
Layers48 (first_k_dense_replace: 1)
Experts128 routed + 1 shared, top-8 active
Attention32 heads / 4 KV heads (GQA), head_dim 128
Context262144 (256K)
Vocab120832
Audio encodermel 128 bins, 24 layers, 16 kHz, max_samples: 480000

That last number divided by 16000 is exactly 30 seconds. The "30-second audio window" isn't a guess from a blog post; it's a config field.

30.4B parameters, roughly 3B active per token. Bidirectional Chinese–English speech-to-speech, official StreamLAAL around 3 seconds.

Do not confuse the two variants. This is where people buy the wrong thing:

  • Hy-SimulMT-30B-A3B — translation-only output
  • Hy-SimulMT-30B-A3B-Interleave — interleaved source text plus translation

The model outputs text. For speech and voice cloning, the official wording is "Speech output and voice cloning require a configured TTS service." You bolt on the TTS yourself. The original survey got this one right: do not assume the download turns into a talking interpreter.

The official comparison on RealSI long-form audio:

SystemFLEURSCoVoST2ACL6060RealSI-Sentence
Qwen3.8-LiveTranslate88.482.180.584.4
Hy-SimulMT90.375.581.284.7
Gemini3.5-LiveTranslate92.474.482.682.2
Confucius4 R2T2→T3PO89.476.876.177.7
SeamlessStreaming72.860.949.856.9

Note the first and third rows. Qwen3.8-LiveTranslate takes CoVoST2 at 82.1, Gemini3.5-LiveTranslate takes FLEURS at 92.4, and both are commercial API routes with no downloadable weights. Of the three systems at the top of this table, only the middle row is something you can download.

Also look at CoVoST2: Hy-SimulMT's 75.5 sits under Qwen's 82.1. It is not first across the board, and Tencent does not claim it is.

3. Do the math on weights and cards

PrecisionActual footprintNote
BF16~57 GiB (official model card)Only an 80G card breathes
FP8~30.4 GB of weights (30.4B params per HF API)Requires Ada or newer
GGUF Q8_033.3 GB (actual file size)—
GGUF Q4_K_M19.7 GB (actual file size)The only 24G option

Three corrections to the circulating table. First, GGUF ships in exactly two sizes, Q4_K_M and Q8_0. There is no Q5_K_M file. Second, an A6000 cannot run FP8 — Ampere has no FP8 tensor cores, so it runs the weights as FP16 and doubles both memory and bandwidth. FP8 needs Ada or later. Third, and worst: the GGUF path is currently broken. Tencent submitted llama.cpp PR #31198 for architecture support, it is not merged, and it only covers llama-mtmd-cli. Server support is "planned." That is not "slower." That is "no road."

4. Three things sold as open-source simultaneous interpretation

Qwen3.8-LiveTranslate is the strongest system here and it is not open source. Released September 19, LAAL down from 2.8s to 2.3s, 60 languages, Thinker–Talker split, real-time speaker diarization, source and translation on the same frame. All of that is real. The weights are not: the announcement offered a demo page and a model API endpoint. The Qwen3.8-27B text model shipped Apache 2.0 in the same period; this line did not. Hence: tier two. Its parameter count is unpublished, so every "3.8B, 5GB FP8, multiple streams on a 24G card" claim is invented. It's a cloud service. Your audio leaves your machine; you pay per token.

Confucius4 has the wrong name and the wrong architecture in that survey. NetEase Youdao open-sourced two genuinely good pieces in September: Confucius4-R2T2, a true streaming ASR with 80ms–2s configurable chunks and append-only output that never rewrites committed text, and Confucius4-T3PO, a 14B text-to-text simultaneous translation model making dynamic READ/WRITE decisions per chunk, tuned with Pareto-aware RL. The catch is in their own README: "does not natively support speech input." So Confucius4 is a pipeline, not an end-to-end SimulST model. You inherit two stacked latencies, error propagation, and two sets of VRAM. The irony: the survey's own pitfall section names "ASR plus LLM pipeline" as the number one trap to avoid, and then recommends one.

Hibiki-Zero is real and has no Chinese. Kyutai shipped this 3B model in February 2026: French, Spanish, Portuguese and German into English, with voice transfer, running in 8GB (12GB comfortable). Code is MIT; the weights carry no stated license, so verify before commercial use. Its selling point is scaling to new languages without word-level alignment data. For Chinese you would need to adapt it yourself — under 1000 hours of speech is the official figure, but that's the cost of adaptation, not the quality of a conference-grade interpreter. Also: there is no Hibiki-M 1.7B. The on-device model is hibiki-1b at 1B; the mainline is hibiki-2b at 2.7B; the new one is hibiki-zero-3b.

5. Three more you should not treat as products

SASST is an AAAI 2026 framework — syntax-aware chunking, frozen Whisper encoder, a decoder-only LLM bolted on. You train it on your own corpus. Calling it a model that trains <WAIT> decision tokens is also off; its mechanism is syntax-aware chunking. The WAIT-token lineage is StreamSpeech and Hikari.

SimulSpeech is a 2020 paper on MuST-C English–Spanish and English–German, using wait-k with CTC segmentation and two levels of knowledge distillation. Good ideas, zero weights. Every "300M / 1.2B dual release, MIT license" detail is fabricated. It's a citation, not an option.

SeamlessM4T-v2 — the survey got this right. It's an offline model; slicing it into a stream on top is fake simultaneity. The full picture: Meta does have a streaming variant called SeamlessStreaming, a separate checkpoint, and it finishes last on the official table with 56.9 on RealSI-Sentence. Streaming is not the same as usable.

6. The table, verified entries only

ModelParamsZh-EnWeightsNative policyLicenseReal minimum hardware
Hy-SimulMT-30B-A3B30.4B / 3B activeBoth waysYesYesApache 2.080G (BF16) / 48G Ada (FP8)
Hy-SimulMT-Interleave30.4B / 3B activeBoth waysYesYesApache 2.0Same, tighter KV
Qwen3.8-LiveTranslateUnpublished60 languagesNo (API)YesClosed serviceNone (cloud)
Confucius4-T3PO14BMultipleYesYes (text layer)Apache 2.0 (code)Needs R2T2, pipeline
Confucius4-R2T2See model cardMultipleYesYes (ASR layer)Apache 2.0 (code)Same as above
Hibiki-Zero3BNo ChineseYesYesCode MIT / weights unstated8G works, 12G safe
SASSTDepends on base modelNeeds trainingFrameworkYes (framework)See paper repoDepends on your training
SimulSpeech—En-Es / En-DeNoYes (paper)No weights—
SeamlessStreaming—MultipleYesYesSee Meta licenseCheck separately

7. What to pick

80G or dual 48G, Chinese–English with bilingual subtitles: Hy-SimulMT-Interleave in FP8. Best open quality today; expect roughly double the token generation, tighter KV cache, conservative concurrency.

Translation only, your own TTS: the base variant. Don't run Interleave and split the languages yourself — that's burning double tokens for nothing.

24G consumer card, Chinese–English: honestly, there is no comfortable answer. Q4_K_M is 19.7GB so the weights fit, but KV cache plus the audio encoder is a squeeze, and the GGUF path is stuck behind an unmerged PR. The only remaining route is the Confucius4 pipeline — R2T2 transcription plus the 14B T3PO for translation. Whether both sets of VRAM fit on one 24G card is something I have not measured, so I won't assert it: do the KV math against your own audio lengths first. Anyone who tells you "it runs on 24G" off the top of their head almost certainly never ran it.

European languages: Hibiki-Zero, 8GB and up, forget Chinese.

No operations, audio may leave the machine: the Qwen3.8-LiveTranslate API. It is the strongest system available today and it is priced as a service. Fine trade — just stop calling it open source.

8. Four details that will bite you

Streaming translation revises itself. Late audio changes earlier output. Your subtitle layer must update a line in place; append-only systems accumulate discarded translations on screen.

Ignore the language tags on Hugging Face. Hy-SimulMT's frontmatter lists 33 languages while the body declares Chinese–English only. Those tags are inherited metadata from the Hunyuan MT line. Read the prose, not the frontmatter.

"StreamLAAL around 3 seconds" is not time-to-first-output. The official figure carries an asterisk: "StreamLAAL is not the same as the time to first speech output." Promise a product on the wrong number and a stopwatch will embarrass you.

GGUF support is half-built. Unmerged PR, CLI only, server mode planned. Anyone who writes "turnkey on a 24G card" has not run it.

9. Verify it yourself in thirty seconds

curl -s "https://huggingface.co/api/models/tencent/Hy-SimulMT-30B-A3B" | grep -o '"total":[0-9]*'
curl -s "https://huggingface.co/tencent/Hy-SimulMT-30B-A3B/raw/main/config.json" \
  | grep -E 'num_hidden_layers|num_experts|max_position|vocab_size'
curl -s "https://huggingface.co/api/models/tencent/Hy-SimulMT-30B-A3B-GGUF/tree/main"

Three commands beat ten "major release" articles.

The question was never which model scores highest. It is which model you can actually install on your own hardware, run, and still have running next month when an unmerged PR is the only thing holding it up.

Benchmark leaderboards are other people's children. config.json is your report card.


No copyright reserved — share it freely. If you feel like it, a credit line to cn-res.vip is appreciated. Or don't. Either way.