TAC2: Timestamped Audio Captioning as composable functions 待解读
新用户看这篇论文该怎么开始
- 先点「赞助解读」，AI 会把论文转成可直接执行的行动清单。
- 看完“可执行改进行动”后，可快速决定是否值得立项。
- 用上面的卡片内容直接发给团队，减少重复阅读。
摘要
Real-world audio applications require precise, queryable representations grounded in physical time. Yet modern Large Audio-Language Models (LALMs) increasingly favor conversational generation, producing free-form descriptions that conflate acoustic perception with higher-level reasoning and lack precise temporal grounding. We introduce TAC2, a foundation model that casts timestamped audio captioning as composable functions to construct granular, speaker-attributed acoustic timelines directly from in-the-wild audio. To avoid prohibitively expensive manual annotation, we programmatically fuse complementary predictions from specialized weak teachers through time-registered IoU matching. We model the resulting multi-granularity schema of timestamped events, speaker turns, and fine-grained speech attributes using an Instruction-Input-Output (IIO) completion format, in which each task is a function of the audio, an instruction, and an optional input. A single call decodes the full description autonomously, while composing calls lets earlier outputs or external priors, such as a reference transcript, condition later ones. We further extend TAC2 with expressive speech attributes to obtain TAC2-S, using parameter-efficient specialists to elicit latent paralinguistic capabilities from the pre-trained backbone and distill them into a unified model. We validate the resulting representation in two complementary settings. Paired with frozen text-only LLMs, it yields strong spoken audio understanding across diverse reasoning backends, demonstrating the utility of decoupling fine-grained acoustic perception from downstream reasoning. Used as paired supervision for text-to-audio generation, it supports semantically and temporally grounded reconstruction, together with explicit zero-shot control over event timing, speaker attributes, and word-level emphasis.
分析报告
暂无报告。点击“分析”开始生成。
个性化解读 与社区共享解读不同
用自己的话告诉 AI 你想要什么样的解读（比如"用大白话讲给非专业人士听"、"重点分析对我们团队 RAG 系统的可迁移性"），生成一份只属于你自己的版本；生成后也可以选择设为"愿意共享"，被更多人看到、点赞。