Tokenizer设计:

网络架构设计:

<aside> 💡

Uni-X 和 UniFork 都是使用VQ-VAE 将图像编码到LLM的embedding空间, 作者均发现不同层的作用有差异, 浅层和中层及深层分别关注不同的特征

HBridge 和 Bagel 基于MoT架构,理解和生成有单独参数,注意力机制共享

</aside>

HBridge: H-Shape Bridging of Heterogeneous Experts for Unified Multimodal Understanding and Generation

UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generation

Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Models

Context Unrolling in Omni Models