Abstract: Multilingual Transformer models offer effective cross-lingual generalization capabilities. However, their architecture suffers from embedding-parameter overhead due to massive vocabulary ...

Heretic is a tool that removes censorship (aka "safety alignment") from transformer-based language models without expensive post-training. It combines an advanced implementation of directional ...

Abstract: This work proposes a novel hybrid deep learning model that incorporates CNNs, Transformer models, and GRU layers, further extended with an MoE model, for advanced sequence analysis. The ...