CEBRA + Self-attention

Attention-Base Architecture

Relu

Fully - connected

flight labes

imput embedding

embedding

Contrastive learning

times labes

Nonlinear encoder

linear

N

X

...

outputs classes

Final layeroutput

Attention Block

CEBRA Architecture（a）

Multi-head attention(b)

feed forward(c)