© 2026 Pattern Engine, Inc.

positional_encoding — Deca

positional_encoding

positional_encoding

Tezos

positional_encoding

Creator

tz1XJfAvEfT2yKWUkqokjXJW8f3aZ7KV7jct

Description

Unlike a recurrent neural network, a Transformer does not have state, and therefore is not sensitive to the passage of time. Information about time (e.g. a token's location in a sequence) is provided explicitly by adding a "positional encoding" vector to the network's input. In "Attention Is All You Need" — the paper that introduced the Transformer — each dimension of the input vector is associated with a clock of a different speed. The cosine and sine of the angle of the clock's hand are added to the input, providing a sense of time without the need for state.

Traits

Tag

Token

Contract: KT1RJ6...xton
Token ID: 546495

View on marketplace

© 2026 Pattern Engine, Inc.