| Quellcodebibliothek Statistik Leitseite products/Sources/formale Sprachen/Java/Threema/domain/protocol/src/ Datei vom 25.3.2026 mit Größe 214 kB |
|
Impressum csp.struct.yml Sprache: unbekannt |
|
|
Spracherkennung für: .yml vermutete Sprache: Unknown {[0] [0] [0]} [Methode: Schwerpunktbildung, einfache Gewichte, sechs Dimensionen] # Meta information
meta: # Document name and ID id: csp name: Chat Server Protocol # References used by the structs references: # A public or secret key key: &key b32 # A random cookie cookie: &cookie b16 # A random nonce nonce: &nonce b24 # A Threema ID identity: &identity b8 # Multiple Threema IDs identities: &identities b8[] # A message ID message-id: &message-id u64-le # Multiple message IDs message-ids: &message-ids u64-le[] # A blob ID blob-id: &blob-id b16 # A poll ID poll-id: &poll-id u64-le # A group ID group-id: &group-id u64-le # Virtual namespace, just containing the below docstring index: &index _doc: |- # Chat Server Protocol The Chat Server Protocol is a custom transport encrypted frame-based protocol, originally designed to operate on top of TCP. it uses the NaCl cryptography library to provide authentication, integrity and encryption. The login [**handshake**](ref:handshake) takes two round trips and establishes ephemeral encryption keys along the way. Authentication is solely based on the secret key associated to a Threema ID. After the handshake process, [**payloads**](ref:payload) can be exchanged bidirectionally although some payload structs may only be used in one direction. A client may now send and receive end-to-end encrypted [**messages**](ref:e2e) (wrapped in [message payload structs](ref:payload.container)). ## Terminology - `CK`: Client Key (permanent secret key associated to the Threema ID) - `SK`: Permanent Server Key - `TCK`: Temporary Client Key - `TSK`: Temporary Server Key - `CCK`: Client Connection Cookie - `SCK`: Server Connection Cookie - `CSN`: Client Sequence Number - `SSN`: Server Sequence Number - `ID`: The client's Threema ID ## General Information **Endianness:** All integers use little-endian encoding. **Encryption cipher:** XSalsa20-Poly1305, unless otherwise specified. **Nonce format:** - a 16 byte cookie (CCK/SCK), followed by - a monotonically increasing sequence number (CSN/SSN, u64-le). **Sequence number:** The sequence number starts with `1` and is counted separately for each direction (i.e. there is one sequence number counter for the client and one for the server). We will use `CSN+` and `SSN+` in this document to denote that the counter should be increased **after** the value has been inserted (i.e. semantically equivalent to `x++` in many languages). ## Size Limitations The chat server protocol currently allows for up to 8192 bytes within a single frame. Because we make heavy use of Protobuf messages, the overhead cannot be calculated reliably ahead of time. Therefore, the total amount of user-defined bytes should be constrained to ~7000 bytes. To achieve this, the maximum recommended size of each property will be defined for each message, so that it's total size roughly matches that constraint. # Handshake structs handshake: &handshake _doc: |- ## Handshake To perform authentication handshake, the following handshake structs have to be exchanged in this order: C -- client-hello -> S C <- server-hello -- S C ---- login ---- -> S C <-- login-ack ---- S Note that handshake structs have no wrapping frame container struct. client-hello: _doc: |- Initial message from the client, containing a server authentication challenge in order to establish transport layer encryption. Direction: Client --> Server fields: - _doc: |- 32 byte temporary public key (`TCK.public`). name: tck type: *key - _doc: |- 16 byte random cookie used for nonces (also acting as server authentication challenge). name: cck type: *cookie server-hello: _doc: |- Initial message from the server, containing the server's authentication challenge response. This concludes establishing transport layer encryption based on `TCK` and `TSK`. Direction: Client <-- Server When creating this message: 1. Ensure that CCK and SCK are not equal. When receiving this message: 1. If CCK and SCK are equal, abort the connection and these steps. 2. If the repeated random cookie of the client does not equal CCK, abort the connection and these steps. fields: - _doc: |- 16 byte random cookie used for nonces (also acting as client authentication challenge) name: sck type: *cookie - _doc: |- The server's challenge response (`server-challenge-response`), encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(SK.secret, TCK.public), nonce=SCK || u64-le(SSN+), ) name: server-challenge-response-box type: b64 server-challenge-response: _doc: |- Authentication challenge response from the server. fields: - _doc: |- 32 byte temporary public key (`TSK.public`) name: tsk type: *key - _doc: |- 16 byte repeated random cookie of the client (acting as the server's challenge response) name: cck type: *cookie login: _doc: |- Login request from the client. IMPORTANT: `CSN` is used and increased for `box` and then for `extension-box`. It must follow this exact order. Direction: Client --> Server fields: - _doc: |- The [`login-data`](ref:handshake.login-data), encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(TCK.secret, TSK.public), nonce=CCK || u64-le(CSN+), ) name: box type: b144 - _doc: |- An optional arbitrary amount of [`extension`](ref:handshake.extension)s, encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(TCK.secret, TSK.public), nonce=CCK || u64-le(CSN+), ) These fields are only present if the [`extension-indicator`](ref:handshake.extension-indicator) of the [`login-data`](ref:handshake.login-data) field is present. If so, extensions should be consumed until the extension indicator `length` field is zero. name: extensions-box type: b* login-data: _doc: |- Login data of the client. fields: - _doc: |- Threema ID of the client. name: identity type: *identity - _doc: |- This is either the old client info field or an extension indicator. If the first 30 bytes of the field start with the string `threema-clever-extension-field`, then parse this field as an [`extension-indicator`](ref:handshake.extension-indicator) and parse `extensions-box` appropriately. Otherwise, this represents an old client info and the content is identical to the content of [`client-info`](ref:handshake.client-info). Since the field has a fixed size, the string is zero-padded. name: client-info-or-extension-indicator type: b32 - _doc: |- 16 byte repeated random cookie of the server (acting as the client's challenge response) name: sck type: *cookie - _doc: |- 24 zero bytes (previously used as vouch nonce, now set to zero indicating that the new vouch format is being used) name: reserved1 type: b24 - _doc: |- The vouch value, calculated as follows: SS1 = X25519HSalsa20(CK.secret, SK.public) SS2 = X25519HSalsa20(CK.secret, TSK.public) VouchKey = BLAKE2b(key=SS1 || SS2, salt='v2', personal='3ma-csp') vouch = BLAKE2b( out-length=32, key=VouchKey, input=SCK || TCK.public, ) name: vouch type: b32 - _doc: |- 16 zero bytes (previously part of the vouch box, now set to zero for compatibility) name: reserved2 type: b16 extension-indicator: _doc: |- Indicates that extensions are present fields: - _doc: |- Magic string: `threema-clever-extension-field` name: magic type: b30 - _doc: |- Amount of encrypted bytes present for extensions. Extension fields need to be consumed until `length` is zero. name: length type: u16-le extension: _doc: |- An extension field. fields: - _doc: |- Type of the extension. Must correspond to the encoded extension struct of the `payload` field: - `0x00`: `client-info` - `0x01`: `csp-device-id` - `0x02`: `message-payload-version` - `0x03`: `device-cookie` name: type type: u8 - _doc: |- Length of the extension's `payload` field. name: length type: u16-le - _doc: |- Extension payload. Needs to be parsed according to the `type` field. name: payload type: b{length} client-info: _doc: |- Client info extension payload. fields: - _doc: |- Client info string in the following format (without line breaks): <app-version>; <platform>; <lang>/<country-code>; <rest> The `<rest>` looks like this for mobile clients (A/I/W): <device-model>; <os-version> The `<rest>` looks like this for web/desktop clients (Q): <renderer>; <renderer-version>; <os-name>; <os-architecture> The `<rest>` looks like this for Bots (B): <os-name>; <os-architecture> The fields may contain the following values: - `app-version`: Arbitrary version string, depending on the platform - `platform`: * `A`: Android * `I`: iOS * `Q`: Desktop/Web * `W`: Windows Phone * `B`: Bot - `lang`: ISO 639-1:2002-ish language code - `country-code`: ISO 3166-1-ish country code - `device-model`: Arbitrary smartphone model - `os-version`: Arbitrary OS version string - `renderer`: Renderer name for Desktop/Web (e.g. `Firefox` or `Electron`) - `renderer-version`: Renderer major version (e.g. `107`) - `os-name`: Name of the operating system (e.g. `Linux` or `Windows`) - `os-architecture`: Architecture of the operating system (e.g. `x64`) name: client-info type: b* csp-device-id: _doc: |- CSP device ID extension payload. fields: - _doc: |- CSP device ID, randomly generated **once** when the device got the Mediator device ID. name: csp-device-id type: u64-le message-payload-version: _doc: |- Message payload struct version to be used. In case this extension is not present, the server must assume that version `0x00` has been selected. In case the server receives an unknown or unsupported protocol version, it shall complete the handshake and then immediately send a `close-error` payload. fields: - _doc: |- Indicates the payload struct version the client will send and expects to receive when exchanging message payload structs with the server: - `0x00`: `legacy-message` - `0x01`: `message-with-metadata-box` name: version type: u8 device-cookie: _doc: |- A 16 byte random value chosen by the client and stored in a secure, device-specific location (not included in any backups etc., not viewable/exportable). Its purpose is to allow detection when a different (rogue) device has connected to the chat server, e.g. because an attacker has obtained the secret key of a user. The server will store the device cookie of the last connection, and if a different cookie is sent by the client, it will set a flag on the identity and send a [`device-cookie-change-indication`](ref:payload.device-cookie-change-indication) payload to the client every time it connects. The client should then show a warning in form of a notification or a dialog to the user. Note that the normal protocol flow should continue regardless of whether the user has acknowledged the warning or not. If this extension is not sent by the client, then the server's behavior depends on whether it has already stored a device cookie for this identity or not. If not, then nothing will happen. If yes, then it will act as if the client had sent an all-zero device cookie. fields: - _doc: |- Device cookie, randomly generated **once** per device. name: device-cookie type: b16 login-ack: _doc: |- Login acknowledgement from the server. Direction: Client <-- Server fields: - _doc: |- Reserved (16 zero bytes), encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(TSK.secret, TCK.public), nonce=SCK || u64-le(SSN+), ) name: reserved-box type: b32 # Payload structs payload: &payload _doc: |- ## Payload After the handshake process, payloads may be sent and received without any strictly defined order. Note that payload structs are mandatory to encrypt and frame. To achieve this, first wrap the payload struct in a [`container`](ref:payload.container) struct, encrypt it and wrap the encrypted bytes in a [`frame`](ref:payload.frame) struct. frame: _group: Header _doc: |- Contains an encrypted [payload](ref:payload#payload) wrapped in a [container](ref:payload.container). Direction: Client <-> Server fields: - _doc: |- Length of the `box` field. name: length type: u16-le - _doc: |- The encrypted [payload](ref:payload#payload). For messages from the server to the client, encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(TSK.secret, TCK.public), nonce=SCK || u64-le(SSN+), ) For messages from the client to the server, encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(TSK.secret, TCK.public), nonce=CCK || u64-le(CSN+), ) name: box type: b{length} container: _group: Header _doc: |- Contains an inner [payload](ref:payload#payload) struct. Direction: Client <-> Server fields: - _doc: |- Type of the payload. Must correspond to the encoded payload struct of the `data` field: - `0x00`: [`echo-request`](ref:payload.echo-request) - `0x80`: [`echo-response`](ref:payload.echo-response) - `0x01`: outgoing [`legacy-message`](ref:payload.legacy-message) or [`message-with-metadata-box`](ref:payload.message-with-metadata-box) - `0x81`: [`message-ack`](ref:payload.message-ack) for an outgoing message - `0x02`: incoming [`legacy-message`](ref:payload.legacy-message) or [`message-with-metadata-box`](ref:payload.message-with-metadata-box) - `0x82`: [`message-ack`](ref:payload.message-ack) for an incoming message - `0x03`: [`unblock-incoming-messages`](ref:payload.unblock-incoming-messages) - `0x20`: [`set-push-notification-token`](ref:payload.set-push-notification-token) - `0x21`: (obsolete, formerly used by iOS to set a push filter) - `0x22`: (obsolete, formerly used by iOS to set a push sound for contacts) - `0x23`: (obsolete, formerly used by iOS to set a push sound for groups) - `0x24`: high-priority token for notifications that require immediate delivery (e.g. for calls) using the same struct as [`set-push-notification-token`](ref:payload.set-push-notification-token) - `0x25`: [`delete-push-notification-token`](ref:payload.delete-push-notification- - `0x30`: [`set-connection-idle-timeout`](ref:payload.set-connection-idle-timeout) - `0x31`: (obsolete, formerly used to ensure that a push message is sent for all messages, regardless of the flag) - `0xd0`: [`queue-send-complete`](ref:payload.queue-send-complete) - `0xd1`: (obsolete, formerly used for a function similar to the device cookie) - `0xd2`: [`device-cookie-change-indication`](ref:payload.device-cookie-change-ind - `0xd3`: [`clear-device-cookie-change-indication`](ref:payload.clear-device-cooki - `0xe0`: [`close-error`](ref:payload.close-error) - `0xe1`: [`alert`](ref:payload.alert) name: type type: u8 - _doc: |- Reserved, currently all zeroes. name: reserved type: b3 - _doc: |- Inner payload. Needs to be parsed according to the `type` field. name: data type: b* echo-request: _group: Payloads _doc: |- An echo request to be answered by a corresponding echo response. Can be used for connection keep-alive or RTT estimation. Direction: Client <-> Server [//]: # "TODO(SE-128)" fields: - _doc: |- Data to be echoed back in the echo response. name: data type: b* echo-response: _group: Payloads _doc: |- An echo response corresponding to an echo request. Direction: Client <-> Server [//]: # "TODO(SE-128)" fields: - _doc: |- Data echoed back from the echo request. name: data type: b* legacy-message: _group: Payloads _doc: |- An end-to-end encrypted Threema message. Direction: Client <-> Server Note: This payload is deprecated and may be phased out eventually. It will only be used in case the [`message-payload-version`](ref:handshake.message-payload-version) was not present during login or was explicitly set to the version `0x00`. Conversion to [`message-with-metadata-box`](ref:payload.message-with-metadata-box) - Copy `legacy-message.sender-nickname` to `message-with-metadata-box.legacy-sender-nickname` - Copy all other fields of `legacy-message` to their respective counterparts in `message-with-metadata-box` - Set `message-with-metadata-box.metadata-length` to `0` - Omit `message-with-metadata-box.metadata-container` (i.e. set it to contain 0 bytes) - Copy `legacy-message.message-nonce` to `message-with-metadata-box.message-and-metadata-nonce`. When sending or receiving this payload, convert it to a `message-with-metadata-box` and handle it as defined by that struct. [//]: # "TODO(SE-128)" fields: - &message-sender-identity _doc: |- The sender's Threema ID. name: sender-identity type: *identity - &message-receiver-identity _doc: |- The receiver's Threema ID. name: receiver-identity type: *identity - &message-message-id _doc: |- Unique message ID for each sender/receiver pair. Used for duplicate detection and for quotes. Messages sent in a group must have the same message ID for each group member. name: message-id type: *message-id - &message-created-at _doc: |- Unix timestamp in seconds for when the message has been created. Messages sent in a group must have the same timestamp for each group member. However, the server overrides this timestamp with the current time if - the declared timestamp is in the future, or - the _short-lived server queuing_ flag was set (`0x20`). Note: The original timestamp is still available in an attached `csp-e2e.MessageMetadata`. name: created-at type: u32-le - &message-flags _doc: |- Flags: - `0x01`: Send push notification. The server will send a push message to the receiver of the message. Only use this for messages that require a notification. For example, do not set this for delivery receipts. - `0x02`: No server queuing. Use this for messages that can be discarded by the chat server in case the receiver is not connected to the chat server, e.g. the _typing_ indicator. - `0x04`: No server acknowledgement. Use this for messages where reliable delivery and acknowledgement is not essential, e.g. the _typing_ indicator. Will not be acknowledged by the chat server when sending. No acknowledgement should be sent by the receiver to the chat server. - `0x10`: Reserved (formerly _group message marker_). - `0x20`: Short-lived server queuing. Messages with this flag will only be queued for 60 seconds. - `0x80`: No automatic delivery receipts. A receiver of a message with this flag must not send automatic delivery receipt of type _received_ (`0x01`) or _read_ (`0x02`). This is not used by the apps but can be used by Threema Gateway IDs which do not necessarily want a delivery receipt for a message. name: flags type: u8 - &message-reserved _doc: |- Reserved, must be set to zero. name: reserved type: u8 - _doc: |- Reserved for header compatibility with metadata message. Must be set to zero by legacy clients. name: reserved-metadata-length type: b2 - _doc: |- The sender's public nickname, padded with zeroes if needed. name: sender-nickname type: b32 - _doc: |- Nonce used for the message box. name: message-nonce type: *nonce - &message-message-box _doc: |- The message, end-to-end encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(<sender.CK>.secret, <receiver.CK>.public), nonce=<message-nonce>, ) name: message-box type: b* message-with-metadata-box: _group: Payloads _doc: |- An end-to-end encrypted Threema message with additional end-to-end encrypted metadata. Direction: Client <-> Server Note: This payload will only be used in case the [`message-payload-version`](ref:handshake.message-payload-version) was set to version `0x01`. Conversion to [`legacy-message`](ref:payload.legacy-message): - Discard `message-with-metadata-box.metadata-length` and `message-with-metadata-box.metadata-container` - Copy `message-with-metadata-box.legacy-sender-nickname` to `legacy-message.sender-nickname` - Copy `message-with-metadata-box.message-and-metadata-nonce` to `legacy-message.message-nonce`. - Copy all other fields of `message-with-metadata-box` to their respective counterparts in `legacy-message` Creating this payload is only allowed as part of the _Bundled Messages Send Steps_. When receiving this payload: 1. (MD) If the device is currently not declared _leader_, exceptionally abort these steps and the connection. 2. If the nonce of `message-and-metadata-nonce` has been used before, log a warning, _Acknowledge_ and discard the message and abort these steps. 3. If `receiver-identity` does not equal the user's Threema ID, log a warning, _Acknowledge_ and discard the message and abort these steps. 4. Run the _Valid Contacts Lookup Steps_ for `sender-identity` and let `contact-or-init` be the result. 5. If `contact-or-init` indicates that the _contact is the user_ or that the _contact is invalid_, log a warning, _Acknowledge_ and discard the message and abort these steps. 6. If `metadata-length` is greater zero, decrypt the `metadata-container` and let `outer-metadata` be the result. If this fails, log a warning, _Acknowledge_ and discard the message and abort these steps. 7. Decrypt the `message-box`, decode it to a [`container`](ref:payload.container) struct and let `outer` be the result. If this fails, log a warning, _Acknowledge_ and discard the message and abort these steps. 8. If `outer.type` is `0xff`, log a warning, _Acknowledge_ and discard the message and abort these steps. (Legacy logic, may be removed in the future.) 9. If `outer.type` is unknown, log a notice, _Acknowledge_ and discard the message and abort these steps. 10. Decode `outer.padded-data` into the message type associated to `outer.type` and let `outer-message` be the result. If this fails, log a warning, _Acknowledge_ and discard the message and abort these steps. 11. If `outer.type` is not `0xa0`, let `inner-metadata` be `outer-metadata`, let `inner-type` be `outer.type` and let `inner-message` be `outer-message`. 12. If `outer.type` is `0xa0`: 1. Run the receive steps associated to `csp-e2e-fs.Envelope` with the decoded `outer-message` and let `inner-metadata`, `inner-type`, `inner-message` and `fs-commit-fn` be the result. If this fails, exceptionally abort these steps and the connection. If the message has been discarded, _Acknowledge_ and abort these steps. 2. If `inner-metadata` is not defined, set `inner-metadata` to `outer-metadata`. 13. If `message-id` does not equal `inner-metadata.message_id`, log a warning, _Acknowledge_ and discard the message and abort these steps. 14. If `message-id` refers to a message that has been received previously from `sender-identity` (including group messages), log a warning, _Acknowledge_ and discard the message and abort these steps. 15. If `inner-type` is not defined (i.e. handling an FS control message), log a notice, _Acknowledge_ and discard the message and abort these steps. 16. If `inner-type` is unknown, log a notice, _Acknowledge_ and discard the message and abort these steps. 17. If `inner-type` is `0xa0` (i.e. FS encapsulation within FS encapsulation), log a warning, _Acknowledge_ and discard the message and abort these steps. 18. If `inner-type` has dedicated blocking exemption steps, run these with `sender-identity` and `inner-message`. If the result indicates that the message should be discarded, _Acknowledge_ the message and abort these steps. 19. If `inner-type` does not have dedicated blocking exemption steps and is not exempted from blocking, run the _Identity Blocked Steps_ for `sender-identity`. If the result indicates that `sender-identity` is blocked, _Acknowledge_ and discard the message and abort these steps. 20. If `sender-identity` equals `*3MAPUSH`: 1. If `inner-type` is not `0xfe`, log a warning, _Acknowledge_ and discard the message and abort these steps. 2. Run the receive steps associated to `inner-type` with `inner-message`. If this fails, exceptionally abort these steps and the connection. If the message has been discarded, _Acknowledge_ the message and abort these steps. 21. If `sender-identity` is not a _Special Contact_: 1. If `inner-metadata.nickname` is defined, let `nickname` be the value of `inner-metadata.nickname`.¹ 2. If `inner-metadata` is not defined and _User Profile Distribution_ was expected for `inner-type`, let `nickname` be the result of decoding the plaintext `legacy-sender-nickname`.¹ 3. If `nickname` is present, trim any excess whitespaces from the beginning and the end of `nickname`. 4. If `contact-or-init` does not refer to an existing contact: 1. If `inner-type` does not require to create an implicit _direct_ contact, log a notice, _Acknowledge_ and discard the message and abort these steps. 2. (MD) Run the following sub-steps (labelled _add-contact_): 1. Begin a transaction with scope `CONTACT_SYNC` and the following precondition: 1. If the contact for `sender-identity` exists, abort the _add-contact_ sub-steps. 2. Reflect a `ContactSync.Create` with `contact` set from `contact-or-init` and the following additional properties: - `created_at` set to now, - `nickname` set to `nickname`, - `acquaintance_level` set to `DIRECT`, - all policies and categories set to their defaults. 3. Commit the transaction and await acknowledgement. 3. If the contact for `sender-identity` does not exist, persist a new contact from `contact-or-init` and `nickname`. 4. TODO(SE-510): Schedule fetching gateway-defined profile picture here, if contact was added and if necessary. 5. If `contact-or-init` does refer to an existing contact: 1. Let `change` be an empty set of properties that need to be updated for `contact-or-init`. 2. If `contact-or-init` has an acquaintance level different to _direct_ and `inner-type` requires to create an implicit _direct_ contact, set `change.acquaintance-level` to _direct_. 3. If the contact's nickname is different to `nickname`, set `change.nickname` to `nickname`. 4. (MD) If `change` is not an empty set: 1. Begin a transaction with scope `CONTACT_SYNC` and the following precondition: 1. If the contact no longer exists, log an error and exceptionally abort these steps and the connection. 2. Reflect a `ContactSync.Update` with `contact` set to `sync.Contact` with the following changes: - `acquaintance_level` set to `change.acquaintance-level`, - `nickname` set to `change.nickname`. 3. Commit the transaction and await acknowledgement. 5. Apply any changes in `change` to `contact-or-init`. 6. If `inner-type` requires to create an implicit _direct_ contact, run the following sub-steps in a loop: 1. If this is the 4th iteration, exceptionally abort these steps and the connection. 2. Lookup the contact associated to `sender-identity` and let `contact` be the result (at this point, `contact` must exist). 3. If `contact` has acquaintance level _direct_, abort the loop. 4. (MD) Run the following sub-steps: 1. Begin a transaction with scope `CONTACT_SYNC` and the following precondition: 1. If the contact no longer exists, log an error and exceptionally abort these steps and the connection. 2. Reflect a `ContactSync.Update` with `contact` including `acquaintance_level` set to `DIRECT`. 3. Commit the transaction and await acknowledgement. 5. Set `contact`'s acquaintance level to _direct_. 7. Run the receive steps associated to `inner-type` with `inner-message`. If this fails, exceptionally abort these steps and the connection. If the message has been discarded, _Acknowledge_ the message and abort these steps. 22. (MD) If the properties associated to `inner-type` require reflecting incoming messages, reflect a `d2d.IncomingMessage` from `outer-type` and `outer-message` and the associated conversation to other devices and wait for reflection acknowledgement.² If this fails, exceptionally abort these steps and the connection.³ 23. If the properties associated to `inner-type` require sending automatic delivery receipts and `flags` does not contain the _no automatic delivery receipts_ (`0x80`) flag, schedule a persistent task to run the _Bundled Messages Send Steps_ with the following properties: - `id` being a random message ID, - `created-at` set to the current timestamp, - `receivers` set to `contact`, - to construct a [`delivery-receipt`](ref:e2e.delivery-receipt) message with status _received_ (`0x01`) and the respective `message-id`. 24. _Acknowledge_ the message. ¹: Note that the `nickname` of `MessageMetadata` may be undefined (leading to no changes) or defined but explicitly empty (leading to the nickname of the contact being removed) which is an important semantic difference. Unlike the legacy nickname field which always contains a value and therefore cannot represent this semantic difference without having to check whether _User Profile Distribution_ was required for the type. ²: We reflect the **outer** message container depending on the unwrapped **inner** message type, so the forward security properties are untouched and all other devices need to go through the same process. ³: Reflection needs to happen after the message has been processed and all side effects have been applied. Otherwise, if the receive process is interrupted and another device takes over, it would discard the message as a duplicate. The following steps are defined as _Acknowledge_ steps for an incoming message: 1. If the steps for this message have already been invoked once, abort these steps. 2. If `flags` does not contain the _no server acknowledgement_ (`0x04`) flag, send a [`message-ack`](ref:payload.message-ack) payload to the chat server with the respective `message-id`. 3. If the properties associated to `inner-type` require protection against replay, mark the nonce of `message-and-metadata-nonce` as used. 4. If `fs-commit-fn` is defined, run it. [//]: # "TODO(SE-128)" fields: - *message-sender-identity - *message-receiver-identity - *message-message-id - *message-created-at - *message-flags - *message-reserved - _doc: |- Length of the metadata box. In case it is zero, no metadata is present (for compatibility with clients using [`legacy-message`](ref:payload.legacy-message)). Note: For outgoing messages, a metadata box should always be present. name: metadata-length type: u16-le - _doc: |- Backwards compatibility field for the sender's public nickname. Padded with zeroes if needed. When sending a message towards a Threema Gateway ID (starts with a `*`), add the same nickname as included in the encrypted metadata box. Otherwise, set it to all zeroes. Note: The backwards compatibility for Threema Gateway IDs will be removed eventually! name: legacy-sender-nickname type: b32 - _doc: |- Metadata associated to the message. Must be ignored in case `metadata-length` is zero. Message Metadata Key (`MMK`) derivation: S = X25519HSalsa20(<sender.CK>.secret, <receiver.CK>.public) MMK = BLAKE2b(key=S, salt='mm', personal='3ma-csp') The encoded `csp-e2e.MessageMetadata` is then encrypted in the following way: XSalsa20-Poly1305( key=MMK, nonce=<message-with-metadata-box.message-and-metadata-nonce>, ) name: metadata-container type: b{metadata-length} - _doc: |- Nonce used for the message and the metadata box. name: message-and-metadata-nonce type: *nonce - *message-message-box message-ack: _group: Payloads _doc: |- Acknowledges that a message has been received. Direction: Client <-> Server [//]: # "TODO(SE-128)" fields: - _doc: |- Identity of the sender for an incoming (`0x82`) message / of the receiver for an outgoing (`0x81`) message. name: identity type: *identity - _doc: |- Refers to the `message-id` of the acknowledged message. name: message-id type: *message-id unblock-incoming-messages: _group: Payloads _doc: |- Unblock incoming messages from the server. Sent by a multi-device capable client once it is nominated to receive incoming messages. Direction: Client --> Server [//]: # "TODO(SE-128)" set-push-notification-token: _group: Payloads _doc: |- Sets the push notification token to be used when sending a push message. Direction: Client --> Server fields: - _doc: |- Type of the push token: - `0x00`: No push - `0x01`: APNs Production - `0x02`: APNs Development - `0x05`: APNs Production with `mutable-content` key - `0x06`: APNs Development with `mutable-content` key - `0x11`: FCM with empty payload - `0x13`: HMS with empty payload name: type type: u8 - _doc: |- Push token, maximum 255 bytes. name: token type: b* delete-push-notification-token: _group: Payloads _doc: |- Deletes push tokens for a Threema ID. Can be used when self-removing or removing another device from a device group. Direction: Client --> Server When receiving this payload: 1. If `csp-device-ids` is empty, delete all tokens for all devices except the device sending the payload and abort these steps. 2. Delete all tokens for the devices specified in `csp-device-ids`. fields: - _doc: |- Delete tokens belonging to a [`csp-device-id`](ref:handshake.csp-device-id) in the same device group. name: csp-device-ids type: u64-le[] set-connection-idle-timeout: _group: Payloads _doc: |- Request a different idle timeout than the default one of 5 minutes. The new setting is valid for the connection only. The client must ensure that it sends echo requests or other traffic frequently to keep the connection alive. Direction: Client --> Server [//]: # "TODO(SE-128)" fields: - _doc: |- Idle timeout in seconds. Minium 30s, maximum 600s. name: timeout type: u16-le queue-send-complete: _group: Payloads _doc: |- Indicates that the incoming message queue on the server has been fully transmitted to the client. A client should not disconnect prior to having received this payload. Direction: Client <-- Server [//]: # "TODO(SE-128)" device-cookie-change-indication: _group: Payloads _doc: |- Indicates to the client that a device cookie mismatch has been detected since the last time that the device cookie change indication has been cleared (using the [`clear-device-cookie-change-indication`](ref:clear-device-cookie-change-indicat payload). The client should display a warning in form of a notification and/or dialog to the user, informing them that a new and potentially unauthorized device has accessed the account. When the user confirms, the client should send a [`clear-device-cookie-change-indication`](ref:clear-device-cookie-change-indicat payload to clear the indication. Direction: Client <-- Server clear-device-cookie-change-indication: _group: Payloads _doc: |- Causes the server to clear the flag that triggers sending the [`device-cookie-change-indication`](ref:device-cookie-change-indication) on each connection. The flag will be set again by the server if another device cookie mismatch is detected. Direction: Client --> Server close-error: _group: Payloads _doc: |- Indicates that the connection has experienced an unrecoverable error and must be closed. Direction: Client <-- Server [//]: # "TODO(SE-128)" fields: - _doc: |- Indicates whether the client is allowed to reconnect automatically after the connection has been severed. This allows the server to prevent infinite loops in case of a recurring error. Set to `0` in case the client may not reconnect automatically or any other value otherwise. name: can-reconnect type: u8 - _doc: |- Error message (UTF-8 encoded) name: message type: b* alert: _group: Payloads _doc: |- Generic alert that should be displayed in the client's user interface. Direction: Client <-- Server [//]: # "TODO(SE-128)" fields: - _doc: |- Alert message (UTF-8 encoded) name: message type: b* # End-to-end encrypted structs e2e: &e2e _doc: |- ## End-to-End Encrypted Messages An end-to-end encrypted message can be sent or received once the handshake was successful. Every end-to-end encrypted message is wrapped inside of a [`container`](ref:payload.container) struct that is then encrypted and wrapped by a payload [`legacy-message`](ref:payload.legacy-message) or [`message-with-metadata-box`](ref:payload.message-with-metadata-box) struct. ### Predefined Contacts A pedefined contacts can be added to the contact list and is automatically initialised with its identity, nickname, hard-coded public key and the verification level _fully verified_. Once a predefined contact is in the contact list, it is treated like any other normal contact (with editable properties like first and last name, etc.). A predefined contact may be marked _special_ meaning it follows special logic. These are also known as _Special Contact_s. Even though special contacts should not normally appear in the contact list, there's nothing stopping a user from adding a special contact to its contact list. While they are treated like normal contacts in the contact list, depending on the special handling logic, it may not be possible to send or receive normal messages from them. The following list contains all predefined contacts: - `*3MAPUSH`: - Nickname: Threema Push - Public Key: - Production: fd711e1a0db0e2f03fcaab6c43da2575b9513664a62a12bd0728d87f7125cc24 - Sandbox: fd711e1a0db0e2f03fcaab6c43da2575b9513664a62a12bd0728d87f7125cc24 - Special: Yes - `*3MATOKN`: - Nickname: Threema Token - Public Key: - Production: 04884d12d668f855d00d71fb1d9d413c95f271312f7e077846af671875c4101b - Special: No - `*3MAWORK`: - Nickname: Threema Work Channel - Public Key: - Production: 9aa0a72a8fb6f0cc53727fea6096f1b7b0ebefcc2650ad39a1e54837bba0bc4b - Sandbox: 9aa0a72a8fb6f0cc53727fea6096f1b7b0ebefcc2650ad39a1e54837bba0bc4b - Special: No - `*BETAFBK`: - Nickname: Threema Beta Feedback - Public Key: - Production: 5684d6dcd32a16488df8371095fc9a1fc25baeb6b97366d99fdf2aba00e2bc5c - Special: No - `*MY3DATA`: - Nickname: My Threema Data - Public Key: - Production: 3b01854f24736e2d0d2dc387eaf2c0273c5049052147132369bf3960d0a0bf02 - Sandbox: 83adfee6558b68ae3cd6bbe2a33f4e4409d5624a7cea23a18975aea6272a0070 - Special: No - `*SUPPORT`: - Nickname: Threema Support - Public Key: - Production: 0f944d18324b2132c61d8e40afce60a0ebd701bb11e89be94972d4229e94722a - Sandbox: 0f944d18324b2132c61d8e40afce60a0ebd701bb11e89be94972d4229e94722a - Special: No - `*THREEMA`: - Nickname: Threema Channel - Public Key: - Production: 3a38650c681435bd1fb8498e213a2919b09388f5803aa44640e0f706326a865c - Sandbox: 3a38650c681435bd1fb8498e213a2919b09388f5803aa44640e0f706326a865c - Special: No Note: OnPrem provisions predefined contacts in the associated OPPF file. ### Mitigating Replay To prevent replay attacks, a client must permanently store used nonces for incoming and outgoing end-to-end encrypted messages. Messages reusing previously used nonces must not be processed and discarded. One nonce store for all end-to-end encrypted messages across different contacts is sufficient. Note that it is still possible for the chat server to replay old messages to a device whose database has been erased (e.g. when restoring a backup). However, this is not applicable to forward security encrypted messages. ### Message ID Each message has an associated message ID. It is crucial to understand that this is not a unique identifier across multiple conversations. Unique identification of a message is determined by: - 1:1 Chats: The message ID in combination with the contact's Threema ID. - Group Chats: The message ID in combination with the group creator's Threema ID and the group ID. - Distribution Lists: The message ID with an artificial distribution list ID. When a message is being quoted, it may only be looked up within the associated conversation. ### Flags For each message, we will define _mandatory_ and _optional_ flags referring to the `flags` field of the payload [`legacy-message`](ref:payload.legacy-message) or [`message-with-metadata-box`](ref:payload.message-with-metadata-box) struct. A flag must be considered _mandatory_ unless it has been explicitly marked _optional_. ### Delivery Receipts There are two types of delivery receipts (sent using the [`delivery-receipt`](ref:e2e.delivery-receipt) message): - Automatic: "received" and "read" - Manual: "acknowledged" and "declined" For each message, we will define whether automatic delivery receipts should be sent and whether it is eligible for sending manual delivery receipts (e.g. acknowledge/decline). However, two general exceptions apply: 1. Automatic delivery receipts are not sent to group members (i.e. when any message struct is wrapped in a `group` message struct). 2. Messages whose flags include `0x80` must not trigger any automatic delivery receipts. ### Blocking The sender Threema ID may be blocked explicitly (i.e. blocking a specific Threema ID) or implicitly (blocking all unknown Threema IDs). This does not require special handling on the server but instead is done entirely by the clients. Note that the protocol does not distinguish between implicitly and explicitly blocked Threema IDs. An implicitly blocked Threema ID (i.e. blocking unknown contacts) must be treated the same as an explicitly blocked Threema ID (i.e. blocking specific contacts). The UI must prevent users from composing or submitting messages towards a blocked contact. In practise, this is only relevant for explicitly blocked contacts. The following steps are defined as the _Identity Blocked Steps_: 1. Let `identity` be the Threema ID to be checked. 2. If `identity` is a _Special Contact_, return that it is not blocked. 3. If `identity` is explicitly blocked, return that it is blocked. 4. If the settings indicate that unknown contacts should not be blocked, return that it is not blocked. 5. If `identity` is a _Predefined Contact_, return that it is not blocked. 6. Let `contact` be the associated contact to `identity`. 7. If `contact` is not defined, return that it is blocked. 8. If `contact` has acquaintance level _direct_, return that it is not blocked. 9. If `contact` is part of a group that is not marked as _left_, return that it is not blocked. 10. Return that it is blocked. ### Contact Flows The following steps are defined as the _Valid Contacts Lookup Steps_: 1. Let `identities` be the identities to look up. 2. Let `contact-or-inits` be an empty map of Threema IDs to a contact, properties to create a contact from, or _contact is the user_ or _contact is invalid_ marker. 3. Let `unknown-identities` be an empty list. 4. For each `identity` of `identities`: 1. If `identity` equals the user's Threema ID, add the information that the _contact is the user_ to the `contact-or-inits` map and abort these sub-steps. 2. If `identity` is a _Special Contact_, add that special contact to the `contact-or-inits` map and abort these sub-steps. 3. Lookup the contact associated to `identity` and let `contact` be the result. 4. If `contact` is defined, add `contact` to the `contact-or-inits` map and abort these sub-steps. 5. Lookup the properties to create a contact from associated to `identity` from the _contact lookup cache_ and let `init` be the result. 6. If `init` is defined, add `init` to the `contact-or-inits` map and abort these sub-steps. 7. Add `identity` to `unknown-identities`. 5. Let `directory-response` be the response of asynchronously looking up `unknown-identities` on the Directory Server. 6. If Work flavour, let `work-directory-response` be the response of asynchronously looking up `unknown-identities` on the Work Contacts API endpoint. 7. Await `directory-response` and `work-directory-response`. 8. Process the result of `directory-response`: 1. If the server could not be reached, exceptionally abort these steps. 2. If the status code is `429`, exceptionally abort these steps and add a minimum delay of 10s before retrying a connection. 3. If the status code is not `200`, exceptionally abort these steps. 4. For each contact entry of the result: 1. Remove the contact from `unknown-identities`. If it was not present in `unknown-identities`, log a warning and abort these sub-steps. 2. If the contact is marked as _invalid_ (never existed or has been revoked), add the information that the _contact is invalid_ to the `contact-or-inits` map and abort these sub-steps. 3. If the contact is a _Predefined Contact_: 1. If the contact's public key does not equal the _Predefined Contact_s public key, log a warning and exceptionally abort these steps. 2. Update the contact information with the following information: - set `verification_level` to `FULLY_VERIFIED`, - set `nickname` to the nickname of the _Predefined Contact_, - if _Predefined Contact_ defines a first name, set it accordingly, - if _Predefined Contact_ defines a last name, set it accordingly, 3. Add the resulting contact information to the `contact-or-inits` map from which a new contact can be created. 5. For each `identity` of `unknown-identities`: 1. Add the information that the _contact is invalid_ to the `contact-or-inits` map for `identity`. 6. Clear `unknown-identities`. 9. If `work-directory-response` is defined, process its result: 1. If the server could not be reached, exceptionally abort these steps. 2. If the status code is `401`, exceptionally abort these steps, notify the user that the Work credentials are invalid and request new ones. The connection should not be retried until new Work credentials have been entered and checked for validity. 3. If the status code is `429`, exceptionally abort these steps and add a minimum delay of 10s before retrying a connection. 4. If the status code is not `200`, exceptionally abort these steps. 5. For each `work-contact` of the result: 1. If an entry for the `work-contact`'s identity does not exist in `contact-or-inits`, log a warning and abort these sub-steps. 2. If `work-contact`'s public key does not equal `contact-or-init`'s public key, log a warning and exceptionally abort these steps. 3. Update the contact entry for `work-contact`'s identity in `contact-or-inits` with the following information: - if `verification_level` is not defined or `UNVERIFIED`, set it to `SERVER_VERIFIED`, - set `work_verification_level` to `WORK_SUBSCRIPTION_VERIFIED`, - if `work-contact.first-name` is defined, set the first name accordingly, - if `work-contact.last-name` is defined, set the last name accordingly, 10. TODO(SE-173): Run the contact import flow for `contact-or-inits` and update the `verification_level` for all whose associated phone number / email could be matched. Import `first_name` and `last_name` (if not already defined) and set `sync_state` to `IMPORTED`. Clarify precedence regarding Work API. 11. For each `init` of `contact-or-inits`: 1. If `init` does not contain properties to create a contact from (i.e. it is a contact or any of the special markers), abort these sub-steps. 2. If `init.sync_state` is not defined, set it to `INITIAL`. 3. If `init.verification_level` is not defined, set it to `UNVERIFIED`. 4. If `init.work_verification_level` is not defined, set it to `NONE`. 12. Update the _contact lookup cache_ with the contents of `contact-or-inits`. Each newly added or updated entry has an expiration time of 10m after which the entry is to be removed from the cache. 13. Return `contact-or-inits`. ### Groups Groups are handled in a decentralised manner. Messages are sent to each group member individually. On a technical level, a group is identified by **both** the Threema ID of the creator and the random group ID the creator chose. A group **must never** be identified by the group ID alone. Group messages are special containers wrapped around normal messages (it is actually just a common header): - [`group-member-container`](ref:e2e.group-member-container): For group message communication between members, including the creator. - [`group-creator-container`](ref:e2e.group-creator-container): For special messages that may only be sent from the creator to normal group members and vice versa. Group messages have special types in order to separate them from other messages. These types also define which container must be used. The group members are determined by the [`group-setup`](ref:e2e.group-setup) message and continuously updated by any following [`group-leave`](ref:e2e.group-leave) messages. Any following [`group-setup`](ref:e2e.group-setup) overrides the previous member state. ### Implicit Contact Creation When the user is added to a group, every unknown member of the group must be added to the contact list with acquaintance level _group_. Messages from a contact with any acquaintance level will not be implicitly blocked by a _block unknown_ setting. The contact remains at the acquaintance level _group_ until a 1:1 conversation with that contact is being started by either side in which case the acquaintance level should be changed to _direct_. A contact with acquaintance level _group_ will remain at that level indefinitely even if the contact is being removed from all groups of the user or if all remaining common groups are marked as _left_. In that case, the contact is implicitly marked as _deleted_ so that it is covered by _block unknown_. ### Notes Group A group is identified as a _notes_ group if all of the following criteria are met: 1. The user is the creator of the group. 2. The group currently has no members (besides the creator). 3. The group is not marked as _left_. Messages in a _notes_ group are synchronised across devices but are not sent to the chat server (since there are no other members). Therefore, it is ideal for "notes to self", hence the name. A group seamlessly transforms into a _notes_ group and out of it given the above criteria. Right now this can happen in three scenarios: - A _notes_ group is created explicitly (i.e. a group with only the user is being created). - The user is the creator of a group and one or more members are being added in which case the _notes_ group transforms into a regular group. - The user is the creator of a group whose members have just been removed (but the group has not been disbanded) in which case the group transforms into a _notes_ group. The UI should signal the _notes_ status of a group to the user. ### Group Flows The following steps are defined as the _Active Group Update Steps_: 1. If the user is not the creator of the group or the group is marked as _left_, log an error and abort these steps. 2. Let `message-ids` be a list of four pre-generated message IDs. 3. Let `changes` be the set of expected changes to the group which may contain the following properties: - `profile-picture` is defined if the group's profile picture is expected to be changed and contains either the new profile picture or a _remove_ mark to remove it - `profile-picture.blob` may contain the associated blob information data in case of a changed profile picture. - `add-members` is a set of new members to be added to the group - `remove-members` is a set of existing members to be removed from the group 4. Let `group` be a snapshot of the current group state. 5. Remove all members from `changes.add-members` that are not in `group.members`. 6. Remove all members from `changes.remove-members` that are in `group.members`. 7. Let `messages` be an empty list. 8. If `changes.remove-members` is not empty, add a message entry to `messages` to remove members to be removed with the following properties: - `id` set to the first message ID of `message-ids`, - `created-at` set to the current timestamp, - `receivers` set to `changes.remove-members`, - to construct a [`group-setup`](ref:e2e.group-setup) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) with an empty members set. 9. If `group.members` is not empty: 1. Add a message entry to `messages` to update the group for the members with the following properties: - `id` set to the first message ID of `message-ids`, - `created-at` set to the current timestamp, - `receivers` set to `group.members`, - to construct a [`group-setup`](ref:e2e.group-setup) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) from `group.members`. 2. Add a message entry to `messages` to announce the group's name to the members with the following properties: - `id` set to the second message ID of `message-ids`, - `created-at` set to the current timestamp, - `receivers` set to `group.members`, - to construct a [`group-name`](ref:e2e.group-name) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) from `group.name`.¹ 3. If `group.profile-picture` is defined: 1. Let `profile-picture-blob` be `changes.profile-picture.blob`. 2. If `group.profile-picture` does not equal `changes.profile-picture`, upload `group.profile-picture` to the blob server with the _persist_ flag and set `profile-picture-blob` to the result. 4. Add a message entry to `messages` to announce the group's profile picture to the members with the following properties: - `id` set to the third message ID of `message-ids`, - `created-at` set to the current timestamp, - `receivers` set to `group.members`, - to construct a [`set-profile-picture`](ref:e2e.set-profile-picture) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) from `profile-picture-blob` if `profile-picture-blob` is defined or [`delete-profile-picture`](ref:e2e.delete-profile-picture) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) otherwise.¹ 5. Let `chosen-call` be the result of the most recent invocation of the _Group Call Refresh Steps_ for the group. 6. If `chosen-call` is defined, add a message entry to `messages` to announce the ongoing group call to newly added members with the following properties: - `id` set to the fourth message ID of `message-ids`, - `created-at` set to the `started_at` value associated to `chosen-call`, - `receivers` set to `changes.add-members`, - to construct a repeat of `csp-e2e.GroupCallStart` (wrapped by [`group-member-container`](ref:e2e.group-member-container)) that is associated to `chosen-call`. 10. Run the _Bundled Messages Send Steps_ with `messages`. ¹: This results in the group name and the group profile picture being distributed to all members regardless of whether it was changed or not. This is deemed acceptable for the sake of implementation simplicity and reusability. The following steps are defined as the _Active Group State Resync Steps_: 1. If the user is not the creator of the group or the group is marked as _left_, log an error and abort these steps. 2. Let `message-ids` be a list of four pre-generated message IDs. 3. Let `target-members` be a set of members to receive the resync. 4. Remove all members from `target-members` that are not in `group.members`. 5. If `target-members` is empty, abort these steps. 6. Let `messages` be an empty list. 7. Add a message entry to `messages` to announce the group composition with the following properties: - `id` set to the first message ID of `message-ids`, - `created-at` set to the current timestamp, - `receivers` set to `target-members`, - to construct a [`group-setup`](ref:e2e.group-setup) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) from `group.members`. 8. Add a message entry to `messages` to announce the group's name with the following properties: - `id` set to the second message ID of `message-ids`, - `created-at` set to the current timestamp, - `receivers` set to `target-members`, - to construct a [`group-name`](ref:e2e.group-name) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) from `group.name`. 9. If `group.profile-picture` is defined, upload `group.profile-picture` to the blob server with the _persist_ flag and let `profile-picture-blob` be the result. 10. Add a message entry to `messages` to announce the group's profile picture with the following properties: - `id` set to the third message ID of `message-ids`, - `created-at` set to the current timestamp, - `receivers` set to `target-members`, - to construct a [`set-profile-picture`](ref:e2e.set-profile-picture) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) from `profile-picture-blob` if `profile-picture-blob` is defined or [`delete-profile-picture`](ref:e2e.delete-profile-picture) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) otherwise. 11. Let `chosen-call` be the result of the most recent invocation of the _Group Call Refresh Steps_ for the group. 12. If `chosen-call` is defined, add a message entry to `messages` to announce the ongoing group call with the following properties: - `id` set to the fourth message ID of `message-ids`, - `created-at` set to the `started_at` value associated to `chosen-call`, - `receivers` set to `target-members`, - to construct a repeat of `csp-e2e.GroupCallStart` (wrapped by [`group-member-container`](ref:e2e.group-member-container)) that is associated to `chosen-call`. 13. Run the _Bundled Messages Send Steps_ with `messages`. 14. For each member of `target-members`, mark the group as _recently resynced_ for 1h. #### Create/Clone Group The following steps must be invoked when the user wants to create or clone a group: 1. Let `init` contain the following properties to create a group: - `name` of the new group or an empty string - `profile-picture` of the new group or undefined - `members` is a set of initial members to be added to the group¹ 2. Let `parameters` be the MDM parameters. If `parameters.th_disable_create_group` is `true`, abort these steps. 3. Let `group-id` be a random group ID. 4. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If a group with `group-id` and the user as creator exists, log an error and abort these steps. 2. If `init.members` contains a member that is not an existing contact, log an error and abort these steps. 5. If `init.profile-picture` is defined, upload `init.profile-picture` to the blob server with the _persist_ flag and set `init.profile-picture.blob` to the result. 6. (MD) Reflect a `GroupSync.Create` with `group` set to contain: - `group_identity` being `group-id` and the user as the creator, - `created_at` set to now, - `name` set to `init.name` - `user_state` set to `MEMBER`, - `profile_picture` set from `init.profile-picture.blob`, - `member_identities` set from `init.members`, - all policies and categories set to their defaults. 7. (MD) Commit the transaction and await acknowledgement. 8. Persist the newly created group from `init` and `group-id` to storage. 9. If `init.members` is empty, abort these steps. 10. Let `message-ids` be a list of four random message IDs. 11. Schedule a persistent task to run the following steps: 1. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_ or the group has no members, log a warning that a group sync race occurred and abort these steps. 2. Let `group` be a snapshot of the current group state. 3. If any of the following cases is true, log a warning that a group sync race occurred: - `init.name` is defined and does not equal `group.name`, - `init.profile-picture` does not equal `group.profile-picture`, - `init.members` does not equal `group.members`. 4. Run the _Active Group Update Steps_ with `message-ids` and the following expected set of `changes`: - `profile-picture` set to `init.profile-picture`, - `add-members` set to `init.members`, - `remove-members` set to an empty list. 5. (MD) Commit the transaction and await acknowledgement. ¹: Note that all contacts must be added before they can be added as initial members of the group. #### Update Group The following steps must be invoked when the user is the creator of a group and intends to apply a change to the group's name, profile picture or add/remove members to/from the group: 1. If the user is not the creator of the group or the group is marked as _left_, log an error and abort these steps. 2. Let `changes` be the set of changes to the group which may contain the following properties: - `name` is defined if the group's name is to be changed and contains the new name or an empty string - `profile-picture` is defined if the group's profile picture is to be changed and contains either the new profile picture or a _remove_ mark to remove it - `add-members` is a set of new members to be added to the group¹ - `remove-members` is a set of existing members to be removed from the group¹ 3. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If `changes.add-members` or `changes.remove-members` contains a member that is not an existing contact, log an error and abort these steps. 2. If the group does not exist or the group is marked as _left_, log a warning and abort these steps. 4. Let `updated-members` be a copy of the current member set of the group. Add all `changes.add-members` to this set that are to be added to the group. Remove all `changes.remove-members` from this set that are to be removed from the group. 5. If `changes.profile-picture` is defined and contains a profile picture, upload `changes.profile-picture` to the blob server with the _persist_ flag and let `changes.profile-picture.blob` be the result. 6. (MD) Reflect a `GroupSync.Update` with `member_state_changes` constructed from `changes.add-members` and `changes.remove-members` and `group` set to contain: - `name` set to `changes.name`, - `profile_picture` set according to `changes.profile-picture` (and `changes.profile-picture.blob`), - `member_identities` set from `updated-members`. 7. (MD) Commit the transaction and await acknowledgement. 8. If the user is currently participating in a group call of this group, remove all `change.remove-members` participants from the group call (handle them as if they left the call). 9. Persist the `updated-members` and other `changes` to the group. 10. If `changes.add-members` or `changes.remove-members` is not empty, run the _Rejected Messages Refresh Steps_ for the group. 11. Let `message-ids` be a list of four random message IDs. 12. Schedule a persistent task to run the following steps: 1. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning that a group sync race occurred and abort these steps. 2. Let `group` be a snapshot of the current group state. 3. If any of the following cases is true, log a warning that a group sync race occurred: - `changes.name` is defined and does not equal `group.name`, - `changes.profile-picture` contains a profile picture and does not equal `group.profile-picture`, - `changes.profile-picture` contains the _remove_ mark and `group.profile-picture` is defined, - `updated-members` does not equal `group.members`. 4. Run the _Active Group Update Steps_ with `message-ids` and `changes`. 5. (MD) Commit the transaction and await acknowledgement. ¹: Note that all contacts must be added before they can be added as members to the group. The same applies to members that are being removed, obviously. #### Disband/Remove Group The following steps must be invoked when the user is the creator of a group and intends to _disband_ or _disband and remove_ the group: 1. Let `intent` be the user's intent which can be either to _disband_ or to _disband and remove_ the group. 2. If the user is not the creator of the group or the group is marked as _left_, log an error and abort these steps. 3. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning and abort these steps. 4. (MD) If `intent` is to _disband_, reflect a `GroupSync.Update` with `group` set to contain `user_state` set to `LEFT`. 5. (MD) If `intent` is to _disband and remove_, reflect a `GroupSync.Delete` for this group. 6. (MD) Commit the transaction and await acknowledgement. 7. If the user is participating in a group call of this group, trigger leaving the call. 8. If the `intent` is to _disband_: 1. Mark the group as _left_. 2. Persist the previous member setup so that the group can be cloned. 3. Run the _Rejected Messages Refresh Steps_ for the group. 9. Let `group` be a snapshot of the current group state. 10. If the `intent` is to _disband and remove_, remove the group and all associated messages from storage. 11. Let `message-id` be a random message ID. 12. Schedule a persistent task to run the following steps: 1. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group exists and is not marked as _left_, log an error that a major group state inconsistency has been detected¹ and abort these steps. 2. Run the _Bundled Messages Send Steps_ with the following properties: - `id` set to `message-id`, - `created-at` set to the current timestamp, - `receivers` set to `group.members`, - to construct a [`group-setup`](ref:e2e.group-setup) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) with an empty members set. 3. (MD) Commit the transaction and await acknowledgement. ¹: Disbanding a group as the creator makes the group strictly non-reusable. #### Leave/Remove Group The following steps must be invoked when the user is not the creator of a group and intends to _leave_ or _leave and remove_ the group: 1. Let `intent` be the user's intent which can be either to _leave_ or to _leave and remove_ the group. 2. If the user is the creator of the group or the group is marked as _left_, log an error and abort these steps. 3. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning and abort these steps. 4. (MD) If `intent` is to _leave_, reflect a `GroupSync.Update` with `group` set to contain `user_state` set to `LEFT`. 5. (MD) If `intent` is to _leave and remove_, reflect a `GroupSync.Delete` for this group. 6. (MD) Commit the transaction and await acknowledgement. 7. If the user is participating in a group call of this group, trigger leaving the call. 8. If the `intent` is to _leave_: 1. Mark the group as _left_. 2. Persist the previous member setup so that the group can be cloned. 3. Run the _Rejected Messages Refresh Steps_ for the group. 9. Let `group` be a snapshot of the current group state. 10. If the `intent` is to _leave and remove_, remove the group and all associated messages from storage. 11. Let `message-id` be a random message ID. 12. Schedule a persistent task to run the following steps: 1. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group exists and is not marked as _left_, log a warning that a group sync race occurred and abort these steps. 2. Run the _Bundled Messages Send Steps_ with the following properties: - `id` set to `message-id`, - `created-at` set to the current timestamp, - `receivers` set to `group.members`, - to construct a [`group-leave`](ref:e2e.group-leave) (wrapped by [`group-member-container`](ref:e2e.group-member-container)) 3. (MD) Commit the transaction and await acknowledgement. #### Remove Group The following steps must be invoked when the user intends to remove a group that is marked as _left_. 1. If the group is not marked as _left_, log an error and abort these steps. 2. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is not marked as _left_, log a warning and abort these steps. 3. (MD) Reflect a `GroupSync.Delete` for this group. 4. (MD) Commit the transaction and await acknowledgement. 5. Remove the group and all associated messages from storage. #### Group Resync The following steps must be invoked when the user is the creator of a group and intends to resync the group manually: 1. If the user is not the creator of the group or the group is marked as _left_, log an error and abort these steps. 2. Let `message-ids` be a list of four random message IDs. 3. Schedule a volatile task to run the following steps: 1. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning and abort these steps. 2. Run the _Active Group State Resync Steps_ with `message-ids` and `target-members` being all current group members.¹ 3. (MD) Commit the transaction and await acknowledgement. ¹: This mechanic intentionally bypasses the 1h _recently resynced_ mark due to an explicit manual request by the user to resync the group. #### Update Conversation The following steps must be invoked when the user intends to change any other synchronised property of the group: 1. Let `change` be one of the following changes to the group as defined by `sync.Group`: - `notification_trigger_policy_override` - `notification_sound_policy_override` - `conversation_category` - `conversation_visibility` 2. Persist the `change` to the group. 3. (MD) Schedule a persistent task to run the following steps: 1. Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist, log a warning and abort these steps. 2. Reflect a `GroupSync.Update` with `group` set to contain the `change`. 3. Commit the transaction and await acknowledgement. 4. Persist the `change` to the group (again). ### Device Flows #### Deactivate Multi-Device Flow The following steps must be invoked when the user wants to deactivate multi-device and continue using the current device. 1. Run the _Drop Devices Steps_ with the intent to _deactivate_ multi-device and keep the user informed regarding the process status and any encountered issues. #### Drop Own Device Flow The following steps must be invoked when the user wants to stop using the current device. 1. If the device does not have multi-device enabled, log an error and abort these steps. 2. Begin a transaction (scope: `DROP_DEVICE`, precondition: none). 3. Send a `DropDevice` with this device's Device ID. 4. Await the corresponding `DropDeviceAck` or the connection closing with close code `4113`. 5. TODO(SE-494): Enter read-only mode persistently. #### Drop Other Devices Flow The following steps must be invoked when the user wants to drop one or more other devices from the device group. 1. Let `device-ids-to-drop` be a set of Device IDs that should be dropped from the device group. 2. If this device's Device ID is contained in `device-ids-to-drop`, log an error and abort these steps. 3. Run the _Drop Devices Steps_ with the intent to _drop specific_ `device-ids-to-drop` and keep the user informed regarding the process status and any encountered issues. #### Drop Devices Steps The following steps are defined as the _Drop Devices Steps_: 1. If the device does not have multi-device enabled¹, run the _Application Setup Steps_ step 2.2. through 2.6. and abort these steps. TODO(SE-199): This shall be removed once multi-device supports FS. 2. Let `intent` be the intent which can be either to _deactivate_ multi-device or _drop specific_ devices, letting `device-ids-to-drop` be that set of specific Device IDs. 3. If `device-ids-to-drop` is defined: 1. If `device-ids-to-drop` is empty, log an error and abort these steps. 2. If `device-ids-to-drop` contains this device's Device ID, log an error and abort these steps. 4. Begin a transaction (scope: `DROP_DEVICE`, precondition: none). 5. Send a `GetDevicesInfo` message. 6. Await the `DevicesInfo` message and let `other-device-ids` be a set of the contained Device IDs excluding this device's Device ID. 7. If `device-ids-to-drop` is defined, remove each Device ID from `device-ids-to-drop` that is not present in `other-device-ids`. 8. If `device-ids-to-drop` is not defined, define it to be a copy of `other-device-ids`. 9. Send a `DropDevice` message for each Device ID of `device-ids-to-drop`. 10. Await all corresponding `DropDeviceAck`s of each Device ID in `device-ids-to-drop`. 11. If `device-ids-to-drop` contains the same Device IDs as `other-device-ids` (i.e. all other devices but this device have been dropped): 1. Send a `DropDevice` with this device's Device ID. 2. Await the corresponding `DropDeviceAck` or the connection closing with close code `4113`. 3. Disable multi-device and purge any existing device group data. 4. Run the _Application Setup Steps_ step 2.2. through 2.6. TODO(SE-199): This shall be removed once multi-device supports FS. ¹: This can happen if the steps are run within a persistent task that is aborted before reactivating FS successfully. TODO(SE-199): This shall be removed once multi-device supports FS. ### Sending The following steps are defined as the _Bundled Messages Send Steps_ and must always be invoked as part of a task in order to send a message: 1. Let `messages` be a list of messages with each having the following properties: - `id` being the associated message ID¹, - `created-at` timestamp, - `receivers` being the set of receivers for the message², - the associated conversation, - all necessary information to construct a _canonical_ message from it, - all necessary information to construct a _specific_ message from it, given the specific receiver. Note: If only one set of informations to construct a message is provided, this is considered both the _canonical_ and the _specific_ message construction variant. 2. For each `message` of `messages`: 1. For each `receiver` of `message.receivers`: 1. If `receiver` is the user, log a warning, remove `receiver` from `receivers` and abort these sub-steps. 2. If `receiver` is marked as _invalid_, remove `receiver` from `receivers and abort these sub-steps. 3. If the properties associated to `message` to be constructed for `receiver` given its feature mask indicates that the message is not exempted from blocking, run the _Identity Blocked Steps_ for `receiver`'s identity. If the result indicates that `receiver` is blocked, remove `receiver` from `receivers` and abort these sub-steps. 4. Construct the _specific_ `message` for `receiver` and attach the constructed message to `receiver`. 5. Run the _Profile Picture Distribution Steps_ with the constructed _specific_ message for `receiver`'s type and `receiver` and extend `messages` with the result. 3. For each `message` of `messages` attach a random nonce for `message` to each receiver of `message.receivers`. 4. (MD) Let `pending-reflect-acks` be an empty list. 5. (MD) For each `message` of `messages`: 1. If the properties associated to the _canonical_ `message` do not require reflecting outgoing messages, abort these sub-steps. 2. Construct a `d2d.OutgoingMessage` from the _canonical_ `message` for the associated conversation and reflect it. 3. Add the pending acknowledgement to `pending-reflect-acks`. 6. (MD) Await all `pending-reflect-acks`. 7. Let `pending-csp-acks` and `fs-commit-fns` be empty lists. 8. For each `message` of `messages`: 1. For each `receiver` of `message.receivers`: 1. If the constructed _specific_ message for `receiver` is of type `0xa0`, let `outer-messages` be a list including only the constructed message for `receiver` and `fs-commit-fns` be an empty list. 2. If the constructed _specific_ message for `receiver` is not of type `0xa0`, run the _FS Encapsulation Steps_ with the constructed _specific_ message for `receiver` and let `outer-messages` and `fs-commit-fns` be the result.³ 3. For each `outer-message` of `outer-messages`: 1. Create a `payload.message-with-metadata-box` for `outer-message` with `receiver` and let `payload` be the result. 2. If `payload.flags` does not contain the _no server acknowledgement_ (`0x04`) flag, add `payload.message-id` to `pending-csp-acks`. 3. If the properties associated to `outer-message` requires protection against replay, mark the nonce of `outer-message` as used. 4. Send `payload`. 9. Await all `pending-csp-acks`. 10. Run each function of `fs-commit-fns`. 11. (MD) Let `pending-reflect-acks` be an empty list. 12. (MD) For each `message` of `messages`: 1. If the properties associated to the _canonical_ `message` is eligible for reflecting `OutgoingMessageUpdate.Sent`: 1. Create an `OutgoingMessageUpdate.Sent` for `message.id` and the associated conversation and reflect it. 2. Add the pending acknowledgement to `pending-reflect-acks`. 13. (MD) Await all `pending-reflect-acks`. 14. For each `message` of `messages`: 1. (MD) If the properties associated to the _canonical_ `message` was eligible for reflecting `OutgoingMessageUpdate.Sent`, mark it as _sent_ with the timestamp from the corresponding `reflect-ack` message. 2. (non-MD) Mark `message` as _sent_ with the current timestamp. ¹: Note that, in groups, this implicitly assigns the same message ID towards each group member which in fact is a requirement of the protocol. ²: Reflecting with `receivers` empty is a legitimate case that occurs when sending a message in a notes group. ³: Always invoking the _FS Encapsulation Steps_ ensures that an FS session is being initiated as soon as possible, so that messages can be protected by FS. Moreover, it ensures that the announced FS session version is up to date (a newer version potentially increasing security or making more messages eligible for FS protection). The following steps are defined as the _Messages Submit Steps_ which must be invoked to submit a user-created message in a conversation: 1. Let `messages` be a list of messages of the user to be added to the conversation with each having the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `blobs` be an optional set of blobs that must be uploaded prior to being able to construct the message, - all necessary information to construct a _canonical_ message from it, - all necessary information to construct a _specific_ message from it, given the specific receiver. Note: If only one set of informations to construct a message is provided, this is considered both the _canonical_ and the _specific_ message construction variant. 2. If the associated conversation is a 1:1 conversation, run the _1:1 Messages Submit Steps_ with `messages`. 3. If the associated conversation is a group conversation, run the _Group Messages Submit Steps_ with `messages`. 4. If the associated conversation is a distribution list conversation, run the _Distribution List Messages Submit Steps_ with `messages`. 5. (Unreachable) The following steps are defined as the _1:1 Messages Submit Steps_ which must be invoked to submit a user-created message in a 1:1 conversation: 1. Let `messages` be a list of messages of the user to be added to the conversation with each having the following properties: - `id` defaulting to a random message ID, - `created-at` defaulting to the current timestamp, - `blobs` be an optional set of blobs that must be uploaded prior to being able to construct the message, - all necessary information to construct a _canonical_ message from it, - all necessary information to construct a _specific_ message from it, given the specific receiver. Note: If only one set of informations to construct a message is provided, this is considered both the _canonical_ and the _specific_ message construction variant. 2. Let `receiver` be the conversation's associated contact. 3. If `receiver` has acquaintance level _deleted_, discard `messages`, log a warning and abort these steps.¹ 4. Run the _Identity Blocked Steps_ for `receiver`'s identity'. If the result indicates that `receiver` is blocked, discard `messages`, log a warning and abort these steps.¹ 5. For each `message` of `messages`, assign a random message ID to `message.id`. 6. Schedule a persistent task to run the following steps:² 1. If the contact for `receiver` no longer exists, log an error, discard `messages` and abort these steps. 2. For each `message` of `messages`: 1. If `message.blob` is not defined, abort these sub-steps. 2. Upload all `message.blobs` to the blob server and update `message` with the result (so that the message can be constructed). 3. (MD) If `receiver`'s acquaintance level is not _direct_ or if at least one of the `messages` associated properties requires to unarchive the conversation and the associated conversation visibility is set to _archived_: 1. Begin a transaction with scope `CONTACT_SYNC` and the following precondition: 1. If the contact for `receiver` no longer exists, log an error, discard `messages` and abort these steps. 2. Let `change` be the following changes as defined by `sync.Contact`: - `acquaintance_level` set to `DIRECT`, - `conversation_visibility` set to `NORMAL` if the associated conversation visibility is currently _archived_, 3. Reflect a `ContactSync.Update` with `contact` set from `change`. 4. Commit the transaction and await acknowledgement. 5. Persist the `change` to the `receiver`.³ 4. Run the _Bundled Messages Send Steps_ for `messages` with the following additional properties added to each message: - `receivers` set to `receiver`. 7. If the the `receiver`'s acquaintance level is not _direct_, update it to _direct_. 8. If at least one of the `messages` associated properties requires to unarchive the conversation and the associated conversation visibility is set to _archived_, set it to _normal_. 9. For each `message` of `messages`, add `message` to the associated conversation or update a stateful message referred to by `message` respectively. 10. If at least one of the `messages` associated properties requires to bump the conversation's _last update_ timestamp, update it to the current timestamp. ¹: While the UI should not allow to submit a message in these states, another device may alter the state just prior to submission, allowing to hit these steps in rare circumstances. ²: Rationale to not check for acquaintance level _deleted_ or whether the receiver is blocked again is the legitimate use case of the user sending a final message prior to blocking or removing a contact. ³: This is intentionally done only for MD since the user may e.g. immediately archive a conversation after submitting a message. This results in both the message and the contact sync being queued as tasks whereas in the non-MD case only the message task would be queued. In the MD case, the conversation briefly flicks back to _unarchived_ once the message has been sent but it immediately flicks back to _archived_ once the contact sync task has been executed. But because no contact sync task is created in the non-MD case, the message task would leave the conversation _unarchived_ which is not intended by the user. The following steps are defined as the _Group Messages Submit Steps_ which must be invoked to submit a user-created message in a group conversation: 1. Let `messages` be a list of messages of the user to be added to the conversation with each having the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `blobs` be an optional set of blobs that must be uploaded prior to being able to construct the message, - all necessary information to construct a _canonical_ message from it, - all necessary information to construct a _specific_ message from it, given the specific receiver. Note: If only one set of informations to construct a message is provided, this is considered both the _canonical_ and the _specific_ message construction variant. 2. If the group is marked as _left_, discard `messages`, log a warning and abort these steps.¹ 3. For each `message` of `messages`, assign a random message ID to `message.id`. 4. Schedule a persistent task to run the following steps:² 1. If the group does not exist or is marked as _left_, log a warning, discard `messages` and abort these steps. 2. For each `message` of `messages`: 1. If `message.blob` is not defined, abort these sub-steps. 2. Upload all `message.blobs` to the blob server with the _persist_ flag and update `message` with the result (so that the message can be constructed). 3. (MD) If at least one of the `messages` associated properties requires to unarchive the conversation and the associated conversation visibility is set to _archived_: 1. Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or is marked as left, log a warning that a group sync race occurred, discard `messages` and abort these steps. 2. Let `change` be the following changes as defined by `sync.Group`: - `conversation_visibility` set to `NORMAL` if the associated conversation visibility is currently _archived_, 3. Reflect a `GroupSync.Update` with `group` set from `change`. 4. Commit the transaction and await acknowledgement. 5. Persist the `change` to the group.³ 4. Run the _Bundled Messages Send Steps_ for `messages` with the following additional properties added to each message: - `receivers` set to the group's members. 5. If at least one of the `messages` associated properties requires to unarchive the conversation and the associated conversation visibility is set to _archived_, set it to _normal_. 6. For each `message` of `messages`, add `message` to the associated conversation or update a stateful message referred to by `message` respectively. 7. If at least one of the `messages` associated properties requires to bump the conversation's _last update_ timestamp, update it to the current timestamp. ¹: While the UI should not allow to submit a message in these states, another device may alter the state just prior to submission, allowing to hit these steps in rare circumstances. ²: Rationale to not check for acquaintance level _deleted_ or whether the receiver is blocked again is the legitimate use case of the user sending a final message prior to removing a group. ³: This is intentionally done only for MD since the user may e.g. immediately archive a conversation after submitting a message. This results in both the message and the group sync being queued as tasks whereas in the non-MD case only the message task would be queued. In the MD case, the conversation briefly flicks back to _unarchived_ once the message has been sent but it immediately flicks back to _archived_ once the group sync task has been executed. But because no group sync task is created in the non-MD case, the message task would leave the conversation _unarchived_ which is not intended by the user. The following steps are defined as the _Distribution List Messages Submit Steps_ which must be invoked to submit a user-composed message in a distribution list conversation: 1. Let `messages` be a list of messages of the user to be added to the conversation with each having the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `blobs` be an optional set of blobs that must be uploaded prior to being able to construct the message, - all necessary information to construct a _canonical_ message from it, - all necessary information to construct a _specific_ message from it, given the specific receiver. Note: If only one set of informations to construct a message is provided, this is considered both the _canonical_ and the _specific_ message construction variant. 2. For each `message` of `messages`, assign a random message ID to `message.id`. 3. Schedule a persistent task to run the following steps: 1. If the distribution list does not exist, log a warning, discard `messages` and abort these steps. 2. For each `message` of `messages`: 1. If `message.blob` is not defined, abort these sub-steps. 2. Upload all `message.blobs` to the blob server with the _persist_ flag and update `message` with the result (so that the message can be constructed). 3. (non-MD) Let `members` be all current members of the distribution list. Remove any members with acquaintance level _deleted_ from `members`.¹ 4. (MD) If at least one of the `messages` associated properties requires to unarchive the conversation and the associated conversation visibility is set to _archived_: 1. Begin a transaction with scope `DISTRIBUTION_LIST_SYNC` and the following precondition: 1. If the distribution list does not exist, log a warning that a distribution list sync race occurred, discard `messages` and abort these steps. 2. Let `members` be all current members of the distribution list. Remove any members with acquaintance level _deleted_ from `members`.¹ 3. Let `change` be the following changes as defined by `sync.DistributionList`: - `member_identities` from `members`, - `conversation_visibility` set to `NORMAL` if the associated conversation visibility is currently _archived_, 4. Reflect a `DistributionList.Update` with `distribution_list` set from `change`. 5. Commit the transaction and await acknowledgement. 6. Persist the `change` to the distribution list.² 5. Run the _Bundled Messages Send Steps_ for `messages` with the following additional properties added to each message: - `receivers` from `members`. 4. If at least one of the `messages` associated properties requires to unarchive the conversation and the associated conversation visibility is set to _archived_, set it to _normal_. 5. For each `message` of `messages`, add `message` to the associated conversation or update a stateful message referred to by `message` respectively. 6. If at least one of the `messages` associated properties requires to bump the conversation's _last update_ timestamp, let `last-update-at` be the current timestamp. 7. If `last-update-at` is defined, update the associated _last update_ timestamp of the conversation to `last-update-at`. 8. For each `member` of the distribution list: 1. Add each message of `messages` to the 1:1 conversation associated to `member`. 2. If `last-update-at` is defined, update the associated _last update_ timestamp of the 1:1 conversation of `member` to `last-update-at`. ¹: There should be no receivers in a distribution list that have acquaintance level _deleted_, so the filtering is only done for safety. ²: This is intentionally done only for MD since the user may e.g. immediately archive a conversation after submitting a message. This results in both the message and the distribution list sync being queued as tasks whereas in the non-MD case only the message task would be queued. In the MD case, the conversation briefly flicks back to _unarchived_ once the message has been sent but it immediately flicks back to _archived_ once the distribution list sync task has been executed. But because no distribution list sync task is created in the non-MD case, the message task would leave the conversation _unarchived_ which is not intended by the user. ### Receiving The following steps are defined as _Common Group Receive Steps_ and will be applied in most cases for group messages: 1. Look up the group. 2. If the group could not be found: 1. If the user is the creator of the group, discard the message and abort these steps. 2. Run the _Identity Blocked Steps_ for the creator of the group. If the result indicates that the creator is blocked, discard the message and abort these steps. 3. Run the _Group Sync Request Steps_ for the group, discard the message and abort these steps. 3. If the group is marked as _left_: 1. If the user is the creator of the group, run the _Bundled Messages Send Steps_ with the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `receivers` set to the sender, - to construct a [`group-setup`](ref:e2e.group-setup) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) with an empty members set 2. If the user is not the creator of the group, run the _Bundled Messages Send Steps_ with the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `receivers` set to the sender, - to construct a [`group-leave`](ref:e2e.group-leave) (wrapped by [`group-member-container`](ref:e2e.group-member-container)) 3. Discard the message and abort these steps. 4. If the sender is not a member of the group: 1. If the user is the creator of the group, run the _Bundled Messages Send Steps_ with the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `receivers` set to the sender, - to construct a [`group-setup`](ref:e2e.group-setup) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) with an empty members set 2. If the user is not the creator of the group, run the _Group Sync Request Steps_, then discard the message and abort these steps. 3. Discard the message and abort these steps. This rule and any exceptions will be referenced/defined explicitly for each message. Note that steps are not allowed to discard messages from blocked contacts prior to running these steps if the message alters group state (group control messages), or is stateful (i.e. introduces a poll, poll vote, or a group call). ### Periodic Group Sync When the creator of a group... - is about to send a group conversation message, or - did just receive a group conversation message, it must trigger a _group sync_ for this group if the last time the _group sync_ was triggered is more than seven days ago. When a _group sync_ is triggered, the creator assumes it has received a [`group-sync-request`](ref:e2e.group-sync-request) from every group member and must now respond accordingly to each member of the group. A newly created group counts as an initial _group sync_ trigger. In other words, the first group sync of a newly created group triggers seven days in the future when one of the above described conditions is met. This provides a form of self-healing in case a device lost its group state (e.g. due to a backup restore) and was unable to correct this mischief. [//]: # "TODO(SE-40): Group states" ### Blobs Since messages have a strict maximum size limitation, large binary blobs are uploaded to the blob server. Blobs currently have a maximum size of 100 MiB. When Multi-Device is activated, all Blobs must be downloaded via the respective Blob Mirror unless explicitly stated otherwise. The following steps are defined as the _Blob Credentials Refresh Steps_: 1. Let `credentials` be the most recently used cached blob credentials. 2. If `credentials` is defined and is not yet expired, return `credentials`. 3. Request Blob Server Credentials via the Directory Server API and set `credentials` to the result. If no credentials could be obtained within 10s or the request was unsuccessful, exceptionally abort these steps. 4. Cache `credentials` with the provided expiration date. 5. Return `credentials`. #### Upload The following steps are defined as the _Blob Upload Steps_: 1. Let `blob` be the following properties: - `data` being the encrypted binary data to be uploaded, - `scope` being either _public_ (for public facing blobs) or _local_ (for device group facing blobs). - `persist` being a mark (primarily for usage within groups). 2. Run the _Blob Credentials Refresh Steps_ and let `credentials` be the result. 3. (non-MD) Upload to the blob server from `blob` and `credentials`. 4. (MD) Upload to the blob mirror server from `blob`, `credentials` and the device group information. 5. If the upload stream stalls for more than 10s, exceptionally abort these steps. 6. Return the resulting blob ID. #### Download The following steps are defined as the _Blob Download Steps_: 1. Let `blob` be the following properties: - `id` being the blob ID, - `scope` being either _public_ (for public facing blobs) or _local_ (for device group facing blobs). 2. Run the _Blob Credentials Refresh Steps_ and let `credentials` be the result. 3. (non-MD) Download the blob data from the blob server using `blob` and `credentials`. 4. (MD) Download the blob data from the blob mirror server using `blob`, `credentials` and the device group information. 5. If the download stream stalls for more than 10s, exceptionally abort these steps. 6. Schedule a volatile background task to run the following steps: 1. Run the _Blob Credentials Refresh Steps_ and let `credentials` be the result. 2. (non-MD) Request to mark the blob download from the blob server of `blob.id` as _done_ with `credentials`. 3. (MD) Request to mark the blob download from the blob mirror server of `blob.id` as _done_ with `credentials` and the device group information. 4. If no response could be obtained within 10s or the request was unsuccessful, log a warning. 7. Return the resulting blob data. ### Image, Audio, Video vs. File Images, as well as audio and video sources can be either send as special media messages or as files. When sending as a file, i.e. a [`file`](ref:e2e.file) message struct with rendering type `0x00` (file), no transcoding is necessary and no media type restrictions apply. Clients should intelligently choose between a media message and a file message but always leave the final choice to the user. The following sections describe what restrictions apply and modifications need to be made in case the source is sent as a media message, i.e. one of the specialised (deprecated) media structs or a [`file`](ref:e2e.file) message struct with rendering type `0x01` (media) or `0x02` (sticker). ### Images Images must be in JPEG format for the legacy [`deprecated-image`](ref:e2e.deprecated-image) message and for profile pictures. When using the [`file`](ref:e2e.file) message struct, the following media types are explicitly supported: - image/gif - image/jpeg - image/png - image/webp The following media types are explicitly not supported: - image/svg+xml Other media types _may_ be supported. Keep the format when resizing images or creating thumbnails, if possible (e.g. if the source is a JPEG, make the thumbnail a JPEG). When the format cannot be kept, use PNG for source images with transparency or lossless encoding (e.g. screenshots) and JPEG for images without transparency or lossy encoding (e.g. photos). Recommended maximum dimensions: - Small: 640x640 - Medium: 1024x1024 - Large: 1600x1600 - Extra Large: 2592x2592 - Original: As is ### Thumbnails Apply the logic described for images to all thumbnails with recommended maximum dimensions of 512x512. ### User Profile Distribution The shareable part of the user profile consists of the user's public nickname and the profile picture: - The nickname is sent along with outgoing messages as `sender-nickname` inside the [`legacy-message`](ref:payload.legacy-message) or as part of the metadata in [`message-with-metadata-box`](ref:payload.message-with-metadata-box). - The profile picture is distributed as described in [Profile Picture Distribution](ref:e2e#profile-picture-distribution). Whether user profile distribution should be triggered by an outgoing message is specified in the description of every message type below. ### Profile Pictures Apply the logic described for images to all profile pictures with recommended maximum dimensions of 512x512 with a square aspect ratio. #### Profile Picture Distribution Every time a message is being sent to a specific contact or a group of contacts, the sender needs to evaluate whether the profile picture needs to be sent. If the receiver of the message is a group, the evaluation needs to be done for each contact of that group. The following steps are defined as the _Profile Picture Distribution Steps_: 1. Let `type` be a message type associated to a message that is about to be sent. 2. Let `receiver` be the receiver of the message. 3. If the properties associated to `type` do not require _User Profile Distribution_, abort these steps without a message. 4. If `receiver`'s Threema ID is `ECHOECHO` or a Threema Gateway ID (starts with a `*`), abort these steps without a message. 5. Let `cache` be the most recently distributed profile picture message variant towards `receiver`, being either - a blob ID if the user's profile picture was distributed via a [`set-profile-picture`](ref:e2e.set-profile-picture) message, or - a _remove_ mark and a timestamp if the user's profile picture was removed via a [`delete-profile-picture`](ref:e2e.delete-profile-picture) message. 6. If the user has no profile picture or the settings indicate that the user's profile picture should be shared with nobody or the settings associated to `receiver` indicate that the profile picture should not be distributed to it: 1. If `cache` is a _remove_ mark and indicates that the most recent [`delete-profile-picture`](ref:e2e.delete-profile-picture) message towards `receiver` was sent less than seven days ago, abort these steps without a message. 2. Update the `cache` for `receiver` to a _remove_ mark with the current timestamp. 3. Return the following properties: - `id` being a random message ID, - `created-at` set to the current timestamp, - `receivers` set to `receiver`, - to construct a [`delete-profile-picture`](ref:e2e.delete-profile-picture). 7. If there is a cached profile picture of the user and the associated blob was uploaded more than seven days ago, remove the cached profile picture. 8. If `cache` contains a blob ID of the most recent [`set-profile-picture`](ref:e2e.set-profile-picture) message sent towards `receiver` that equals the blob ID of the cached profile picture, abort these steps without a message. 9. If there is no cached profile picture, encrypt the user's profile picture with a random symmetric key and upload it to the blob server. Store the key and the resulting blob ID as the cached profile picture of the user. 10. Update the `cache` for `receiver` to the blob ID of the cached profile picture. 11. Return the following properties: - `id` being a random message ID, - `created-at` set to the current timestamp, - `receivers` set to `receiver`, - to construct a [`set-profile-picture`](ref:e2e.set-profile-picture) message using the cached profile picture's blob ID and key. When the user changes the profile picture, run the steps associated to update the user's profile with the new profile picture or the newly removed profile picture. #### Profile Picture Sharing Settings In the client settings, there are three profile picture sharing options that the user can choose from: - Share with nobody - Share with everybody you write to - Share with selected contacts only The default is to share the profile picture with everyone. #### Contact Profile Picture Precedence There are three different sources of profile pictures, ordered by precedence: 1. _contact-defined_: Set by the contact, distributed through a [`set-profile-picture`](ref:e2e.set-profile-picture) message. 2. _gateway-defined_: Set by the creator in the Threema Gateway (or Threema Broadcast) control panel and distributed through `avatar.threema.ch`. Only applicable to Threema Gateway IDs (starting with a `*`). 3. _user-defined_: Set by the app user for this contact or imported from the address book. Applicable to all Threema IDs which are not Threema Gateway IDs. The following steps are defined as _Contact Profile Picture Selection Steps_ and will be applied to determine the contact's profile picture that should be displayed: 1. Let `id` be the Threema ID of the contact. 2. If the _contact-defined_ picture is set for the contact, apply it and abort these steps. 3. If `id` starts with a `*` (is a Threema Gateway ID) and the _gateway-defined_ picture is set for the contact, apply it and abort these steps. 4. If `id` does not start with `*` and the _user-defined_ picture is set for the contact, apply it and abort these steps. 5. Apply a fallback picture. #### Recurring Gateway Contact Profile Picture Refresh For contacts with a Threema Gateway ID (starting with a `*`), the profile picture needs to be fetched recurringly: 1. Fetch the profile picture for the ID from `avatar.threema.ch`. 2. If no profile picture could be found, schedule the next refresh in 24h and abort these steps. 3. Store the profile picture as the _gateway-defined_ picture. 4. Schedule the next refresh according to the `expires` header of the HTTP response. 5. Run the _Contact Profile Picture Selection Steps_ for this contact. ### Audio Audio must be in AAC format. If the source is already in AAC, no transcoding is necessary. Otherwise, the recommended transcoding settings are: Bitrate 128 kbit/s, 2 channels. When recording audio (i.e. a voice message), the recommended recording settings are: Sample rate 44.1 kHz, bitrate 32 kbit/s, 1 channel. ### Video Videos must be encoded in H.264 and the MP4 container format. Recommended encoding settings for all videos: - Low: 480x480, scale by maintaining aspect ratio to nearest multiple of 16px. Video bitrate 384 kbit/s, audio bitrate 32 kbit/s (2 channels). Baseline Profile, Level 3.1. - High: 848x848, scale by maintaining aspect ratio to nearest multiple of 16px. Video bitrate 1500 kbit/s, audio bitrate 64 kbit/s (2 channels). Baseline Profile, Level 3.1. - Original: As is. Still needs transcoding in case a different codec has been used. When recording a video, the following recording settings are recommended to avoid post-reencoding: 1280x720 / 720x1280. Video bitrate 2000 kbit/s at 30 fps, audio bitrate 128 kbit/s (2 channels). ### Call Features Call features are transmitted within either a [`call-offer`](ref:e2e.call-offer) or a [`call-answer`](ref:e2e.call-answer) message. It is an optional object containing the below defined fields. If the object is not provided, assume an empty features object. - Video Support (`'video'`): Set this field to `null` or an empty object if video calls are enabled. If either side omits this field, video support is disabled for the upcoming call. ### Application Entrypoints #### Work Credentials URL The following steps are defined as _Application Work Credentials URL Entrypoint Steps_ and must be run when the application is built for the _Work_ flavour and is invoked by a Work credentials URL: 1. Decode the Work credentials URL and let `work-credentials` be the result. 2. Run the _Common Application Entrypoint Steps_ with `work-credentials`. #### OnPrem Server/License URL The following steps are defined as _Application OnPrem Server/License URL Entrypoint Steps_ and must be run when the application is built for the _OnPrem_ flavour and is invoked via an OnPrem server/license URL: 1. Decode the OnPrem server/license URL¹ and let `on-prem-server-url` and `work-credentials` be the result. 2. Run the _Common Application Entrypoint Steps_ with `on-prem-server-url` and `work-credentials`. ¹: While the OnPrem server URL only provides the path to the OPPF file, the OnPrem license URL additionally provides the Work credentials. #### Default The following steps are defined as the _Common Application Entrypoint Steps_ and resemble the default entrypoint if no specific entrypoint was invoked by a user interaction: 1. If identity data exists: 1. (OnPrem: Refresh the OPPF file and apply its configuration to the application. TODO(SE-137): Specify more clearly.) 2. [...] 3. Abort these steps. 2. (Identity data is missing at this point.) 3. (If the application is built for the _Consumer_ flavour, request/verify the license. TODO(SE-137): Specify more clearly.) 4. If the application is built for the _Work_ flavour: 1. Let `work-credentials` be the provided parameters. 2. If `work-credentials` is not defined, request the user to provide this information and update `work-credentials` with the result. 3. (Verify the Work license. TODO(SE-137): Specify more clearly.) 5. If the application is built for the _OnPrem_ flavour: 1. Let `on-prem-server-url` and `work-credentials` be the provided parameters. 2. If the application is built for the regular _OnPrem_ flavour: 1. If the MDM parameter `th_onprem_server` is defined: 1. If `on-prem-server-url` is defined and its canonical¹ representation does not equal the canonical¹ representation of the MDM parameter `th_onprem_server`, show an error to the user that an incorrect OnPrem server/license URL has been used and abort these steps. 2. Set `on-prem-server-url` to the MDM parameter `th_onprem_server`. 2. If `on-prem-server-url` or `work-credentials` is not defined, request the user to provide the missing information and update `on-prem-server-url` and `work-credentials` with the result. 3. If the application is built for the _White-Labeled OnPrem_ flavour²: 1. If `on-prem-server-url` is defined and its canonical¹ representation does not equal the canonical¹ representation of the preconfigured OnPrem server URL, show an error to the user that an incorrect OnPrem server/license URL has been used and abort these steps. 2. Set `on-prem-server-url` to the preconfigured OnPrem server URL. 3. If `work-credentials` is not defined, request the user to provide the Work credentials and update `work-credentials` with the result. 4. (Verify the OnPrem/Work license. TODO(SE-137): Specify more clearly.) 6. Run the _Application Setup Steps_. ¹: The canonical URL is constructed by appending `/prov/config.oppf` if the URL does not end with `.oppf`. ²: Note that the `th_onprem_server` parameter is intentionally being ignored in this case. ### Application Setup The following steps are defined as _Application Setup Steps_ and must be run when no identity data exists (i.e. the application is installed for the first time or the Threema ID and associated identity data has been removed): 1. [...] 2. (The application allows to create a new Threema ID or restore a backup here. TODO(SE-137): Specify more clearly.) 3. If OnPrem sub-flavour and the MDM parameter `th_enable_remote_secret` is `true`, run the _Remote Secret Activate Steps_. 4. If application state has not been set up by the _Device Join Protocol_ (meaning that multi-device is deactivated), run the following steps: 1. [...] 2. Update the user's feature mask on the directory server. 3. Let `contacts` be the list of all contacts, including those with an acquaintance level different than _direct_. 4. Call the _Work Sync_ endpoint with `contacts` and update `contacts` and the settings with the result. 5. Refresh the state, type and feature mask of all `contacts` from the directory server and make any changes persistent. 6. Let `solicited-contacts` be a copy of `contacts` filtered in the following way. For each `contact`: 1. If the `contact`'s activity state is _invalid_ (i.e. it does not exist or has been revoked), remove `contact` from the list and abort these sub-steps. 2. If `contact` is part of a group that is not marked as _left_, add `contact` to the list and abort these sub-steps. 3. Lookup the 1:1 conversation with `contact` and let `last-update` be the associated _last update_ timestamp. 4. If `last-update` is defined, add `contact` to the list and abort these sub-steps. 5. Remove `contact` from the list. 7. If FS is supported by the client, run the _FS Refresh Steps_ with `solicited-contacts`. 8. For each `contact` of `solicited-contacts` run the _Bundled Messages Send Steps_ with the following properties: - `id` being a random message ID, - `created-at` set to the current timestamp, - `receivers` set to `contact`, - to construct a [`contact-request-profile-picture`](ref:e2e.contact-request-profile-picture) 9. For each group not marked as _left_: 1. If the user is the creator of the group, trigger a _group sync_ for that group. 2. If the user is not the creator of the group, run the _Group Sync Request Steps_ for the group. 10. [...] 5. Commit the application state with the updated `contacts` and settings, outer storage potentially protected by a passphrase (if provided) and inner storage potentially protected by RS (if created). ### Application Update The following steps are defined as _Application Update Steps_ and must be run as a persistent task when the application has just been updated to a new version or downgraded to a previous version: 1. [...] 2. Update the user's feature mask on the directory server. 3. Let `contacts` be the list of all contacts (regardless of the acquaintance level). 4. Refresh the state, type and feature mask of all `contacts` from the directory server and make any changes persistent. 5. For each `contact` of `contacts`: 1. If an associated FS session with `contact` exists and any of the FS states is unknown or any of the stored FS versions (local or remote) is unknown, terminate the FS session by running the _Bundled Messages Send Steps_ with the following properties: - `id` being a random message ID, - `created-at` set to the current timestamp, - `receivers` set to `contact`, - to construct a `csp-e2e-fs.Terminate` message with cause `RESET`. 6. [...] Note: Reactivation of FS due to disabling multi-device should run the _Application Setup Steps_ step 2.2. through 2.6. TODO(SE-199): This note will be removed once multi-device supports FS. ### Application Start The following steps are defined as _Application Start Steps_ and must be run as a blocking task when the application starts before running any further sequences: 1. [...] 2. Attempt to unlock the outer storage, potentially protected by a passphrase (if provided). 3. If the Remote Secret feature is active, run the _Remote Secret Monitor Steps_ until it yields RS and let it continue as a volatile background task bound to the application. 4. Unlock the inner storage, optionally protected by RS (if activated). 5. [...] 6. Initialise the application from the unlocked storage. container: _group: Header _doc: |- Contains an end-to-end encrypted message. fields: - _doc: |- Type of the message (`common.CspE2eMessageType`). name: type type: u8 - _doc: |- Inner message. Needs to be parsed according to the `type` field. Padded with a random amount from 1 to 255 bytes in [PKCS#7 format](https://datatracker.ietf.org/doc/html/rfc5652#section-6.3). Additionally, for security reasons, the total size of `padded-data` should be at least 32 bytes, to avoid leaking information about the contents. Example padding (hex representation): - 1 byte: `01` - 3 bytes: `030303` - 10 bytes: `0A0A0A0A0A0A0A0A0A0A` To add padding without information leaks, run the following steps: 1. Let `data` be the data to be padded. 2. Let `pad-length` be a random number between (inclusive) 1 and 255. 3. If the sum of the byte length of `data` and `pad-length` is less than 32, update `pad-length` so the sum is precisely 32. 4. Let `pad-byte` be the encoded unsigned 8-bit integer representation of `pad-length`. 5. Let `padded-data` be the padded data by adding `pad-length` trailing `pad-byte` bytes to `data`. To remove padding: 1. Let `pad-length` be the decoded unsigned 8-bit integer representation of the last byte of `padded-data`. 2. Let `data` be the unpadded data by ignoring the trailing `pad-length` bytes of `padded-data`. name: padded-data type: b* group-creator-container: _group: Header _doc: |- Container that is wrapped around some special group messages sent by the creator to normal group members and vice versa. fields: - _doc: |- 8 byte random group ID. Uniquely identifies the group when combined with the creator's Threema ID. name: group-id type: *group-id - _doc: |- Inner message struct. name: inner-data type: b* group-member-container: _group: Header _doc: |- Container that is wrapped around most messages sent by group members to other group members. fields: - _doc: |- The group creator's Threema ID. name: creator-identity type: *identity - _doc: |- 8 byte random group ID assigned to the group by the creator. name: group-id type: *group-id - _doc: |- Inner message struct. name: inner-data type: b* empty: _doc: |- An empty message (duh). Only used when encapsulated by an `csp_e2e_fs.Envelope` to announce a new FS version without explicit renegotiation. **Properties**: - Kind: 1:1 - Flags: None - User profile distribution: No - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: No - Outgoing: No - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Send to Threema Gateway ID group creator: N/A When the user submits this message in a 1:1, group or distribution list, _Rick Astley - Never Gonna Give You Up_ must be played and looped indefinitely on the user's device. When receiving this message, a mechanical finger must be unlatched from the device and boop the receiving user on the nose. text: _group: Conversation Messages _doc: |- A text message. **Properties (1:1)**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: Yes - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: Text - Deletable by: User and sender - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: N/A - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: Text - Deletable by: User and sender - Send to Threema Gateway ID group creator: If capture is enabled When the user submits this message in a 1:1, group or distribution list conversation: 1. Split the provided text into multiple segments with at most 6000 UTF-8 encoded bytes while preserving grapheme clusters at boundaries and let `text-segments` be the result.¹ 2. Run the _Messages Submit Steps_ with `messages` set from `text-segments` (for a group, wrapped by [`group-member-container`](ref:e2e.group-member-container)). ¹: The UI may warn the user that the message will be split if the UTF-8 encoded text exceeds 6000 bytes and request confirmation before submission. When reflected from another device as an incoming or outgoing 1:1 message: 1. Add the message to the associated 1:1 conversation. When receiving this message as a 1:1 message: 1. Add the message to the associated 1:1 conversation. When reflected from another device as an incoming or outgoing group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Add the message to the associated group conversation. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Add the message to the associated group conversation. fields: - _doc: |- UTF-8 encoded text. name: text type: b* deprecated-image: _group: Conversation Messages _doc: |- An image message. Note: This message is deprecated and may be phased out eventually. When sending images, use the [`file`](ref:e2e.file) message with the rendering type `0x01` (media). **Properties**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: N/A - _Sent_ update: Yes - Delivery receipts: Yes - Reactions: Yes - When rejected: N/A (deprecated message is not being sent) - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: N/A The image must be in JPEG format, is uploaded to the blob server and encrypted by: XSalsa20-Poly1305( key=X25519HSalsa20(<sender.CK>.secret, <receiver.CK>.public), nonce=<deprecated-image.nonce>, ) This message is deprecated and may no longer be submitted. When reflected from another device as an incoming message: 1. Run the _Common Deprecated Image Receive Steps_. When receiving this message: 1. Run the _Common Deprecated Image Receive Steps_. The following steps are defined as the _Common Deprecated Image Receive Steps_: 1. Add the message to the associated 1:1 conversation. 2. If this message is eligible for auto-download, schedule downloading the image data from the blob server and request the blob to be removed. fields: - _doc: |- Blob ID to obtain the image data. name: image-blob-id type: *blob-id - _doc: |- Image size in bytes. name: image-size type: u32-le - _doc: |- Random nonce used to encrypt the image data. name: nonce type: *nonce deprecated-group-image: _group: Conversation Messages _doc: |- An image message (only used by groups). Note: This message is deprecated and may be phased out eventually. When sending images, use the [`file`](ref:e2e.file) message with the rendering type `0x01` (media). **Properties**: - Kind: Group - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: N/A - _Sent_ update: Yes - Delivery receipts: N/A - Reactions: Yes - When rejected: N/A (deprecated message is not being sent) - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: If capture is enabled The image must be in JPEG format, is uploaded to the blob server and encrypted by: XSalsa20-Poly1305(key=<deprecated-group-image.key>, nonce=00..01) This message is deprecated and may no longer be submitted. When reflected from another device as an incoming message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the _Common Deprecated Group Image Receive Steps_. When receiving this message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Run the _Common Deprecated Group Image Receive Steps_. The following steps are defined as the _Common Deprecated Group Image Receive Steps_: 1. Add the message to the associated group conversation. 2. If this message is eligible for auto-download, schedule downloading the image data from the blob server but do not request the blob to be removed. fields: - _doc: |- Blob ID to obtain the image data. name: image-blob-id type: *blob-id - _doc: |- Image size in bytes. name: image-size type: u32-le - _doc: |- Random symmetric key used to encrypt the image data. name: key type: *key location: _group: Conversation Messages _doc: |- A location message. **Properties (1:1)**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: Yes - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: N/A - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: If capture is enabled When the user submits this message in a 1:1, group or distribution list conversation: 1. Let `location` be the provided location with the following properties: - `latitude` being a WGS-84 string, - `longitude` being a WGS-84 string, - `accuracy` being a floating point number or undefined, - `address` being an address of a point of interest or undefined, - `name` being a name for the point of interest or undefined, 2. If `location.name` is defined but `location.address` is not defined, discard `location`, log an error and abort these steps.¹ 3. If the UTF-8 encoded bytes of `location.address` or `location.name` exceed 512 bytes, discard `location`, log an error and abort these steps.¹ 4. Run the _Messages Submit Steps_ with `messages` set from `location` (for a group, wrapped by [`group-member-container`](ref:e2e.group-member-container)). ¹: The UI should prevent submission in these cases. When reflected from another device as an incoming or outgoing 1:1 message: 1. Add the message to the associated 1:1 conversation. When receiving this message as a 1:1 message: 1. Add the message to the associated 1:1 conversation. When reflected from another device as an incoming or outgoing group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Add the message to the associated group conversation. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Add the message to the associated group conversation. fields: - _doc: |- Location coordinates and meta information encoded in comma- and line-separated UTF-8: <latitude>,<longitude>[,<accuracy>] or <latitude>,<longitude>[,<accuracy>] <address> or <latitude>,<longitude>[,<accuracy>] <name> <address> Values: - `latitude` and `longitude` are the geographic coordinates represented in a WGS-84 string. - `accuracy` is the accuracy in meters represented by a floating point number formatted as a string. - `address` is a full address. If it contains multiple lines, each line feed must be escaped (literal `\n`). - `name` is the name of a point of interest. _Latitude_ and _longitude_ must always be present while _accuracy_ is optional and should only be provided when the current location of the device is being sent. These values are comma-separated. Following values are optional and separated by line feeds (`\n`). This may be either: - a single line containing the _address_ representing the closest address matching the coordinates, or - two lines containing a point of interest _name_ and _address_ (which means that the coordinates refer to the point of interest). name: location type: b* deprecated-audio: _group: Conversation Messages _doc: |- An audio message. Note: This message is deprecated and may be phased out eventually. When sending audio, use the [`file`](ref:e2e.file) message with the rendering type `0x01` (media). **Properties (1:1)**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: N/A - _Sent_ update: Yes - Delivery receipts: Yes - Reactions: Yes - When rejected: N/A (deprecated message is not being sent) - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: N/A - _Sent_ update: Yes - Delivery receipts: N/A - Reactions: Yes - When rejected: N/A (deprecated message is not being sent) - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: If capture is enabled The audio is uploaded to the blob server and encrypted by: XSalsa20-Poly1305(key=<deprecated-audio.key>, nonce=00..01) This message is deprecated and may no longer be submitted. When reflected from another device as an incoming 1:1 message: 1. Run the _Common Deprecated Audio Receive Steps_. When receiving this message as a 1:1 message: 1. Run the _Common Deprecated Audio Receive Steps_. When reflected from another device as an incoming group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the _Common Deprecated Audio Receive Steps_. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Run the _Common Deprecated Audio Receive Steps_. The following steps are defined as the _Common Deprecated Audio Receive Steps_: 1. Add the message to the associated conversation. 2. If this message is eligible for auto-download, schedule downloading the audio data from the blob server. Only request the blob to be removed if the associated conversation is a 1:1 conversation. fields: - _doc: |- Audio duration in seconds. name: duration type: u16-le - _doc: |- Blob ID to obtain the audio data. name: audio-blob-id type: *blob-id - _doc: |- Audio size in bytes. name: audio-size type: u32-le - _doc: |- Random symmetric key used to encrypt the audio data. name: key type: *key deprecated-video: _group: Conversation Messages _doc: |- A video message. Note: This message is deprecated and may be phased out eventually. When sending video, use the [`file`](ref:e2e.file) message with the rendering type `0x01` (media). **Properties (1:1)**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: N/A - _Sent_ update: Yes - Delivery receipts: Yes - Reactions: Yes - When rejected: N/A (deprecated message is not being sent) - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: N/A - _Sent_ update: Yes - Delivery receipts: N/A - Reactions: Yes - When rejected: N/A (deprecated message is not being sent) - Edit applies to: N/A - Deletable by: User and sender - Send to Threema Gateway ID group creator: If capture is enabled The video is uploaded to the blob server and encrypted by: XSalsa20-Poly1305(key=<deprecated-video.key>, nonce=00..01) The thumbnail must be in JPEG format, is uploaded to the blob server and encrypted by: XSalsa20-Poly1305(key=<deprecated-video.key>, nonce=00..02) This message is deprecated and may no longer be submitted. When reflected from another device as an incoming 1:1 message: 1. Run the _Common Deprecated Audio Receive Steps_. When receiving this message as a 1:1 message: 1. Run the _Common Deprecated Audio Receive Steps_. When reflected from another device as an incoming group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the _Common Deprecated Audio Receive Steps_. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Run the _Common Deprecated Audio Receive Steps_. The following steps are defined as the _Common Deprecated Video Receive Steps_: 1. Add the message to the associated conversation. 2. If this message is eligible for auto-download, schedule downloading the video data from the blob server. Only request the blob to be removed if the associated conversation is a 1:1 conversation. fields: - _doc: |- Video duration in seconds. name: duration type: u16-le - _doc: |- Blob ID to obtain the video data. name: video-blob-id type: *blob-id - _doc: |- Video size in bytes. name: video-size type: u32-le - _doc: |- Blob ID to obtain the thumbnail in JPEG format. name: thumbnail-blob-id type: *blob-id - _doc: |- Thumbnail size in bytes. name: thumbnail-size type: u32-le - _doc: |- Random symmetric key used to encrypt the video and thumbnail data. name: key type: *key file: _group: Conversation Messages _doc: |- A file or media message. **Properties (1:1)**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: Yes - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: Caption - Deletable by: User and sender - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: N/A - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: Caption - Deletable by: User and sender - Send to Threema Gateway ID group creator: If capture is enabled The file is uploaded to the blob server and encrypted by: XSalsa20-Poly1305(key=<file.key>, nonce=00..01) The thumbnail is uploaded to the blob server and encrypted by: XSalsa20-Poly1305(key=<file.key>, nonce=00..02) When the user submits this message in a 1:1, group or distribution list conversation: 1. Let `file` be all necessary properties to construct this message. 2. If the UTF-8 encoded bytes of `file.name` exceed 256 bytes, discard `file`, log an error and abort these steps.¹ 3. If the UTF-8 encoded bytes of `file.caption` exceed 6000 bytes, discard `file`, log an error and abort these steps.¹ 4. Run the _Messages Submit Steps_ with `messages` set from `file` (for a group, wrapped by [`group-member-container`](ref:e2e.group-member-container)). ¹: The UI should prevent submission in these cases. When reflected from another device as an incoming or outgoing 1:1 message: 1. Run the _Common File Receive Steps_. When receiving this message as a 1:1 message: 1. Run the _Common File Receive Steps_. When reflected from another device as an incoming or outgoing group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the _Common File Receive Steps_. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Run the _Common File Receive Steps_. The following steps are defined as the _Common File Receive Steps_: 1. Add the message to the associated conversation. 2. Schedule auto-downloading the thumbnail data from the blob server. Only request the blob to be removed if the associated conversation is a 1:1 conversation. 3. If this message is eligible for auto-download, schedule downloading the file data from the blob server. Only request the blob to be removed if the associated conversation is a 1:1 conversation. fields: - _doc: |- UTF-8, JSON-encoded object with the following fields: - Rendering type (`'j'`): - `0`: Render as a file. - `1`: Render as media (e.g. an image, audio or video). - `2`: Render as a sticker (for transparent images). If this field is not set, fall back to the value of `'i'`. If no value could be determined or the rendering type is unassigned, assume `0`. - Deprecated (`'i'`): Set this to the integer `1` if the rendering type is `1` or `2`, otherwise set this to the integer `0`. - Encryption key (`'k'`): Random symmetric key used to encrypt the blobs (file and thumbnail data) in lowercase hex string. - File blob ID (`'b'`): Blob ID in lowercase hex string representation to obtain the file data. - File media type (`'m'`): The media type of the file. - File name (`'n'`): Optional filename of the file. - File size (`'s'`): File size in bytes. - Thumbnail Blob ID (`'t'`): Optional blob containing the thumbnail file data. - Thumbnail media type (`'p'`): Media type of the thumbnail. If not set, assume `image/jpeg`. - Caption (`'d'`): Optional caption text. - Correlation ID (`'c'`): Optional random 32 byte ASCII string to collocate multiple media files. - Metadata (`'x'`): An optional metadata object as defined below. Metadata object fields depend on the media type of the file. All fields are optional but recommended to set in order to determine the layout logic while the file is being downloaded. Once the file has been parsed, the parsed data supersedes this object. For images: - Width (`'w'`): The width as an integer in px. - Height (`'h'`): The height as an integer in px. - Animated (`'a'`): Set this to the boolean `true` if the image is animated (e.g. an animated GIF). For audio: - Duration (`'d'`): The duration as a float in seconds. For video: - Width (`'w'`): The width as an integer in px. - Height (`'h'`): The height as an integer in px. - Duration (`'d'`): The duration as a float in seconds. Note that the rendering logic depends on three key fields which should be set accordingly: - Media type, - Rendering type, - Animated flag in the metadata object. name: file type: b* poll-setup: _group: Conversation Messages _doc: |- Creates a new poll or finalises an existing poll. During the lifecycle of a poll, this message will be used exactly twice: Once to create the poll, and once to close it. **Properties (1:1)**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: Yes - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: N/A - Deletable by: User only (TODO(SE-383)) - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: - `0x01`: Send push notification. - User profile distribution: Yes - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: Yes - Delivery receipts: N/A - Reactions: Yes - When rejected: Re-send after confirmation - Edit applies to: N/A - Deletable by: User only (TODO(SE-383)) - Send to Threema Gateway ID group creator: If capture is enabled When the user submits this message in a 1:1 or group conversation: 1. Let `poll` be all necessary properties to construct this message. 2. If the UTF-8 encoded bytes of `poll.description` exceed 256 bytes, discard `poll`, log an error and abort these steps.¹ 3. If the UTF-8 encoded bytes of any of the `poll.choices` exceeds 256 bytes, discard `poll`, log an error and abort these steps.¹ 4. JSON encode `poll` and let `encoded-poll` be the UTF-8 encoded result. 5. If `encoded-poll` exceeds 6000 bytes, discard `poll` (and `encoded-poll`), log an error and abort these steps.¹ 6. Run the _Messages Submit Steps_ with `messages` set from `encoded-poll` (for a group, wrapped by [`group-member-container`](ref:e2e.group-member-container)). ¹: The UI should prevent submission in these cases. When reflected from another device as an incoming or outgoing 1:1 message: 1. Run the _Common Poll Setup Receive Steps_. When receiving this message as a 1:1 message: 1. Run the _Common Poll Setup Receive Steps_. When reflected from another device as an incoming or outgoing group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the _Common Poll Setup Receive Steps_. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Run the _Common Poll Setup Receive Steps_. The following steps are defined as the _Common Poll Setup Receive Steps_: 1. Let `state` be the _State_ field of the message. Let `participants` be the _Participants_ field of the message. 2. Look up the poll with the given ID within the conversation. 3. If no associated poll could be found: 1. If `state` is `1` (closed), discard the message and abort these steps. 2. Add the poll to the associated conversation with the provided fields of the message and abort these steps. 4. If the associated poll is closed, discard the message and abort these steps. 5. If `state` is `0` (open), discard the message and abort these steps. 6. Close the poll with the given `participants`, ignore any other fields of the message. fields: - _doc: |- Random unique (per creator within this conversation) ID of the poll. name: id type: *poll-id - _doc: |- UTF-8, JSON-encoded object with the following fields: - Description (`'d'`): A short description/topic string for the poll. - State (`'s'`): - `0`: Poll is _open_ for votes. - `1`: Poll has been _closed_. A state transition from _closed_ to _open_ is illegal and must be ignored by the receiving client. - Answer type (`'a'`): - `0`: Single choice poll. - `1`: Multiple choice poll. Any transition from one of the types to another is illegal and must be ignored by the receiving client. - Announce type (`'t'`): - `0`: Announce votes in form of the `poll-vote` message only to the creator of the ballot. Results are invisible until the creator closes the vote and reports the final results. - `1`: Announce votes in form of the `poll-vote` message to everyone in the conversation. Interim results are therefore visible to everyone. Any transition from one of the types to another is illegal and must be ignored by the receiving client. - Display mode (`'u'`): - `0`: List mode. List choices of all participants as presented by this message. - `1`: Summary mode. Only display the total amount of votes per choice and the user's vote (if any). If the field is not present, assume _list_ mode (`0`). Any transition from one of the modes to another is illegal and must be ignored by the receiving client. - Choices type (`'o'`, DEPRECATED): Always set this to the integer `0`. - Participants (`'p'`): A list of Threema IDs that participated in the poll (i.e. they cast a vote). This field must only be present if the poll is being _closed_. In display mode _summary_, this field should be an empty list and must be ignored by the receiver. - Choices (`'c'`): A list of choice objects as defined below. Choice object fields: - Choice ID (`'i'`): A per-poll unique ID of the choice in form of an integer. Used when casting a vote. - Description (`'n'`): Choice description in form of a string. - Sort key (`'o'`, DEPRECATED): Set this to the index of the choice object within the _choices_ list. - Participant votes (`'r'`): A list of votes for this choice in the same order as the `participants` (i.e. mapped by their associated index). The integer `0` indicates that the participant did not vote for this choice. Any integer value other than `0` indicates that the participant voted for this choice. Must be of same length as `participants`. This field must only be present if the poll is being _closed_. In display mode _summary_ this should be an empty list and must be ignored by the receiver. - Total amount of votes (`'t'`): The total amount of votes for this choice. This field must only be present if the poll is being _closed_. In display mode _normal_ this field should not be present and must be ignored by the receiver. name: poll type: b* poll-vote: _group: Conversation Messages _doc: |- Cast a vote on a poll. **Properties (1:1)**: - Kind: 1:1 - Flags: None - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No¹ - Bump _last update_: No¹ - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A (can just send another `poll-vote`) - Deletable by: User only (TODO(SE-383)) - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: None - User profile distribution: Yes - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No¹ - Bump _last update_: No¹ - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: N/A - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A (can just send another `poll-vote`) - Deletable by: User only (TODO(SE-383)) - Send to Threema Gateway ID group creator: If capture is enabled ¹: A [`poll-vote`](ref:e2e.poll-vote) updates the poll state initiated by a corresponding [`poll-setup`](ref:e2e.poll-setup), meaning it does not produce a new visible message. When the user submits this message in a 1:1 or group conversation by casting a vote: 1. Let `vote` be all necessary properties to construct this message. 2. JSON encode `vote` and let `encoded-vote` be the UTF-8 encoded result. 3. If `encoded-vote` exceeds 6000 bytes, discard `vote` (and `encoded-vote`), log an error and abort these steps.¹ 4. Run the _Messages Submit Steps_ with `messages` set from `encoded-vote` (for a group, wrapped by [`group-member-container`](ref:e2e.group-member-container)). ¹: The UI should prevent submission in these cases. When reflected from another device as an incoming or outgoing 1:1 message: 1. Run the _Common Poll Vote Receive Steps_. When receiving this message as a 1:1 message: 1. Run the _Common Poll Vote Receive Steps_. When reflected from another device as an incoming or outgoing group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the _Common Poll Vote Receive Steps_. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Run the _Common Poll Vote Receive Steps_. The following steps are defined as the _Common Poll Vote Receive Steps_. 1. Look up the poll with the given ID within the conversation. 2. If no associated poll could be found or if the associated poll is closed, discard the message and abort these steps. 3. Update the poll with the provided choices of the sender. fields: - _doc: |- The Threema ID of the creator of the poll. name: creator-identity type: *identity - _doc: |- ID of the associated poll. name: poll-id type: *poll-id - _doc: |- UTF-8, JSON-encoded list containing one or more choice tuples. Each choice tuple contains the following two integer values: - Choice ID, referring to the Choice ID defined in the `poll-setup` message. - Selected: - `0`: The choice has not been selected. - `1`: The choice has been selected. Note: For protocol simplicity, a vote must always include all possible choices, whether or not they have been selected. name: choices type: b* call-offer: _group: Conversation Messages _doc: |- Initiates a call. **Properties**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - `0x20`: Short-lived server queuing. - User profile distribution: Yes - Exempt from blocking: No - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: Yes - Bump _last update_: Yes - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: Abort call - Edit applies to: N/A - Deletable by: User only (TODO(SE-384)) - Send to Threema Gateway ID group creator: N/A [//]: # "When submit / receiving / reflected: TODO(SE-102)" fields: - _doc: |- UTF-8, JSON-encoded object with the following fields: - Call ID (`'callId'`): Random 32 bit unsigned integer greater than 0 that uniquely identifies a call throughout its lifetime. Assume `0` if not set. - WebRTC Offer (`'offer'`): An offer object. - Feature negotiation (`'features'`): Optional Call Features object. Offer object fields: - WebRTC Offer SDP type (`'sdpType'`): Set this to `'offer'` and ignore offers with other types. - WebRTC Offer SDP (`'sdp'`): Opaque string containing the SDP. name: offer type: b* call-answer: _group: Conversation Messages _doc: |- Answer or reject a call. **Properties**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - `0x20`: Short-lived server queuing. - User profile distribution: Only if accepted (`action`: `1`) - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No¹ - Bump _last update_: No¹ - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: Abort call - Edit applies to: N/A - Deletable by: User only (TODO(SE-384)) - Send to Threema Gateway ID group creator: N/A ¹: A [`call-answer`](ref:e2e.call-answer) updates the call state initiated by a corresponding [`call-offer`](ref:e2e.call-offer), meaning it does not produce a new visible message. [//]: # "When submit / receiving / reflected: TODO(SE-102)" fields: - _doc: |- UTF-8, JSON-encoded object with the following fields: - Call ID (`'callId'`): Random 32 bit unsigned integer greater than 0 that uniquely identifies a call throughout its lifetime. Assume `0` if not set. - Required action (`'action'`): - `0`: The call has been rejected and needs to be aborted. - `1`: The call has been accepted and a connection needs to be established. - Rejection reason (`'rejectReason'`): If the call has been rejected, this field contains a reject reason: - `0`: Generic or unspecified rejection. - `1`: The callee is busy (another call is active). - `2`: The callee did not accept the call in time. - `3`: The callee explicitly rejected the call. - `4`: The callee disabled calls. - `5`: The callee was called during an off-hour period. - WebRTC Answer (`'answer'`): An answer object. - Feature negotiation (`'features'`): Optional Call Features object. Answer object fields: - WebRTC Answer SDP type (`'sdpType'`): Set this to `'answer'` and ignore answers with other types. - WebRTC Answer SDP (`'sdp'`): Opaque string containing the SDP. name: answer type: b* call-ice-candidate: _group: Conversation Messages _doc: |- An ICE candidate for an ongoing call. **Properties**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - `0x20`: Short-lived server queuing. - User profile distribution: No - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: No¹ - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: No - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A ¹: This message does not trigger any kind of reaction and adding ICE candidates again has no ill-effect. [//]: # "When submit / receiving / reflected: TODO(SE-102)" fields: - _doc: |- UTF-8, JSON-encoded object with the following fields: - Call ID (`'callId'`): Random 32 bit unsigned integer greater than 0 that uniquely identifies a call throughout its lifetime. Assume `0` if not set. - Deprecated (`'removed'`): Always set this to `false` and ignore messages with this field set to `true`. - WebRTC Candidates (`'candidates'`): An array of candidate objects. Candidate object fields: - WebRTC Candidate SDP (`'candidate'`): Opaque string containing the ICE candidate SDP. - WebRTC MID (`'sdpMid'`): Media stream identification string or `null`. - WebRTC Media Line Index (`'sdpMLineIndex'`): Media description line index integer or `null`. - WebRTC Username Fragment (`'ufrag'`): ICE username fragment or `null`. name: candidates type: b* call-hangup: _group: Conversation Messages _doc: |- Hang up a call. **Properties**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - User profile distribution: No - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: If no corresponding `call-offer` can be found¹ - Bump _last update_: If no corresponding `call-offer` can be found¹ - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A - Deletable by: User only (TODO(SE-384)) - Send to Threema Gateway ID group creator: N/A ¹: A [`call-hangup`](ref:e2e.call-hangup) usually updates the call state initiated by a corresponding [`call-offer`](ref:e2e.call-offer), meaning it does not produce a new visible message. However, the [`call-offer`](ref:e2e.call-offer) uses the _Short-lived server queuing_ flag and therefore may be lost. In that case, the [`call-hangup`](ref:e2e.call-hangup) does create a visible _call missed_ message. [//]: # "When submit / receiving / reflected: TODO(SE-102)" fields: - _doc: |- UTF-8, JSON-encoded object. If this field contains zero bytes, assume an empty object. Contains the following fields: - Call ID (`'callId'`): Random 32 bit unsigned integer greater than 0 that uniquely identifies a call throughout its lifetime. Assume `0` if not set. name: hangup type: b* call-ringing: _group: Conversation Messages _doc: |- Sent by the callee to indicate that the call is ringing. **Properties**: - Kind: 1:1 - Flags: - `0x01`: Send push notification. - `0x20`: Short-lived server queuing. - User profile distribution: No - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No¹ - Bump _last update_: No¹ - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: Abort call - Edit applies to: N/A - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A ¹: A [`call-ringing`](ref:e2e.call-ringing) updates the call state initiated by a corresponding [`call-offer`](ref:e2e.call-offer), meaning it does not produce a new visible message. [//]: # "When submit / receiving / reflected: TODO(SE-102)" fields: - _doc: |- UTF-8, JSON-encoded object. If this field contains zero bytes, assume an empty object. Contains the following fields: - Call ID (`'callId'`): Random 32 bit unsigned integer greater than 0 that uniquely identifies a call throughout its lifetime. Assume `0` if not set. name: hangup type: b* delivery-receipt: _group: Status Updates _doc: |- Confirms reception or delivers detailed status updates of a message. **Properties (1:1)**: - Kind: 1:1 - Flags: None - User profile distribution: Only for reactions - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Only for reactions¹ - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes² - _Sent_ update: No - Delivery receipts: No, that would be silly! - Reactions: No (also silly) - When rejected: N/A (ignored) - Edit applies to: N/A (can just send another `delivery-receipt`) - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A ¹: Repeating a status of type _received_ or _read_ has no ill-effects. ²: When the message is being _read_ and _read_ receipts are disabled, an `d2d.IncomingMessageUpdate` will be reflected instead. **Properties (Group)**: - Kind: Group - Flags: None - User profile distribution: Only for reactions - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Only for reactions¹ - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes. When the message is being _read_ and _read_ receipts are disabled, reflect an `d2d.IncomingMessageUpdate` (since no `delivery-receipt` is sent in this case). - _Sent_ update: No - Delivery receipts: No, that would be silly! - Reactions: No (also silly) - When rejected: N/A (ignored) - Edit applies to: N/A (can just send another `delivery-receipt`) - Deletable by: N/A - Send to Threema Gateway ID group creator: If capture is enabled ¹: Repeating a status of type _received_ or _read_ has no ill-effects. When the user opens a 1:1 conversation or marks it as _read_ by another mechanism: 1. Let `message-ids` be all message IDs associated to messages of the conversation that require _automatic_ delivery receipts and which are not yet marked as _read_. 2. Split the provided `message-ids` into bundles of at most 875 message IDs and let `message-ids-bundles` be the result.¹ 3. Run the _Messages Submit Steps_ with `messages` set to create one or more [`delivery-receipt`](ref:e2e.delivery-receipt)s from `message-ids-bundles` with status `0x02`. ¹: Each message ID has 8 bytes, divided by at most 7000 bytes. When reflected from another device as an incoming 1:1 message: 1. Run the _Common Incoming Delivery Receipt Steps_. When reflected from another device as an outgoing 1:1 message: 1. Run the _Common Outgoing Delivery Receipt Steps_. When receiving this message as a 1:1 message: 1. Run the _Common Incoming Delivery Receipt Steps_. When reflected from another device as an incoming group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. If `status` is not `0x03` or `0x04` (i.e. not a reaction), log a notice, discard the message and abort these steps. 2. Run the _Common Incoming Delivery Receipt Steps_. When reflected from another device as an outgoing group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. If `status` is not `0x03` or `0x04` (i.e. not a reaction), log a notice, discard the message and abort these steps. 2. Run the _Common Outgoing Delivery Receipt Steps_. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. If `status` is not `0x03` or `0x04` (i.e. not a reaction), log a notice, discard the message and abort these steps. 3. Run the _Common Incoming Delivery Receipt Steps_. The following steps are defined as the _Common Incoming Delivery Receipt Steps_: 1. For each `message-id` of `message-ids`: 1. Lookup the associated message for `message-id` in the associated conversation and let `referred-message` be the result. 2. If `referred-message` is not defined, abort these sub-steps. 3. If the associated conversation is a 1:1 conversation and the original sender of `referred-message` is not the user, abort these sub-steps. 4. If `status` is `0x01` or `0x02` (i.e. a delivery receipt) and `referred-message` allows for delivery receipts (see the associated _Delivery receipts_ property), apply and replace the delivery receipt of the sender to `referred-message` with the associated timestamp set to the message's (the `delivery-receipt`'s) `created-at`. 5. If `status` is `0x03` or `0x04` (i.e. a reaction) and `referred-message` is reactable (see the associated _Reactions_ property), apply and replace any existing reaction of the sender to `referred-message` with the reaction timestamp set to the message's (the `delivery-receipt`'s) `created-at`.¹ The following steps are defined as the _Common Outgoing Delivery Receipt Steps_: 1. For each `message-id` of `message-ids`: 1. Lookup the associated message for `message-id` in the associated conversation and let `referred-message` be the result. 2. If `referred-message` is not defined, abort these sub-steps. 3. If the associated conversation is a 1:1 conversation and the original sender of `referred-message` is the user, abort these sub-steps. 4. If `status` is `0x01` or `0x02` (i.e. a delivery receipt) and `referred-message` allows for delivery receipts (see the associated _Delivery receipts_ property), apply and replace the delivery receipt of the user to `referred-message` with the associated timestamp set to the message's (the `delivery-receipt`'s) `created-at`. 5. If `status` is `0x03` or `0x04` (i.e. a reaction) and `referred-message` is reactable (see the associated _Reactions_ property), apply and replace any existing reaction of the user to `referred-message` with the reaction timestamp set to the message's (the `delivery-receipt`'s) `created-at`.¹ ¹: Note that the deprecated reactions transmitted by a `delivery-receipt` always replace **all existing reactions** of the respective sender, including new-style reactions transmitted by a `csp_e2e.Reaction` message. fields: - _doc: |- Message status: - `0x01`: Message was received. - `0x02`: Message was read. - `0x03`: **Deprecated** Maps to the �� emoji (`1F44D`). - `0x04`: **Deprecated** Maps to the �� emoji (`1F44E`). Note that only the `0x01` and `0x02` variants are considered true _delivery receipts_ whereas the deprecated `0x03` and `0x04` variants are considered _reactions_ (just like `csp_e2e.Reaction`). The following replacement logic is to be applied on a message's status when displayed: 1. `0x02` replaces groups listed below, 2. `0x01` replaces the unlisted _created_ status. name: status type: u8 - _doc: |- One or more `message-id`s whose status should be updated. name: message-ids type: *message-ids typing-indicator: _group: Status Updates _doc: |- Indicates whether a contact is currently typing. **Properties**: - Kind: 1:1 - Flags: - `0x02`: No server queuing. - `0x04`: No server acknowledgement. - User profile distribution: No - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: No¹ - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: No - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A ¹: It is deemed acceptable if the _typing_ indicator in the UI is replayed since there is no further consequence. When the user is currently _typing_ while composing a **new**¹ message in an associated conversation: 1. Schedule a volatile task to run the _Bundled Messages Send Steps_ with the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `receivers` set to the targeted receiver, - to construct this message with `is-typing` set to `1`. 2. Start a _user is typing_ timer in the conversation to rerun these steps in 10s. When the user stopped _typing_ while composing a message in an associated conversation, or when the user left the conversation view: 1. If no _user is typing_ timer is running for the conversation, abort these steps. 2. Stop the _user is typing_ timer of the conversation. 3. Schedule a volatile task to run the _Bundled Messages Send Steps_ with the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `receivers` set to the targeted receiver, - to construct this message with `is-typing` set to `0`. When reflected from another device as an incoming message: 1. Run the _Common Typing Indicator Receive Steps_. When receiving this message: 1. Run the _Common Typing Indicator Receive Steps_. The following steps are defined as the _Common Typing Indicator Receive Steps_: 1. If `is-typing` is `1`, start a timer to display that the sender is typing in the associated conversation for the next 15s. 2. If `is-typing` is `0`, cancel any running timer displaying that the sender is typing in the associated conversation. ¹: Editing a message may not trigger _typing_. fields: - _doc: |- Set to `1` in case the contact is currently typing or `0` in case the contact stopped typing. Other values are invalid. name: is-typing type: u8 set-profile-picture: _group: Contact and Group Control _doc: |- Set the profile picture of a contact or a group. **Properties (1:1)**: - Kind: 1:1 - Flags: None - User profile distribution: No (obviously) - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A (can just send another `set-profile-picture`) - Deletable by: N/A (can just send a `delete-profile-picture`) - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: None - User profile distribution: No (obviously) - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: N/A - Reactions: No - When rejected: N/A¹ - Edit applies to: N/A (can just send another `set-profile-picture`) - Deletable by: N/A (can just send a `delete-profile-picture`) - Send to Threema Gateway ID group creator: N/A ¹: For the group creator it will be handled as if `group-sync-request` was received, re-sending the group profile picture state, implicitly triggered by FS `Reject` receive steps. The profile picture must be in JPEG format, is uploaded to the blob server and encrypted by: XSalsa20-Poly1305(key=<set-profile-picture.key>, nonce=00..01) When reflected from another device as an outgoing 1:1 message: 1. Update the most recently distributed profile picture cache for the contact to the enclosed blob ID. When receiving this message as a 1:1 message: 1. Download the picture from the blob server but do not request the blob to be removed. Store the profile picture. 2. Store the picture as the _contact-defined_ profile picture and run the _Contact Profile Picture Selection Steps_. When receiving this message as a group message (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Download the picture from the blob server but do not request the blob to be removed. Let `profile-picture` be the result. 3. Let `group` be a snapshot of the current group state. 4. If `group.profile-picture` is defined and equals `profile-picture` (i.e. no changes), discard the message and abort these steps. 5. (MD) Run the following sub-steps: 1. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning that a group sync race occurred, discard the message and abort these steps. 2. (MD) Let `group` be a snapshot of the current group state. 3. (MD) If `group.profile-picture` is defined and equals `profile-picture`, log a warning that a group sync race occurred. 4. (MD) Reflect a `GroupSync.Update` with `group` set to contain `profile_picture` set to `profile-picture. 5. (MD) Commit the transaction and await acknowledgement. 6. Store the profile picture and and apply it to the group. fields: - _doc: |- Blob ID to obtain the image data. name: picture-blob-id type: *blob-id - _doc: |- Profile picture size in bytes. name: picture-size type: u32-le - _doc: |- Random symmetric key used to encrypt the image data. name: key type: *key delete-profile-picture: _group: Contact and Group Control _doc: |- Delete the profile picture of a contact. **Properties (1:1)**: - Kind: 1:1 - Flags: None - User profile distribution: No (obviously) - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A (can just send another `delete-profile-picture`) - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A **Properties (Group)**: - Kind: Group - Flags: None - User profile distribution: No (obviously) - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: N/A - Reactions: No - When rejected: N/A¹ - Edit applies to: N/A (can just send another `delete-profile-picture`) - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A ¹: For the group creator it will be handled as if `group-sync-request` was received, re-sending the group profile picture state, implicitly triggered by FS `Reject` receive steps. When reflected from another device as an incoming 1:1 message: 1. Remove the _contact-defined_ profile picture and run the _Contact Profile Picture Selection Steps_. When reflected from another device as an outgoing 1:1 message: 1. Update the most recently distributed profile picture cache for the contact to a _remove_ mark with the reflected timestamp. When receiving this message as a 1:1 message: 1. Remove the _contact-defined_ profile picture and run the _Contact Profile Picture Selection Steps_. When receiving this message as a group message (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Let `group` be a snapshot of the current group state. 3. If `group.profile-picture` is not defined (i.e. no change), discard the message and abort these steps. 4. (MD) Run the following sub-steps: 1. Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning that a group sync race occurred, discard the message and abort these steps. 2. Let `group` be a snapshot of the current group state. 3. If `group.profile-picture` is not defined, log a warning that a group sync race occurred. 4. Reflect a `GroupSync.Update` with `group` set to contain `profile_picture` set to be removed. 5. Commit the transaction and await acknowledgement. 5. Remove the profile picture of the group. contact-request-profile-picture: _group: Contact and Group Control _doc: |- Request a contact's profile picture. Note that this message does not result in the profile picture being sent immediately in reply to this message. Instead, it will be sent the next time that contact sends a message to the user (if one is set, and if the user is eligible for receiving the profile picture). **Properties**: - Kind: 1:1 - Flags: None - User profile distribution: No - Exempt from blocking: No - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: No - _Sent_ update: No - Delivery receipts: No - Reactions: No - When rejected: N/A (ignored) - Edit applies to: N/A - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A When reflected from another device as an incoming message: 1. Run the _Common Request Profile Picture Receive Steps_. When receiving this message: 1. Run the _Common Request Profile Picture Receive Steps_. The following steps are defined as the _Common Request Profile Picture Receive Steps_: 1. Purge the most recently distributed profile picture cache for the sender. group-setup: _group: Contact and Group Control _doc: |- Announces the group setup to all participants. The group creator is always a member of the group and must not be included in the member list. This is sent by the creator to create a new group, as well as update and disband an existing group. The group creator sends this message to all current (including those to be removed) and newly added group members. The group creator may also send this to a single receiver in special cases. Since the group creator is not allowed to leave the group, the only way for it to stop being a member is by sending a `group-setup` with an empty members list and thereby disbanding the group. **Properties**: - Kind: Group - Flags: None - User profile distribution: Yes - Exempt from blocking: See dedicated steps - Implicit _direct_ contact creation: Yes - Protect against replay: Yes - Unarchive: No¹ - Bump _last update_: No² - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: N/A - Reactions: No - When rejected: N/A³ - Edit applies to: N/A (can just send another `group-setup`) - Deletable by: N/A (can just send another `group-setup`) - Send to Threema Gateway ID group creator: N/A ¹: A newly created group's conversation visibility is implicitly _normal_ and therefore not _archived_. For the sake of simplicity and sender/receiver symmetry, further updates to the group should not alter the conversation visibility. ²: A newly created group is implicitly created with _last update_ set to the current timestamp. For the sake of simplicity and sender/receiver symmetry, no further updates to the group should bump _last update_. ³: For the group creator it will be handled as if `group-sync-request` was received, re-sending the group state, implicitly triggered by FS `Reject` receive steps. The following steps are the dedicated blocking exemption steps for this message as a group message (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)): 1. Look up the group. 2. If the group could be found, return that the message passed the blocking check. 3. Run the _Identity Blocked Steps_ for the creator. If the result indicates that the creator is not blocked, return that the message passed the blocking check. Otherwise return that the message needs to be discarded. When receiving this message as a group message (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)): 1. Let `members` be the given member list. Remove all duplicate entries from `members`. Remove the sender (creator) from `members` if present. 2. Look up the group. 3. If the group could be found: 1. Let `group` be a snapshot of the current group state. 2. If the group is marked as _left_ and `members` is empty (i.e. no change), discard the message and abort these steps. 3. If the group is not marked as _left_: 1. Let `current-members` be a copy of `group.members`. 2. Add the user to `current-members`. 3. If `current-members` equals `members` (i.e. no change), discard the message and abort these steps. 4. If `members` does not include the user: 1. If the group could not be found, discard the message and abort these steps. 2. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, discard the message and abort these steps. 3. (MD) Reflect a `GroupSync.Update` with `group` set to contain the `user_state` set to `KICKED`. 4. (MD) Commit the transaction and await acknowledgement. 5. If the user is currently participating in a group call of this group, trigger leaving the call. 6. Mark the group as _left_. 7. Persist the previous member setup so that the group can be cloned. 8. Run the _Rejected Messages Refresh Steps_ for the group. 5. If `members` includes the user. 1. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the sender (creator) contact does not exist, log an error, discard the message and abort these steps. 2. Run the _Valid Contacts Lookup Steps_ for `members` and overwrite `members` with the result. 3. For each `contact-or-init` of `members`: 1. If `contact-or-init` indicates that the _contact is the user_, remove the entry from `members` and abort these sub-steps. 2. If `contact-or-init` indicates that the _contact is invalid_, remove the entry from `members`, log a warning and abort these sub-steps. 4. (MD) Let `pending-reflect-acks` be an empty list. 5. For each `contact-or-init` of `members`: 1. If `contact-or-init` is an existing contact, abort these sub-steps. 2. (MD) Reflect a `ContactSync.Create` with `contact` set from `contact-or-init` and the following additional properties: - `created_at` set to now, - `acquaintance_level` set to `GROUP`, - all policies and categories set to their defaults. 3. (MD) Add the pending reflect acknowledgement to `pending-reflect-acks`. 6. (MD) Await all `pending-reflect-acks`. 7. Let `added-members` be a copy of `members`. 8. Let `group` be a snapshot of the current group state or undefined if the group does not exist. 9. If `group` is not defined: 1. Let `removed-members` be an empty list. 2. (MD) Reflect a `GroupSync.Create` with `group` set to contain: - `group_identity`, - `created_at`, - `name` empty, - `user_state` set to `MEMBER`, - `profile_picture` empty, - `member_identities` from `members`, - all policies and categories set to their defaults. 10. If `group` is defined: 1. Remove all members from `added-members` that are in `group.members`. 2. Let `removed-members` be a copy of `group.members`. 3. Remove all members from `removed-members` that are in `members`. 4. (MD) Reflect a `GroupSync.Update` with `member_state_changes` constructed from `added-members` and `removed-members` and `group` set to contain the following additional properties: - `user_state` set to `MEMBER`, - `member_identities` from `members`. 11. (MD) Commit the transaction and await acknowledgement. 12. If the user is currently participating in a group call of this group, remove all `removed-members` participants from the group call (handle them as if they left the call) and unblock all pending group call flows for `added-members`. 13. Persist newly added contacts from `members`. 14. Persist the newly created group or the member changes to the group. If the group was previously marked as _left_, remove the _left_ mark. 15. TODO(SE-510): Schedule fetching gateway-defined profile picture here for each newly added contact from `members`, if necessary. 16. If `added-members` or `removed-members` is not empty, run the _Rejected Messages Refresh Steps_ for the group. fields: - _doc: |- A set of Threema IDs defining group membership. The creator's Threema ID is always inferred and must not be included in this set. name: members type: *identities group-name: _group: Contact and Group Control _doc: |- Name (or rename) a group. Sent to all group members when the group is being created for the first time or the group is being renamed. May also be sent to a single receiver as a response to a [`group-sync-request`](ref:e2e.group-sync-request) message. **Properties**: - Kind: Group - Flags: None - User profile distribution: No - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: N/A - Reactions: No - When rejected: N/A¹ - Edit applies to: N/A (can just send another `group-name`) - Deletable by: N/A (can just send an empty name) - Send to Threema Gateway ID group creator: N/A ¹: For the group creator it will be handled as if `group-sync-request` was received, re-sending the group name, implicitly triggered by FS `Reject` receive steps. When receiving this message as a group message (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)): 1. Run the [_Common Group Receive Steps_](ref:e2e#receiving). If the message has been discarded, abort these steps. 2. Let `group` be a snapshot of the current group state. 3. If `group.name` equals `name` (i.e. no change), discard the message and abort these steps. 4. (MD) Run the following sub-steps: 1. Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning that a group sync race occurred, discard the message and abort these steps. 2. Let `group` be a snapshot of the current group state. 3. If `group.name` equals `name`, log a warning that a group sync race occurred. 4. Reflect a `GroupSync.Update` with `group` set to contain `name` set to `name`. 5. Commit the transaction and await acknowledgement. 5. Update the group's name with `name`. fields: - _doc: |- UTF-8 encoded string containing the group's name. name: name type: b* group-leave: _group: Contact and Group Control _doc: |- Sent by a group member... * that is leaving the group. The message is sent to all other group members and the creator. * in direct reply to a group message for a group that it has marked as left. Note: The group creator is not allowed to leave the group. **Properties**: - Kind: Group - Flags: None - User profile distribution: No - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: N/A - Reactions: No - When rejected: N/A¹ - Edit applies to: N/A - Deletable by: N/A - Send to Threema Gateway ID group creator: Yes ¹: Re-send of `group-leave` implicitly triggered by FS `Reject` receive steps due to _Common Group Receive Steps_ invocation. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. If the sender is the creator of the group, log a warning, discard the message and abort these steps. 2. Look up the group. 3. If the group could not be found or is marked as _left_: 1. If the user is the creator of the group (as alleged by the message), discard the message and abort these steps. 2. Run the _Identity Blocked Steps_ for the creator of the group. If the result indicates that the creator is blocked, discard the message and abort these steps. 3. Run the _Group Sync Request Steps_ for the group, discard the message and abort these steps. 4. Let `group` be a snapshot of the current group state. 5. If `group.members` does not include the sender, discard the message and abort these steps. 6. (MD) Run the following sub-steps: 1. Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist or the group is marked as _left_, log a warning that a group sync race occurred, discard the message and abort these steps. 2. Let `group` be a snapshot of the current group state. 3. If `group.members` does not include the sender, log a warning that a group sync race occurred. 4. Let `updated-members` be a copy of `group.members`. 5. Remove the sender from `updated-members`. 6. Reflect a `GroupSync.Update` with `member_state_changes` set to the single entry of the sender leaving and `group` set to contain `member_identities` set from `updated-members`. 7. Commit the transaction and await acknowledgement. 7. If the user is currently participating in a group call of this group, remove the sender from the group call (handle it as if the sender left the call). 8. Remove the sender from the group. 9. Run the _Rejected Messages Refresh Steps_ for the group. group-sync-request: _group: Contact and Group Control _doc: |- Sent by a group member (or a device assuming to be part of the group) to the group creator. **Properties**: - Kind: Group - Flags: None - User profile distribution: No - Exempt from blocking: Yes - Implicit _direct_ contact creation: No - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: Yes - Outgoing: Yes - _Sent_ update: No - Delivery receipts: N/A - Reactions: No - When rejected: N/A¹ - Edit applies to: N/A - Deletable by: N/A - Send to Threema Gateway ID group creator: Yes ¹: Implicitly ignored by FS `Reject` receive steps. The following steps are defined as the _Group Sync Request Steps_: 1. If the user is the creator of the group, log an error and abort these steps. 2. If the group is marked as _recently resynced_ for the user, log a notice and abort these steps.¹ 3. Run the _Bundled Messages Send Steps_ with the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `receivers` set to the creator of the group, - to construct this message (wrapped by [`group-member-container`](ref:e2e.group-member-container)) 4. Mark the group as _recently resynced_ for the user for 1h. When receiving this message as a group message (wrapped by [`group-member-container`](ref:e2e.group-member-container)): 1. Look up the group. If the group could not be found, discard the message and abort these steps. 2. If the user is not the creator of the group, discard the message and abort these steps. 3. If the group is marked as _recently resynced_ for the sender, log a notice, discard the message and abort these steps.¹ 4. (MD) Begin a transaction with scope `GROUP_SYNC` and the following precondition: 1. If the group does not exist, log a warning that a group sync race occurred, discard the message and abort these steps. 5. If the group is marked as _left_ or the sender is not a member of the group, run the _Bundled Messages Send Steps_ with the following properties: - `id` set to a random message ID, - `created-at` set to the current timestamp, - `receivers` set to the sender, - to construct a [`group-setup`](ref:e2e.group-setup) (wrapped by [`group-creator-container`](ref:e2e.group-creator-container)) with an empty members set. 6. If the group is not marked as _left_ and the sender is a member of the group, run the _Active Group State Resync Steps_ with four random message IDs and `target-members` set to the sender. 7. (MD) Commit the transaction and await acknowledgement. ¹: This is a precaution since a `group-sync-request` is automatically triggered and creates an automatic response. This can easily lead to message loops. Limiting `group-sync-request`s to once an hour per group per sender/receiver breaks a potential infinite loop. web-session-resume: _group: Push Control _doc: |- A control message from Threema Web, requesting a session to be resumed. **Properties (1:1)**: - Kind: 1:1 - Flags: - `0x20`: Short-lived server queuing. - User profile distribution: N/A (not sent by apps) - Exempt from blocking: Yes - Implicit _direct_ contact creation: N/A (blocking is circumvented) - Protect against replay: Yes - Unarchive: No - Bump _last update_: No - Reflect: - Incoming: No - Outgoing: No - _Sent_ update: No - Delivery receipts: N/A - Reactions: N/A - When rejected: N/A (not sent by clients) - Edit applies to: N/A - Deletable by: N/A - Send to Threema Gateway ID group creator: N/A When receiving this message: 1. If the sender is not `*3MAPUSH`, discard the message and abort these steps. 2. Lookup the web client session associated to `wcs` and attempt to resume it. fields: - _doc: |- UTF-8, JSON-encoded object with the following fields: - Webclient session (`'wcs'`): SHA256 hash (hex encoded) of the public permanent key of the session initiator, string. - Affiliation ID (`'wca'`): An optional identifier for affiliating consecutive pushes, `string` or `null`. - Affiliation ID (`'wct'`): Unix epoch timestamp of the request in seconds, `i64`. - Protocol version (`'wcv'`): Version of the Threema Web protocol, `u16`. All fields must be part of the JSON object, even if their values are nullable. name: push-payload type: b* # Parsed struct namespaces (mapped into separate files) namespaces: index: *index handshake: *handshake payload: *payload e2e: *e2e [Seitenstruktur0.185Druckenetwas mehr zur Ethik2026-04-27] |
2026-05-26