Native mobile and audio interfaces

This is an embedder reference for the source interfaces, not a phone installation guide or a promise that a particular distribution includes a calling application. Begin with the native integration contract and use the header from the same revision as your library. The host supplies its own interface, device permissions, capture, playback, authentication presentation and lifecycle supervision.

Choose the correct interface

Mobile admission, completion and ownership

Every asynchronous mobile call has two results. The immediate return is 0 when admitted, 1 for invalid arguments or 3 when unavailable; a nonzero return promises no callback. After admission, retain the callback and its context until its terminal callback. A callback can arrive before the initiating function returns: prepare state first, then call. Callbacks are serialized within one operation but independent operations may run concurrently on runtime threads.

Inputs are copied before return. Text is UTF-8, at most 4096 input bytes. Callback byte buffers are borrowed only during that invocation; a null pointer is permitted at length zero. Copy retained data and dispatch UI work asynchronously. Session open returns a nonzero owned handle and the inherited gateway base URL in value. Relay open returns another owned handle. Register returns a runner ID in value; receive returns frame bytes. Other successful fields are empty or zero. Neither access tokens nor raw HTTP exceptions cross this ABI.

Example: inspect a pairing and open its relay

1. Initialize the native runtime and sign in through the ordinary gateway workflow. Call ha_mobile_session_open with a retained callback context. On immediate 0, wait for terminal 0 and store its session handle. Opening does not register a runner.
2. If this host is registering as a runner, call ha_mobile_runner_register with a stable device UUID and a nonempty display name of at most 160 characters. The UUID is 36 characters with hyphens at positions 8, 13, 18 and 23 and hexadecimal digits elsewhere. Retain the returned runner ID on success.
3. Call ha_mobile_pairings_list. Collect status-4 rows into a new list and publish it only after terminal 0. An empty list is a valid result, not permission to invent a pairing ID. The implementation rejects responses over 1024 rows or with malformed IDs.
4. Let the user select an existing pairing ID. Optionally call ha_mobile_pairing_wake and await its terminal response. A successful wake request does not prove the runner or mobile application is online.
5. Call ha_mobile_relay_open with that same pairing ID. On terminal 0 retain the relay handle, then issue a receive operation with its own retained completion context. Do not treat pairing IDs, runner IDs and numeric relay handles as interchangeable.
6. Send only frames produced by the consuming application's authenticated encryption protocol. A successful send means admission to the bounded outgoing queue, not delivery or acknowledgement by the remote application. Correlate application-level acknowledgements inside that protocol before retrying a digital action.
7. On user closure, stop issuing operations, close each relay handle, then close the session handle. Drain already admitted callbacks before freeing their contexts or exiting the runtime.

session_open -> admitted 0 -> callback 0, session handle
pairings_list(session) -> admitted 0 -> callback 4, pairing ID/name
                                      -> callback 0, list complete
relay_open(session, selected ID) -> admitted 0 -> callback 0, relay handle
relay_receive(relay) -> admitted 0 -> callback 0, encrypted frame
handle_close(relay); drain its pending callbacks
handle_close(session); drain its pending callbacks

The trace is an example of control flow, not a literal wire payload. The library exposes no mobile pairing-creation or end-to-end encryption command here. Those responsibilities remain with the gateway and consuming application; passing arbitrary JSON to the relay is not a valid substitute.

Revocation, bounds and uncertain outcomes

ha_mobile_pairing_revoke takes the selected pairing UUID and awaits a terminal result. The HTTP implementation accepts an already absent pairing (DELETE 404) as success. After an uncertain response, list pairings again instead of reporting success. Closing a mobile session leaves the main gateway credentials intact; disconnecting or replacing that account invalidates sessions that inherited its authority.

Relay frames are at most 1 MiB. The internal incoming and outgoing queues each hold eight frames, so consumers must drain receive results and bound their own queues. HTTP operations, relay establishment and enqueueing a send have a 20-second outer bound; HTTP response timeout is 15 seconds. Receive has no independent idle timeout: it waits for data or closure. Close its relay to release a pending receive. Do not free a receive's context merely because the UI stopped waiting.

ha_mobile_handle_close is synchronous and idempotent. Closing a session stops its relays, but each returned relay handle still needs closing. Close invalidates operations; it is not a callback-drain barrier or a guarantee that an already queued remote effect did not happen. Keep operation contexts until their terminal callback. After account changes, create fresh sessions rather than reusing old numeric handles. If an already admitted open completes during shutdown, close its returned handle too; do not discard a successful completion just because its view has disappeared.

Example: stage one native voice turn

1. Choose a unique nonempty UTF-8 turn ID. Voice staging currently limits this ID to 512 bytes, stricter than the ordinary turn-option limit. Allocate a callback context owning a bounded playback queue, capture-worker lifecycle and a synchronized optional call handle.
2. Call ha_engine_stage_turn_options for this turn, then ha_engine_stage_voice with the same ID. Check each immediate result. Voice staging requires existing options and changes this turn into a voice call instead of using its initial text prompt.
3. Submit the matching turn.start through the normal request path. Admission is not connection. If submission is abandoned, discard staging for that turn; do not leave an audio callback retained for an unrelated future request.
4. Only on audio event 0, after server acknowledgement, start capture and publish the callback's call handle to that worker. Configure signed little-endian PCM16, 24,000 Hz, mono. Device setup must be bounded.
5. Copy incoming event-1 audio into bounded playback storage and schedule it on the host audio device. Do not retain the borrowed callback pointer or synchronously wait for GUI-thread work.
6. On event 3 discard scheduled playback without stopping capture. This resets playback for interruption; it is not the end of the call.
7. On event 2 stop and join capture, clear all shared references to the call handle, and stop playback before returning. The bridge frees the handle immediately after this callback. Then observe the ordinary turn outcome; stopping media does not undo delegated tool effects.

A 20 ms capture frame contains 480 samples, or 960 bytes. The 24,000-byte ceiling is a maximum, not a recommended device-buffer size. On queue rejection stop capture; do not build an unbounded retry buffer. The call handle is valid only between start and the return from stop. A stale pointer is not a recoverable “closed” probe: never call through one.

Host acceptance cases

Before exposing a calling control, test device-denied start followed by stop, playback interruption without capture cancellation, queue overrun, close during capture, account replacement and staging discard. Verify that capture has joined before stop returns and no audio outlives the owner. The ABI's validation tests do not exercise real devices. Provider access and organization authority are separate from microphone permission; see the voice-call workflow.

WebRTC negotiation and scoped audio

The agent-webrtc Haskell library exports an opaque Peer. Acquire it only with withPeer; all users and child workers must finish before its scope exits. The offerer calls createOffer, sends the returned SDP through the host's own signaling transport, receives an answer and calls setRemoteAnswer. The answerer calls setRemoteOffer before createAnswer. Both call awaitConnected before starting normal capture/playback. Creating a description also installs the local description and waits for gathering; do not add a second invented local-description API.

The following in-process example connects two peers and transports silence. It needs the pinned package's native media dependencies but no account, microphone or external signaling service. It is a consumer example, not evidence of a live native application test.

import Agent.WebRTC
import Control.Concurrent (threadDelay)
import Control.Concurrent.Async (concurrently)
import Control.Monad (replicateM_)
import qualified Data.ByteString as BS
import System.Timeout (timeout)

receiveSilence :: IO (Maybe Int)
receiveSilence = withPeer $ \offerer -> withPeer $ \answerer -> do
    offer <- createOffer offerer
    setRemoteOffer answerer offer
    answer <- createAnswer answerer
    setRemoteAnswer offerer answer
    awaitConnected offerer
    awaitConnected answerer
    let send = replicateM_ 50 $ do
            pushAudio offerer (BS.replicate 960 0)
            threadDelay 20000
    result <- timeout 5000000 (concurrently send (pullAudio answerer))
    pure (BS.length . snd <$> result)

Include agent-webrtc, async and bytestring in the consuming package. The repository's Nix package supplies the native GStreamer dependencies; do not replace them with an unrelated global installation. A returned Just byte count demonstrates receipt, not a particular audio-device latency or microphone permission. Nothing indicates the example's five-second media deadline expired.

Media validation and shutdown

Remote SDP must be nonempty UTF-8, contain no NUL byte and occupy at most 65,536 encoded bytes. Negotiation polling and connection establishment use 15-second bounds. They raise failures for unavailable plugins, invalid encoding or size, disconnection, negotiation failure or timeout. These failures are not provider-authentication errors.

pushAudio accepts nonempty, even-sized PCM16 frames up to 24,000 bytes; queue failure raises an exception rather than silently accumulating data. pullAudio waits for a nonempty frame and has no application-level idle deadline. Wrap long-running consumers in the host's cancellation scope and join them before leaving withPeer. The example's structured concurrency ensures its sender is cancelled and joined if receive fails or times out. Do not retain Peer past release or create detached audio workers.

Source and validation boundary

The canonical contracts are HaskellAgentBridge.h, MobileGateway.hs, MobileGatewayBridge.hs, Voice.hs and Agent.WebRTC. Select the same repository revision as your binary. Tests in MobileGatewaySpec.hs cover credential invalidation and invalid inputs; the WebRTC package's native peer checks cover local media transport. Neither certifies an external application's pairing UI, encryption protocol, microphone permissions, account access or deployed relay connectivity.