Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

streamcoreai/rust-sdk

Folders and files

NameName
Last commit message
Last commit date

Latest commit

History

3 Commits

Repository files navigation

streamcore-rust-sdk

Rust SDK for connecting to a StreamCoreAI server via WebRTC + WHIP.

Installation

[dependencies]
streamcoreai-voice-agent-sdk = { git = "https://github.com/streamcoreai/rust-sdk" }

Or add to your Cargo.toml:

streamcore-rust-sdk = "0.1"

Quick Start

use std::sync::Arc;
use streamcore_rust_sdk::{Client, Config, EventHandler, FRAME_SIZE};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
 let client = Arc::new(Client::new(
 Config {
 whip_endpoint: "http://localhost:8080/whip".into(),
 ..Default::default()
 },
 EventHandler {
 on_status_change: Some(Box::new(|status| {
 println!("[status] {}", status);
 })),
 on_transcript: Some(Box::new(|entry, _all| {
 println!("[{}] {}", entry.role, entry.text);
 })),
 on_error: Some(Box::new(|err| {
 eprintln!("[error] {}", err);
 })),
 on_data_channel_message: None,
 },
 ));
 client.connect().await?;
 // Send microphone audio (f32 PCM, mono, 48 kHz, 960 samples per frame)
 let client_tx = Arc::clone(&client);
 tokio::spawn(async move {
 let pcm = vec![0.0f32; FRAME_SIZE]; // replace with real mic capture
 loop {
 client_tx.send_pcm(&pcm).await.unwrap();
 }
 });
 // Receive agent audio
 let client_rx = Arc::clone(&client);
 tokio::spawn(async move {
 let mut pcm = vec![0.0f32; FRAME_SIZE];
 loop {
 let n = client_rx.recv_pcm(&mut pcm).await.unwrap();
 // Play pcm[..n] through speakers
 let _ = &pcm[..n];
 }
 });
 tokio::signal::ctrl_c().await?;
 client.disconnect().await;
 Ok(())
}

API

Client::new(config, events)

Creates a new client instance.

Config

Field Type Default Description
whip_endpoint String "http://localhost:8080/whip" WHIP signaling endpoint URL
ice_servers Vec<String> ["stun:stun.l.google.com:19302"] ICE server URLs

EventHandler

Callback Signature Description
on_status_change Option<Box<dyn Fn(ConnectionStatus) + Send + Sync>> Fired when connection status changes
on_transcript Option<Box<dyn Fn(TranscriptEntry, Vec<TranscriptEntry>)>> Fired on new or updated transcript
on_error Option<Box<dyn Fn(String) + Send + Sync>> Fired on connection/server errors
on_data_channel_message Option<Box<dyn Fn(DataChannelMessage)>> Fired for every raw DC message

Client Methods

Method Returns Description
connect() Result<(), ClientError> Establish WebRTC + WHIP session
disconnect().await Tear down connection, free resources
send_pcm(&pcm) Result<(), ClientError> Encode f32 PCM → Opus → RTP and send to server
recv_pcm(&mut pcm) Result<usize, ClientError> Receive + decode one frame of agent audio
status() ConnectionStatus Current connection status
transcript() Vec<TranscriptEntry> Full conversation history (copy)

Audio Constants

Constant Value Description
SAMPLE_RATE 48000 Audio sample rate in Hz (Opus)
CHANNELS 1 Number of audio channels (mono)
FRAME_SIZE 960 Samples per 20 ms frame at 48 kHz

Client Fields (after connect())

Field Type Description
local_track Arc<TrackLocalStaticRTP> Write RTP packets here to send audio to server
remote_track_notify Arc<Notify> Notifies when remote_track is available
remote_track Arc<Mutex<Option<TrackRemote>>> Agent's audio track (check after notify fires)

Types

pub enum ConnectionStatus { Idle, Connecting, Connected, Error, Disconnected }
pub struct TranscriptEntry {
 pub role: String, // "user" or "assistant"
 pub text: String,
 pub partial: bool,
}
pub struct DataChannelMessage {
 pub msg_type: String, // "transcript", "response", or "error"
 pub text: String,
 pub r#final: bool,
 pub message: String,
}

Audio I/O

The SDK handles Opus encoding/decoding and RTP packetization internally. You only work with raw PCM f32 samples:

  • Sending audio: Call client.send_pcm(&pcm) with a FRAME_SIZE-length slice of mono f32 samples
  • Receiving audio: Call client.recv_pcm(&mut pcm) to get the next decoded frame from the agent

For microphone capture and speaker playback, use a library like cpal.

Requirements

  • Rust 1.87+

Dependencies

  • webrtc 0.17 — Pion WebRTC bindings
  • audiopus — Opus audio codec
  • tokio — Async runtime
  • reqwest — HTTP client for WHIP signaling
  • serde — JSON serialization for data channel messages

License

MIT

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

Contributors

Languages

AltStyle によって変換されたページ (->オリジナル) /