Audio API Reference

Base URL: https://platform.dataeyes.ai
Authentication: Authorization: Bearer <your-api-key>
Document Version: 2026-04

1. Overview

This platform provides a full suite of audio API capabilities covering the following scenarios:

Scenario	Endpoint	Description
Text-to-Speech (TTS)	`/v1/audio/speech`	Synthesize text into an audio file
Speech-to-Text (STT)	`/v1/audio/transcriptions`	Transcribe an audio file to text
Audio Translation	`/v1/audio/translations`	Translate audio to English text
Audio in Chat	`/v1/chat/completions`	Audio input/output within multimodal conversations
Realtime Voice Chat	`/v1/realtime`	Bidirectional real-time voice streaming over WebSocket

Supported Providers

Provider	TTS	STT	Realtime	Notes
OpenAI	Yes	Yes	Yes	Full support including streaming
VolcEngine (ByteDance)	Yes	No	No	Supports WebSocket streaming TTS; maps to Chinese voice presets
MiniMax	Yes	No	No	Supports emotion, speed, and volume control
Cloudflare	No	Yes	No	Transcription and translation only

2. TTS — Text-to-Speech

Synthesize text into an audio file.

2.1 Endpoint

POST /v1/audio/speech

2.2 Request Body (JSON)

Field	Type	Required	Description
`model`	string	Yes	TTS model name
`input`	string	Yes	The text to synthesize
`voice`	string	Yes	Voice ID
`instructions`	string	No	Voice style instructions (supported by `gpt-4o-mini-tts`)
`response_format`	string	No	Output audio format; default `mp3`
`speed`	float	No	Speed multiplier, range 0.25–4.0; default 1.0
`stream_format`	string	No	Set to `"sse"` to enable streaming output
`metadata`	object	No	Provider-specific parameters

2.3 Available Voices

OpenAI Standard Voices:

Voice ID	Style Description
`alloy`	Neutral, balanced
`echo`	Male, steady
`fable`	Female, gentle
`onyx`	Male, deep
`nova`	Female, lively
`shimmer`	Female, soft

VolcEngine automatically maps the above voice IDs to corresponding Chinese voice presets:
alloy → Male conversational voice
echo → Male composed voice
fable → Female sweet voice
onyx → Male intellectual voice
nova → Female energetic voice
shimmer → Female bright voice

2.4 Supported Audio Formats

Format	Content-Type	Notes
`mp3`	audio/mpeg	Default format; best compatibility
`wav`	audio/wav	Lossless; larger file size
`aac`	audio/aac	Common on iOS platforms
`flac`	audio/flac	Lossless compression
`opus`	audio/opus	High quality at low bitrate
`pcm`	audio/pcm	Raw PCM data (24kHz, 16-bit, mono)

2.5 Request Examples

Basic TTS:

HD TTS:

GPT-4o-mini-TTS (with style instructions):

Streaming TTS (SSE):

2.6 Response

Non-streaming: Returns raw audio binary data directly. The Content-Type header matches the corresponding audio MIME type.

Streaming: Returns audio data in chunked SSE format.

2.7 TTS Model List

Model	Description	Provider
`tts-1`	Standard TTS	OpenAI
`tts-1-hd`	HD TTS	OpenAI
`tts-1-1106`	TTS version 1106	OpenAI
`tts-1-hd-1106`	HD TTS version 1106	OpenAI
`gpt-4o-mini-tts`	GPT-4o Mini TTS (supports style instructions)	OpenAI
`gpt-4o-mini-tts-2025-03-20`	GPT-4o Mini TTS specific version	OpenAI
`gpt-4o-mini-tts-2025-12-15`	GPT-4o Mini TTS specific version	OpenAI
`speech-2.5-hd-preview`	MiniMax 2.5 HD Preview	MiniMax
`speech-2.5-turbo-preview`	MiniMax 2.5 Turbo Preview	MiniMax
`speech-02-hd`	MiniMax HD	MiniMax
`speech-02-turbo`	MiniMax Turbo	MiniMax
`speech-01-hd`	MiniMax HD v1	MiniMax
`speech-01-turbo`	MiniMax Turbo v1	MiniMax

3. STT — Speech-to-Text (Transcription)

Transcribe an audio file to text.

3.1 Endpoint

POST /v1/audio/transcriptions

3.2 Request Format

Content-Type: multipart/form-data

Field	Type	Required	Description
`file`	file	Yes	Audio file
`model`	string	Yes	STT model name
`language`	string	No	Language code, e.g. `zh`, `en`, `ja`
`prompt`	string	No	Context hint to help the model better understand the content
`response_format`	string	No	Response format; default `json`
`temperature`	float	No	Temperature parameter, range 0–1

3.3 Supported Audio File Formats

mp3, mp4, mpeg, mpga, m4a, wav, webm, ogg, flac

3.4 response_format Options

Format	whisper-1	gpt-4o-transcribe series	Description
`json`	Yes	Yes	Simple JSON containing a `text` field (default)
`text`	Yes	Yes	Plain text
`srt`	Yes	No	SRT subtitle format
`verbose_json`	Yes	No	Verbose JSON with timestamps and segment information
`vtt`	Yes	No	WebVTT subtitle format

Note: The gpt-4o-transcribe and gpt-4o-mini-transcribe model series only support the json and text response formats. Using verbose_json, srt, or vtt with these models will return an unsupported_value error. For subtitle or timestamp functionality, use the whisper-1 model.

3.5 Request Examples

Basic Transcription (Whisper):

GPT-4o-transcribe (High-Accuracy Transcription):

Speaker Diarization Transcription:

Generate Subtitle File:

3.6 Response Body

json format (whisper-1):

{
  "text": "Hello, welcome to our platform."
}

json format (gpt-4o-transcribe series):

{
  "text": "Hello, welcome to our platform.",
  "usage": {
    "type": "tokens",
    "total_tokens": 34,
    "input_tokens": 30,
    "input_token_details": {
      "text_tokens": 0,
      "audio_tokens": 30
    },
    "output_tokens": 4
  }
}

The gpt-4o-transcribe series additionally returns a usage field with a detailed token consumption breakdown. Audio token consumption is approximately 1,000 audio tokens per minute.

verbose_json format (whisper-1 only):

{
  "task": "transcribe",
  "language": "chinese",
  "duration": 5.32,
  "text": "Hello, welcome to our platform.",
  "segments": [
    {
      "id": 0,
      "seek": 0,
      "start": 0.0,
      "end": 2.5,
      "text": "Hello,",
      "tokens": [1, 2, 3],
      "temperature": 0.0,
      "avg_logprob": -0.25,
      "compression_ratio": 0.8,
      "no_speech_prob": 0.01
    },
    {
      "id": 1,
      "seek": 250,
      "start": 2.5,
      "end": 5.32,
      "text": "welcome to our platform.",
      "tokens": [4, 5, 6, 7, 8],
      "temperature": 0.0,
      "avg_logprob": -0.18,
      "compression_ratio": 0.9,
      "no_speech_prob": 0.02
    }
  ]
}

srt format:

1
00:00:00,000 --> 00:00:02,500
Hello,

2
00:00:02,500 --> 00:00:05,320
welcome to our platform.

3.7 STT Model List

Model	Description	Supported response_format	Notes
`whisper-1`	Whisper standard model	json, text, srt, verbose_json, vtt	General-purpose, stable; supports subtitle output
`gpt-4o-transcribe`	GPT-4o transcription	json, text	High accuracy with better contextual understanding
`gpt-4o-transcribe-diarize`	GPT-4o transcription with speaker diarization	json, text	Automatically identifies different speakers
`gpt-4o-mini-transcribe`	GPT-4o Mini transcription	json, text	Cost-effective
`gpt-4o-mini-transcribe-2025-03-20`	GPT-4o Mini transcription specific version	json, text	Specific dated version
`gpt-4o-mini-transcribe-2025-12-15`	GPT-4o Mini transcription specific version	json, text	Specific dated version

4. Audio Translation (Translate to English)

Translate audio in any language to English text.

4.1 Endpoint

POST /v1/audio/translations

4.2 Request Format

Content-Type: multipart/form-data

Field	Type	Required	Description
`file`	file	Yes	Audio file
`model`	string	Yes	Model name (default `whisper-1`)
`prompt`	string	No	Context hint
`response_format`	string	No	Response format; default `json`
`temperature`	float	No	Temperature parameter

4.3 Request Example

4.4 Response Body

{
  "text": "Hello, welcome to our platform."
}

5. Chat Completions with Audio Input/Output

Use audio capabilities within the standard Chat Completions interface.

5.1 Endpoint

POST /v1/chat/completions

5.2 Audio Input (Embed Audio in a Message)

5.3 Audio Output

To enable audio output, set the modalities and audio parameters:

5.4 Audio Chat Models

Model	Description
`gpt-4o-audio-preview`	GPT-4o audio preview
`gpt-4o-audio-preview-2024-12-17`	Specific version
`gpt-4o-mini-audio-preview`	GPT-4o Mini audio preview
`gpt-audio`	GPT audio general availability
`gpt-audio-mini`	GPT audio Mini

5.5 Audio Tokens in Usage

Audio chat returns a detailed token breakdown in usage:

{
  "usage": {
    "prompt_tokens": 100,
    "completion_tokens": 200,
    "total_tokens": 300,
    "prompt_tokens_details": {
      "cached_tokens": 0,
      "text_tokens": 50,
      "audio_tokens": 50,
      "image_tokens": 0
    },
    "completion_tokens_details": {
      "text_tokens": 100,
      "audio_tokens": 100,
      "reasoning_tokens": 0
    }
  }
}

6. Realtime — Live Voice Conversation (WebSocket)

Supports bidirectional real-time voice streaming. Suitable for voice assistants, live translation, and similar scenarios.

6.1 Endpoint

GET /v1/realtime (WebSocket Upgrade)

6.2 Connection

wss://cloud.dataeyes.ai/v1/realtime?model=gpt-4o-realtime-preview

WebSocket Headers:

Authorization: Bearer <your-api-key>
Sec-WebSocket-Protocol: realtime

6.3 Session Initialization

After connecting, send a session.update event to configure the session:

{
  "event_id": "evt_001",
  "type": "session.update",
  "session": {
    "modalities": ["text", "audio"],
    "instructions": "You are a friendly Chinese-English translation assistant. Translate Chinese to English and English to Chinese.",
    "voice": "alloy",
    "input_audio_format": "pcm16",
    "output_audio_format": "pcm16",
    "input_audio_transcription": {
      "model": "whisper-1"
    },
    "turn_detection": {
      "type": "server_vad",
      "threshold": 0.5,
      "prefix_padding_ms": 300,
      "silence_duration_ms": 500
    },
    "temperature": 0.8,
    "tools": [],
    "tool_choice": "auto"
  }
}

6.4 Audio Formats

Format	Parameter Value	Description
PCM 16-bit	`pcm16`	Raw PCM (24kHz, 16-bit, mono)
G.711 u-law	`g711_ulaw`	Telephony audio format
G.711 A-law	`g711_alaw`	Telephony audio format

6.5 Sending Audio Data

Send audio data Base64-encoded:

{
  "type": "input_audio_buffer.append",
  "audio": "<base64-encoded PCM audio data>"
}

6.6 Triggering a Response

{
  "type": "response.create",
  "response": {
    "modalities": ["text", "audio"]
  }
}

6.7 Receiving Response Events

Session Created:

{
  "type": "session.created",
  "session": {
    "modalities": ["text", "audio"],
    "voice": "alloy",
    ...
  }
}

Audio Data Delta:

{
  "type": "response.audio.delta",
  "delta": "<base64-encoded audio chunk>"
}

Audio Transcript Delta:

{
  "type": "response.audio_transcript.delta",
  "delta": "Hello"
}

Response Done (with Usage):

{
  "type": "response.done",
  "response": {
    "usage": {
      "total_tokens": 500,
      "input_tokens": 200,
      "output_tokens": 300,
      "input_token_details": {
        "cached_tokens": 0,
        "text_tokens": 50,
        "audio_tokens": 150
      },
      "output_token_details": {
        "text_tokens": 100,
        "audio_tokens": 200
      }
    }
  }
}

Error Event:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "Audio format not supported"
  }
}

6.8 Full Event Interaction Flow

Client                                      Server
  │                                           │
  │ ──── WebSocket Connect ───────────────→   │
  │ ←──── session.created ────────────────   │
  │                                           │
  │ ──── session.update ──────────────────→   │
  │ ←──── session.updated ────────────────   │
  │                                           │
  │ ──── input_audio_buffer.append ───────→   │
  │ ──── input_audio_buffer.append ───────→   │
  │ ──── input_audio_buffer.append ───────→   │
  │                                           │
  │ ──── response.create ─────────────────→   │
  │ ←──── response.audio.delta ───────────   │
  │ ←──── response.audio.delta ───────────   │
  │ ←──── response.audio_transcript.delta    │
  │ ←──── response.done ─────────────────   │
  │                                           │

6.9 Realtime Model List

Model	Description
`gpt-4o-realtime-preview`	GPT-4o realtime preview
`gpt-4o-realtime-preview-2024-10-01`	Specific version
`gpt-4o-realtime-preview-2024-12-17`	Specific version
`gpt-4o-realtime-preview-2025-06-03`	Specific version
`gpt-4o-mini-realtime-preview`	GPT-4o Mini realtime preview
`gpt-4o-mini-realtime-preview-2024-12-17`	Specific version
`gpt-realtime`	GPT realtime general availability
`gpt-realtime-2025-08-28`	Specific version
`gpt-realtime-mini`	GPT realtime Mini
`gpt-realtime-mini-2025-10-06`	Specific version
`gpt-realtime-mini-2025-12-15`	Specific version

7. Provider Details

7.1 OpenAI

Full support for TTS + STT + Realtime.

TTS supports 6 standard voices plus gpt-4o-mini-tts with style instructions

STT supports whisper-1 and the gpt-4o-transcribe series

Realtime supports PCM16 and G.711 audio formats

Audio token calculation: approximately 1,000 tokens per minute of audio

7.2 VolcEngine (ByteDance)

Supports TTS only.

Voice Mapping (OpenAI voice IDs to VolcEngine Chinese voice presets):

OpenAI Voice	VolcEngine Voice ID	Style
`alloy`	zh_male_M392_conversation_wvae_bigtts	Male conversational
`echo`	zh_male_wenhao_mars_bigtts	Male composed
`fable`	zh_female_tianmei_mars_bigtts	Female sweet
`onyx`	zh_male_zhibei_mars_bigtts	Male intellectual
`nova`	zh_female_shuangkuaisisi_mars_bigtts	Female energetic
`shimmer`	zh_female_cancan_mars_bigtts	Female bright

Audio Format Mapping:

Requested Format	VolcEngine Encoding
`mp3`	mp3
`opus`	ogg_opus
`aac` / `flac`	mp3 (fallback)
`wav`	wav
`pcm`	pcm

Authentication Format: The API Key uses appid|access_token format.

Request Example (TTS via VolcEngine channel):

Note: The actual available model names depend on the channel configuration in the platform.

7.3 MiniMax

Supports TTS only.

Distinctive Features:

Emotion control (emotion)

Fine-grained control over speed, volume, and pitch

Voice timbre weight blending

Language enhancement (language_boost)

Subtitle generation

MiniMax TTS Request Parameters (via metadata):

Parameter	Type	Description
`voice_setting.speed`	float	Speech speed
`voice_setting.vol`	float	Volume
`voice_setting.pitch`	float	Pitch
`voice_setting.emotion`	string	Emotion
`language_boost`	string	Language enhancement
`subtitle_enable`	bool	Enable subtitle generation
`output_format`	string	Output format

MiniMax TTS Models:

Model	Description
`speech-2.5-hd-preview`	2.5 HD Preview
`speech-2.5-turbo-preview`	2.5 Turbo Preview
`speech-02-hd`	HD version
`speech-02-turbo`	Turbo version
`speech-01-hd`	v1 HD version
`speech-01-turbo`	v1 Turbo version

7.4 Cloudflare

Supports STT only (transcription and translation).

Simple file-upload relay mode, supports the whisper-1 model.

8. Complete Model List

8.1 TTS Models

Model	Provider	Description
`tts-1`	OpenAI	Standard TTS
`tts-1-hd`	OpenAI	HD TTS
`tts-1-1106`	OpenAI	TTS version 1106
`tts-1-hd-1106`	OpenAI	HD version 1106
`gpt-4o-mini-tts`	OpenAI	GPT-4o Mini TTS (supports style instructions)
`gpt-4o-mini-tts-2025-03-20`	OpenAI	Specific version
`gpt-4o-mini-tts-2025-12-15`	OpenAI	Specific version
`speech-2.5-hd-preview`	MiniMax	MiniMax 2.5 HD
`speech-2.5-turbo-preview`	MiniMax	MiniMax 2.5 Turbo
`speech-02-hd`	MiniMax	MiniMax HD
`speech-02-turbo`	MiniMax	MiniMax Turbo
`speech-01-hd`	MiniMax	MiniMax v1 HD
`speech-01-turbo`	MiniMax	MiniMax v1 Turbo

8.2 STT Models (Transcription / Translation)

Model	Provider	Description
`whisper-1`	OpenAI	Whisper standard; general-purpose transcription
`gpt-4o-transcribe`	OpenAI	GPT-4o high-accuracy transcription
`gpt-4o-transcribe-diarize`	OpenAI	GPT-4o transcription with speaker diarization
`gpt-4o-mini-transcribe`	OpenAI	GPT-4o Mini transcription
`gpt-4o-mini-transcribe-2025-03-20`	OpenAI	Specific version
`gpt-4o-mini-transcribe-2025-12-15`	OpenAI	Specific version

8.3 Audio Preview Models (Audio Input/Output in Chat)

Model	Provider	Description
`gpt-4o-audio-preview`	OpenAI	Audio chat preview
`gpt-4o-audio-preview-2024-10-01`	OpenAI	Specific version
`gpt-4o-audio-preview-2024-12-17`	OpenAI	Specific version
`gpt-4o-audio-preview-2025-06-03`	OpenAI	Specific version
`gpt-4o-mini-audio-preview`	OpenAI	Mini audio chat preview
`gpt-4o-mini-audio-preview-2024-12-17`	OpenAI	Specific version
`gpt-audio`	OpenAI	Audio chat general availability
`gpt-audio-2025-08-28`	OpenAI	Specific version
`gpt-audio-mini`	OpenAI	Mini audio chat
`gpt-audio-mini-2025-10-06`	OpenAI	Specific version
`gpt-audio-mini-2025-12-15`	OpenAI	Specific version

8.4 Realtime Models (Live Voice Streaming)

Model	Provider	Description
`gpt-4o-realtime-preview`	OpenAI	GPT-4o realtime preview
`gpt-4o-realtime-preview-2024-10-01`	OpenAI	Specific version
`gpt-4o-realtime-preview-2024-12-17`	OpenAI	Specific version
`gpt-4o-realtime-preview-2025-06-03`	OpenAI	Specific version
`gpt-4o-mini-realtime-preview`	OpenAI	Mini realtime preview
`gpt-4o-mini-realtime-preview-2024-12-17`	OpenAI	Specific version
`gpt-realtime`	OpenAI	Realtime general availability
`gpt-realtime-2025-08-28`	OpenAI	Specific version
`gpt-realtime-mini`	OpenAI	Realtime Mini
`gpt-realtime-mini-2025-10-06`	OpenAI	Specific version
`gpt-realtime-mini-2025-12-15`	OpenAI	Specific version

9. Error Handling

9.1 Error Response Format

{
  "error": {
    "message": "Invalid audio format. Supported formats: mp3, mp4, mpeg, mpga, m4a, wav, webm, ogg, flac",
    "type": "invalid_request_error",
    "code": "invalid_audio_format"
  }
}

9.2 Common Errors

HTTP Status	Scenario	Description
400	Invalid parameters	Missing required fields (e.g. `file`, `model`) or unsupported format
400	File too large	Audio file exceeds size limit (generally 25 MB)
401	Authentication failure	Invalid API Key
403	Insufficient permissions	No access to the specified model
413	Request too large	Request body exceeds limit
429	Rate limit	Too many requests in a short period
500	Server error	Upstream provider error

9.3 TTS-Specific Errors

model is missing or unsupported → 400

input is empty → 400

voice is invalid → 400

Text exceeds the model's length limit → 400

9.4 STT-Specific Errors

file field is missing → 400

Audio file format is unsupported → 400

Audio file is corrupted or cannot be decoded → 400

language code is invalid → 400

Appendix: Endpoint Quick Reference

Method	Path	Content-Type	Description
POST	`/v1/audio/speech`	application/json	TTS — Text-to-Speech
POST	`/v1/audio/transcriptions`	multipart/form-data	STT — Speech-to-Text
POST	`/v1/audio/translations`	multipart/form-data	Audio translation (to English)
POST	`/v1/chat/completions`	application/json	Audio input/output in chat
GET	`/v1/realtime`	WebSocket Upgrade	Live voice conversation

Quick Selection Guide

Use Case	Recommended Model	Endpoint
Simple text-to-speech	`tts-1`	`/v1/audio/speech`
High-definition text-to-speech	`tts-1-hd`	`/v1/audio/speech`
TTS with emotion or style	`gpt-4o-mini-tts`	`/v1/audio/speech`
General-purpose transcription	`whisper-1`	`/v1/audio/transcriptions`
High-accuracy transcription	`gpt-4o-transcribe`	`/v1/audio/transcriptions`
Meeting transcription (multi-speaker)	`gpt-4o-transcribe-diarize`	`/v1/audio/transcriptions`
Cost-effective transcription	`gpt-4o-mini-transcribe`	`/v1/audio/transcriptions`
Translate speech to English	`whisper-1`	`/v1/audio/translations`
Understand audio in conversation	`gpt-4o-audio-preview`	`/v1/chat/completions`
Real-time voice assistant	`gpt-4o-realtime-preview`	`/v1/realtime`

Audio API

Audio API Reference#

Table of Contents#

1. Overview#

Supported Providers#

2. TTS — Text-to-Speech#

2.1 Endpoint#

2.2 Request Body (JSON)#

2.3 Available Voices#

2.4 Supported Audio Formats#

2.5 Request Examples#

2.6 Response#

2.7 TTS Model List#

3. STT — Speech-to-Text (Transcription)#

3.1 Endpoint#

3.2 Request Format#

3.3 Supported Audio File Formats#

3.4 response_format Options#

3.5 Request Examples#

3.6 Response Body#

3.7 STT Model List#

4. Audio Translation (Translate to English)#

4.1 Endpoint#

4.2 Request Format#

4.3 Request Example#

4.4 Response Body#

5. Chat Completions with Audio Input/Output#

5.1 Endpoint#

5.2 Audio Input (Embed Audio in a Message)#

5.3 Audio Output#

5.4 Audio Chat Models#

5.5 Audio Tokens in Usage#

6. Realtime — Live Voice Conversation (WebSocket)#

6.1 Endpoint#

6.2 Connection#

6.3 Session Initialization#

6.4 Audio Formats#

6.5 Sending Audio Data#

6.6 Triggering a Response#

6.7 Receiving Response Events#

6.8 Full Event Interaction Flow#

6.9 Realtime Model List#

7. Provider Details#

7.1 OpenAI#

7.2 VolcEngine (ByteDance)#

7.3 MiniMax#

7.4 Cloudflare#

8. Complete Model List#

8.1 TTS Models#

8.2 STT Models (Transcription / Translation)#

8.3 Audio Preview Models (Audio Input/Output in Chat)#

8.4 Realtime Models (Live Voice Streaming)#

9. Error Handling#

9.1 Error Response Format#

9.2 Common Errors#

9.3 TTS-Specific Errors#

9.4 STT-Specific Errors#

Appendix: Endpoint Quick Reference#

Quick Selection Guide#