DataEyesAI
Official SiteConsoleDocs HomeGetting StartedDeveloper ToolsAI Models API
Official SiteConsoleDocs HomeGetting StartedDeveloper ToolsAI Models API
  1. Audio API
  • OpenAI format (supports major original models)
    • Chat (Response)
      • Create Network Search
      • Create Model Response GPT-5 Enable Thinking
      • Create Function Call
      • Create Model Response
      • Create Model Response (Streaming Return)
      • Create Model Response (Control Thinking Length)
    • ChatGPT Interface
      • Audio
        • Audio to text gpt-4o-transcribe
        • GPT-4o-audio
        • Audio to text whisper-1
        • Audio to text gpt-4o-transcribe
        • Create voice gpt-4o-mini-tts
      • Chat
        • Create chat-based image recognition (non-streaming)
        • Create chat-based image recognition (streaming)
        • Create chat-based image recognition (streaming) best64
        • Official N test
        • Create structured output
        • Control the effort level of the inference model
        • Create chat function call
        • deepseek-ocr recognition
        • Create chat completion (non-stream)
      • Completions
        • ChatGPT automatic completion
        • Create completion
    • Image
      • Edit image
      • Create chat completion (streaming)
      • Create chat completion (qwen-mt-turbo)
      • Create chat completion with deepseek v3.1 level of reasoning (streaming)
    • Audio
      • Speech recognition
      • Speech synthesis
      • Official Function Calling invocation
      • Create chat-generated images (non-streaming)
    • Embedding
      • Text embeddings
  • Anthropic format
    • Chat
    • Chat(prompt cache)
    • Streaming response
    • Chat (deep reasoning)
    • Tool invocation (function call)
    • Analyze image
  • Google Gemini interface
    • Native format
      • Text-to-image + control over aspect ratio + clarity
      • Generate image
      • Text generation
      • Text generation - stream
      • Text generation + reasoning - stream
      • Image generation
      • Formatted output
      • Function call
      • Document understanding
      • URL context [native format]
      • Code execution
      • Video understanding
      • URL context
      • Video understanding - url [native format]
      • Imagen 4
      • Audio understanding
      • Embeddings
      • Chat
      • Edit image
    • Image-to-image Base64 request method
      • Multi-image fusion slice generation with gemini-3-pro-image-preview, controlling aspect ratio and clarity
      • Image editing
      • Single image gemini-3-pro-image-preview, controlling aspect ratio and clarity.
      • Image generation( gemini-2.5-flash-image)
      • Image generation gemini-2.5-flash-image, controlling aspect ratio.
      • Image understanding
    • Image-to-image URL request returns URL request format OpenAI
      • Single image generation with gemini-3-pro-image-preview, controlling aspect ratio and clarity.
      • Multi-image fusion slice generation with gemini-3-pro-image-preview, controlling aspect ratio and clarity.
      • Image understanding
  • NanoBanana
    • OpenAI request
      • Edit image
      • OpenAI image format
    • Gemini request
      • Generate image
      • Edit image
  • Midjourney format
    • Midjourney API Reference
    • Task query interface
    • Upload image
    • Get seed (Seed)
    • Submit Imagine task
    • Query tasks based on ID list
    • FaceSwap
    • Execute Action operation
    • /mj/submit/blend
    • Submit Describe task
    • Submit Modal
    • Refresh link
    • Edit image
    • Query task status by task ID
    • Get the seed of the task image
  • Doubao - Painting
    • doubao-seededit-3-0-i2i-250628
    • doubao-seedream-4-0-250828 - text-to-image
    • doubao-seedream-4-0-250828 - image-to-image
    • doubao-seedream-4-0-250828 - multi-image generation
  • Rerank Reordering Model
    • Rerank
  • Video Model
    • Grok Video Generation
      • 00-Overview
      • 01-Text-to-Video
      • 02-Image-to-Video
      • 03-Reference-to-Video
      • 04-Video-Editing
      • 05-Video-Extension
    • Seedance Video Generation
      • 00-Overview
      • 01-Create-Video-Generation-Task
      • 02-Query-Video-Generation-Task
      • 03-Query-Video-Generation-Task-List
      • 04-Cancel-or-Delete-Task
      • Seedance Private Asset Library API Documentation
    • MiniMax-H3 Video Generation
      • 00-Overview
      • 01-Create-Video-Generation
      • 02-Create-Video-Regeneration
      • 03-Create-H3-Context-IR
      • 04-Query-Task
      • 05-List-Tasks
      • 06-Cancel-or-Delete-Task
    • Hailuo Video Generation
      • 00-Overview
      • 01-Text-to-Video-T2V
      • 02-Image-to-Video-I2V
      • 03-First-Last-Frame-FL2V
      • 04-Subject-Reference-S2V
      • 05-Query-Task-Status
      • 06-Video-Download
      • 99-Appendix-Camera-Movement-and-Webhooks
    • Jimeng Video Generation
      • 00-Overview
      • 01-3.0-Pro-Video-Generation
      • 02-720P-Text-to-Video
      • 03-720P-Image-to-Video-First-Frame
      • 04-720P-Image-to-Video-Start-End-Frame
      • 05-720P-Image-to-Video-Camera
      • 06-1080P-Text-to-Video
      • 07-1080P-Image-to-Video-First-Frame
      • 08-1080P-Image-to-Video-Start-End-Frame
      • 09-Error-Codes
    • Kling AI Video Generation
      • 00-Overview
      • 01-Text-to-Video
      • 02-Image-to-Video
      • 03-Omni-Video
      • 04-Multi-Image-to-Video
      • 05-Motion-Control
      • 06-Multi-Elements
      • 07-Video-Extension
      • 08-Lip-Sync
      • 09-Avatar
      • 10-Text-to-Audio
      • 11-Video-to-Audio
      • 12-TTS
      • 13-Custom-Voices
      • 14-Image-Recognition
      • 15-Element-Management
      • 16-Video-Effects
    • Vidu Video Generation
      • 00-Overview
      • 01-Text-to-Video
      • 02-Image-to-Video
      • 03-Reference-to-Video
      • 04-Start-End-Frame
      • 05-Multi-Frame
      • 06-Scene-Template
      • 07-Template-Story
      • 08-Query-Tasks
    • HappyHorse
      • HappyHorse Text-to-Video
      • HappyHorse Image-to-Video (First Frame)
      • HappyHorse Reference-to-Video
      • HappyHorse Video Editing
    • Wan Video Generation
      • 00-Overview.md
      • 01-Text-to-Video
      • 02-Image-to-Video
      • 03-Reference-to-Video
      • 04-Video-Editing
      • 05-First-Last-Frame-to-Video
      • 06-Motion-Transfer-and-Character-Swap
      • 07-Digital-Human-Video
      • 08-VACE-Video-Editing
      • 09-Query-Task
  • Audio API
    • Audio API
    • Gemini TTS API
    • Google DeepMind Lyria API
    • Elevenlabs Speech to Text API Reference
    • Text-to-Music Suno
      • Task Submission
        • Generate Song (Inspiration Mode)
        • Generate Song (Custom Mode)
        • Generate Song (Continuation Mode)
        • Generate Song (Singer Style)
        • Generate Song (Secondary Creation from Uploaded Song)
        • Generate Song (Song Stitching)
        • Generate Lyrics
        • Song Stitching
      • Query Interface
        • Batch Retrieve Tasks
        • Query Single Task
  1. Audio API

Google DeepMind Lyria API

Music Generation API Reference#

Version: v1.0.0  |  Last Updated: 2026-06-18  |  Model: lyria-3-pro-preview  |  Status: Preview
This document is based on live production request/response testing. All field names, types, and values are strictly consistent with actual API responses.

Table of Contents#

1. Overview
2. Authentication
3. Available Models
4. Generate Content
4.1 Endpoint
4.2 Request Body
4.3 Response Body
4.4 Lyrics Format Specification
4.5 Music Analysis Metadata
5. Complete Examples
5.1 cURL
5.2 Python
5.3 Node.js
6. Error Codes
7. Rate Limits
8. Best Practices
9. Changelog

1. Overview#

The Music Generation API invokes Google DeepMind Lyria series models via a Gemini-compatible interface (generateContent) to deliver high-fidelity AI music generation.
CapabilityDescription
Text-to-MusicGenerate complete songs from natural language descriptions
Full-length SongGenerate complete tracks up to approximately 150 seconds in a single call
Vocal + InstrumentalSupports lead vocals and multi-track instrumental arrangement
Auto LyricsAutomatically generates lyrics with precise timestamps and section structure annotations
Music AnalysisReturns metadata including BPM, duration, and per-section instrumentation analysis
MP3 OutputOutput in standard MP3 format (audio/mpeg), ready for direct playback
Protocol Compatibility: Request/response format is fully compatible with the Google Gemini API generateContent interface.

2. Authentication#

All requests must carry the API Key in the HTTP header:
Authorization: Bearer YOUR_API_KEY
Required Request Headers:
HeaderValueRequired
AuthorizationBearer <API_KEY>Yes
Content-Typeapplication/jsonYes
Security Notice: The API Key is a sensitive credential. Never expose it in client-side code, public repositories, or logs. Inject it via environment variables or a secrets management service.

3. Available Models#

Model IDDescriptionOutput FormatMax DurationStatus
lyria-3-pro-previewGoogle DeepMind flagship music generation model. Supports high-fidelity full-song creation with lead vocals, multi-track instrumental arrangement, automatic lyrics generation, and song structure analysis.audio/mpeg (MP3)~150 secondsPreview

4. Generate Content#

4.1 Endpoint#

POST https://platform.dataeyes.ai/v1/models/{model_id}:generateContent
Path Parameters:
ParameterTypeDescription
model_idstringModel identifier. Currently available value: lyria-3-pro-preview

4.2 Request Body#

Parameter Reference#

ParameterTypeRequiredDescription
contentsarray<object>YesArray of conversation content objects.
contents[].partsarray<object>YesArray of content parts. Must contain at least one text part.
contents[].parts[].textstringYesThe generation prompt. Use natural language to describe the desired music style, instruments, mood, tempo, vocal style, etc.
generationConfigobjectNoGeneration configuration.
generationConfig.responseModalitiesarray<string>YesList of response modalities. Music generation must include "AUDIO". Including "TEXT" is recommended to also receive lyrics and analysis. Accepted values: "AUDIO", "TEXT".

Full Schema#

{
  "contents": [
    {
      "parts": [
        {
          "text": "<music generation prompt>"
        }
      ]
    }
  ],
  "generationConfig": {
    "responseModalities": ["AUDIO", "TEXT"]
  }
}

4.3 Response Body#

A successful request returns HTTP 200.

Top-Level Fields#

FieldTypeDescription
candidatesarray<object>Array of candidate results. Currently always returns exactly 1 candidate.
usageMetadataobjectToken usage statistics.
modelVersionstringThe actual model version used, e.g. "lyria-3-pro-preview".
createTimestring (ISO 8601)Response creation time, e.g. "2026-06-18T11:59:11.049072Z".
responseIdstringUnique identifier for this response, useful for troubleshooting and log correlation.

candidates[] Structure#

FieldTypeDescription
content.rolestringFixed value "model".
content.partsarray<object>Array of content parts. See detailed description below.
finishReasonstringReason generation stopped.
finishReason Values:
ValueMeaning
STOPNormal completion; output is complete.
MAX_TOKENSToken limit reached; output may be truncated.
SAFETYContent safety policy triggered; request was blocked.

content.parts[] Detailed Description#

A music generation response always contains 3 parts (verified by live testing):
IndexTypeFieldDescriptionObserved Size
Part 0TexttextLyrics: Contains section identifiers [[X0]] and precise timestamps [sec:]. See 4.4 Lyrics Format Specification.1,497 characters
Part 1TexttextMusic Analysis: BPM, duration, quality score, and per-section instrumentation/vocal/mood analysis. See 4.5 Music Analysis Metadata.7,263 characters
Part 2BinaryinlineDataAudio File: Base64-encoded MP3 data.3.39 MB (decoded)

inlineData Object#

FieldTypeDescription
mimeTypestringMIME type, always "audio/mpeg".
datastringBase64-encoded audio data. Decodes to a standard MP3 file.

usageMetadata Object#

FieldTypeDescription
promptTokenCountintegerTokens consumed by the input prompt.
candidatesTokenCountintegerTokens consumed by the generated output (includes audio tokens).
totalTokenCountintegerTotal = promptTokenCount + candidatesTokenCount.
trafficTypestringTraffic type, e.g. "ON_DEMAND".
promptTokensDetailsarray<object>Breakdown of input tokens by modality. Each entry contains modality (e.g. "TEXT") and tokenCount.
candidatesTokensDetailsarray<object>Breakdown of output tokens by modality. Each entry contains modality (e.g. "AUDIO") and tokenCount.

Live Response Example#

The following is a real response recorded on 2026-06-18 (audio data truncated):
{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": "[[A0]]\n[[B1]]\n[10.9:] The kettle's singing in the morning light,\n[:] Dusting off the shadows of the night.\n[:] My boots are heavy but my heart is high,\n[:] Matching every color in the eastern sky.\n..."
          },
          {
            "text": "mosic: 4.5\nbpm: 110.0\nduration_secs: 150.5\ngood_crop: 1.0\n[[A0]]\n[0.0:] A bright and inviting Acoustic Folk intro defined by its rhythmic clarity..."
          },
          {
            "inlineData": {
              "mimeType": "audio/mpeg",
              "data": "<Base64-encoded, 3,554,545 bytes / 3.39 MB when decoded>"
            }
          }
        ]
      },
      "finishReason": "STOP"
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 13,
    "candidatesTokenCount": 3199,
    "totalTokenCount": 3212,
    "trafficType": "ON_DEMAND",
    "promptTokensDetails": [
      { "modality": "TEXT", "tokenCount": 13 }
    ],
    "candidatesTokensDetails": [
      { "modality": "AUDIO", "tokenCount": 3199 }
    ]
  },
  "modelVersion": "lyria-3-pro-preview",
  "createTime": "2026-06-18T11:59:11.049072Z",
  "responseId": "j90zarD_Asaa9LsP5t7F8QU"
}

4.4 Lyrics Format Specification#

The text field in Part 0 uses a structured annotation format containing two types of markers: section identifiers and timestamps.

Section Identifiers [[LetterNumber]]#

The letter indicates the functional type of the section; the number is the occurrence index of that section within the full song (zero-based).
LetterTypeDescriptionObserved in Testing
AIntro / Interlude / SoloInstrumental section (intro, interlude, or solo) — no vocals[[A0]], [[A5]]
BVerseVerse section with narrative vocals[[B1]], [[B3]]
CChorusChorus section — highest energy, melodic repetition[[C2]], [[C4]], [[C6]]
DOutroClosing section — fades out and resolves[[D7]]
Note: The A type does not exclusively mean "intro." In live testing, [[A0]] was the intro while [[A5]] was a harmonica solo interlude mid-song. The letter indicates the functional role (instrumental/no vocals), not a fixed position in the song.

Timestamp Format#

MarkerMeaningExample
[seconds:]The absolute start time (in seconds) for that lyric line[10.9:] = at the 10.9-second mark
[:]Immediately follows the previous line; no independent timestamp—

Live Lyrics Structure Example#

[[A0]]                          <- Intro (instrumental only, no lyric lines)
[[B1]]                          <- Verse 1
[10.9:] The kettle's singing    <- starts at 10.9 seconds
[:] in the morning light,       <- continues immediately
[:] Dusting off the shadows...
[[C2]]                          <- Chorus
[37.1:] Oh, let the good...    <- starts at 37.1 seconds
[[B3]]                          <- Verse 2
[54.6:] I met a friend...      <- starts at 54.6 seconds
[[C4]]                          <- Chorus (repeat)
[80.8:] Oh, let the good...
[[A5]]                          <- Interlude / harmonica solo (instrumental only)
[[C6]]                          <- Chorus (final)
[111.3:] Oh, let the good...
[[D7]]                          <- Outro
[128.8:] The luckiest man...

4.5 Music Analysis Metadata#

The text field in Part 1 is a music analysis automatically generated by the model, structured as a key-value header followed by per-section descriptions.

Global Metrics (First 4 Lines of Header)#

FieldTypeDescriptionObserved Value
mosicfloatMusic Quality Score — model self-assessment, range 0–54.5
bpmfloatBeats Per Minute110.0
duration_secsfloatTotal audio duration in seconds150.5
good_cropfloatUsable crop ratio; 1.0 means the entire track is usable1.0
Note: The field name mosic is the actual value returned by the API (it is not a typo of music). Parse it as-is in your code.

Per-Section Descriptions#

After the header, each section identifier (e.g. [[A0]]) is followed by a detailed musicological analysis of that section, covering:
Analysis DimensionDescriptionExample Excerpt
InstrumentationInstruments used and how they are played"steel-string acoustic guitar playing a steady, rhythmic strumming pattern"
RhythmBeat patterns and groove characteristics"rhythmic hand-clapping on the two and four beats"
Vocal CharacteristicsVoice type, range, and singing style"male tenor with a warm, crystalline, and slightly raspy timbre"
Mood / AtmosphereEmotional tone of the section"jubilant atmosphere of collective celebration"
Musical ReferenceSimilar styles or scene associations"reminiscent of traveling singer-songwriters"
Each section description also includes a timestamp [sec:] marking the section's start time in the audio.

5. Complete Examples#

5.1 cURL#

Tip: Music generation takes considerable time (30–120 seconds). When using curl, add --max-time 180 to prevent timeout.

5.2 Python#

5.3 Node.js#


6. Error Codes#

HTTP StatusError CodeDescriptionRecommended Action
400INVALID_ARGUMENTInvalid request parameters. Missing contents, malformed parts, or responseModalities does not include "AUDIO".Verify the JSON structure against the schema in 4.2 Request Body.
401UNAUTHENTICATEDAPI Key is missing, malformed, or has expired.Confirm the header format is Authorization: Bearer <key> and that the key has not expired or been revoked.
403PERMISSION_DENIEDThe current key does not have access to the specified model.Verify that music generation permissions are enabled for your account, or contact an administrator to adjust model authorization.
429RESOURCE_EXHAUSTEDRequest rate limit exceeded or token quota exhausted.Reduce request frequency; review rate limit information in the response headers; upgrade your subscription plan if necessary.
500INTERNALInternal server error.Retry with exponential backoff (see 7. Rate Limits). If the issue persists, contact technical support with the responseId.
503UNAVAILABLEService temporarily unavailable or overloaded.Retry with exponential backoff.

7. Rate Limits#

Music generation is a compute-intensive task subject to the following constraints (specific limits depend on your subscription plan):
MetricDescription
RPM (Requests Per Minute)Maximum number of requests per minute. Exceeding the limit returns 429.
TPM (Tokens Per Minute)Maximum token consumption per minute. Audio output is measured in AUDIO tokens.
Concurrent RequestsMaximum number of simultaneously processed requests. A single generation takes 30–120 seconds; manage concurrency accordingly.

Retry Strategy#

When encountering 429 or 503, use Exponential Backoff with Jitter:
wait = min(base * 2^attempt + random(0, 1), max_wait)
ParameterRecommended Value
base (initial wait)1 second
max_wait (maximum wait)32 seconds
max_attempts (maximum retries)5

8. Best Practices#

8.1 Writing Prompts#

Be specific: Explicitly state the style (acoustic folk, electronic, jazz), instruments (guitar, piano, drums), and mood (cheerful, melancholic, energetic).
Specify vocals: You can indicate voice type and style (e.g. "male tenor vocal", "female soprano, breathy tone").
Avoid vague prompts: "make a song" produces far inferior results compared to "Create a melancholic jazz ballad with piano and saxophone, slow tempo, female vocal".

8.2 Audio Handling#

The audio in the response is Base64-encoded MP3 data. Decode it with Base64 and write the result to an .mp3 file.
In live testing, a single song is approximately 3–5 MB (decoded). Ensure the client has enough memory to handle the full response body.
Always set the HTTP timeout to at least 120 seconds; 180 seconds is recommended. Music generation requires significant model inference time.

8.3 Response Parsing#

Process parts by type (text / inlineData) individually. Do not hardcode the index or length of the parts array — future model versions may change the number of parts.
Always check finishReason; only "STOP" indicates a complete output.
Log the responseId to facilitate troubleshooting and correlation with platform records.

8.4 Cost Optimization#

Track token consumption via usageMetadata. The candidatesTokenCount for the AUDIO modality is the primary cost driver.
In production, consider deduplication or caching for identical prompts to avoid redundant generation.
During development and testing, use short prompts to keep costs low.

9. Changelog#

DateVersionChanges
2026-06-18v1.0.0Initial release. Supports lyria-3-pro-preview model.

This document is based on live testing conducted on 2026-06-18. Test prompt: "Create a cheerful acoustic folk song with guitar and harmonica, bright mood". Live response: promptTokenCount=13, candidatesTokenCount=3199, totalTokenCount=3212, audio 3.39 MB, duration 150.5 seconds.
Previous
Gemini TTS API
Next
Elevenlabs Speech to Text API Reference