Batch Operations

Submit multiple jobs as a batch for efficient processing with centralized tracking and notifications.

Create Batch

POST /api/v1/jobs/batch

Submit multiple jobs as a batch with optional webhook notifications.

Request Body

{
  "batch_id": "my-custom-batch-id",
  "jobs": [
    {
      "type": "image_resize",
      "source_bucket": "my-bucket",
      "source_key": "image1.jpg",
      "dest_bucket": "my-bucket",
      "dest_key": "image1-resized.jpg",
      "parameters": {
        "width": 800,
        "height": 600
      }
    },
    {
      "type": "image_optimize",
      "source_bucket": "my-bucket",
      "source_key": "image2.jpg",
      "dest_bucket": "my-bucket",
      "dest_key": "image2-optimized.webp",
      "parameters": {
        "format": "webp",
        "quality": 85
      }
    }
  ],
  "options": {
    "webhook_url": "https://your-domain.com/webhook",
    "fail_fast": false
  }
}

Parameters

Parameter	Type	Required	Description
batch_id	string	❌	Custom batch ID (auto-generated if not provided)
jobs	array	✅	Array of job requests (1-100 jobs)
options	object	❌	Batch processing options

Job Object

Field	Type	Required	Description
type	string	✅	Job type (see Job Types)
source_bucket	string	✅	Source S3 bucket
source_key	string	✅	Source S3 key
dest_bucket	string	✅	Destination S3 bucket
dest_key	string	✅	Destination S3 key
parameters	object	❌	Job-specific parameters

Batch Options

Option	Type	Description
webhook_url	string	URL to call when batch completes
fail_fast	boolean	Stop processing on first failure (default: false)

Response

{
  "batch_id": "batch-550e8400-e29b-41d4-a716-446655440000",
  "total_jobs": 2,
  "job_ids": [
    "job-1-550e8400-e29b-41d4-a716-446655440001",
    "job-2-550e8400-e29b-41d4-a716-446655440002"
  ],
  "status": "queued"
}

Job Types

image_resize

Resize images to specific dimensions

image_optimize

Optimize and compress images

image_watermark

Add watermarks to images

image_colors

Extract dominant colors

image_ai_keywords

Generate AI keywords

video_thumbnail

Extract video thumbnails

audio_convert

Convert audio formats

analyze

Send to external AI services

Get Batch Status

GET /api/v1/jobs/batch/{id}

Get the status of a batch and optionally include job details.

Query Parameters

Parameter	Type	Description
include_jobs	boolean	Include individual job details (default: false)

Response

{
  "batch": {
    "id": "batch-550e8400-e29b-41d4-a716-446655440000",
    "status": "processing",
    "total_jobs": 10,
    "pending": 3,
    "completed": 6,
    "failed": 1,
    "created_at": "2025-10-18T22:00:00Z",
    "updated_at": "2025-10-18T22:05:00Z"
  },
  "jobs": [
    {
      "id": "job-1",
      "type": "image_resize",
      "status": "completed",
      "created_at": "2025-10-18T22:00:00Z"
    }
  ]
}

Batch Status Values

queued

Batch created, jobs are being queued

processing

Jobs are being processed

completed

All jobs completed successfully

failed

All jobs failed

partial

Some jobs succeeded, some failed

List Batches

GET /api/v1/jobs/batch

List all batches with optional filtering.

Query Parameters

Parameter	Type	Description
status	string	Filter by status (queued, processing, completed, failed, partial)
limit	integer	Maximum results to return (default: 50, max: 100)

Example

curl -H "X-API-Key: your_api_key_here" \
  "http://localhost:8080/api/v1/jobs/batch?status=completed&limit=10"

Examples

Resize Multiple Images

cURL

curl -X POST http://localhost:8080/api/v1/jobs/batch \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your_api_key_here" \
  -d '{
    "jobs": [
      {
        "type": "image_resize",
        "source_bucket": "my-bucket",
        "source_key": "photo1.jpg",
        "dest_bucket": "my-bucket",
        "dest_key": "photo1-thumb.jpg",
        "parameters": { "width": 300, "height": 300 }
      },
      {
        "type": "image_resize",
        "source_bucket": "my-bucket",
        "source_key": "photo2.jpg",
        "dest_bucket": "my-bucket",
        "dest_key": "photo2-thumb.jpg",
        "parameters": { "width": 300, "height": 300 }
      }
    ],
    "options": {
      "webhook_url": "https://myapp.com/batch-complete"
    }
  }'

JavaScript

const response = await fetch('http://localhost:8080/api/v1/jobs/batch', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-API-Key': 'your_api_key_here'
  },
  body: JSON.stringify({
    jobs: [
      {
        type: 'image_resize',
        source_bucket: 'my-bucket',
        source_key: 'photo1.jpg',
        dest_bucket: 'my-bucket',
        dest_key: 'photo1-thumb.jpg',
        parameters: { width: 300, height: 300 }
      },
      {
        type: 'image_resize',
        source_bucket: 'my-bucket',
        source_key: 'photo2.jpg',
        dest_bucket: 'my-bucket',
        dest_key: 'photo2-thumb.jpg',
        parameters: { width: 300, height: 300 }
      }
    ],
    options: {
      webhook_url: 'https://myapp.com/batch-complete'
    }
  })
});
const data = await response.json();

Mixed Job Types

Process different types of media in a single batch:

{
  "jobs": [
    {
      "type": "image_resize",
      "source_bucket": "my-bucket",
      "source_key": "photo.jpg",
      "dest_bucket": "my-bucket",
      "dest_key": "photo-small.jpg",
      "parameters": { "width": 800, "height": 600 }
    },
    {
      "type": "video_thumbnail",
      "source_bucket": "my-bucket",
      "source_key": "video.mp4",
      "dest_bucket": "my-bucket",
      "dest_key": "video-thumb.jpg",
      "parameters": { "timestamp": "00:00:05" }
    },
    {
      "type": "audio_convert",
      "source_bucket": "my-bucket",
      "source_key": "audio.wav",
      "dest_bucket": "my-bucket",
      "dest_key": "audio.mp3",
      "parameters": { "format": "mp3", "bitrate": "192k" }
    }
  ]
}

Fail-Fast Mode

When to Use Fail-Fast

Enable fail_fast when jobs are dependent on each other or when you want to stop processing immediately if something goes wrong.

{
  "jobs": [...],
  "options": {
    "fail_fast": true
  }
}

With fail_fast: true:

• Processing stops on the first job failure
• Remaining jobs are cancelled
• Batch status becomes failed

With fail_fast: false (default):

• All jobs are processed regardless of failures
• Batch status becomes partial if some jobs fail
• Batch status becomes completed only if all jobs succeed

Webhook Notifications

Receive a webhook when the batch completes:

{
  "event": "batch.completed",
  "timestamp": "2025-10-18T22:10:00Z",
  "batch": {
    "id": "batch-550e8400",
    "status": "completed",
    "total_jobs": 10,
    "completed": 10,
    "failed": 0
  }
}

Learn more in the Webhooks Guide.

Use Cases

Bulk Image Processing

Resize or optimize hundreds of images at once

Media Library Migration

Convert entire media libraries to new formats

Automated Workflows

Process uploads from users or external systems

Scheduled Jobs

Run batch processing on a schedule (cron)

Limits

Batch Limits

• Maximum jobs per batch: 100
• Minimum jobs per batch: 1
• Concurrent batches: No limit (subject to worker capacity)

Best Practices

1. Use Custom Batch IDs

   Provide meaningful batch IDs for easier tracking:

   "batch_id": "user-123-upload-2025-10-18"

2. Enable Webhooks

   Get notified when batches complete instead of polling

3. Monitor Progress

   Check batch status periodically for long-running batches

4. Handle Partial Failures

   Check individual job statuses when batch status is partial

Error Handling

Common Errors

• 400 Bad Request: Invalid job configuration or too many jobs
• 401 Unauthorized: Missing or invalid API key
• 500 Internal Error: Server error during batch creation

Next Steps

• Learn about Job Management
• Set up Webhooks for notifications
• Explore Image Operations