ActionCue AI Service Integration Guide

Complete technical guide for integrating external AI services with ActionCue. This documentation covers the full integration process, API specifications, call flows, and configuration requirements for AI service providers.

Target Audience: Technical administrators, developers, AI service providers
Prerequisites: Understanding of REST APIs, authentication methods, and multimedia processing
Integration Scope: External AI service providers, MangoApps Forms integration, file processing workflows

🏗️ Integration Architecture Overview

System Components

ActionCue operates as an orchestration layer between MangoApps and external AI processing services, managing the complete workflow from file upload to form population.

Integration Layers

1. User Interface Layer

• File upload interface with drag-and-drop support
• Processing status monitoring and real-time updates
• Result review interface with confidence scoring
• Form field mapping and approval workflow

2. Orchestration Layer

• Processing job management and status tracking
• File security and access control
• Form schema analysis and field mapping
• Error handling and retry mechanisms

3. AI Integration Layer

• External AI service communication
• Authentication and security management
• Request/response transformation
• Webhook handling for asynchronous processing

4. Data Layer

• Secure file storage with presigned URLs
• Processing job state management
• Extracted results storage and versioning
• Integration with MangoApps Forms database

🔄 Complete Processing Call Flow

Phase 1: Job Initiation and File Handling

Phase 2: AI Service Processing

Phase 3: Result Processing and Form Integration

🔌 AI Service Provider Requirements

Technical Prerequisites

Endpoint Requirements:

• HTTPS-accessible REST API endpoint
• JSON request/response handling capability
• Support for Bearer token or custom authentication
• File download capability from secure URLs
• Processing timeout support (120-600 seconds)

Processing Capabilities:

• Multimedia file analysis (video, audio, documents, images)
• Structured data extraction for form fields
• Confidence scoring for extraction quality
• Source attribution for extracted data

API Contract Specification

Main Processing Endpoint

Endpoint: POST /process (or your custom path)

Request Headers:

Content-Type: application/json
Authorization: Bearer {api_key}
User-Agent: ActionCue/1.0 (MangoApps)

Request Payload Structure:

{
  "processing_job": {
    "job_id": "550e8400-e29b-41d4-a716-446655440000",
    "business_context": {
      "business_name": "ACME Corporation",
      "industry": "Technology",
      "context": "Employee onboarding processes"
    },
    "processing_options": {
      "batch_processing": true,
      "custom_prompts": false,
      "require_approval": false
    }
  },
  
  "input_data": {
    "files": [
      {
        "file_id": 123,
        "type": "video",
        "url": "https://secure-download.workforce.mangoapps.com/files/abc123",
        "filename": "onboarding_video.mp4",
        "size_bytes": 52428800,
        "mime_type": "video/mp4",
        "duration_seconds": 180,
        "metadata": {
          "duration": 180,
          "has_audio": true,
          "resolution": "1920x1080"
        }
      }
    ],
    "text_input": "Please extract employee information from the onboarding video",
    "user_instructions": "Focus on personal details, job title, and start date information"
  },
  
  "target_form": {
    "form_title": "Employee Onboarding Form",
    "form_url": "https://company.workforce.mangoapps.com/forms/employee-onboarding",
    "form_type": "forms_app",
    "fields": [
      {
        "id": "employee_name",
        "name": "employee_name",
        "label": "Full Name",
        "type": "text",
        "required": true,
        "description": "Employee's full legal name",
        "ai_hints": ["Look for name mentioned in speech", "Check for name on documents"]
      }
    ]
  },
  
  "processing_options": {
    "confidence_threshold": 0.75,
    "language": "en",
    "prioritize_accuracy": true,
    "api_timeout": 120
  }
}

Required Response Format:

{
  "processing_result": {
    "job_id": "550e8400-e29b-41d4-a716-446655440000",
    "status": "completed",
    "processing_time_ms": 15000,
    "overall_confidence": 0.87,
    "external_job_id": "ext-job-789"
  },
  
  "extracted_data": [
    {
      "field_id": "employee_name",
      "extracted_value": "John Smith",
      "confidence_score": 0.95,
      "data_source": "speech",
      "extraction_context": "Name mentioned at 0:15 in video: 'Hi, I'm John Smith'",
      "source_metadata": {
        "timestamp": "00:00:15",
        "location": "video audio track"
      }
    }
  ],
  
  "processing_report": {
    "summary": "Successfully extracted 4 of 4 form fields with high confidence",
    "recommendations": [
      "All fields extracted successfully",
      "Video audio quality was excellent for speech recognition"
    ],
    "processing_metadata": {
      "files_processed": 1,
      "total_fields": 4,
      "fields_extracted": 4,
      "average_confidence": 0.9,
      "processing_duration_ms": 15000
    }
  }
}

⚙️ ActionCue Configuration Setup

Marketplace App Configuration

Enable ActionCue:

1. Navigate to Apps → Marketplace
2. Find ActionCue in AI Tools category
3. Click Enable for your business
4. Access configuration through Apps → ActionCue → Configure

Core Configuration Fields:

{
  "api_provider": "custom",
  "api_endpoint": "https://your-ai-service.com/api/v1/process",
  "api_key": "your-secure-api-key",
  "api_timeout": 120,
  "auth_header": "Authorization",
  "auth_format": "Bearer",
  
  "processing_mode": "async",
  "async_webhook_enabled": true,
  "webhook_secret": "your-webhook-secret",
  "polling_interval_seconds": 30,
  
  "max_file_size_mb": 100,
  "supported_video_formats": ["mp4", "mov", "avi", "mkv"],
  "supported_audio_formats": ["mp3", "wav", "m4a", "aac"],
  "supported_document_formats": ["pdf", "docx", "txt"],
  "supported_image_formats": ["jpg", "jpeg", "png", "gif"],
  "confidence_threshold": 75,
  
  "forms_app_integration": true,
  "allow_external_forms": true,
  "require_approval": false,
  "enable_batch_processing": false,
  "data_retention_days": 30
}

Authentication Configuration

Bearer Token Authentication (Recommended):

{
  "auth_header": "Authorization",
  "auth_format": "Bearer",
  "api_key": "your-api-token"
}

Custom Authentication Headers:

{
  "auth_header": "X-API-Key",
  "auth_format": "",
  "api_key": "your-custom-key"
}

Enhanced Security with Secrets:

{
  "auth_header": "Authorization",
  "auth_format": "Bearer",
  "api_key": "your-api-token",
  "additional_headers": {
    "X-API-Secret": "your-api-secret"
  }
}

🔒 Security and File Access

File Security Model

Secure File URLs:

• Files stored in AWS S3 with enterprise-grade encryption
• Time-limited presigned URLs (24-hour expiration)
• Unique URLs generated per processing request
• No permanent storage required on AI service side

Example Presigned URL:

https://mangoops-files.s3.amazonaws.com/files/abc123?
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Credential=...&
X-Amz-Date=20250115T120000Z&
X-Amz-Expires=86400&
X-Amz-Signature=...

File Access Requirements:

# Example file download implementation
import requests

def download_file(file_url, local_path):
    response = requests.get(file_url, stream=True)
    response.raise_for_status()
    
    with open(local_path, 'wb') as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)
    
    return local_path

Data Privacy Requirements

Processing Guidelines:

• Process files in memory when possible
• Delete temporary files immediately after processing
• Do not log sensitive extracted data
• Implement data retention policies (maximum 30 days)
• Comply with GDPR, CCPA, and other privacy regulations

Security Checklist:

• HTTPS endpoints with valid SSL certificates
• Secure API key storage and rotation
• File download over encrypted connections
• Immediate cleanup of temporary processing files
• Audit logging for processing activities
• Data encryption at rest and in transit

🔄 Asynchronous Processing (Optional)

Webhook Integration

For long-running processing jobs, ActionCue supports webhook notifications:

Webhook Endpoint:

POST https://your-mangoops-instance.com/api/v1/actioncue/webhooks/job_status/{job_uuid}

Webhook Payload:

{
  "job_uuid": "550e8400-e29b-41d4-a716-446655440000",
  "external_job_id": "ext-job-789",
  "status": "completed",
  "progress": 100,
  "confidence_score": 0.87,
  "error_message": null,
  "completed_at": "2025-01-15T12:05:00Z",
  "extracted_results": {
    // Same format as main processing response
  }
}

Webhook Authentication:

// HMAC-SHA256 signature verification
const crypto = require('crypto');

function verifyWebhook(payload, signature, secret) {
  const expectedSignature = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex');
    
  return `sha256=${expectedSignature}` === signature;
}

Status Polling

Polling Endpoint:

GET /jobs/{external_job_id}/status

Response Format:

{
  "status": "processing|completed|failed",
  "progress": 75,
  "confidence_score": 0.85,
  "error_message": null,
  "completed_at": "2025-01-15T12:05:00Z",
  "processing_time_ms": 300000,
  "extracted_results": {
    // Complete results when status is "completed"
  }
}

📋 Form Field Type Support

Complete Field Type Reference

ActionCue supports all MangoApps Forms field types. Critical field types and their output requirements:

Text Fields:

{
  "field_id": "employee_name",
  "field_type": "text",
  "extracted_value": "John Smith",
  "confidence_score": 0.95,
  "data_source": "speech"
}

Date Fields:

{
  "field_id": "start_date",
  "field_type": "date",
  "extracted_value": "2025-02-01",
  "confidence_score": 0.92,
  "data_source": "combined"
}

Selection Fields:

{
  "field_id": "department",
  "field_type": "select",
  "extracted_value": "Engineering",
  "confidence_score": 0.85,
  "data_source": "contextual",
  "available_options": ["Engineering", "Sales", "Marketing", "HR"]
}

Multi-Selection Fields:

{
  "field_id": "skills",
  "field_type": "checkbox",
  "extracted_value": ["JavaScript", "Python", "React"],
  "confidence_score": 0.88,
  "data_source": "text"
}

Field Validation Requirements

Data Format Validation:

• Date fields: YYYY-MM-DD format
• DateTime fields: ISO 8601 format
• Time fields: HH:MM:SS format
• Email fields: Valid email format
• Phone fields: Normalized phone format
• URLs: Valid URL format with protocol

Selection Validation:

• Single select: Must match one of the provided option values
• Multi-select: Array of values, each matching provided options
• Case-sensitive matching required
• Invalid selections should be flagged with low confidence

🚨 Error Handling and Troubleshooting

Common Error Scenarios

File Access Errors:

{
  "error": {
    "code": "FILE_ACCESS_ERROR",
    "message": "Unable to download file from provided URL",
    "details": {
      "file_url": "https://...",
      "http_status": 403,
      "timestamp": "2025-01-15T12:00:00Z"
    }
  }
}

Processing Timeout:

{
  "error": {
    "code": "PROCESSING_TIMEOUT",
    "message": "Processing exceeded maximum allowed time",
    "details": {
      "timeout_seconds": 120,
      "partial_results": {
        // Any results processed before timeout
      }
    }
  }
}

Unsupported File Format:

{
  "error": {
    "code": "UNSUPPORTED_FORMAT",
    "message": "File format not supported for processing",
    "details": {
      "file_type": "wmv",
      "supported_formats": ["mp4", "mov", "avi", "webm"]
    }
  }
}

Retry and Recovery Mechanisms

ActionCue Retry Logic:

• Automatic retry for temporary failures (network issues, rate limits)
• Exponential backoff with jitter (2s, 4s, 8s intervals)
• Maximum 3 retry attempts for recoverable errors
• No retry for permanent failures (authentication, unsupported formats)

Rate Limit Handling:

{
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "Too many requests",
    "retry_after": 3600
  }
}

ActionCue will automatically respect Retry-After headers and implement appropriate delays.

🧪 Testing and Validation

Connection Testing

Test Payload:

{
  "processing_job": {
    "job_id": "test-connection",
    "business_context": {
      "business_name": "Test Business"
    }
  },
  "input_data": {
    "text_input": "This is a test connection.",
    "user_instructions": "Test API connectivity"
  },
  "target_form": {
    "form_title": "Test Form",
    "fields": [
      {
        "id": "test_field",
        "name": "test_field",
        "type": "text",
        "label": "Test Field"
      }
    ]
  }
}

Expected Response:

{
  "processing_result": {
    "job_id": "test-connection",
    "status": "completed",
    "processing_time_ms": 100
  },
  "extracted_data": [
    {
      "field_id": "test_field",
      "extracted_value": "test connection successful",
      "confidence_score": 1.0,
      "data_source": "text",
      "extraction_context": "API connectivity test"
    }
  ]
}

Integration Testing Checklist

Prerequisites:

• ActionCue marketplace app enabled
• AI service endpoint accessible via HTTPS
• API credentials configured and tested
• Test files prepared (small samples of each supported type)

Basic Connectivity:

• Health check endpoint responds correctly
• Authentication accepts provided credentials
• Test processing request returns expected format
• Error responses include proper HTTP status codes

File Processing:

• Video files process successfully
• Audio files extract speech-to-text
• Documents extract text content
• Images process with visual recognition
• Confidence scores are realistic (0.0-1.0 range)

Integration Flow:

• Form field mapping works correctly
• Extracted data formats match field types
• Processing reports include useful metadata
• Webhook notifications work (if enabled)
• Error scenarios are handled gracefully

📈 Performance and Optimization

Performance Guidelines

Response Time Targets:

• Simple text extraction: < 30 seconds
• Document OCR: < 60 seconds
• Audio transcription: < 90 seconds
• Video analysis: < 120 seconds

File Size Recommendations:

• Video files: Up to 500MB (configurable)
• Audio files: Up to 100MB
• Documents: Up to 50MB
• Images: Up to 20MB per file

Optimization Strategies:

• Process files in parallel when possible
• Use appropriate AI model sizes for speed vs. accuracy tradeoffs
• Implement efficient file download and caching
• Provide early confidence estimates for long-running jobs

Monitoring and Analytics

Key Metrics to Track:

• Processing success rate
• Average processing time by file type
• Confidence score distribution
• Error rates and common failure modes
• User satisfaction with extraction accuracy

ActionCue provides:

• Processing job status dashboard
• Quality metrics and confidence scoring trends
• File type and size analytics
• Integration performance monitoring

🔗 Advanced Integration Options

Custom Request Formatting

For AI services requiring specific payload formats:

{
  "custom_formatter": "actioncue_standard",
  "custom_transformations": {
    "flatten_file_metadata": true,
    "include_business_context": false,
    "custom_field_mapping": {
      "job_id": "request_id",
      "files": "input_files"
    }
  }
}

Batch Processing Support

{
  "enable_batch_processing": true,
  "batch_size_limit": 10,
  "batch_timeout_seconds": 600,
  "supports_batch_polling": true
}

Batch Request Format:

{
  "batch_processing": true,
  "processing_jobs": [
    {
      "job_id": "batch-job-1",
      // Standard processing job structure
    },
    {
      "job_id": "batch-job-2",
      // Another job in the batch
    }
  ]
}

Custom AI Prompts

{
  "custom_ai_prompts": true,
  "prompt_templates": {
    "incident_analysis": "Focus on extracting incident details including time, location, people involved, and safety concerns",
    "employee_onboarding": "Extract personal information, job details, and required training completion status"
  }
}

📞 Support and Resources

Implementation Support

Getting Started:

1. Review this complete integration guide
2. Set up test environment with sample files
3. Implement basic /process endpoint
4. Test with ActionCue connection validator
5. Configure production settings and go live

Technical Support:

• Integration Questions: MangoApps developer support team
• API Issues: Submit support tickets with detailed error logs
• Performance Optimization: Consult with AI service provider and MangoApps
• Security Reviews: Contact MangoApps security team

Additional Resources

Documentation Links:

• MangoApps Forms API Documentation
• API Authentication Guide
• Security Best Practices
• ActionCue Overview for business context

Developer Tools:

• OpenAPI specification for ActionCue external provider contract
• Postman collection for API testing
• Sample implementations in Python, Node.js, and other languages
• Test file samples for each supported format

This integration guide provides complete technical specifications for ActionCue AI service integration. For business context and user guidance, refer to the ActionCue Overview article.