🧬 Copilot-LD

An intelligent agent leveraging GitHub Copilot and Linked Data

Deployment Guide

This guide covers how to deploy and run Copilot-LD in production using either Docker Compose or AWS CloudFormation. Choose the deployment method that best fits your infrastructure requirements.

Prerequisites

Note: For GitHub Actions workflows, ensure S3 access credentials are configured as repository secrets.

Docker Compose Deployment

The Docker Compose deployment provides a complete containerized stack with application load balancing, object storage, and all microservices.

Architecture Overview

The Docker Compose stack includes:

SSL Certificate Setup

Configure SSL certificates for secure HTTPS access. See the SSL Certificate Configuration section in the Configuration Guide for detailed setup instructions including self-signed certificate generation and production certificate installation.

Configuration

Configure environment variables and service settings. See the Configuration Guide for complete details on all environment variables and YAML configuration options.

Deploy the Stack

The platform uses a unified Docker build process with a single Dockerfile that handles all components (services, extensions, and tools). Building the containers requires a GitHub Personal Access Token (PAT) with at minimum the read:packages scope. Put the token in config/.build_token and deploy the stack like this: 

# Build all container images using unified Dockerfile
GITHUB_TOKEN=$(cat config/.build_token) docker compose build

# Start the complete stack
docker compose up -d

# Check service status
docker compose ps

Access Points

Once deployed, access the system via:

AWS CloudFormation Deployment

Deploy Copilot-LD on AWS using ECS Fargate with CloudFormation for production-grade scalability and managed infrastructure.

Infrastructure Naming Standard

All AWS resources follow a consistent naming pattern: ${EnvironmentName}-cld-<Component>-<resourcetype>-cf

Component Examples
EnvironmentName Environment identifier (e.g., demo, prod)
cld Copilot-LD project identifier
component Single string: network, agent, web, public1
resourcetype vpc, subnet, cluster, service, taskdef
cf CloudFormation deployment indicator

Example: demo-cld-agent-service-cf

AWS OIDC Setup for GitHub Actions

Before using the GitHub Actions workflows for automated deployment, you must configure AWS OpenID Connect (OIDC) to allow secure, keyless authentication from GitHub Actions to your AWS account.

1. Create GitHub OIDC Identity Provider in AWS

First, create an OIDC identity provider in your AWS account:

# Using AWS CLI
aws iam create-open-id-connect-provider \
  --url https://token.actions.githubusercontent.com \
  --client-id-list sts.amazonaws.com \
  --thumbprint-list 6938fd4d98bab03faadb97b34396831e3780aea1 \
  --thumbprint-list 1c58a3a8518e8759bf075b76b750d4f2df264fcd

Or create via AWS Console:

Reference: GitHub Docs: Configuring OIDC in AWS

2. Create IAM Role for GitHub Actions

Create an IAM role that GitHub Actions can assume. Save this as github-actions-role.json: 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Principal": {
                "Federated": "arn:aws:iam::YOUR_ACCOUNT_ID:oidc-provider/token.actions.githubusercontent.com"
            },
            "Condition": {
                "StringEquals": {
                    "token.actions.githubusercontent.com:aud": [
                        "sts.amazonaws.com"
                    ]
                },
                "StringLike": {
                    "token.actions.githubusercontent.com:sub": [
                        "repo:copilot-ld/copilot-ld:*"
                    ]
                }
            }
        }
    ]
}

Create the role and attach policies:

# Create the role
aws iam create-role \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --assume-role-policy-document file://github-actions-role.json

# Attach CloudFormation permissions
aws iam attach-role-policy \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --policy-arn arn:aws:iam::aws:policy/CloudFormationFullAccess

# Attach EC2 permissions for network infrastructure deployment
aws iam attach-role-policy \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess

# Attach ECS permissions for service deployment
aws iam attach-role-policy \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --policy-arn arn:aws:iam::aws:policy/AmazonECS_FullAccess

# Attach IAM permissions for role creation
aws iam attach-role-policy \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --policy-arn arn:aws:iam::aws:policy/IAMFullAccess

# Attach S3 permissions for data management
aws iam attach-role-policy \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess

# Attach Secrets Manager permissions
aws iam attach-role-policy \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --policy-arn arn:aws:iam::aws:policy/SecretsManagerReadWrite

# Attach CloudWatch Logs permissions for log group management
aws iam attach-role-policy \
  --role-name GitHubActions-CopilotLD-Demo-Deploy \
  --policy-arn arn:aws:iam::aws:policy/CloudWatchLogsFullAccess

Security Note: For production environments, replace *FullAccess policies with more restrictive, principle-of-least-privilege policies tailored to your specific deployment needs.

3. Configure GitHub Repository Secrets

Add the following secrets to your GitHub repository (Settings → Secrets and variables → Actions):

4. Workflow Usage

The configured OIDC role enables secure access in four deployment workflows:

For detailed CloudFormation deployment commands and parameters, refer to the actual workflow files in .github/workflows/. These workflows contain the most current deployment procedures and parameter configurations.

Reference: AWS Docs: Creating OIDC Identity Providers

Automated Deployment

Use the GitHub Actions workflows for automated deployment. The workflows handle all CloudFormation stack creation, parameter management, and deployment orchestration. See the workflow files for specific implementation details and current deployment procedures.

Deployment Order

Deploy the CloudFormation stacks in the following order using the GitHub Actions workflows:

  1. Network Infrastructure: Deploy demo-network.yml workflow first to create the VPC, subnets, NAT gateways, and routing infrastructure
  2. Secrets Management: Deploy demo-secrets.yml workflow to create AWS Secrets Manager resources for secure credential storage
  3. Data Storage: Deploy demo-data.yml workflow to create S3 bucket and IAM roles for data access
  4. Services: Deploy demo-services.yml workflow to create ECS services, load balancer, and application infrastructure

Each stack outputs the necessary parameters for the next stack in the deployment chain. The network stack provides VPC and subnet IDs required by the services stack.

Data Upload

Upload your processed knowledge base to S3 for deployment. For complete data management instructions including upload configuration, download utilities, and storage management, see the Data Management Utilities section in the Processing Guide.

Production Considerations

Security

Troubleshooting

Docker Compose Issues

AWS CloudFormation Issues

GitHub Actions OIDC Issues

Debug OIDC authentication issues:

# Validate OIDC provider exists
aws iam get-open-id-connect-provider \
  --open-id-connect-provider-arn arn:aws:iam::YOUR_ACCOUNT_ID:oidc-provider/token.actions.githubusercontent.com

# Check role trust policy
aws iam get-role --role-name GitHubActions-CopilotLD-Deploy --query 'Role.AssumeRolePolicyDocument'

# Test role assumption locally (requires AWS STS GetCallerIdentity)
aws sts get-caller-identity

Common Deployment Validation

Validate your deployment:

# Docker Compose validation
docker-compose ps
curl -k https://localhost/web/health

# AWS ECS validation
aws ecs list-services --cluster copilot-ld
aws elbv2 describe-target-health --target-group-arn arn:aws:elasticloadbalancing:...

Maintenance

Updates

Backup