Building a Continuous AI Workflow with PostHog and GitHub

cn -p "Help me set up my environment for PostHog session analysis. Please:
1. Check if I have the required secrets configured in Continue CLI:
   - POSTHOG_API_KEY
   - POSTHOG_PROJECT_ID
   - POSTHOG_HOST
   - GITHUB_PAT
2. If any are missing, guide me through:
   - How to get PostHog Personal API Key from PostHog settings
   - How to find my PostHog Project ID from the URL
   - How to create a GitHub Personal Access Token with issues:write permission
   - How to add these as secrets in Continue CLI
3. Help me identify my GitHub repository owner and name by:
   - Checking git remote origin URL
   - Or asking me to provide them
4. Test the API connections by making a simple API call to both PostHog and GitHub to verify the credentials work.
Please walk me through this setup process step by step."

cn -p "I need you to help me analyze PostHog session recordings to identify UX issues. Please:

1. First, fetch session recordings from PostHog using these details:
   - Use the POSTHOG_API_KEY, POSTHOG_PROJECT_ID, and POSTHOG_HOST from my secrets
   - API endpoint: {POSTHOG_HOST}/api/projects/{POSTHOG_PROJECT_ID}/session_recordings/?limit=20
   - Include Authorization header: 'Bearer {POSTHOG_API_KEY}'

2. Filter the sessions to find problematic ones with either:
   - console_error_count > 0 (JavaScript errors)
   - recording_duration > 300 (sessions longer than 5 minutes)

3. Analyze the filtered sessions and create exactly 3 GitHub issues for the most critical UX problems.

4. Format each issue like this:
### 1. **Issue Title Here**
**Problem**: One sentence describing the user problem
**Technical Causes**: Brief technical explanation
**Affected Pages**: List the URLs
**Recommended Fix**:
- Specific action item 1
- Specific action item 2
**Priority**: **HIGH** or **MEDIUM** or **LOW**
**Impact**: Quantify the impact (e.g., 'Affects 80% of sessions')

Please make the API call, analyze the data, and provide the 3 formatted issues. Base all recommendations on actual patterns in the session data."

cn -p "I have analyzed PostHog session data and identified UX issues. Now I need you to create GitHub issues using the GitHub API.

Use these details from my secrets:
- GITHUB_PAT: My GitHub Personal Access Token
- Repository: Extract from git remote or ask me to specify

For each issue in my analysis:
1. Parse the issue title, body content, and priority level
2. Create a GitHub issue via API POST to: https://api.github.com/repos/{owner}/{repo}/issues
3. Use these headers:
   - Authorization: token {GITHUB_PAT}
   - Accept: application/vnd.github.v3+json
   - Content-Type: application/json
4. Set labels based on priority:
   - HIGH: [\"bug\", \"high-priority\", \"user-experience\", \"automated\"]
   - MEDIUM: [\"enhancement\", \"medium-priority\", \"user-experience\", \"automated\"]
   - LOW: [\"low-priority\", \"user-experience\", \"automated\"]
5. Format the issue title as: \"🔍 UX Issue: {original title}\"

Please create the GitHub issues and confirm they were successfully created with issue numbers and URLs.
"

name: PostHog Session Analysis with Continue CLI

on:
  schedule:
    - cron: "0 6 * * *" # Run at 6 AM UTC daily
  workflow_dispatch: # Allow manual triggering

jobs:
  analyze:
    runs-on: ubuntu-latest

    permissions:
      contents: read
      issues: write

    steps:
      - uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "18"

      - name: Install Continue CLI
        run: |
          npm install -g @continuedev/cli
          echo "✅ Continue CLI installed"
          cn --version

      - name: Set up environment variables
        run: |
          echo "POSTHOG_API_KEY=${{ secrets.POSTHOG_API_KEY }}" >> $GITHUB_ENV
          echo "POSTHOG_PROJECT_ID=${{ secrets.POSTHOG_PROJECT_ID }}" >> $GITHUB_ENV
          echo "POSTHOG_HOST=${{ secrets.POSTHOG_HOST || 'https://us.posthog.com' }}" >> $GITHUB_ENV
          echo "GITHUB_PAT=${{ secrets.GITHUB_PAT }}" >> $GITHUB_ENV
          echo "GITHUB_OWNER=${{ github.repository_owner }}" >> $GITHUB_ENV
          echo "GITHUB_REPO=$(echo ${{ github.repository }} | cut -d'/' -f2)" >> $GITHUB_ENV

      - name: Authenticate with Continue CLI
        run: |
          # Set Continue API key
          export CONTINUE_API_KEY="${{ secrets.CONTINUE_API_KEY }}"
          # Login to Continue CLI
          if ! cn auth login --api-key "$CONTINUE_API_KEY"; then
            echo "❌ Failed to authenticate with Continue CLI"
            exit 1
          fi
          echo "✅ Continue CLI authenticated"

      - name: Run PostHog Session Analysis
        run: |
          echo "🎬 Starting PostHog session analysis..."
          # Create the analysis prompt
          cat > analysis_prompt.txt << 'EOF'
          I need you to help me analyze PostHog session recordings to identify UX issues. Please:

          1. First, fetch session recordings from PostHog using these details:
             - Use the POSTHOG_API_KEY, POSTHOG_PROJECT_ID, and POSTHOG_HOST from environment variables
             - API endpoint: $POSTHOG_HOST/api/projects/$POSTHOG_PROJECT_ID/session_recordings/?limit=20
             - Include Authorization header: "Bearer $POSTHOG_API_KEY"

          2. Filter the sessions to find problematic ones with either:
             - console_error_count > 0 (JavaScript errors)
             - recording_duration > 300 (sessions longer than 5 minutes)

          3. Analyze the filtered sessions and create exactly 3 GitHub issues for the most critical UX problems.

          4. Format each issue like this:
          ### 1. **Issue Title Here**
          **Problem**: One sentence describing the user problem
          **Technical Causes**: Brief technical explanation
          **Affected Pages**: List the URLs
          **Recommended Fix**:
          - Specific action item 1
          - Specific action item 2
          **Priority**: **HIGH** or **MEDIUM** or **LOW**
          **Impact**: Quantify the impact (e.g., 'Affects 80% of sessions')

          Please make the API call, analyze the data, and provide the 3 formatted issues. Base all recommendations on actual patterns in the session data.

          Environment context:
          - POSTHOG_API_KEY: $POSTHOG_API_KEY
          - POSTHOG_PROJECT_ID: $POSTHOG_PROJECT_ID
          - POSTHOG_HOST: $POSTHOG_HOST
          - GitHub Repo: $GITHUB_OWNER/$GITHUB_REPO
          EOF

          # Run the analysis
          cn < analysis_prompt.txt > analysis-results.txt 2>&1

          # Check if analysis completed
          if [ ! -s analysis-results.txt ]; then
            echo "❌ Analysis failed or produced no output"
            exit 1
          fi

          echo "✅ Analysis completed"
          echo "📄 Analysis results preview:"
          head -20 analysis-results.txt

      - name: Create GitHub Issues from Analysis
        run: |
          echo "📝 Creating GitHub issues from analysis..."
          # Create the issue creation prompt
          cat > issue_prompt.txt << 'EOF'
          I have analyzed PostHog session data and identified UX issues. Now I need you to create GitHub issues using the GitHub API.

          Use these details from environment variables:
          - GITHUB_PAT: My GitHub Personal Access Token
          - Repository: $GITHUB_OWNER/$GITHUB_REPO

          For each issue in my analysis:
          1. Parse the issue title, body content, and priority level
          2. Create a GitHub issue via API POST to: https://api.github.com/repos/$GITHUB_OWNER/$GITHUB_REPO/issues
          3. Use these headers:
             - Authorization: token $GITHUB_PAT
             - Accept: application/vnd.github.v3+json
             - Content-Type: application/json
          4. Set labels based on priority:
             - HIGH: ["bug", "high-priority", "user-experience", "automated"]
             - MEDIUM: ["enhancement", "medium-priority", "user-experience", "automated"]
             - LOW: ["low-priority", "user-experience", "automated"]
          5. Format the issue title as: "🔍 UX Issue: {original title}"

          Please create the GitHub issues and confirm they were successfully created with issue numbers and URLs.

          Here's my analysis to process:
          EOF

          # Append the analysis results to the prompt
          cat analysis-results.txt >> issue_prompt.txt

          # Run the issue creation
          cn < issue_prompt.txt > issue-creation-results.txt 2>&1

          echo "✅ Issue creation completed"
          echo "📄 Issue creation results:"
          cat issue-creation-results.txt

      - name: Upload Analysis Artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: analysis-results-${{ github.run_number }}
          path: |
            analysis-results.txt
            issue-creation-results.txt
          retention-days: 30

      - name: Summary
        run: |
          echo "✅ PostHog analysis workflow completed!"
          echo "📊 Check the artifacts for detailed analysis results"
          echo "🐛 Check your repository issues for newly created UX issues"

cn -p "[Paste the PostHog analysis prompt from Step 2]"

cn -p "[Paste the GitHub issue creation prompt from Step 3 with your analysis results]"

cn -p "Help me set up my environment for PostHog session analysis. Please:

1. Check if I have the required secrets configured in Continue CLI:
   - POSTHOG_API_KEY
   - POSTHOG_PROJECT_ID
   - POSTHOG_HOST
   - GITHUB_PAT

2. If any are missing, guide me through:
   - How to get PostHog Personal API Key from PostHog settings
   - How to find my PostHog Project ID from the URL
   - How to create a GitHub Personal Access Token with issues:write permission
   - How to add these as secrets in Continue CLI

3. Help me identify my GitHub repository owner and name by:
   - Checking git remote origin URL
   - Or asking me to provide them

4. Test the API connections by making a simple API call to both PostHog and GitHub to verify the credentials work.

Please walk me through this setup process step by step."