c9c629767e
Creates a new skill that automates smoke testing of the myAi CV Matcher: - Starts Docker Compose and waits for app health check - Uploads CV.pdf and job description via Selenium WebDriver - Verifies CV analysis results display (score, strengths, gaps) - Confirms match email was sent by checking container logs - Returns pass/fail summary with any failures detailed Includes SKILL.md documentation and run_smoke_test.py automation script with hardcoded test data (CV file path, job description). Can be extended to test against different CVs/job descriptions via environment variables. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
188 lines
7.0 KiB
Markdown
188 lines
7.0 KiB
Markdown
---
|
|
name: myai-smoke-test
|
|
description: Run a smoke test of the myAi CV Matcher application locally. Starts Docker Compose, uploads a CV, submits a job description, verifies CV analysis results display, and confirms the match email was sent. Use this skill whenever you need to validate that the core CV matching workflow is functioning correctly after code changes.
|
|
compatibility: Requires Docker, Docker Compose, Chrome/Chromium, Python 3.8+, Selenium WebDriver
|
|
---
|
|
|
|
# myAi Smoke Test Skill
|
|
|
|
## Purpose
|
|
|
|
This skill automates a minimal end-to-end test of the myAi CV Matcher application running locally. It verifies:
|
|
1. Application starts and health check passes
|
|
2. CV upload and parsing works
|
|
3. Job description input and matching is triggered
|
|
4. CV analysis results display (score, strengths, gaps, evidence)
|
|
5. Match email is sent to the configured recipient
|
|
|
|
Use this after significant code changes to ensure the core workflow remains functional.
|
|
|
|
## Prerequisites
|
|
|
|
- **Docker & Docker Compose** installed and running
|
|
- **Chrome or Chromium** browser installed (`chromedriver` auto-downloads via Selenium)
|
|
- **Python 3.8+** with pip
|
|
- **CV.pdf** file in the project root (`C:\Apps\easySoft\AI\myAi\CV.pdf`)
|
|
- The myAi solution source code locally available
|
|
- Port 5140 available (web app), 8080 (api), 8081 (rag-api), 8082 (cv-matcher-api), 5432 (database), 1025 (mailhog)
|
|
|
|
## Workflow
|
|
|
|
### 1. Start Application (45-second timeout)
|
|
|
|
The script launches Docker Compose from the project root:
|
|
```bash
|
|
docker compose -f docker-compose/docker-compose.yml up --build -d
|
|
```
|
|
|
|
Then waits up to 45 seconds for the web application at `http://localhost:5140` to become ready (HTTP 200 on root path).
|
|
|
|
**Success criteria**: GET `/` returns HTTP 200
|
|
**Failure**: Application does not respond within 45 seconds → test fails with "Application startup timeout"
|
|
|
|
### 2. Open Browser & Navigate
|
|
|
|
Opens Chrome (in headless or visible mode, configurable) and navigates to the home page.
|
|
|
|
### 3. Upload CV (Hardcoded CV.pdf)
|
|
|
|
Uploads the CV.pdf file from the project root to the CV file input on the home page.
|
|
|
|
**Expected behavior**:
|
|
- File input accepts the PDF
|
|
- JavaScript triggers the file change handler
|
|
- Filename displays in the UI (e.g., "CV.pdf selected")
|
|
|
|
### 4. Fill Job Description (Hardcoded)
|
|
|
|
Fills the job description input with:
|
|
```
|
|
Senior Full Stack Engineer - 5+ years experience with C# ASP.NET Core,
|
|
React, and cloud deployment. Experience with CI/CD pipelines, Docker,
|
|
and agile teams required.
|
|
```
|
|
|
|
**Expected behavior**:
|
|
- Textarea accepts input
|
|
- Text is visible in the field
|
|
|
|
### 5. Submit Form (Completes reCaptcha if needed)
|
|
|
|
Clicks the "Submit" button to trigger the CV match API call.
|
|
|
|
**Behavior**:
|
|
- If reCaptcha is required, the script waits for it to be marked `data-sitekey` complete
|
|
- Form submission triggers async `postCv()` function
|
|
- API call goes to `/api/cv/match` with CV file and job description
|
|
- Response includes `matchScore`, `strengths`, `gaps`, `evidence`
|
|
|
|
### 6. Verify Results Display (30-second timeout)
|
|
|
|
Waits for results to appear on the page:
|
|
- Checks for `.match-result` container visibility
|
|
- Verifies match score badge displays (0-100%)
|
|
- Checks for strengths list (ul.strengths with li items)
|
|
- Checks for gaps list (ul.gaps with li items)
|
|
- Checks for evidence section (div.evidence with bullets)
|
|
|
|
**Success criteria**: All result elements present and visible within 30 seconds
|
|
**Failure**: Results do not display → test fails with "Results did not display within timeout"
|
|
|
|
### 7. Verify Email Sent (Log Check)
|
|
|
|
Checks Docker container logs for email delivery confirmation:
|
|
- Inspects `email-api` container logs
|
|
- Searches for "Message sent" confirmation message
|
|
- Verifies the recipient includes the job candidate email
|
|
|
|
**Success criteria**: Log contains email send confirmation
|
|
**Failure**: No email confirmation found → test fails with "Email was not sent"
|
|
|
|
### 8. Report Results
|
|
|
|
Reports one of:
|
|
- **All passed**: "✓ Smoke test passed. CV uploaded, matched (score: X%), results displayed, email sent."
|
|
- **Partial failure**: Details which step failed (e.g., "✗ Results did not display within timeout")
|
|
- **Critical failure**: "✗ Application failed to start" or "✗ File upload failed"
|
|
|
|
## Hardcoded Values
|
|
|
|
These are built into the skill and do not require environment variables:
|
|
|
|
| Value | Usage |
|
|
|-------|-------|
|
|
| `CV.pdf` (from root) | CV file upload |
|
|
| `Senior Full Stack Engineer...` (job desc) | Job matching criteria |
|
|
| `http://localhost:5140` | Web app home |
|
|
| 45 seconds | App startup timeout |
|
|
| 30 seconds | Results display timeout |
|
|
|
|
## Environment Variables (Optional)
|
|
|
|
| Variable | Default | Purpose |
|
|
|----------|---------|---------|
|
|
| `HEADLESS_CHROME` | `true` | Run Chrome in headless mode (no visible window) |
|
|
| `DOCKER_COMPOSE_FILE` | `docker-compose/docker-compose.yml` | Path to docker-compose.yml relative to project root |
|
|
| `APP_PORT` | `5140` | Port the web app runs on |
|
|
| `APP_STARTUP_TIMEOUT` | `45` | Seconds to wait for app health check |
|
|
| `RESULTS_DISPLAY_TIMEOUT` | `30` | Seconds to wait for results to appear |
|
|
| `PROJECT_ROOT` | Current directory | Root of myAi solution |
|
|
|
|
## Output
|
|
|
|
On success:
|
|
```
|
|
✓ Smoke test passed. CV uploaded, matched (score: 87%), results displayed, email sent.
|
|
```
|
|
|
|
On failure:
|
|
```
|
|
✗ Application failed to start (timeout after 45 seconds)
|
|
[Details of failed step]
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### "Application startup timeout"
|
|
- Verify Docker Compose is running: `docker ps`
|
|
- Check logs: `docker logs myai-web-1`
|
|
- Ensure port 5140 is not in use: `netstat -an | grep 5140` (Windows) or `lsof -i :5140` (Mac/Linux)
|
|
|
|
### "File upload failed"
|
|
- Verify CV.pdf exists in project root
|
|
- Check file permissions (readable by your user)
|
|
- Ensure the file input element exists in the form
|
|
|
|
### "Results did not display"
|
|
- Check browser console for JavaScript errors: `docker logs myai-web-1`
|
|
- Verify the `/api/cv/match` endpoint is responding: manually test with Postman
|
|
- Check cv-matcher-api logs: `docker logs myai-cv-matcher-api-1`
|
|
|
|
### "Email was not sent"
|
|
- Verify email-api container is running: `docker ps | grep email-api`
|
|
- Check email-api logs: `docker logs myai-email-api-1`
|
|
- If using MailHog for local testing, verify it's accessible: `http://localhost:1025`
|
|
|
|
### Chrome/Chromedriver issues
|
|
- Selenium automatically downloads chromedriver on first run
|
|
- If Chrome is not in PATH, install from https://chromedriver.chromium.org/
|
|
- Verify Chrome version matches chromedriver version
|
|
|
|
## Running the Skill
|
|
|
|
```bash
|
|
python scripts/run_smoke_test.py
|
|
```
|
|
|
|
Or from within Claude:
|
|
- Trigger the skill with: "Run the myAi smoke test"
|
|
- Or: "Execute a smoke test of the CV matching workflow"
|
|
|
|
## Implementation Notes
|
|
|
|
- Uses Selenium WebDriver with Chrome (auto-downloaded by webdriver-manager)
|
|
- Handles async JavaScript (waits for API response before checking results)
|
|
- Logs all steps to console with timestamps
|
|
- Automatically stops Docker Compose containers on test completion (can be disabled with `KEEP_CONTAINERS=true`)
|
|
- Tests are idempotent — can run multiple times without side effects
|