Add internet job search feature (cv-search-job)

- New cv-search-models shared library: EF entities + CvSearchDbContext for cvSearch schema (JobSearchTokens, JobSearchSessions, JobSearchResults tables) - New cv-search-job worker service: polls DB for pending sessions, scrapes job boards via configurable HTML scraping, runs LLM scoring via cv-matcher-api, emails ranked results - cv-matcher-api: JobTokenService creates one-time tokens; JobSearchController handles link clicks and creates sessions - api: proxies job-search start endpoint, appends job search link to match result email - CI workflow updated to build and push myai-cv-search-job:staging image - CLAUDE.md documentation added for all affected services Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 17:56:23 +03:00
parent a0ae262afc
commit 6293fa89e3
38 changed files with 2074 additions and 18 deletions
@@ -0,0 +1,158 @@
+# myAi — Solution Guide
+
+## Infrastructure URLs
+
+| Purpose | URL |
+|---------|-----|
+| Staging app | https://myai.easysoft.ro |
+| Production app | https://myai.ro |
+| Portainer (container management) | https://portainer.easysoft.ro/#!/auth |
+| Grafana (logs) | https://grafana.easysoft.ro/login |
+| Gitea (source control) | https://git.easysoft.ro |
+
+The Gitea instance has two deployment repos:
+- **staging repo** → auto-deploys to `myai.easysoft.ro`
+- **production repo** → auto-deploys to `myai.ro`
+
+## Staging browser testing
+
+To verify a feature against staging use the `verify` skill pointed at `https://myai.easysoft.ro`.  
+Portainer at `portainer.easysoft.ro` can restart containers or inspect live state.  
+Grafana at `grafana.easysoft.ro` shows structured logs from all containers.
+
+## Feature workflow (plan → ship)
+
+When a plan is approved and implementation begins:
+1. Add the plan as a **Gitea Wiki page** in the relevant repository (under a `Features/` or `Plans/` namespace)
+2. Create **Gitea Issues** — one per logical work chunk — and link them to the Wiki page
+3. Reference the issue number in commit messages (`Closes #N`)
+4. Issues are closed automatically (or manually) when the code is merged
+
+This applies to both the staging and production repos as appropriate.
+
+## Tech stack
+- .NET 10, ASP.NET Core, Worker Service
+- Entity Framework Core + SQL Server (multi-schema)
+- Refit for typed HTTP clients between services
+- Serilog (JSON structured logging, Console + File + Email sinks)
+- MailKit for SMTP
+- Docker Compose for local and production deployment
+- Watchtower for automatic container updates in production
+
+## Solution layout
+
+```
+Apis/
+  api/                    Public-facing proxy API (port 8080). Handles CORS, rate limiting, captcha, email.
+  api-models/             DTOs and settings shared by api only.
+  cv-matcher-api/         Internal CV match engine (port 8082). Owns cvMatcher + cvSearch DB schemas.
+  cv-matcher-api-models/  DTOs shared between api and cv-matcher-api.
+  cv-search-models/       EF entities + DbContext for cvSearch schema. Shared by cv-matcher-api and cv-search-job.
+  rag-api/                Internal RAG/vector-search service (port 8081). Owns rag DB schema.
+  rag-api-models/         DTOs shared with rag-api.
+  shared-models/          Cross-service shared models (DatabaseSettings, etc.).
+Helpers/
+  startup-helpers/        Shared Program.cs bootstrap: Serilog, Swagger, .env loading, Azure Key Vault, middleware.
+  common-helpers/         Utility helpers.
+Jobs/
+  job-scheduler/          IJobTask + JobSchedulerHostedService — the reusable scheduled-job engine.
+  cv-cleanup-job/         Worker: deletes old CVs from file storage. Runs hourly.
+  cv-search-job/          Worker: picks up pending job search sessions, scrapes providers, emails results.
+web/                      Razor Pages / Blazor front-end (port 5000).
+docker-compose/           docker-compose.yml + .env file.
+```
+
+## Build & restore
+
+```powershell
+dotnet restore myAi.sln
+dotnet build myAi.sln
+```
+
+## Running locally with Docker
+
+```powershell
+docker compose -f docker-compose/docker-compose.yml up --build
+```
+
+Config lives in `docker-compose/.env`. All env vars use `${VAR:-default}` fallback syntax.
+
+## Database schemas
+
+| Schema      | Owner DbContext       | Migrations assembly   |
+|-------------|----------------------|-----------------------|
+| `cvMatcher` | `CvMatcherDbContext` | `cv-matcher-api`      |
+| `rag`       | `RagDbContext`       | `rag-api`             |
+| `cvSearch`  | `CvSearchDbContext`  | `cv-search-models`    |
+
+Both `cv-matcher-api` and `cv-search-job` register `CvSearchDbContext` and call `db.Database.Migrate()` on startup (idempotent — safe for both to run).
+
+## EF Core migrations
+
+```powershell
+# Add a migration to cv-search-models
+dotnet ef migrations add <MigrationName> \
+  --context CvSearchDbContext \
+  --project Apis/cv-search-models \
+  --startup-project Apis/cv-matcher-api
+
+# Add a migration to cv-matcher-api
+dotnet ef migrations add <MigrationName> \
+  --context CvMatcherDbContext \
+  --project Apis/cv-matcher-api
+```
+
+EF tools version warning ("older than runtime") is expected and harmless. The `HostAbortedException` output during migration scaffolding is normal — EF starts the host to discover DbContext then aborts it.
+
+## Service dependency chain
+
+```
+web → api → cv-matcher-api → rag-api
+               ↑
+         cv-search-job
+```
+
+`api` never talks directly to `rag-api` — always via `cv-matcher-api`.
+
+## Internal API key auth
+
+All internal service-to-service calls require the `X-Internal-Api-Key` header.  
+The key is shared via the `CvMatcherApi__InternalApiKey` and `RagApi__InternalApiKey` env vars.  
+`startup-helpers` provides `UseInternalApiKeyProtection()` middleware that enforces it on `cv-matcher-api` and `rag-api`.
+
+## Shared file storage
+
+CV PDFs are written by `api` to `Apis/api/Files/` and read by `cv-cleanup-job` and `cv-search-job`.  
+All three containers mount the same bind volume:
+```yaml
+- ../Apis/api/Files:/app/Files
+```
+The path inside containers is controlled by `FileStorage__Path` (default: `Files`).
+
+## Job task pattern
+
+Every background worker uses the same pattern from `job-scheduler`:
+1. Implement `IJobTask` (has `TaskType` string + `ExecuteAsync(CancellationToken)`)
+2. Register as singleton: `services.AddSingleton<IEnumerable<IJobTask>>(sp => new IJobTask[] { ... })`
+3. Register `JobSchedulerHostedService` as hosted service
+4. Configure in appsettings under `Jobs:Tasks` array: `TaskType`, `Enabled`, `Interval`
+
+## Program.cs conventions
+
+Every service follows this structure:
+1. `StartupExtensions.LoadDotEnvFile()` — must be first, loads `docker-compose/.env`
+2. `StartupExtensions.GetApplicationVersion(Assembly.GetExecutingAssembly())`
+3. `builder.ConfigureJsonSerilog(ServiceName, appVersion)` from startup-helpers
+4. `builder.AddAzureKeyVaultIfConfigured()` (APIs only)
+5. `app.UseDefaultSerilogRequestLogging()`
+6. `app.UseJsonExceptionHandler(ServiceName)`
+7. EF migrations in a scoped block before `app.Run()`
+
+## Coding conventions
+
+- No XML doc comments on internal code; Swagger annotations on public controller actions
+- No explanatory inline comments — code should be self-describing
+- Use `$$"""..."""` raw string literals (not `$"""`) when the content contains CSS or other curly-brace-heavy text — avoids CS9006 brace-escaping errors
+- `sealed` on all concrete service classes
+- Settings classes injected via `IOptions<T>` — registered with `Configure<T>(config.GetSection("..."))`
+- Refit clients configured via a shared local function when multiple clients share the same base URL and auth header (see `api/Program.cs` → `ConfigureCvMatcherApiClient`)