claude
e95ed36647
Phases 1-10 of the planned refactoring:
Phase 1: rename shared-models -> common
- namespace Shared.Models -> Common throughout
- remove stale AspNetCore.Http.Features 5.0 reference
Phase 2: create shared-data with abstract BaseEntity
- BaseEntity: required string Id { get; init; } + DateTime CreatedAt { get; init; }
Phase 3: rename myai-models -> myai-data
- namespace MyAi.Models -> MyAi.Data
- MigrationsAssembly("myai-data")
Phase 4: rename cv-search-models -> cv-search-data
- namespace CvSearch.Models -> CvSearch.Data
- move JobSearchSettings to cv-matcher-api-models
- JobSearch*Entity now inherits BaseEntity
Phase 5: extract rag-data from rag-api
- new project: Apis/rag-data with RagDbContext + entities + migrations
- RagDocumentEntity inherits BaseEntity; cache entities use CacheKey PK
- fix duplicate AddHttpClient<RagAiClient>/AddScoped registrations in rag-api
- MigrationsAssembly("rag-data")
Phase 6: extract cv-matcher-data from cv-matcher-api
- new project: Apis/cv-matcher-data with CvMatcherDbContext + entities + migrations
- CvMatchResultEntity inherits BaseEntity; CvMatcherChatCacheEntity uses CacheKey PK
- MigrationsAssembly("cv-matcher-data")
Phase 7: create empty cv-cleanup-job-models and cv-search-job-models
Phase 8: update all 5 Dockerfiles for renamed/new projects
Phase 9: reorganise .sln virtual folders (Apis/Jobs/Models/Data/Helpers)
- update root CLAUDE.md with new project taxonomy and migration commands
- update cv-matcher-api/CLAUDE.md and cv-search-job/CLAUDE.md
Phase 10: add Directory.Packages.props for centralised NuGet versions
- remove Version= from all PackageReference elements in active .csproj files
No database changes. No runtime behaviour changes.
All MigrationId strings in __EFMigrationsHistory are unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
91 lines
4.0 KiB
Markdown
91 lines
4.0 KiB
Markdown
# cv-search-job — Internet Job Search Worker
|
|
|
|
Background worker. Polls the database every 30 s for pending job search sessions and processes them.
|
|
|
|
## What it does (per session)
|
|
|
|
1. Reads session from DB (`Status = Pending`)
|
|
2. Sets `Status = Processing`
|
|
3. Deserializes `ProviderConfigJson` (snapshot of provider configs taken at token-start time)
|
|
4. For each enabled provider: calls `HtmlJobSearcher` to scrape job URLs
|
|
5. Deduplicates URLs across providers, caps at `MaxJobsToMatch` (default 15)
|
|
6. Calls `cv-matcher-api POST /api/cv/match-job` for each URL (uses existing LLM scoring)
|
|
7. Saves each result as `JobSearchResultEntity`
|
|
8. Filters to `Score >= MinMatchScore` (default 15)
|
|
9. Sets `Status = Done`, saves keywords + provider snapshot to session
|
|
10. Sends ranked results email via `CvSearchEmailSender` (dual-recipient: user + `Contact:ToEmail`)
|
|
11. Attaches CV PDF from shared file storage if it exists
|
|
|
|
## Crash recovery
|
|
|
|
On every tick, sessions with `Status = Processing` AND `CreatedAt < UtcNow - 10 min` are reset to `Pending`. This handles container restarts mid-processing.
|
|
|
|
## HtmlJobSearcher — generic HTML scraper
|
|
|
|
No per-provider logic. Config-driven. For each provider:
|
|
1. Combines `provider.InitialKeywords` + CV keywords from session, URL-encodes as space-joined string
|
|
2. `GET {SearchUrlTemplate}` with keyword substitution
|
|
3. Regex-parses all `<a href="..." >text</a>` tags
|
|
4. Two-stage filter:
|
|
- Stage 1: `href` must contain `JobLinkContains`
|
|
- Stage 2: anchor text must contain at least one CV keyword
|
|
5. Makes hrefs absolute, deduplicates, returns up to `MaxResults` URLs
|
|
|
|
## Provider config
|
|
|
|
Defined under `JobSearch:Providers` in appsettings / docker-compose env vars. Three providers ship as defaults (all `Enabled: false`):
|
|
|
|
| Name | Notes |
|
|
|------|-------|
|
|
| `ejobs.ro` | Romanian job board; reliable HTML structure |
|
|
| `bestjobs.eu` | Romanian job board |
|
|
| `linkedin.com` | Likely to return empty results due to bot detection |
|
|
|
|
Provider config is snapshotted to `JobSearchSessionEntity.ProviderConfigJson` at session creation time (in `cv-matcher-api`), so changes to config do not affect in-flight sessions.
|
|
|
|
To enable a provider via docker-compose env var (index-based):
|
|
```
|
|
JobSearch__Providers__0__Enabled=true # ejobs.ro
|
|
JobSearch__Providers__1__Enabled=true # bestjobs.eu
|
|
JobSearch__Providers__2__Enabled=true # linkedin.com
|
|
```
|
|
|
|
## Email
|
|
|
|
`CvSearchEmailSender` reads SMTP config directly from `IConfiguration` (same `Smtp:*` keys as `api`).
|
|
Sends to both `toEmail` (from session) and `Contact:ToEmail` (operator copy).
|
|
CV PDF attached from `{FileStorage:Path}/{cvDocumentId}.pdf` if the file exists.
|
|
|
|
## Shared volume
|
|
|
|
`../Apis/api/Files:/app/Files` — same bind mount as `api` and `cv-cleanup-job`.
|
|
CV PDFs written by `api` are readable here without any API call.
|
|
|
|
## Key settings
|
|
|
|
| Section | Env var | Notes |
|
|
|---------|---------|-------|
|
|
| `Database` | `Database__*` | Same SQL Server as other services |
|
|
| `CvMatcherApi` | `CvMatcherApi__BaseUrl`, `CvMatcherApi__InternalApiKey` | Internal call to match-job endpoint |
|
|
| `Smtp` | `Smtp__*` | Same vars as `api` |
|
|
| `Contact` | `Contact__ToEmail` | Operator copy recipient |
|
|
| `FileStorage` | `FileStorage__Path` | Must match the shared volume mount path |
|
|
| `JobSearch` | `JobSearch__Enabled`, `MinMatchScore`, `MaxJobsToMatch` | Core search limits |
|
|
| `Jobs:Tasks:0` | `Jobs__Tasks__0__Interval` | Poll interval (default `00:00:30`) |
|
|
|
|
## Logging
|
|
|
|
Follows the same scheme as `cv-cleanup-job`:
|
|
- **Console** — `[HH:mm:ss LVL] SourceContext: Message`
|
|
- **File** — `logs/cv-search-job-.log`, daily rolling, 30-day retention
|
|
- **Email** (index 2) — Errors only, wired via `Serilog__WriteTo__2__Args__*` env vars in docker-compose
|
|
- **Enrich** — `FromLogContext`, `WithMachineName`, `WithEnvironmentName`
|
|
|
|
`Serilog.Sinks.Email` is available transitively through `startup-helpers` — no extra package needed in the csproj.
|
|
|
|
## EF migrations
|
|
|
|
This project runs `CvSearchDbContext.Database.Migrate()` on startup.
|
|
Migrations live in `Apis/cv-search-data/Migrations/`.
|
|
To add a migration: see root CLAUDE.md.
|