64 Commits

Author SHA1 Message Date
claude 9a33dc019a Promote staging->production: BuildKit CI, Compact JSON logging, consolidated compose + latest app code
Build and Push Docker Images / build (push) Successful in 3m23s
2026-06-18 19:14:59 +03:00
claude 62654978af ci: re-enable BuildKit (runner now has buildx plugin)
Build and Push Docker Images / build (push) Successful in 43m53s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 18:09:35 +03:00
claude 7da084c174 ci: revert DOCKER_BUILDKIT (runner job env has no buildx plugin -> build failed)
Keep the cache win (no base-image nuke); back to the legacy builder which the job
context supports. BuildKit needs buildx installed in the runner before it can be used.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 18:02:29 +03:00
claude 27f4cfe21e ci: enable BuildKit + let its GC manage the layer cache
DOCKER_BUILDKIT=1 (explicit) so the restore layer is cached across builds; drop the
explicit 'docker builder prune' (it was wiping that cache) and rely on BuildKit's own
GC + 86GB headroom. Cleanup keeps dangling-image removal only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 17:58:00 +03:00
claude 903fbcd143 ci: preserve Docker build cache (faster runner)
Stop wiping the layer cache + base images after every build (the host has 86GB free).
Keep base images (prune dangling only) and ~2 weeks of build cache, so the cache-friendly
Dockerfiles (COPY *.csproj + restore before source) actually benefit -> warm rebuilds skip
restore and base-image pulls.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 17:36:59 +03:00
claude c5e1b7f687 ci: branch-driven deploys (staging/production branches), build the pushed commit
main = day-to-day work (no deploy). Merge into staging -> :staging, into
production -> :production; IMAGE_TAG = branch name. Also fixes the checkout to
build the PUSHED commit (git checkout $GITHUB_SHA) instead of always cloning
the default branch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 16:29:07 +03:00
claude 9b33876c11 docs: replace template README with a proper one (intro, run, deploy, logging)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 16:04:23 +03:00
claude 2d5572725d docs: add Observability section to CLAUDE.md (Compact JSON logs->Loki); gitignore ACCESS.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 15:46:46 +03:00
claude da1f90449e Merge observability/compact-json: standardize Serilog to Compact JSON (Application/Environment/AppVersion enrichment) 2026-06-18 10:59:13 +03:00
claude 2192c3f4c5 Logs: Compact JSON + aligned enrichment in shared StartupExtensions
CompactJsonFormatter in both ConfigureJsonSerilog overloads; rename Service->Application,
EnvironmentName->Environment (keep AppVersion). Applies to all myAi services.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:03:55 +03:00
claude 492859f17f ci: prune images + build cache after build (prevent runner disk exhaustion)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 22:51:31 +03:00
claude 1920885835 Merge main into production: PR #55 — Fix CV keyword extraction prompt
Build and Push Docker Images Staging / build (push) Successful in 52s
2026-06-09 16:41:03 +03:00
claude a3567ce8e9 Merge pull request 'Fix CV keyword extraction — derive from candidate CV, not matched job' (#55) from feature/fix-keyword-extraction-prompt into main
Build and Push Docker Images Staging / build (push) Successful in 2m23s
Merge PR #55: Fix CV keyword extraction — derive from candidate CV, not matched job
2026-06-09 13:40:10 +00:00
claude b52ef8ddff Fix CV keyword extraction to reflect candidate identity, not matched job
The AI prompt now instructs the LLM to derive keywords entirely from the
candidate's CV (seniority level, primary role title, core technologies they
emphasize) rather than from the job description being matched. This ensures
the job-board search keywords used by cv-search-job represent who the
candidate actually is, not a mirror of the job they happened to match against.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:38:02 +03:00
claude eb83d28ed5 Merge staging into production: PR #54 — Fix hardcoded user-facing strings
Build and Push Docker Images Staging / build (push) Successful in 10m36s
2026-06-08 22:34:35 +03:00
claude d2b12e39ec Merge PR #54: Fix hardcoded user-facing strings (Issue #53)
Build and Push Docker Images Staging / build (push) Successful in 46s
2026-06-08 22:34:28 +03:00
claude 1e8758796e Fix hardcoded user-facing strings — localize email fallbacks, API errors, AI parse messages
- Frontend: update extractApiError to check body.code first via i18n 'error.<code>' keys;
  add en/ro translations for cv_file_missing, captcha_verification_failed, request_cancelled
- email-data migration: seed 6 fallback template keys (match N/A, subject label, unknown IP,
  job search results empty states for keywords/providers/location)
- EmailApiEmailSender: replace "N/A", "Job", "Unknown" literals with template lookups
- CvSearchEmailSender: replace "none detected", "none", "-" literals with template lookups
- cv-matcher-data migration: seed parse-error.summary and parse-error.recommendation in AiPrompts
- CvMatcherService: look up localized parse-error messages from AiPrompts before calling ParseResult

Closes #53

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 22:32:03 +03:00
claude b6d9aea3bc Merge branch 'main' into production
Build and Push Docker Images Staging / build (push) Successful in 1m5s
2026-06-08 22:10:26 +03:00
claude ef2793448a Fix: declare language before jobLabel in CvMatcherController
Build and Push Docker Images Staging / build (push) Successful in 8m20s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 22:10:22 +03:00
claude 2b9132a3a9 Merge branch 'main' into production
Build and Push Docker Images Staging / build (push) Failing after 14s
2026-06-08 22:08:32 +03:00
claude cbf06031e8 Merge pull request 'Fix language consistency in job search and match emails' (#52) from feature/language-consistency into main
Build and Push Docker Images Staging / build (push) Failing after 32s
Merge PR #52: Fix language consistency in job search and match emails
2026-06-08 19:08:28 +00:00
claude 90f540139a Fix language consistency in job search and match emails
1. Pass session.Language to MatchJobRequest in cv-search-job so the LLM
   uses the correct language-specific prompt for job titles and summaries.

2. Replace hardcoded "Manual job description" with a template-driven label
   (email.match.manual-job-label) seeded in English and Romanian, so the
   match email subject and Job row reflect the user's language.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 22:07:15 +03:00
claude e5bf56cc4d Merge branch 'main' into production
Build and Push Docker Images Staging / build (push) Successful in 42s
2026-06-08 21:48:24 +03:00
claude 71d5ac8e06 Remove environment name prefix from email subjects
Build and Push Docker Images Staging / build (push) Successful in 1m26s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:47:21 +03:00
claude c2082d6729 Suppress environment prefix in email subjects on Production
[ENV_NAME] prefix is now only prepended in non-production environments
(Development, Staging, etc.). Production emails get a clean subject line.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:45:46 +03:00
claude 8f58708cd9 Revert "Suppress environment prefix in email subjects on Production"
Build and Push Docker Images Staging / build (push) Successful in 1m35s
This reverts commit 06dd0140d6.
2026-06-08 21:45:45 +03:00
claude 06dd0140d6 Suppress environment prefix in email subjects on Production
[ENV_NAME] prefix is now only prepended in non-production environments
(Development, Staging, etc.). Production emails get a clean subject line.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:45:29 +03:00
claude 0aee7c4ed6 Changes
Build and Push Docker Images Staging / build (push) Successful in 45s
2026-06-08 21:31:35 +03:00
claude cd661fe613 Merge pull request 'Staging to Production' (#51) from main into production
Merge staging to production
2026-06-08 18:28:46 +00:00
claude 6f1d8992ab Merge pull request 'Link pageFetcher.PageFetches to cvSearch.JobSearchSessions' (#50) from feature/page-fetch-session-link into main
Build and Push Docker Images Staging / build (push) Successful in 17m39s
Merge PR #50: Link pageFetcher.PageFetches to cvSearch.JobSearchSessions
2026-06-08 16:57:07 +00:00
claude 2d9ffc9c2b Link pageFetcher.PageFetches to cvSearch.JobSearchSessions
Adds nullable JobSearchSessionId to PageFetchEntity and FetchPageRequest.
cv-search-job passes session.Id on every fetch so all Playwright page
loads for a job search session can be traced back to their session.
Includes index on JobSearchSessionId for efficient lookup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:56:13 +03:00
claude 9fbad722fc Merge pull request 'Add Email and ClientIpAddress audit fields to cvSearch.JobSearchSessions and JobSearchResults' (#48) from feature/job-search-audit-fields into main
Build and Push Docker Images Staging / build (push) Successful in 39s
Merge PR #48: Add Email and ClientIpAddress audit fields to cvSearch
2026-06-08 16:21:46 +00:00
claude 473c36d65f Store match-time ClientIpAddress on cvSearch.JobSearchTokens
Captures the IP when the user submits the CV match form and stores it on
the token, giving a full audit trail: token holds the match-site IP,
session holds the email link-click IP.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:20:02 +03:00
claude 292d19d5ed Populate JobText from fetched page content in JobSearchResults
Previously always stored empty string; now stores the full page text
returned by page-fetcher-api, which is already in scope at save time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:13:10 +03:00
claude d56729de42 Add Email and ClientIpAddress audit fields to cvSearch.JobSearchSessions and JobSearchResults
Captures client IP at job-search link-click time and threads it through to the session.
Both Email and ClientIpAddress are copied from session to each result row during processing.
Closes #47

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:11:50 +03:00
claude 79a3dec679 Merge pull request 'Add Email and ClientIpAddress audit fields to cvMatcher.Results' (#46) from feature/result-email-and-ip into main
Build and Push Docker Images Staging / build (push) Successful in 16m43s
Merge PR #46: Add Email and ClientIpAddress audit fields to cvMatcher.Results
2026-06-08 15:58:18 +00:00
claude 02d2b1e510 Add Email and ClientIpAddress audit fields to cvMatcher.Results
Threads the caller's email and client IP through the match pipeline so
every Results row records who triggered the match and from where.
Closes #45

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:56:36 +03:00
claude 3c3451b198 Merge branch 'main' into staging
Build and Push Docker Images Staging / build (push) Successful in 17m15s
2026-06-08 18:43:56 +03:00
claude a83f6f705f Remove UseHeadlessBrowser from JobProvider — all fetches now go via page-fetcher-api
page-fetcher-api always uses Playwright (networkidle by default), so the
per-provider flag that chose between headless and plain HTTP is obsolete.

- Removed from JobProviderEntity, CvSearchDbContext, JobProviderConfig, JobTokenService
- HtmlJobSearcher no longer passes WaitFor (uses page-fetcher-api default)
- EF migration drops the column from cvSearch.JobProviders

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:43:42 +03:00
claude b68cf942a8 Merge branch 'main' into staging
Build and Push Docker Images Staging / build (push) Successful in 28m0s
2026-06-08 18:37:00 +03:00
claude 61805e2fb5 Merge pull request 'feat: page-fetcher-api centralised Playwright page fetcher' (#44) from feature/page-fetcher-api into main
Merge feature/page-fetcher-api into main
2026-06-08 15:36:44 +00:00
claude dcfc50ff32 Fix Docker builds: upgrade Refit to 11.0.1, add page-fetcher-api-models to Dockerfiles
- Refit 10.1.6 signing certificate was revoked; upgraded to 11.0.1 in Directory.Packages.props
- cv-matcher-api/Dockerfile and cv-search-job/Dockerfile were missing COPY steps
  for page-fetcher-api-models (added in this feature branch)

All 8 images now build cleanly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:35:41 +03:00
claude b1ed1cb201 Rename EmailApi.Models.* namespace to Email.Models.* in email-api-models
Removes the spurious Api segment to match the pattern used by all other
models projects: CvMatcher.Models.*, Rag.Models.*, PageFetcher.Models.*.

Updated all consumers: email-api, api, cv-search-job.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:06:38 +03:00
claude e1f171168e Align email-api and page-fetcher-api namespaces to Api.* convention
Fixes inconsistency where email-api used EmailApi.* and page-fetcher-api
used PageFetcherApi.*, while cv-matcher-api and rag-api use the generic
Api.* namespace. All four API projects now follow the same pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:04:03 +03:00
claude ae2bc9b902 Move SmtpSettings and PageFetcherSettings into their respective models projects
Settings classes now live in the -models project alongside DTOs and client
interfaces, eliminating the Settings/ folder from both API projects.

- SmtpSettings: email-api/Settings/ → email-api-models/Settings/ (namespace EmailApi.Models.Settings)
- PageFetcherSettings: page-fetcher-api/Settings/ → page-fetcher-api-models/Settings/ (namespace PageFetcher.Models.Settings)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:00:44 +03:00
claude 30a8df431f Move PageFetcherSettings back to page-fetcher-api/Settings/, matching SmtpSettings pattern
Server-side-only settings (internal config not needed by callers) belong in
the API project itself, not in the models project. PageFetcherSettings
(DefaultWaitFor, TimeoutSeconds, MaxTextChars) mirrors SmtpSettings in
email-api/Settings/ — callers never reference these.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:56:21 +03:00
claude 95b0cfa0a9 Move PageFetcherSettings to page-fetcher-api-models, consistent with EmailApiSettings pattern
Settings class now lives in Apis/page-fetcher-api-models/Settings/ with
namespace PageFetcher.Models.Settings, matching how EmailApiSettings is
placed in email-api-models/Settings/.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:54:08 +03:00
claude 20b13647de Move PageFetcherSettings to Settings/ folder, consistent with email-api pattern
Settings classes belong in Settings/ with namespace PageFetcherApi.Settings,
not Services/. Matches the SmtpSettings placement in email-api.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:51:51 +03:00
claude df011f2a03 Fix PageFetcherApi BaseUrl default to use Docker service name, not container name
Use http://page-fetcher-api:8080 (the Compose service key) for Docker DNS
resolution, consistent with all other internal service URLs (rag-api,
email-api, cv-matcher-api).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:49:00 +03:00
claude 3414c61cea Commit 2026-06-08 17:47:17 +03:00
claude 898dd09d50 feat: add page-fetcher-api — centralised Playwright page fetcher
Introduces page-fetcher-api, a new internal ASP.NET Core service that
centralises all web-page fetching through a single Playwright (headless
Chromium) browser instance. All fetches are persisted to the pageFetcher
SQL schema for auditing.

New projects:
- Apis/page-fetcher-api-models: FetchPageRequest, FetchPageResponse, IPageFetcherApiClient
- Apis/page-fetcher-data: PageFetchDbContext, PageFetchEntity, InitialSchema migration (schema: pageFetcher)
- Apis/page-fetcher-api: PlaywrightBrowserService (singleton), PageFetcherService, PageController

Changes to existing services:
- cv-matcher-api: JobTextExtractor now calls IPageFetcherApiClient instead of HttpClient
- cv-search-job: HtmlJobSearcher uses IPageFetcherApiClient (removes inline Playwright);
  CvSearchJobTask fetches individual job pages and applies keyword pre-filter before
  LLM call; passes pre-fetched JobDescription to cv-matcher-api to skip re-fetch
- common: add PageFetcherApiSettings
- docker-compose.yml, build.yml: add new service + env vars for callers

Closes #43

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:43:56 +03:00
claude 4de6f1db45 Merge branch 'main' into staging
Build and Push Docker Images Staging / build (push) Successful in 7m25s
2026-06-08 17:30:49 +03:00
claude 1222a86eb7 Fix file:// URL bug in HtmlJobSearcher — skip non-HTTP(S) URLs
After resolving relative hrefs against the base search URL, some ejobs.ro
links were producing file:/// URIs (e.g. file:///user/locuri-de-munca/...).
These were sent to cv-matcher-api and rejected with HTTP 400, causing 0 matches.

Added a scheme guard after URI resolution to skip any URL that is not
http:// or https://, preventing malformed URLs from reaching the matcher.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:30:45 +03:00
claude 2e9069cbdb Fix file:// URL bug in HtmlJobSearcher — skip non-HTTP(S) URLs
Build and Push Docker Images Staging / build (push) Successful in 35s
After resolving relative hrefs against the base search URL, some ejobs.ro
links were producing file:/// URIs (e.g. file:///user/locuri-de-munca/...).
These were sent to cv-matcher-api and rejected with HTTP 400, causing 0 matches.

Added a scheme guard after URI resolution to skip any URL that is not
http:// or https://, preventing malformed URLs from reaching the matcher.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 16:57:52 +03:00
claude c89df975bd Add searched location to job search results email
Build and Push Docker Images Staging / build (push) Successful in 14m42s
Show the candidate's location in the scan summary block of the results email
alongside keywords and providers, for both en and ro templates.

- CvSearchEmailSender.SendResultsAsync accepts location and passes it to BuildScanSummary
- BuildScanSummary passes {{location}} to the template (falls back to '-' when absent)
- CvSearchJobTask passes session.Location to SendResultsAsync
- New migration AddLocationToScanSummaryTemplate updates both language variants of
  email.search-results.scan-summary to include a 'Location / Locație căutată' row

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 15:54:38 +03:00
claude 709c0ac4c3 Merge pull request 'Fix job search: location filtering, keyword quality, anchor filter bypass' (#42) from feature/job-search-location-keywords into main
Merge PR #42: Fix job search — location filtering, keyword quality, anchor filter bypass
2026-06-08 12:51:16 +00:00
claude 99e5cfb76b Fix job search: location filtering, keyword quality, anchor filter bypass
Closes #41

- Add RequireKeywordInAnchor per-provider flag (default true); set false for
  ejobs.ro and bestjobs.eu so Stage 2 anchor-text filter is skipped — their
  search URL already filters by relevance server-side
- Update AI system prompts (en + ro) to extract concise job-board-friendly
  keywords (role title + key tech, not abstract concepts) and candidate location
- Propagate location through JobMatchResponse -> CreateJobSearchTokenRequest ->
  JobSearchTokenEntity -> JobSearchSessionEntity
- Add {location} and {location-slug} substitution in HtmlJobSearcher
- Update provider SearchUrlTemplates to include location:
    ejobs.ro:    /locuri-de-munca/{location-slug}?q={keywords}
    bestjobs.eu: /ro/locuri-de-munca-in-{location-slug}?keywords={keywords}
    linkedin.com: ?keywords={keywords}&location={location}
- Three new migrations: AddRequireKeywordInAnchorAndLocation,
  ImproveKeywordsAndAddLocation, AddLocationToProviders

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 15:45:45 +03:00
claude 91b2baa445 Fix email-api middleware order: API key check before swagger
Build and Push Docker Images Staging / build (push) Successful in 17m14s
UseInternalApiKeyProtection was registered after UseSwaggerInDevelopment,
allowing unauthenticated access to /swagger. Swapped order to match
rag-api and cv-matcher-api.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 22:41:29 +03:00
claude 0f64cb8d99 Fix email-api Dockerfile: add missing shared-data COPY
email-data references shared-data but the email-api Dockerfile never
copied it into the build context, causing MSB9008 during Docker build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 22:30:25 +03:00
claude b67e926c5f Fix Serilog email sink: configure in code, not JSON config
Serilog.Settings.Configuration cannot deserialize NetworkCredential or
MailKit's SecureSocketOptions from JSON, causing an InvalidOperationException
in the binder and preventing containers from starting.

Fix: remove Email from the WriteTo JSON array entirely and wire it in code
inside ConfigureJsonSerilog using a dedicated SerilogEmail:* config section.
The sink is skipped when From/To/Host are absent, so local dev is unaffected.

Also renames the docker-compose env vars from the verbose
Serilog__WriteTo__2__Args__* prefix to the clean SerilogEmail__* prefix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 22:25:26 +03:00
claude f7d856147e Escalate provider fetch failures to Error for alert emails
HTTP and Playwright fetch failures in HtmlJobSearcher are now logged at
Error so that Serilog's email sink triggers an alert when a job provider
is unreachable. Per-URL match failures remain at Warning (expected noise).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 22:03:46 +03:00
claude 8679bd1efd Fix Serilog email sink config for v4 API breaking changes
Serilog.Sinks.Email v4 renamed all configuration parameters from their
v2 names. The old names were silently ignored, so no error alert emails
were ever sent.

Parameter renames applied across all 6 appsettings.json and docker-compose:
  fromEmail → from
  toEmail   → to
  mailServer → host
  networkCredential → credentials
  enableSsl: true → connectionSecurity: StartTls
  emailSubject → subject
  outputTemplate → body
  batchPostingLimit / period removed (v4 batching uses a separate overload)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 21:57:06 +03:00
claude 1bcf95d8d4 Add download rate limit policy to template and docker-compose
Build and Push Docker Images Staging / build (push) Failing after 1m38s
5 requests / 1 min per IP. docker-compose.yml wired with ${VAR:-default}.
Staging and production .env files updated locally (gitignored).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 20:40:25 +03:00
claude 73f67d1342 Protect FileDownloadController with reCAPTCHA v3 and rate limiting
- Require captchaToken query param on initial (non-range) download requests
- Range requests (HTTP resume) bypass captcha — they are continuations of an already-validated download
- Add download rate limit policy: 5 requests / 1 min per IP (configured in .env)
- Inject ICaptchaVerifier; action name is file_download

UI change required: execute grecaptcha.execute(siteKey, {action: 'file_download'})
before triggering the download and append ?captchaToken=<token> to the URL.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 20:37:44 +03:00
111 changed files with 4469 additions and 390 deletions
+25 -4
View File
@@ -1,13 +1,19 @@
name: Build and Push Docker Images Staging
name: Build and Push Docker Images
# Branch-driven deploys — no yaml edits to switch environment:
# merge into `staging` -> tag :staging (staging Watchtower deploys)
# merge into `production` -> tag :production (production Watchtower deploys)
# `main` is the day-to-day work branch and deploys nothing.
on:
push:
branches:
- staging
- production
env:
GIT_HOST: git.easysoft.ro
REGISTRY_HOST: registry.easysoft.ro
DOCKER_BUILDKIT: "1"
API_IMAGE: apps/myai-api
CV_MATCHER_API_IMAGE: apps/myai-cv-matcher-api
RAG_API_IMAGE: apps/myai-rag-api
@@ -15,18 +21,20 @@ env:
WEB_IMAGE: apps/myai-web
CV_CLEANUP_JOB_IMAGE: apps/myai-cv-cleanup-job
CV_SEARCH_JOB_IMAGE: apps/myai-cv-search-job
IMAGE_TAG: staging
PAGE_FETCHER_API_IMAGE: apps/myai-page-fetcher-api
IMAGE_TAG: ${{ github.ref_name }} # branch name == image tag (staging | production)
jobs:
build:
runs-on: host
steps:
- name: Checkout repository
- name: Checkout the pushed commit
env:
TOKEN: ${{ secrets.REPO_TOKEN }}
run: |
git clone "http://gelu:${TOKEN}@${GIT_HOST}:3000/${GITHUB_REPOSITORY}.git" .
git checkout "${{ github.sha }}"
- name: Login to registry
run: |
@@ -62,6 +70,10 @@ jobs:
run: |
docker build -f Jobs/cv-search-job/Dockerfile -t "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}" .
- name: Build Page Fetcher API image
run: |
docker build -f Apis/page-fetcher-api/Dockerfile -t "${REGISTRY_HOST}/${PAGE_FETCHER_API_IMAGE}:${IMAGE_TAG}" .
- name: Push API image
run: |
docker push "${REGISTRY_HOST}/${API_IMAGE}:${IMAGE_TAG}"
@@ -88,4 +100,13 @@ jobs:
- name: Push CV search job image
run: |
docker push "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}"
docker push "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}"
- name: Push Page Fetcher API image
run: |
docker push "${REGISTRY_HOST}/${PAGE_FETCHER_API_IMAGE}:${IMAGE_TAG}"
- name: Reclaim disk space (keep recent build cache)
if: always()
run: |
docker image prune -f # dangling only (keep base images)
+3
View File
@@ -376,3 +376,6 @@ files/
/docker-compose/.env.production
/docker-compose/.env.staging
# local infra access notes (secrets) — never commit
ACCESS.md
@@ -10,4 +10,6 @@ public sealed class JobMatchRequest
public string? CaptchaToken { get; set; }
/// <summary>ISO 639-1 language code for the match result (e.g. "en", "ro"). Defaults to "en".</summary>
public string? Language { get; set; }
/// <summary>Client IP address — set by the api layer from the HTTP context before forwarding. Not supplied by the browser.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -10,5 +10,5 @@ public interface IJobSearchApi
Task<CreateJobSearchTokenResponse> CreateTokenAsync([Body] CreateJobSearchTokenRequest request, CancellationToken ct);
[Post("/api/cv/job-search/token/{tokenId}/start")]
Task<StartJobSearchResponse> StartSearchAsync(string tokenId, CancellationToken ct);
Task<StartJobSearchResponse> StartSearchAsync(string tokenId, [Body] StartJobSearchRequest request, CancellationToken ct);
}
+6 -5
View File
@@ -163,17 +163,17 @@ public sealed class CvMatcherController : ControllerBase
return BadRequest(new ErrorResponse { Error = "Captcha verification failed.", Code = "captcha_verification_failed" });
}
request.ClientIpAddress = userIp;
_logger.LogInformation("Proxying job match request to cv-matcher-api. CvDocumentId={CvDocumentId}, HasJobUrl={HasJobUrl}, HasJobDescription={HasJobDescription}",
request.CvDocumentId,
!string.IsNullOrWhiteSpace(request.JobUrl),
!string.IsNullOrWhiteSpace(request.JobDescription));
var res = await _cvApi.MatchJob(request, ct);
var attachmentPath = TryGetCachedCvPath(request.CvDocumentId);
var language = NormalizeLanguage(request.Language);
var jobLabel = !string.IsNullOrWhiteSpace(request.JobUrl)
? request.JobUrl
: "Manual job description";
var language = NormalizeLanguage(request.Language);
: _emailSender.GetManualJobLabel(language);
string? jobSearchLink = null;
if (!string.IsNullOrWhiteSpace(request.Email) && !string.IsNullOrWhiteSpace(request.CvDocumentId))
@@ -181,7 +181,7 @@ public sealed class CvMatcherController : ControllerBase
try
{
var tokenResp = await _jobSearchApi.CreateTokenAsync(
new CreateJobSearchTokenRequest { CvDocumentId = request.CvDocumentId, Email = request.Email, Language = language, Keywords = res.Keywords },
new CreateJobSearchTokenRequest { CvDocumentId = request.CvDocumentId, Email = request.Email, Language = language, Keywords = res.Keywords, Location = res.Location, ClientIpAddress = userIp },
ct);
if (!string.IsNullOrWhiteSpace(tokenResp.TokenId))
{
@@ -244,7 +244,8 @@ public sealed class CvMatcherController : ControllerBase
{
try
{
var result = await _jobSearchApi.StartSearchAsync(t, ct);
var userIp = HttpContext.Connection.RemoteIpAddress?.ToString();
var result = await _jobSearchApi.StartSearchAsync(t, new StartJobSearchRequest { ClientIpAddress = userIp }, ct);
var lang = "en";
var (title, message) = result.Status switch
{
+33 -7
View File
@@ -7,6 +7,7 @@ using Microsoft.Extensions.Options;
using Microsoft.Net.Http.Headers;
using Swashbuckle.AspNetCore.Annotations;
using Common.Responses;
using Microsoft.AspNetCore.RateLimiting;
namespace Api.Controllers
{
@@ -17,38 +18,44 @@ namespace Api.Controllers
[ApiController]
[Route("api/[controller]")]
[EnableCors("FrontendOnly")]
[EnableRateLimiting("download")]
public sealed class FileDownloadController : ControllerBase
{
private readonly ILogger<FileDownloadController> _logger;
private readonly FileStorageSettings _fileStorageSettings;
private readonly IContentTypeProvider _contentTypeProvider;
private readonly IEmailSender _emailSender;
private const int BufferSize = 81920; // 80 KB buffer for optimal streaming performance
private readonly ICaptchaVerifier _captcha;
private const int BufferSize = 81920;
public FileDownloadController(
ILogger<FileDownloadController> logger,
IOptions<FileStorageSettings> fileStorageSettings,
IContentTypeProvider contentTypeProvider,
IEmailSender emailSender)
IEmailSender emailSender,
ICaptchaVerifier captcha)
{
_logger = logger;
_fileStorageSettings = fileStorageSettings.Value;
_contentTypeProvider = contentTypeProvider;
_emailSender = emailSender;
_captcha = captcha;
}
/// <summary>
/// Downloads a file with support for resume (range requests) and chunked transfer.
/// Supports HTTP 206 Partial Content for efficient downloads and resume capability.
/// Requires a valid reCAPTCHA v3 token on the initial (non-range) request.
/// Sends email notification when download starts.
/// </summary>
/// <param name="fileName">The name of the file to download (optional - uses default from settings if not provided)</param>
/// <param name="fileName">The name of the file to download (optional - uses default from settings if not provided).</param>
/// <param name="captchaToken">reCAPTCHA v3 token — required on the initial download request; omit on subsequent range requests.</param>
/// <returns>File stream with appropriate headers for resumable downloads</returns>
[HttpGet("{fileName?}")]
[SwaggerOperation(Summary = "Download file", Description = "Downloads a file with support for full and ranged (resumable) transfers.")]
[SwaggerOperation(Summary = "Download file", Description = "Downloads a file. Requires a reCAPTCHA v3 token on the initial request. Range requests for resume do not require a token.")]
[SwaggerResponse(StatusCodes.Status200OK, "Full file content returned")]
[SwaggerResponse(StatusCodes.Status206PartialContent, "Partial file content returned for a range request")]
[SwaggerResponse(StatusCodes.Status400BadRequest, "No file name provided and no default configured")]
[SwaggerResponse(StatusCodes.Status400BadRequest, "Missing/invalid captcha token, no file name, or no default configured")]
[SwaggerResponse(StatusCodes.Status404NotFound, "Requested file was not found")]
[SwaggerResponse(StatusCodes.Status416RangeNotSatisfiable, "Requested byte range is invalid")]
[SwaggerResponse(StatusCodes.Status500InternalServerError, "Unexpected server error while downloading")]
@@ -58,10 +65,30 @@ namespace Api.Controllers
[ProducesResponseType(typeof(ErrorResponse), StatusCodes.Status404NotFound)]
[ProducesResponseType(StatusCodes.Status416RangeNotSatisfiable)]
[ProducesResponseType(typeof(ErrorResponse), StatusCodes.Status500InternalServerError)]
public async Task<IActionResult> DownloadFile(string? fileName = null)
public async Task<IActionResult> DownloadFile(string? fileName = null, [FromQuery] string? captchaToken = null)
{
try
{
var userIp = HttpContext.Connection.RemoteIpAddress?.ToString();
// Captcha required on the initial (full) download only — range requests are resume continuations.
var isRangeRequest = !string.IsNullOrEmpty(Request.Headers[HeaderNames.Range].ToString());
if (!isRangeRequest)
{
if (string.IsNullOrWhiteSpace(captchaToken))
{
_logger.LogWarning("Download attempt without captcha token from IP={IP}", HttpContext.Connection.RemoteIpAddress);
return BadRequest(new ErrorResponse { Error = "Captcha token is required.", Code = "captcha_token_missing" });
}
var verdict = await _captcha.VerifyAsync(captchaToken, userIp, "file_download", CancellationToken.None);
if (!verdict.Success)
{
_logger.LogWarning("Download blocked by captcha. IP={IP} Score={Score}", userIp, verdict.Score);
return BadRequest(new ErrorResponse { Error = "Captcha verification failed.", Code = "captcha_verification_failed" });
}
}
if (string.IsNullOrWhiteSpace(fileName))
{
fileName = _fileStorageSettings.DefaultFileName;
@@ -99,7 +126,6 @@ namespace Api.Controllers
if (!_contentTypeProvider.TryGetContentType(filePath, out var contentType))
contentType = "application/octet-stream";
var userIp = HttpContext.Connection.RemoteIpAddress?.ToString();
_ = Task.Run(async () =>
{
try
+2 -2
View File
@@ -5,8 +5,8 @@ using Email.Data;
using Email.Data.Repositories;
using Email.Data.Repositories.Contracts;
using Email.Data.Services;
using EmailApi.Models.Clients;
using EmailApi.Models.Settings;
using Email.Models.Clients;
using Email.Models.Settings;
using Microsoft.EntityFrameworkCore;
using Models.Settings;
using MyAi.Data;
@@ -61,5 +61,11 @@ namespace Api.Services.Contracts
/// <param name="expiryDays">Number of days until the job-search link expires (shown in the footer copy).</param>
/// <returns>Rendered HTML body string.</returns>
string BuildMatchEmailBody(string cvDocumentId, JobMatchResponse result, string? jobLabel, string language, string? jobSearchLink = null, int expiryDays = 7);
/// <summary>
/// Returns the localised label for a manually-entered job description (no URL provided).
/// </summary>
/// <param name="language">Two-letter language code.</param>
string GetManualJobLabel(string language);
}
}
+9 -6
View File
@@ -1,8 +1,8 @@
using Api.Services.Contracts;
using CvMatcher.Models.Responses;
using Email.Data.Services;
using EmailApi.Models.Clients;
using EmailApi.Models.Requests;
using Email.Models.Clients;
using Email.Models.Requests;
using Microsoft.Extensions.Options;
using Models.Requests;
using Models.Settings;
@@ -138,7 +138,7 @@ public sealed class EmailApiEmailSender : IEmailSender
</tr>
<tr style="background:#f8f9fa">
<td style="font-weight:600;border:1px solid #dee2e6;color:#495057">IP Address</td>
<td style="border:1px solid #dee2e6">{userIp ?? "Unknown"}</td>
<td style="border:1px solid #dee2e6">{userIp ?? _emailTemplates.Get("email.notification.unknown-ip", "en")}</td>
</tr>
</table>
""";
@@ -215,8 +215,8 @@ public sealed class EmailApiEmailSender : IEmailSender
// email.match.body is now stored as HTML in the database
var body = _emailTemplates.Render("email.match.body", language,
("cvDocumentId", cvDocumentId),
("jobLabel", jobLabel ?? "N/A"),
("jobUrl", result.JobUrl ?? "N/A"),
("jobLabel", jobLabel ?? _emailTemplates.Get("email.match.fallback-na", language)),
("jobUrl", result.JobUrl ?? _emailTemplates.Get("email.match.fallback-na", language)),
("score", result.Score.ToString()),
("summary", WebUtility.HtmlEncode(result.Summary ?? string.Empty)),
("strengths", strengths),
@@ -238,5 +238,8 @@ public sealed class EmailApiEmailSender : IEmailSender
public string BuildMatchEmailSubject(int score, string? jobLabel, string language) =>
_emailTemplates.Render("email.match.subject", language,
("score", score.ToString()),
("jobLabel", jobLabel ?? "Job"));
("jobLabel", jobLabel ?? _emailTemplates.Get("email.match.subject-fallback-label", language)));
public string GetManualJobLabel(string language) =>
_emailTemplates.Get("email.match.manual-job-label", language);
}
+1 -21
View File
@@ -2,8 +2,7 @@
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File",
"Serilog.Sinks.Email"
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
@@ -30,25 +29,6 @@
"retainedFileCountLimit": 30,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "Email",
"Args": {
"restrictedToMinimumLevel": "Error",
"fromEmail": "",
"toEmail": "",
"mailServer": "",
"networkCredential": {
"userName": "",
"password": ""
},
"port": 587,
"enableSsl": true,
"emailSubject": "[mihes.ro API] Error Alert",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}",
"batchPostingLimit": 10,
"period": "0.00:05:00"
}
}
],
"Enrich": [
@@ -0,0 +1,11 @@
namespace Common.Settings;
/// <summary>
/// Connection settings for the internal page-fetcher-api service.
/// Bound from the <c>PageFetcherApi</c> configuration section.
/// </summary>
public sealed class PageFetcherApiSettings
{
public string BaseUrl { get; set; } = string.Empty;
public string InternalApiKey { get; set; } = string.Empty;
}
@@ -6,4 +6,7 @@ public sealed class CreateJobSearchTokenRequest
public string Email { get; set; } = string.Empty;
public string Language { get; set; } = "en";
public List<string> Keywords { get; set; } = [];
public string? Location { get; set; }
/// <summary>Client IP address forwarded by the api layer at CV match time. Null when not available.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -9,5 +9,7 @@
public string? Email { get; set; }
/// <summary>ISO 639-1 language code for the match result (e.g. "en", "ro"). Defaults to "en".</summary>
public string? Language { get; set; }
/// <summary>Client IP address forwarded by the api layer. Null when called from a background job.</summary>
public string? ClientIpAddress { get; set; }
}
}
@@ -0,0 +1,11 @@
namespace CvMatcher.Models.Requests;
/// <summary>
/// Request body sent by <c>api</c> when activating a one-time job-search link.
/// Carries the caller's IP address so it can be persisted on the session for auditing.
/// </summary>
public sealed class StartJobSearchRequest
{
/// <summary>Client IP address forwarded by the api layer. Null when not available.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -9,6 +9,7 @@
public List<string> Recommendations { get; set; } = [];
public List<string> Evidence { get; set; } = [];
public List<string> Keywords { get; set; } = [];
public string? Location { get; set; }
public bool Cached { get; set; }
public string? JobDocumentId { get; set; }
public string? JobUrl { get; set; }
@@ -21,6 +21,9 @@ public sealed class JobProviderConfig
public string JobLinkContains { get; set; } = string.Empty;
public List<string> InitialKeywords { get; set; } = [];
public int MaxResults { get; set; } = 20;
/// <summary>When true the scraper uses a headless Chromium browser to render JS-heavy pages.</summary>
public bool UseHeadlessBrowser { get; set; }
/// <summary>
/// When false, the Stage 2 anchor-text keyword filter is skipped.
/// Set to false for providers whose search URL already filters by relevance server-side.
/// </summary>
public bool RequireKeywordInAnchor { get; set; } = true;
}
@@ -53,7 +53,7 @@ public sealed class JobSearchController : ControllerBase
if (string.IsNullOrWhiteSpace(request.CvDocumentId) || string.IsNullOrWhiteSpace(request.Email))
return BadRequest(new ErrorResponse { Error = "CvDocumentId and Email are required.", Code = "invalid_request" });
var tokenId = await _tokenService.CreateTokenAsync(request.CvDocumentId, request.Email, request.Language, request.Keywords, ct);
var tokenId = await _tokenService.CreateTokenAsync(request.CvDocumentId, request.Email, request.Language, request.Keywords, request.Location, request.ClientIpAddress, ct);
return Ok(new CreateJobSearchTokenResponse { TokenId = tokenId });
}
catch (Exception ex)
@@ -81,11 +81,11 @@ public sealed class JobSearchController : ControllerBase
[SwaggerResponse(StatusCodes.Status500InternalServerError, "Session creation failed", typeof(ErrorResponse))]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(typeof(ErrorResponse), StatusCodes.Status500InternalServerError)]
public async Task<ActionResult<StartJobSearchResponse>> Start(string tokenId, CancellationToken ct)
public async Task<ActionResult<StartJobSearchResponse>> Start(string tokenId, [FromBody] StartJobSearchRequest? request, CancellationToken ct)
{
try
{
var status = await _tokenService.TriggerStartAsync(tokenId, ct);
var status = await _tokenService.TriggerStartAsync(tokenId, request?.ClientIpAddress, ct);
return Ok(new StartJobSearchResponse { Status = status });
}
catch (Exception ex)
+2
View File
@@ -8,6 +8,7 @@ COPY Apis/cv-search-data/cv-search-data.csproj Apis/cv-search-data/
COPY Apis/cv-matcher-data/cv-matcher-data.csproj Apis/cv-matcher-data/
COPY Apis/common/common.csproj Apis/common/
COPY Apis/cv-matcher-api-models/cv-matcher-api-models.csproj Apis/cv-matcher-api-models/
COPY Apis/page-fetcher-api-models/page-fetcher-api-models.csproj Apis/page-fetcher-api-models/
COPY Apis/myai-data/myai-data.csproj Apis/myai-data/
COPY Apis/shared-data/shared-data.csproj Apis/shared-data/
COPY Helpers/common-helpers/common-helpers.csproj Helpers/common-helpers/
@@ -20,6 +21,7 @@ COPY Apis/cv-search-data/ Apis/cv-search-data/
COPY Apis/cv-matcher-data/ Apis/cv-matcher-data/
COPY Apis/common/ Apis/common/
COPY Apis/cv-matcher-api-models/ Apis/cv-matcher-api-models/
COPY Apis/page-fetcher-api-models/ Apis/page-fetcher-api-models/
COPY Apis/myai-data/ Apis/myai-data/
COPY Apis/shared-data/ Apis/shared-data/
COPY Helpers/common-helpers/ Helpers/common-helpers/
+12 -1
View File
@@ -13,6 +13,7 @@ using Microsoft.EntityFrameworkCore;
using Refit;
using Serilog;
using Common.Settings;
using PageFetcher.Models;
using StartupHelpers;
using System.Reflection;
@@ -36,6 +37,16 @@ try
builder.Services.Configure<CvMatcher.Models.Settings.AiSettings>(builder.Configuration.GetSection("Ai"));
builder.Services.Configure<MatcherSettings>(builder.Configuration.GetSection("Matcher"));
builder.Services.Configure<JobSearchSettings>(builder.Configuration.GetSection("JobSearch"));
builder.Services.Configure<PageFetcherApiSettings>(builder.Configuration.GetSection("PageFetcherApi"));
builder.Services.AddRefitClient<IPageFetcherApiClient>()
.ConfigureHttpClient((sp, c) =>
{
var settings = sp.GetRequiredService<Microsoft.Extensions.Options.IOptions<PageFetcherApiSettings>>().Value;
c.BaseAddress = new Uri(settings.BaseUrl.TrimEnd('/') + "/");
if (!string.IsNullOrWhiteSpace(settings.InternalApiKey))
c.DefaultRequestHeaders.Add("X-Internal-Api-Key", settings.InternalApiKey);
});
builder.Services.AddRefitClient<IRefitRagApi>()
.ConfigureHttpClient((sp, c) =>
@@ -50,7 +61,7 @@ try
builder.Services.AddScoped<IRagApiClient, RagApiClient>();
builder.Services.AddHttpClient<IMatcherAiClient, MatcherAiClient>();
builder.Services.AddHttpClient<IJobTextExtractor, JobTextExtractor>();
builder.Services.AddScoped<IJobTextExtractor, JobTextExtractor>();
builder.Services.AddDbContext<CvMatcherDbContext>(options =>
{
@@ -13,21 +13,23 @@ public interface IJobTokenService
/// <param name="email">Email address of the user who will receive the results.</param>
/// <param name="language">Preferred language for result emails (e.g. <c>"en"</c>, <c>"ro"</c>).</param>
/// <param name="keywords">Job search keywords extracted by the LLM during the match call.</param>
/// <param name="location">Candidate location extracted from the CV (e.g. "Cluj-Napoca, Romania"). Null if not available.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>
/// The generated token ID to embed in the one-click job search link,
/// or <c>null</c> when no job providers are currently enabled (link should be suppressed).
/// </returns>
Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, CancellationToken ct);
Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, string? location, string? clientIpAddress, CancellationToken ct);
/// <summary>
/// Validates the token and, if valid, marks it as used and creates a <c>Pending</c> job search session.
/// </summary>
/// <param name="tokenId">The token ID from the one-click link.</param>
/// <param name="clientIpAddress">Client IP address forwarded by the api layer. Null when not available.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>
/// One of the <c>StartJobSearchStatus</c> string constants:
/// <c>Started</c>, <c>AlreadyUsed</c>, <c>Expired</c>, or <c>NotFound</c>.
/// </returns>
Task<string> TriggerStartAsync(string tokenId, CancellationToken ct);
Task<string> TriggerStartAsync(string tokenId, string? clientIpAddress, CancellationToken ct);
}
@@ -77,7 +77,7 @@ public sealed class CvMatcherService : ICvMatcherService
{
var job = await _rag.GetDocumentAsync(result.DocumentId, ct);
if (job is null) continue;
jobs.Add(await ScorePairAsync(cv, job, result.MatchedChunks.Select(x => x.Text).ToArray(), request.Email, NormalizeLanguage(null), ct));
jobs.Add(await ScorePairAsync(cv, job, result.MatchedChunks.Select(x => x.Text).ToArray(), request.Email, null, NormalizeLanguage(null), ct));
}
return new FindJobsResponse { CvDocumentId = request.CvDocumentId, Jobs = jobs };
@@ -107,7 +107,7 @@ public sealed class CvMatcherService : ICvMatcherService
.FirstOrDefault(x => x.DocumentId == job.DocumentId)?
.MatchedChunks.Select(x => x.Text).ToArray() ?? [];
return await ScorePairAsync(cv, jobDocument, matchedChunks, request.Email, NormalizeLanguage(request.Language), ct);
return await ScorePairAsync(cv, jobDocument, matchedChunks, request.Email, request.ClientIpAddress, NormalizeLanguage(request.Language), ct);
}
/// <summary>
@@ -115,7 +115,7 @@ public sealed class CvMatcherService : ICvMatcherService
/// Returns a cached result immediately when the same (CV, job, language) triple has been scored before.
/// When no evidence chunks are available from the vector search, falls back to the raw job text.
/// </summary>
private async Task<JobMatchResponse> ScorePairAsync(RagDocumentDetails cv, RagDocumentDetails job, IReadOnlyList<string> evidenceChunks, string? email, string language, CancellationToken ct)
private async Task<JobMatchResponse> ScorePairAsync(RagDocumentDetails cv, RagDocumentDetails job, IReadOnlyList<string> evidenceChunks, string? email, string? clientIpAddress, string language, CancellationToken ct)
{
var cached = await _repository.GetMatchAsync(cv.Id, job.Id, language, ct);
if (cached is not null) return cached;
@@ -141,11 +141,13 @@ public sealed class CvMatcherService : ICvMatcherService
""";
var json = await _ai.CreateChatCompletionAsync(systemPrompt, userPrompt, 0.2m, ct);
var result = ParseResult(json);
var errorSummary = await _aiPrompts.GetAsync("parse-error.summary", language, ct);
var errorRec = await _aiPrompts.GetAsync("parse-error.recommendation", language, ct);
var result = ParseResult(json, errorSummary, errorRec);
result.JobDocumentId = job.Id;
result.JobUrl = job.SourceUrl;
result.Cached = false;
await _repository.SaveMatchAsync(cv.Id, job.Id, language, result, ct);
await _repository.SaveMatchAsync(cv.Id, job.Id, language, result, email, clientIpAddress, ct);
return result;
}
@@ -153,7 +155,10 @@ public sealed class CvMatcherService : ICvMatcherService
/// Deserialises the LLM's JSON output into a <see cref="JobMatchResponse"/>.
/// Returns a safe fallback response instead of throwing when the JSON cannot be parsed.
/// </summary>
private static JobMatchResponse ParseResult(string json)
private static JobMatchResponse ParseResult(
string json,
string? errorSummary = null,
string? errorRec = null)
{
try
{
@@ -168,8 +173,8 @@ public sealed class CvMatcherService : ICvMatcherService
return new JobMatchResponse
{
Score = 0,
Summary = "The AI response could not be parsed as structured JSON.",
Recommendations = ["Inspect the raw model output and tune the scoring prompt."]
Summary = errorSummary ?? "The AI response could not be parsed as structured JSON.",
Recommendations = [errorRec ?? "Inspect the raw model output and tune the scoring prompt."]
};
}
@@ -1,26 +1,23 @@
using System.Net;
using System.Text.RegularExpressions;
using CvMatcher.Models.Settings;
using Api.Services.Contracts;
using Microsoft.Extensions.Options;
using PageFetcher.Models;
namespace Api.Services;
/// <summary>
/// Extracts normalised plain text from a job posting, either from a pasted description or by
/// fetching and stripping the HTML of the job page URL.
/// fetching the job page text via <c>page-fetcher-api</c> (headless Chromium rendering).
/// </summary>
public sealed class JobTextExtractor : IJobTextExtractor
{
private readonly HttpClient _http;
private readonly IPageFetcherApiClient _pageFetcher;
private readonly MatcherSettings _settings;
public JobTextExtractor(HttpClient http, IOptions<MatcherSettings> options)
public JobTextExtractor(IPageFetcherApiClient pageFetcher, IOptions<MatcherSettings> options)
{
_http = http;
_pageFetcher = pageFetcher;
_settings = options.Value;
_http.Timeout = TimeSpan.FromSeconds(25);
_http.DefaultRequestHeaders.UserAgent.ParseAdd("MyAi.ro CV Matcher/1.0");
}
/// <inheritdoc />
@@ -31,15 +28,18 @@ public sealed class JobTextExtractor : IJobTextExtractor
if (string.IsNullOrWhiteSpace(jobUrl)) return string.Empty;
if (!Uri.TryCreate(jobUrl, UriKind.Absolute, out var uri) || uri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException("Invalid job URL.");
}
var html = await _http.GetStringAsync(uri, ct);
html = Regex.Replace(html, "<script[\\s\\S]*?</script>", " ", RegexOptions.IgnoreCase);
html = Regex.Replace(html, "<style[\\s\\S]*?</style>", " ", RegexOptions.IgnoreCase);
html = Regex.Replace(html, "<[^>]+>", " ");
return Limit(Normalize(WebUtility.HtmlDecode(html)));
var response = await _pageFetcher.FetchAsync(new FetchPageRequest
{
Url = jobUrl,
CallerService = "cv-matcher-api"
}, ct);
if (!response.Success)
throw new InvalidOperationException($"Failed to fetch job page: {response.Error}");
return Limit(Normalize(response.Text));
}
/// <summary>Truncates text to the configured maximum character count.</summary>
@@ -34,7 +34,7 @@ public sealed class JobTokenService : IJobTokenService
}
/// <inheritdoc />
public async Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, CancellationToken ct)
public async Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, string? location, string? clientIpAddress, CancellationToken ct)
{
var hasEnabledProviders = await _db.JobProviders.AnyAsync(p => p.Enabled, ct);
if (!hasEnabledProviders)
@@ -50,6 +50,8 @@ public sealed class JobTokenService : IJobTokenService
Email = email,
Language = language,
Keywords = string.Join(",", keywords),
Location = location,
ClientIpAddress = clientIpAddress,
ExpiresAt = DateTime.UtcNow.AddDays(_settings.TokenExpiryDays),
Used = false,
CreatedAt = DateTime.UtcNow
@@ -57,12 +59,12 @@ public sealed class JobTokenService : IJobTokenService
_db.JobSearchTokens.Add(token);
await _db.SaveChangesAsync(ct);
_logger.LogInformation("Job search token created. TokenId={TokenId}, CvDocumentId={CvDocumentId}, Keywords={Keywords}", token.Id, cvDocumentId, token.Keywords);
_logger.LogInformation("Job search token created. TokenId={TokenId}, CvDocumentId={CvDocumentId}, Keywords={Keywords}, Location={Location}", token.Id, cvDocumentId, token.Keywords, token.Location);
return token.Id;
}
/// <inheritdoc />
public async Task<string> TriggerStartAsync(string tokenId, CancellationToken ct)
public async Task<string> TriggerStartAsync(string tokenId, string? clientIpAddress, CancellationToken ct)
{
var token = await _db.JobSearchTokens.FirstOrDefaultAsync(x => x.Id == tokenId, ct);
if (token is null) return StartJobSearchStatus.NotFound;
@@ -92,6 +94,8 @@ public sealed class JobTokenService : IJobTokenService
Language = token.Language,
Status = JobSearchStatus.Pending,
Keywords = keywords,
Location = token.Location,
ClientIpAddress = clientIpAddress,
ProviderConfigJson = providerConfigJson,
CreatedAt = DateTime.UtcNow
};
@@ -126,7 +130,7 @@ public sealed class JobTokenService : IJobTokenService
JobLinkContains = entity.JobLinkContains,
InitialKeywords = keywords,
MaxResults = entity.MaxResults,
UseHeadlessBrowser = entity.UseHeadlessBrowser
RequireKeywordInAnchor = entity.RequireKeywordInAnchor
};
}
+1 -21
View File
@@ -2,8 +2,7 @@
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File",
"Serilog.Sinks.Email"
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
@@ -30,25 +29,6 @@
"retainedFileCountLimit": 30,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "Email",
"Args": {
"restrictedToMinimumLevel": "Error",
"fromEmail": "",
"toEmail": "",
"mailServer": "",
"networkCredential": {
"userName": "",
"password": ""
},
"port": 587,
"enableSsl": true,
"emailSubject": "[mihes.ro API] Error Alert",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}",
"batchPostingLimit": 10,
"period": "0.00:05:00"
}
}
],
"Enrich": [
@@ -82,6 +82,7 @@
<ProjectReference Include="..\cv-search-data\cv-search-data.csproj" />
<ProjectReference Include="..\cv-matcher-data\cv-matcher-data.csproj" />
<ProjectReference Include="..\common\common.csproj" />
<ProjectReference Include="..\page-fetcher-api-models\page-fetcher-api-models.csproj" />
<ProjectReference Include="..\..\Helpers\startup-helpers\startup-helpers.csproj" />
</ItemGroup>
</Project>
@@ -36,6 +36,8 @@ public sealed class CvMatcherDbContext : DbContext
entity.Property(x => x.JobDocumentId).HasMaxLength(64).IsRequired();
entity.Property(x => x.ResultJson).IsRequired();
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.Property(x => x.Email).HasMaxLength(256);
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.HasIndex(x => new { x.CvDocumentId, x.JobDocumentId, x.Language }).IsUnique();
});
@@ -9,4 +9,6 @@ public sealed class CvMatchResultEntity : BaseEntity
public string Language { get; set; } = "en";
public string ResultJson { get; set; } = string.Empty;
public int Score { get; set; }
public string? Email { get; set; }
public string? ClientIpAddress { get; set; }
}
@@ -0,0 +1,130 @@
// <auto-generated />
using System;
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvMatcher.Data.Migrations
{
[DbContext(typeof(CvMatcherDbContext))]
[Migration("20260608124331_ImproveKeywordsAndAddLocation")]
partial class ImproveKeywordsAndAddLocation
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvMatcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvMatcher.Data.Entities.AiPromptEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("AiPrompts", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Language")
.IsRequired()
.HasColumnType("nvarchar(450)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("CvDocumentId", "JobDocumentId", "Language")
.IsUnique();
b.ToTable("Results", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatcherChatCacheEntity", b =>
{
b.Property<string>("CacheKey")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Model")
.IsRequired()
.HasMaxLength(120)
.HasColumnType("nvarchar(120)");
b.Property<string>("ResponseText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<decimal>("Temperature")
.HasColumnType("decimal(4,2)");
b.HasKey("CacheKey");
b.ToTable("ChatCache", "cvMatcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,65 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvMatcher.Data.Migrations
{
/// <inheritdoc />
public partial class ImproveKeywordsAndAddLocation : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
// Update English prompt: tighter keywords instruction (job-board search terms, not abstract
// concepts) and add location field so the LLM extracts the candidate's city/country.
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "en"],
columns: ["Value", "Description"],
values: [
"You are a strict CV-to-job matching engine. Return JSON only. Score realistically from 0 to 100. Penalize missing required skills. Do not invent experience. Use concise business language. All text fields in the JSON response must be in English.\nJSON shape: {\"score\":number,\"summary\":\"one-line summary in English\",\"strengths\":[\"strength 1 in English\"],\"gaps\":[\"gap 1 in English\"],\"recommendations\":[\"recommendation 1 in English\"],\"evidence\":[\"evidence 1 in English\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"City, Country\"}.\nFor 'keywords': extract 2-4 short, concrete terms a recruiter would search for on a job board — the candidate's primary role title and key technologies (e.g. 'Senior .NET Developer', 'C#', 'Azure'). Avoid abstract concepts like 'leadership', 'cloud', or 'microservices'.\nFor 'location': extract the candidate's city and country from the CV (e.g. 'Cluj-Napoca, Romania'). Use an empty string if not found.",
"System prompt for CV-to-job matching in English. Extracts job-board-friendly keywords (role title + key tech) and candidate location."
]);
// Update Romanian prompt: same improvements.
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "ro"],
columns: ["Value", "Description"],
values: [
"Ești un motor strict de potrivire CV-job. Returnează doar JSON. Punctează realist între 0 și 100. Penalizează abilitățile lipsă necesare. Nu inventa experiență. Folosește limbaj profesional concis. Toate câmpurile text din răspunsul JSON trebuie să fie în limba română.\nJSON shape: {\"score\":number,\"summary\":\"rezumat pe o linie în română\",\"strengths\":[\"punct forte 1 în română\"],\"gaps\":[\"lipsă 1 în română\"],\"recommendations\":[\"recomandare 1 în română\"],\"evidence\":[\"dovadă 1 în română\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"Oraș, Țară\"}.\nPentru 'keywords': extrage 2-4 termeni scurți și concreți pe care un recrutor i-ar căuta pe un site de joburi — titlul principal al rolului și tehnologiile cheie (ex. 'Senior .NET Developer', 'C#', 'Azure'). Evită concepte abstracte precum 'leadership', 'cloud' sau 'microservicii'.\nPentru 'location': extrage orașul și țara candidatului din CV (ex. 'Cluj-Napoca, România'). Folosește string gol dacă nu se găsește.",
"System prompt pentru potrivire CV-job în limba română. Extrage cuvinte cheie prietenoase pentru site-uri de joburi (titlu rol + tehnologii cheie) și locația candidatului."
]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "en"],
columns: ["Value", "Description"],
values: [
"You are a strict CV-to-job matching engine. Return JSON only. Score realistically from 0 to 100. Penalize missing required skills. Do not invent experience. Use concise business language. All text fields in the JSON response must be in English.\nJSON shape: {\"score\":number,\"summary\":\"one-line summary in English\",\"strengths\":[\"strength 1 in English\"],\"gaps\":[\"gap 1 in English\"],\"recommendations\":[\"recommendation 1 in English\"],\"evidence\":[\"evidence 1 in English\"],\"keywords\":[\"keyword1\",\"keyword2\",\"keyword3\"]}",
"System prompt for CV-to-job matching in English. Instructs LLM to return JSON with CV strengths, gaps, and recommendations relative to the job."
]);
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "ro"],
columns: ["Value", "Description"],
values: [
"Ești un motor strict de potrivire CV-job. Returnează doar JSON. Punctează realist între 0 și 100. Penalizează abilitățile lipsă necesare. Nu inventa experiență. Folosește limbaj profesional concis. Toate câmpurile text din răspunsul JSON trebuie să fie în limba română.\nJSON shape: {\"score\":number,\"summary\":\"rezumat pe o linie în română\",\"strengths\":[\"punct forte 1 în română\"],\"gaps\":[\"lipsă 1 în română\"],\"recommendations\":[\"recomandare 1 în română\"],\"evidence\":[\"dovadă 1 în română\"],\"keywords\":[\"cuvant1\",\"cuvant2\",\"cuvant3\"]}",
"System prompt pentru potrivire CV-job în limba română. Instruiește LLM-ul să returneze JSON cu punctele forte ale CV-ului, lacunele și recomandări relative la job."
]);
}
}
}
@@ -0,0 +1,138 @@
// <auto-generated />
using System;
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvMatcher.Data.Migrations
{
[DbContext(typeof(CvMatcherDbContext))]
[Migration("20260608155310_AddEmailAndIpToResults")]
partial class AddEmailAndIpToResults
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvMatcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvMatcher.Data.Entities.AiPromptEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("AiPrompts", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Language")
.IsRequired()
.HasColumnType("nvarchar(450)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("CvDocumentId", "JobDocumentId", "Language")
.IsUnique();
b.ToTable("Results", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatcherChatCacheEntity", b =>
{
b.Property<string>("CacheKey")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Model")
.IsRequired()
.HasMaxLength(120)
.HasColumnType("nvarchar(120)");
b.Property<string>("ResponseText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<decimal>("Temperature")
.HasColumnType("decimal(4,2)");
b.HasKey("CacheKey");
b.ToTable("ChatCache", "cvMatcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,45 @@
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvMatcher.Data.Migrations
{
/// <inheritdoc />
public partial class AddEmailAndIpToResults : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "Results",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
migrationBuilder.AddColumn<string>(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "Results",
type: "nvarchar(256)",
maxLength: 256,
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "Results");
migrationBuilder.DropColumn(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "Results");
}
}
}
@@ -0,0 +1,138 @@
// <auto-generated />
using System;
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvMatcher.Data.Migrations
{
[DbContext(typeof(CvMatcherDbContext))]
[Migration("20260608193046_AddParseErrorPrompts")]
partial class AddParseErrorPrompts
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvMatcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvMatcher.Data.Entities.AiPromptEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("AiPrompts", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Language")
.IsRequired()
.HasColumnType("nvarchar(450)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("CvDocumentId", "JobDocumentId", "Language")
.IsUnique();
b.ToTable("Results", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatcherChatCacheEntity", b =>
{
b.Property<string>("CacheKey")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Model")
.IsRequired()
.HasMaxLength(120)
.HasColumnType("nvarchar(120)");
b.Property<string>("ResponseText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<decimal>("Temperature")
.HasColumnType("decimal(4,2)");
b.HasKey("CacheKey");
b.ToTable("ChatCache", "cvMatcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,67 @@
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvMatcher.Data.Migrations
{
/// <inheritdoc />
public partial class AddParseErrorPrompts : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.summary", "en", "The AI response could not be parsed. Please try again.", "Summary shown in match email when the AI returns an unparseable response"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.summary", "ro", "Răspunsul AI nu a putut fi interpretat. Vă rugăm să încercați din nou.", "Sumar afișat în emailul de potrivire când AI returnează un răspuns care nu poate fi interpretat"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.recommendation", "en", "If the problem persists, try a different job link or description.", "Recommendation shown in match email when the AI returns an unparseable response"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.recommendation", "ro", "Dacă problema persistă, încercați un alt link sau descriere de job.", "Recomandare afișată în emailul de potrivire când AI returnează un răspuns care nu poate fi interpretat"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.summary", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.summary", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.recommendation", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.recommendation", "ro"]);
}
}
}
@@ -0,0 +1,138 @@
// <auto-generated />
using System;
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvMatcher.Data.Migrations
{
[DbContext(typeof(CvMatcherDbContext))]
[Migration("20260609133623_FixKeywordExtractionPrompt")]
partial class FixKeywordExtractionPrompt
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvMatcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvMatcher.Data.Entities.AiPromptEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("AiPrompts", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Language")
.IsRequired()
.HasColumnType("nvarchar(450)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("CvDocumentId", "JobDocumentId", "Language")
.IsUnique();
b.ToTable("Results", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatcherChatCacheEntity", b =>
{
b.Property<string>("CacheKey")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Model")
.IsRequired()
.HasMaxLength(120)
.HasColumnType("nvarchar(120)");
b.Property<string>("ResponseText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<decimal>("Temperature")
.HasColumnType("decimal(4,2)");
b.HasKey("CacheKey");
b.ToTable("ChatCache", "cvMatcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,93 @@
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvMatcher.Data.Migrations
{
/// <inheritdoc />
public partial class FixKeywordExtractionPrompt : Migration
{
// Full prompt values — only the 'keywords' instruction changes vs. the previous migration.
// Stored in full so Down() can restore the previous version exactly.
private const string EnNew =
"You are a strict CV-to-job matching engine. Return JSON only. Score realistically from 0 to 100. Penalize missing required skills. Do not invent experience. Use concise business language. All text fields in the JSON response must be in English.\n" +
"JSON shape: {\"score\":number,\"summary\":\"one-line summary in English\",\"strengths\":[\"strength 1 in English\"],\"gaps\":[\"gap 1 in English\"],\"recommendations\":[\"recommendation 1 in English\"],\"evidence\":[\"evidence 1 in English\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"City, Country\"}.\n" +
"For 'keywords': extract 2-4 job-board search terms that represent the candidate's professional identity as shown in their CV — their seniority level and primary role title (e.g. 'Software Architect', 'Engineering Manager', 'Senior .NET Developer') plus 1-2 core technologies they genuinely emphasize throughout the CV. Derive these entirely from the CV — do not use the job title or job technologies unless they independently match the candidate's actual positioning. Avoid generic terms like 'developer', 'engineer', 'cloud', or 'leadership'.\n" +
"For 'location': extract the candidate's city and country from the CV (e.g. 'Cluj-Napoca, Romania'). Use an empty string if not found.";
private const string EnPrev =
"You are a strict CV-to-job matching engine. Return JSON only. Score realistically from 0 to 100. Penalize missing required skills. Do not invent experience. Use concise business language. All text fields in the JSON response must be in English.\n" +
"JSON shape: {\"score\":number,\"summary\":\"one-line summary in English\",\"strengths\":[\"strength 1 in English\"],\"gaps\":[\"gap 1 in English\"],\"recommendations\":[\"recommendation 1 in English\"],\"evidence\":[\"evidence 1 in English\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"City, Country\"}.\n" +
"For 'keywords': extract 2-4 short, concrete terms a recruiter would search for on a job board — the candidate's primary role title and key technologies (e.g. 'Senior .NET Developer', 'C#', 'Azure'). Avoid abstract concepts like 'leadership', 'cloud', or 'microservices'.\n" +
"For 'location': extract the candidate's city and country from the CV (e.g. 'Cluj-Napoca, Romania'). Use an empty string if not found.";
private const string RoNew =
"Ești un motor strict de potrivire CV-job. Returnează doar JSON. Punctează realist între 0 și 100. Penalizează abilitățile lipsă necesare. Nu inventa experiență. Folosește limbaj profesional concis. Toate câmpurile text din răspunsul JSON trebuie să fie în limba română.\n" +
"JSON shape: {\"score\":number,\"summary\":\"rezumat pe o linie în română\",\"strengths\":[\"punct forte 1 în română\"],\"gaps\":[\"lipsă 1 în română\"],\"recommendations\":[\"recomandare 1 în română\"],\"evidence\":[\"dovadă 1 în română\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"Oraș, Țară\"}.\n" +
"Pentru 'keywords': extrage 2-4 termeni de căutare pe site-uri de joburi care reprezintă identitatea profesională a candidatului conform CV-ului — nivelul de senioritate și titlul principal de rol (ex. 'Software Architect', 'Engineering Manager', 'Senior .NET Developer') și 1-2 tehnologii de bază pe care candidatul le evidențiază cu adevărat în CV. Derivă aceștia exclusiv din CV — nu folosi titlul jobului sau tehnologiile din job dacă nu corespund poziționării reale a candidatului. Evită termeni generici precum 'developer', 'engineer', 'cloud' sau 'leadership'.\n" +
"Pentru 'location': extrage orașul și țara candidatului din CV (ex. 'Cluj-Napoca, România'). Folosește string gol dacă nu se găsește.";
private const string RoPrev =
"Ești un motor strict de potrivire CV-job. Returnează doar JSON. Punctează realist între 0 și 100. Penalizează abilitățile lipsă necesare. Nu inventa experiență. Folosește limbaj profesional concis. Toate câmpurile text din răspunsul JSON trebuie să fie în limba română.\n" +
"JSON shape: {\"score\":number,\"summary\":\"rezumat pe o linie în română\",\"strengths\":[\"punct forte 1 în română\"],\"gaps\":[\"lipsă 1 în română\"],\"recommendations\":[\"recomandare 1 în română\"],\"evidence\":[\"dovadă 1 în română\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"Oraș, Țară\"}.\n" +
"Pentru 'keywords': extrage 2-4 termeni scurți și concreți pe care un recrutor i-ar căuta pe un site de joburi — titlul principal al rolului și tehnologiile cheie (ex. 'Senior .NET Developer', 'C#', 'Azure'). Evită concepte abstracte precum 'leadership', 'cloud' sau 'microservicii'.\n" +
"Pentru 'location': extrage orașul și țara candidatului din CV (ex. 'Cluj-Napoca, România'). Folosește string gol dacă nu se găsește.";
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
// Update English prompt: keywords must now be derived from the CV only,
// not influenced by the job description being matched against.
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "en"],
columns: ["Value", "Description"],
values: [
EnNew,
"System prompt for CV-to-job matching in English. Keywords represent the candidate's CV identity (seniority + role + core tech), not the job being matched."
]);
// Update Romanian prompt: same improvement.
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "ro"],
columns: ["Value", "Description"],
values: [
RoNew,
"System prompt pentru potrivire CV-job în română. Cuvintele cheie reprezintă identitatea CV-ului candidatului (senioritate + rol + tehnologii cheie), nu jobul cu care se face potrivirea."
]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "en"],
columns: ["Value", "Description"],
values: [
EnPrev,
"System prompt for CV-to-job matching in English. Extracts job-board-friendly keywords (role title + key tech) and candidate location."
]);
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "ro"],
columns: ["Value", "Description"],
values: [
RoPrev,
"System prompt pentru potrivire CV-job în limba română. Extrage cuvinte cheie prietenoase pentru site-uri de joburi (titlu rol + tehnologii cheie) și locația candidatului."
]);
}
}
}
@@ -60,6 +60,10 @@ namespace CvMatcher.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
@@ -70,6 +74,10 @@ namespace CvMatcher.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
@@ -6,7 +6,7 @@ public interface IMatcherRepository
{
Task InitializeAsync(CancellationToken ct);
Task<JobMatchResponse?> GetMatchAsync(string cvDocumentId, string jobDocumentId, string language, CancellationToken ct);
Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, CancellationToken ct);
Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, string? email, string? clientIpAddress, CancellationToken ct);
Task<string?> GetChatCompletionAsync(string cacheKey, CancellationToken ct);
Task SaveChatCompletionAsync(string cacheKey, string model, decimal temperature, string responseText, CancellationToken ct);
}
@@ -40,7 +40,7 @@ public sealed class EfMatcherRepository : IMatcherRepository
return result;
}
public async Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, CancellationToken ct)
public async Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, string? email, string? clientIpAddress, CancellationToken ct)
{
var exists = await _db.CvMatchResults.AnyAsync(
x => x.CvDocumentId == cvDocumentId && x.JobDocumentId == jobDocumentId && x.Language == language,
@@ -58,6 +58,8 @@ public sealed class EfMatcherRepository : IMatcherRepository
Language = language,
ResultJson = JsonSerializer.Serialize(response, new JsonSerializerOptions(JsonSerializerDefaults.Web)),
Score = response.Score,
Email = email,
ClientIpAddress = clientIpAddress,
CreatedAt = DateTime.UtcNow
});
@@ -36,6 +36,7 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.Language).HasMaxLength(8).HasDefaultValue("en").IsRequired();
entity.Property(x => x.Keywords).HasMaxLength(1000).HasDefaultValue(string.Empty);
entity.Property(x => x.Used).HasDefaultValue(false);
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
});
@@ -51,6 +52,7 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.Keywords).HasMaxLength(1000);
entity.Property(x => x.ProviderConfigJson).IsRequired(false);
entity.Property(x => x.Language).HasMaxLength(8).HasDefaultValue("en").IsRequired();
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.HasIndex(x => x.Status);
});
@@ -64,6 +66,8 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.ProviderName).HasMaxLength(128);
entity.Property(x => x.JobUrl).HasMaxLength(2048);
entity.Property(x => x.JobTitle).HasMaxLength(512);
entity.Property(x => x.Email).HasMaxLength(256);
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.HasIndex(x => x.SessionId);
});
@@ -79,7 +83,6 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.InitialKeywordsJson).HasMaxLength(2000).HasDefaultValue("[]").IsRequired();
entity.Property(x => x.MaxResults).HasDefaultValue(20);
entity.Property(x => x.DisplayOrder).HasDefaultValue(0);
entity.Property(x => x.UseHeadlessBrowser).HasDefaultValue(false);
});
}
}
@@ -31,6 +31,9 @@ public sealed class JobProviderEntity
/// <summary>Controls display ordering in future admin UIs.</summary>
public int DisplayOrder { get; set; }
/// <summary>When true, the scraper renders the page with headless Chromium instead of a plain HTTP GET.</summary>
public bool UseHeadlessBrowser { get; set; }
/// <summary>
/// When false, the Stage 2 anchor-text keyword filter is skipped.
/// Set to false for providers whose search URL already filters by relevance server-side (ejobs.ro, bestjobs.eu).
/// </summary>
public bool RequireKeywordInAnchor { get; set; } = true;
}
@@ -11,4 +11,8 @@ public sealed class JobSearchResultEntity : BaseEntity
public string JobText { get; set; } = string.Empty;
public int Score { get; set; }
public string ResultJson { get; set; } = string.Empty;
/// <summary>Email address of the user who triggered the search. Copied from the parent session.</summary>
public string? Email { get; set; }
/// <summary>Client IP address at link-click time. Copied from the parent session.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -9,6 +9,9 @@ public sealed class JobSearchSessionEntity : BaseEntity
public string Email { get; set; } = string.Empty;
public string Status { get; set; } = JobSearchStatus.Pending;
public string Keywords { get; set; } = string.Empty;
public string? Location { get; set; }
/// <summary>Client IP address captured when the user clicked the one-time job-search link. Null for sessions created before this field was added.</summary>
public string? ClientIpAddress { get; set; }
public string? ProviderConfigJson { get; set; }
public string Language { get; set; } = "en";
}
@@ -10,4 +10,7 @@ public sealed class JobSearchTokenEntity : BaseEntity
public DateTime ExpiresAt { get; set; }
public bool Used { get; set; }
public string Keywords { get; set; } = string.Empty;
public string? Location { get; set; }
/// <summary>Client IP address captured when the user submitted the CV match request. Null for tokens created before this field was added.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -0,0 +1,243 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608124304_AddRequireKeywordInAnchorAndLocation")]
partial class AddRequireKeywordInAnchorAndLocation
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.Property<bool>("UseHeadlessBrowser")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,74 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class AddRequireKeywordInAnchorAndLocation : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "Location",
schema: "cvSearch",
table: "JobSearchTokens",
type: "nvarchar(max)",
nullable: true);
migrationBuilder.AddColumn<string>(
name: "Location",
schema: "cvSearch",
table: "JobSearchSessions",
type: "nvarchar(max)",
nullable: true);
migrationBuilder.AddColumn<bool>(
name: "RequireKeywordInAnchor",
schema: "cvSearch",
table: "JobProviders",
type: "bit",
nullable: false,
defaultValue: true);
// ejobs.ro (Id=1) and bestjobs.eu (Id=2) do server-side keyword filtering via their
// search URL — the Stage 2 anchor-text filter rejects all Romanian job titles because
// they rarely contain abstract LLM keywords.
migrationBuilder.UpdateData(
schema: "cvSearch",
table: "JobProviders",
keyColumn: "Id",
keyValue: 1,
column: "RequireKeywordInAnchor",
value: false);
migrationBuilder.UpdateData(
schema: "cvSearch",
table: "JobProviders",
keyColumn: "Id",
keyValue: 2,
column: "RequireKeywordInAnchor",
value: false);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "Location",
schema: "cvSearch",
table: "JobSearchTokens");
migrationBuilder.DropColumn(
name: "Location",
schema: "cvSearch",
table: "JobSearchSessions");
migrationBuilder.DropColumn(
name: "RequireKeywordInAnchor",
schema: "cvSearch",
table: "JobProviders");
}
}
}
@@ -0,0 +1,243 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608124452_AddLocationToProviders")]
partial class AddLocationToProviders
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.Property<bool>("UseHeadlessBrowser")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,71 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class AddLocationToProviders : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
// ejobs.ro (Id=1): location in URL path as slug, keywords via q= param.
// Verified URL structure: /locuri-de-munca/{location-slug}?q={keywords}
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "JobProviders",
keyColumn: "Id",
keyValue: 1,
column: "SearchUrlTemplate",
value: "https://www.ejobs.ro/locuri-de-munca/{location-slug}?q={keywords}");
// bestjobs.eu (Id=2): location in URL path as slug, keywords via query param.
// Verified URL structure: /ro/locuri-de-munca-in-{location-slug}?keywords={keywords}
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "JobProviders",
keyColumn: "Id",
keyValue: 2,
column: "SearchUrlTemplate",
value: "https://bestjobs.eu/ro/locuri-de-munca-in-{location-slug}?keywords={keywords}");
// linkedin.com (Id=3): location as query parameter.
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "JobProviders",
keyColumn: "Id",
keyValue: 3,
column: "SearchUrlTemplate",
value: "https://www.linkedin.com/jobs/search/?keywords={keywords}&location={location}");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "JobProviders",
keyColumn: "Id",
keyValue: 1,
column: "SearchUrlTemplate",
value: "https://www.ejobs.ro/locuri-de-munca?q={keywords}");
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "JobProviders",
keyColumn: "Id",
keyValue: 2,
column: "SearchUrlTemplate",
value: "https://www.bestjobs.eu/ro/locuri-de-munca?keywords={keywords}");
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "JobProviders",
keyColumn: "Id",
keyValue: 3,
column: "SearchUrlTemplate",
value: "https://www.linkedin.com/jobs/search/?keywords={keywords}");
}
}
}
@@ -0,0 +1,238 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608154221_RemoveUseHeadlessBrowser")]
partial class RemoveUseHeadlessBrowser
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,32 @@
using CvSearch.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class RemoveUseHeadlessBrowser : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "UseHeadlessBrowser",
schema: MigrationConstants.SchemaName,
table: "JobProviders");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<bool>(
name: "UseHeadlessBrowser",
schema: MigrationConstants.SchemaName,
table: "JobProviders",
type: "bit",
nullable: false,
defaultValue: false);
}
}
}
@@ -0,0 +1,250 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608161102_AddEmailIpToSessionAndResults")]
partial class AddEmailIpToSessionAndResults
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,58 @@
using CvSearch.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class AddEmailIpToSessionAndResults : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchSessions",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
migrationBuilder.AddColumn<string>(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults",
type: "nvarchar(256)",
maxLength: 256,
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchSessions");
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults");
migrationBuilder.DropColumn(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults");
}
}
}
@@ -0,0 +1,254 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608161930_AddClientIpToJobSearchTokens")]
partial class AddClientIpToJobSearchTokens
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,32 @@
using CvSearch.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class AddClientIpToJobSearchTokens : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchTokens",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchTokens");
}
}
}
@@ -61,16 +61,14 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.Property<bool>("UseHeadlessBrowser")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
@@ -82,11 +80,19 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
@@ -131,6 +137,10 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
@@ -158,6 +168,9 @@ namespace CvSearch.Data.Migrations
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
@@ -184,6 +197,10 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
@@ -216,6 +233,9 @@ namespace CvSearch.Data.Migrations
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
@@ -1,7 +1,7 @@
using EmailApi.Models.Requests;
using Email.Models.Requests;
using Refit;
namespace EmailApi.Models.Clients;
namespace Email.Models.Clients;
public interface IEmailApiClient
{
@@ -1,4 +1,4 @@
namespace EmailApi.Models.Requests;
namespace Email.Models.Requests;
public sealed class SendEmailRequest
{
@@ -1,4 +1,4 @@
namespace EmailApi.Models.Settings;
namespace Email.Models.Settings;
public sealed class EmailApiSettings
{
@@ -1,4 +1,4 @@
namespace Models.Settings;
namespace Email.Models.Settings;
public sealed class SmtpSettings
{
@@ -1,9 +1,9 @@
using EmailApi.Models.Requests;
using EmailApi.Services;
using Api.Services;
using Email.Models.Requests;
using Microsoft.AspNetCore.Mvc;
using Swashbuckle.AspNetCore.Annotations;
namespace EmailApi.Controllers;
namespace Api.Controllers;
/// <summary>
/// Internal email relay. Accepts an HTML body fragment from trusted callers
+2
View File
@@ -4,6 +4,7 @@ WORKDIR /src
COPY Apis/email-api/email-api.csproj Apis/email-api/
COPY Apis/email-data/email-data.csproj Apis/email-data/
COPY Apis/shared-data/shared-data.csproj Apis/shared-data/
COPY Apis/email-api-models/email-api-models.csproj Apis/email-api-models/
COPY Apis/api-models/api-models.csproj Apis/api-models/
COPY Apis/common/common.csproj Apis/common/
@@ -15,6 +16,7 @@ RUN dotnet restore Apis/email-api/email-api.csproj
COPY Apis/email-api/ Apis/email-api/
COPY Apis/email-data/ Apis/email-data/
COPY Apis/shared-data/ Apis/shared-data/
COPY Apis/email-api-models/ Apis/email-api-models/
COPY Apis/api-models/ Apis/api-models/
COPY Apis/common/ Apis/common/
+3 -3
View File
@@ -3,8 +3,9 @@ using Email.Data;
using Email.Data.Repositories;
using Email.Data.Repositories.Contracts;
using Email.Data.Services;
using EmailApi.Services;
using Api.Services;
using Microsoft.EntityFrameworkCore;
using Email.Models.Settings;
using Models.Settings;
using Serilog;
using StartupHelpers;
@@ -50,9 +51,8 @@ try
app.UseDefaultSerilogRequestLogging();
app.UseJsonExceptionHandler(ServiceName);
app.UseSwaggerInDevelopment("Email API", "EmailAPI");
app.UseInternalApiKeyProtection();
app.UseSwaggerInDevelopment("Email API", "EmailAPI");
app.UseRouting();
app.UseAuthorization();
@@ -1,12 +1,13 @@
using Email.Data.Services;
using EmailApi.Models.Requests;
using Email.Models.Requests;
using MailKit.Net.Smtp;
using MailKit.Security;
using Microsoft.Extensions.Options;
using MimeKit;
using Email.Models.Settings;
using Models.Settings;
namespace EmailApi.Services;
namespace Api.Services;
/// <summary>
/// Wraps an HTML body fragment in the branded HTML shell and sends the resulting email via SMTP using MailKit.
@@ -57,7 +58,7 @@ public sealed class SmtpEmailDispatcher
if (!string.IsNullOrWhiteSpace(req.ReplyTo))
msg.ReplyTo.Add(MailboxAddress.Parse(req.ReplyTo));
msg.Subject = $"[{_environmentName}] {req.Subject}".Trim();
msg.Subject = req.Subject.Trim();
var shellStart = _templates.Get("email.html-shell.start", "*");
var shellEnd = _templates.Get("email.html-shell.end", "*");
+1 -21
View File
@@ -2,8 +2,7 @@
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File",
"Serilog.Sinks.Email"
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
@@ -30,25 +29,6 @@
"retainedFileCountLimit": 30,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "Email",
"Args": {
"restrictedToMinimumLevel": "Error",
"fromEmail": "",
"toEmail": "",
"mailServer": "",
"networkCredential": {
"userName": "",
"password": ""
},
"port": 587,
"enableSsl": true,
"emailSubject": "[myAi] Email API Error Alert",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}",
"batchPostingLimit": 10,
"period": "0.00:05:00"
}
}
],
"Enrich": [
@@ -0,0 +1,69 @@
// <auto-generated />
using System;
using Email.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace Email.Data.Migrations
{
[DbContext(typeof(EmailDbContext))]
[Migration("20260608125339_AddLocationToScanSummaryTemplate")]
partial class AddLocationToScanSummaryTemplate
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("email")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("Email.Data.Entities.EmailTemplateEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<string>("OperatorCopy")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("Templates", "email");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,80 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace Email.Data.Migrations
{
/// <inheritdoc />
public partial class AddLocationToScanSummaryTemplate : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "en"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Keywords used:</strong>&nbsp;{{keywordsHtml}}</div>
<div style=""margin-bottom: 8px;""><strong>Location:</strong>&nbsp;{{location}}</div>
<div><strong>Providers scanned:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "ro"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Cuvinte cheie folosite:</strong>&nbsp;{{keywordsHtml}}</div>
<div style=""margin-bottom: 8px;""><strong>Locație căutată:</strong>&nbsp;{{location}}</div>
<div><strong>Furnizori scanați:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "en"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Keywords used:</strong>&nbsp;{{keywordsHtml}}</div>
<div><strong>Providers scanned:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "ro"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Cuvinte cheie folosite:</strong>&nbsp;{{keywordsHtml}}</div>
<div><strong>Furnizori scanați:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
}
}
}
@@ -0,0 +1,69 @@
// <auto-generated />
using System;
using Email.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace Email.Data.Migrations
{
[DbContext(typeof(EmailDbContext))]
[Migration("20260608190611_AddManualJobLabelTemplate")]
partial class AddManualJobLabelTemplate
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("email")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("Email.Data.Entities.EmailTemplateEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<string>("OperatorCopy")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("Templates", "email");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,43 @@
using Email.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace Email.Data.Migrations
{
/// <inheritdoc />
public partial class AddManualJobLabelTemplate : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.manual-job-label", "en", "Manual job description", "Label used in the match email subject and body when no job URL was provided"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.manual-job-label", "ro", "Descriere manuală a jobului", "Etichetă folosită în subiectul și corpul emailului când nu a fost furnizat un URL de job"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.manual-job-label", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.manual-job-label", "ro"]);
}
}
}
@@ -0,0 +1,69 @@
// <auto-generated />
using System;
using Email.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace Email.Data.Migrations
{
[DbContext(typeof(EmailDbContext))]
[Migration("20260608192938_AddFallbackStringTemplates")]
partial class AddFallbackStringTemplates
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("email")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("Email.Data.Entities.EmailTemplateEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<string>("OperatorCopy")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("Templates", "email");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,163 @@
using Email.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace Email.Data.Migrations
{
/// <inheritdoc />
public partial class AddFallbackStringTemplates : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.fallback-na", "en", "N/A", "Fallback when a match email field (job label or URL) has no value"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.fallback-na", "ro", "N/A", "Fallback când un câmp al emailului de potrivire (etichetă job sau URL) nu are valoare"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.subject-fallback-label", "en", "Job", "Fallback job label used in match email subject when no specific label is available"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.subject-fallback-label", "ro", "Job", "Etichetă fallback pentru subiectul emailului de potrivire când nu există o etichetă specifică"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.notification.unknown-ip", "en", "Unknown", "Fallback IP address label in operator notification emails"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.notification.unknown-ip", "ro", "Necunoscut", "Etichetă fallback pentru adresa IP în emailurile de notificare operator"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.keywords-empty", "en", "none detected", "Text shown in job search results email when no CV keywords were extracted"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.keywords-empty", "ro", "niciunul detectat", "Text afișat în emailul cu rezultatele căutării când nu au fost extrase cuvinte cheie din CV"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.providers-empty", "en", "none", "Text shown in job search results email when no providers were searched"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.providers-empty", "ro", "niciunul", "Text afișat în emailul cu rezultatele căutării când nu au fost căutați furnizori"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.location-empty", "en", "-", "Fallback location display in job search results email scan summary"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.location-empty", "ro", "-", "Afișaj fallback pentru locație în sumarului de scanare al emailului cu rezultatele căutării"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.fallback-na", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.fallback-na", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.subject-fallback-label", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.subject-fallback-label", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.notification.unknown-ip", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.notification.unknown-ip", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.keywords-empty", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.keywords-empty", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.providers-empty", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.providers-empty", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.location-empty", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.location-empty", "ro"]);
}
}
}
@@ -0,0 +1,26 @@
namespace PageFetcher.Models;
/// <summary>
/// Request to fetch a web page via the page-fetcher-api.
/// </summary>
public sealed class FetchPageRequest
{
/// <summary>Absolute HTTP or HTTPS URL to fetch.</summary>
public string Url { get; set; } = string.Empty;
/// <summary>
/// Playwright wait condition. Accepted values: <c>networkidle</c> (default), <c>domcontentloaded</c>, <c>load</c>.
/// </summary>
public string WaitFor { get; set; } = "networkidle";
/// <summary>
/// Identifies the calling service for audit purposes (e.g. <c>cv-matcher-api</c>, <c>cv-search-job</c>).
/// </summary>
public string CallerService { get; set; } = string.Empty;
/// <summary>
/// Optional reference to the job search session that triggered this fetch.
/// Stored on <c>pageFetcher.PageFetches</c> for cross-schema audit queries.
/// </summary>
public string? JobSearchSessionId { get; set; }
}
@@ -0,0 +1,25 @@
namespace PageFetcher.Models;
/// <summary>
/// Result of a page fetch operation.
/// </summary>
public sealed class FetchPageResponse
{
/// <summary>Final URL after any redirects.</summary>
public string Url { get; set; } = string.Empty;
/// <summary>HTTP status code returned by the page. <c>0</c> on network failure.</summary>
public int StatusCode { get; set; }
/// <summary>Full rendered HTML as returned by Playwright.</summary>
public string Html { get; set; } = string.Empty;
/// <summary>Plain text extracted from the HTML (script/style stripped, whitespace normalised).</summary>
public string Text { get; set; } = string.Empty;
/// <summary>Whether the fetch succeeded. <c>false</c> on timeout or network error.</summary>
public bool Success { get; set; }
/// <summary>Exception message when <see cref="Success"/> is <c>false</c>.</summary>
public string? Error { get; set; }
}
@@ -0,0 +1,16 @@
using Refit;
namespace PageFetcher.Models;
/// <summary>
/// Refit client for the internal page-fetcher-api service.
/// All calls require the <c>X-Internal-Api-Key</c> header, configured at registration time.
/// </summary>
public interface IPageFetcherApiClient
{
/// <summary>
/// Fetches a web page via headless Chromium and returns the rendered HTML and extracted plain text.
/// </summary>
[Post("/api/page/fetch")]
Task<FetchPageResponse> FetchAsync([Body] FetchPageRequest request, CancellationToken ct = default);
}
@@ -0,0 +1,17 @@
namespace PageFetcher.Models.Settings;
/// <summary>
/// Runtime settings for the page-fetcher service.
/// Bound from the <c>PageFetcher</c> configuration section.
/// </summary>
public sealed class PageFetcherSettings
{
/// <summary>Default Playwright wait condition (<c>networkidle</c>, <c>load</c>, <c>domcontentloaded</c>).</summary>
public string DefaultWaitFor { get; set; } = "networkidle";
/// <summary>Page navigation timeout in seconds.</summary>
public int TimeoutSeconds { get; set; } = 30;
/// <summary>Maximum characters stored/returned in the extracted text field.</summary>
public int MaxTextChars { get; set; } = 60_000;
}
@@ -0,0 +1,12 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AssemblyName>page-fetcher-api-models</AssemblyName>
<RootNamespace>PageFetcher.Models</RootNamespace>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Refit.HttpClientFactory" />
</ItemGroup>
</Project>
@@ -0,0 +1,47 @@
using Api.Services;
using Microsoft.AspNetCore.Mvc;
using PageFetcher.Models;
using Swashbuckle.AspNetCore.Annotations;
namespace Api.Controllers;
/// <summary>
/// Handles page-fetch requests: navigates to the URL via Playwright and returns rendered HTML and extracted text.
/// </summary>
[ApiController]
[Route("api/page")]
public sealed class PageController : ControllerBase
{
private readonly PageFetcherService _service;
private readonly ILogger<PageController> _logger;
public PageController(PageFetcherService service, ILogger<PageController> logger)
{
_service = service;
_logger = logger;
}
/// <summary>
/// Fetches a web page via headless Chromium.
/// Returns rendered HTML and extracted plain text.
/// </summary>
[HttpPost("fetch")]
[SwaggerOperation(Summary = "Fetch a web page", Description = "Navigates to the given URL using Playwright, returns rendered HTML and stripped plain text.")]
[SwaggerResponse(StatusCodes.Status200OK, "Page fetched successfully", typeof(FetchPageResponse))]
[SwaggerResponse(StatusCodes.Status400BadRequest, "Invalid or non-HTTP(S) URL")]
public async Task<ActionResult<FetchPageResponse>> Fetch([FromBody] FetchPageRequest request, CancellationToken ct)
{
if (string.IsNullOrWhiteSpace(request.Url))
return BadRequest(new { Error = "Url is required." });
if (!Uri.TryCreate(request.Url, UriKind.Absolute, out var uri) ||
(uri.Scheme != Uri.UriSchemeHttp && uri.Scheme != Uri.UriSchemeHttps))
return BadRequest(new { Error = "Url must be an absolute HTTP or HTTPS URL." });
_logger.LogInformation("Fetch request: {Url} | caller={Caller} | waitFor={WaitFor}",
request.Url, request.CallerService, request.WaitFor);
var result = await _service.FetchAsync(request, ct);
return Ok(result);
}
}
+50
View File
@@ -0,0 +1,50 @@
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY Directory.Packages.props ./
COPY Apis/page-fetcher-api/page-fetcher-api.csproj Apis/page-fetcher-api/
COPY Apis/page-fetcher-data/page-fetcher-data.csproj Apis/page-fetcher-data/
COPY Apis/page-fetcher-api-models/page-fetcher-api-models.csproj Apis/page-fetcher-api-models/
COPY Apis/common/common.csproj Apis/common/
COPY Apis/shared-data/shared-data.csproj Apis/shared-data/
COPY Helpers/startup-helpers/startup-helpers.csproj Helpers/startup-helpers/
COPY Helpers/common-helpers/common-helpers.csproj Helpers/common-helpers/
RUN dotnet restore Apis/page-fetcher-api/page-fetcher-api.csproj
COPY Apis/page-fetcher-api/ Apis/page-fetcher-api/
COPY Apis/page-fetcher-data/ Apis/page-fetcher-data/
COPY Apis/page-fetcher-api-models/ Apis/page-fetcher-api-models/
COPY Apis/common/ Apis/common/
COPY Apis/shared-data/ Apis/shared-data/
COPY Helpers/startup-helpers/ Helpers/startup-helpers/
COPY Helpers/common-helpers/ Helpers/common-helpers/
RUN dotnet publish Apis/page-fetcher-api/page-fetcher-api.csproj -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false
# Download Playwright Chromium browser in the build stage.
# Node.js is only needed here to run npx — it is not copied to the final image.
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
RUN apt-get update && apt-get install -y --no-install-recommends nodejs npm \
&& npx --yes playwright@1.60.0 install chromium \
&& rm -rf /var/lib/apt/lists/*
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS final
WORKDIR /app
# System libraries required by Chromium on Debian bookworm
RUN apt-get update && apt-get install -y --no-install-recommends \
libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 \
libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 \
libgbm1 libasound2t64 libpango-1.0-0 libcairo2 libatspi2.0-0 \
libwayland-client0 libx11-xcb1 libx11-6 libxcb1 libxext6 \
&& rm -rf /var/lib/apt/lists/*
# Copy the Playwright Chromium browser from the build stage
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
COPY --from=build /ms-playwright /ms-playwright
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "page-fetcher-api.dll"]
+75
View File
@@ -0,0 +1,75 @@
using System.Reflection;
using Microsoft.EntityFrameworkCore;
using PageFetcher.Data;
using Api.Services;
using PageFetcher.Models.Settings;
using Serilog;
using StartupHelpers;
StartupExtensions.LoadDotEnvFile();
const string ServiceName = "page-fetcher-api";
var appVersion = StartupExtensions.GetApplicationVersion(Assembly.GetExecutingAssembly());
try
{
var builder = WebApplication.CreateBuilder(args);
builder.ConfigureJsonSerilog(ServiceName, appVersion);
Log.Information("Starting {Service} version {AppVersion}", ServiceName, appVersion);
builder.AddAzureKeyVaultIfConfigured();
builder.Services.Configure<PageFetcherSettings>(builder.Configuration.GetSection("PageFetcher"));
builder.Services.AddDbContext<PageFetchDbContext>(options =>
{
var connectionString = builder.Services.GetConfiguredDbConnectionString(builder.Configuration);
options.UseSqlServer(connectionString, sql =>
{
sql.MigrationsHistoryTable(PageFetchDbContext.MigrationTableName, PageFetchDbContext.SchemaName);
sql.MigrationsAssembly("page-fetcher-data");
});
});
// Playwright browser: singleton hosted service, shared across all requests
builder.Services.AddSingleton<PlaywrightBrowserService>();
builder.Services.AddHostedService(sp => sp.GetRequiredService<PlaywrightBrowserService>());
builder.Services.AddScoped<PageFetcherService>();
builder.Services.AddControllers();
builder.Services.AddSwaggerWithXmlComments(Assembly.GetExecutingAssembly(), "Page Fetcher API");
var app = builder.Build();
app.LogStartupDiagnostics(ServiceName);
app.UseDefaultSerilogRequestLogging();
app.UseJsonExceptionHandler(ServiceName);
app.UseInternalApiKeyProtection();
app.UseSwaggerInDevelopment("Page Fetcher API", "PageFetcherAPI");
app.UseRouting();
app.UseAuthorization();
app.MapControllers();
Log.Information("Running EF Core migrations if any");
using (var scope = app.Services.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<PageFetchDbContext>();
db.Database.Migrate();
}
Log.Information("{Service} startup complete. Listening for requests...", ServiceName);
app.Run();
}
catch (Exception ex)
{
Log.Fatal(ex, "{Service} terminated unexpectedly", ServiceName);
}
finally
{
Log.Information("Shutting down {Service}", ServiceName);
Log.CloseAndFlush();
}
@@ -0,0 +1,12 @@
{
"profiles": {
"page-fetcher-api": {
"commandName": "Project",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"applicationUrl": "https://localhost:50268;http://localhost:50269"
}
}
}
@@ -0,0 +1,145 @@
using System.Diagnostics;
using System.Net;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Options;
using Microsoft.Playwright;
using PageFetcher.Data;
using PageFetcher.Data.Entities;
using PageFetcher.Models;
using PageFetcher.Models.Settings;
namespace Api.Services;
/// <summary>
/// Fetches a web page via Playwright, extracts plain text, persists the result to the database,
/// and returns a <see cref="FetchPageResponse"/>.
/// </summary>
public sealed class PageFetcherService
{
private readonly PlaywrightBrowserService _browserService;
private readonly PageFetchDbContext _db;
private readonly PageFetcherSettings _settings;
private readonly ILogger<PageFetcherService> _logger;
public PageFetcherService(
PlaywrightBrowserService browserService,
PageFetchDbContext db,
IOptions<PageFetcherSettings> settings,
ILogger<PageFetcherService> logger)
{
_browserService = browserService;
_db = db;
_settings = settings.Value;
_logger = logger;
}
/// <summary>
/// Fetches the page at <paramref name="request.Url"/> using Playwright, saves the fetch record,
/// and returns the HTML and extracted text.
/// Returns a failed response (with <see cref="FetchPageResponse.Success"/> = false) rather than throwing
/// on network or navigation errors.
/// </summary>
public async Task<FetchPageResponse> FetchAsync(FetchPageRequest request, CancellationToken ct)
{
var sw = Stopwatch.StartNew();
string html = string.Empty;
string text = string.Empty;
int? statusCode = null;
bool success = false;
string? errorMessage = null;
string finalUrl = request.Url;
try
{
var page = await _browserService.Browser.NewPageAsync();
await using var _ = page.ConfigureAwait(false);
var waitUntil = request.WaitFor?.ToLowerInvariant() switch
{
"load" => WaitUntilState.Load,
"domcontentloaded" => WaitUntilState.DOMContentLoaded,
_ => WaitUntilState.NetworkIdle
};
IResponse? response;
try
{
response = await page.GotoAsync(request.Url, new PageGotoOptions
{
WaitUntil = waitUntil,
Timeout = _settings.TimeoutSeconds * 1_000
});
}
catch (TimeoutException)
{
_logger.LogWarning("Playwright NetworkIdle timeout for {Url}, using partial content", request.Url);
response = null;
}
statusCode = response?.Status;
finalUrl = page.Url;
html = await page.ContentAsync();
text = ExtractText(html);
success = true;
_logger.LogInformation("Fetched {Url} → HTTP {Status} | HTML {HtmlLen} chars | text {TextLen} chars | {DurationMs} ms",
request.Url, statusCode?.ToString() ?? "timeout", html.Length, text.Length, sw.ElapsedMilliseconds);
}
catch (Exception ex)
{
errorMessage = ex.Message;
_logger.LogError(ex, "Failed to fetch {Url}", request.Url);
}
finally
{
sw.Stop();
}
// Persist fetch record
var entity = new PageFetchEntity
{
Id = Guid.NewGuid().ToString("N"),
Url = request.Url,
CallerService = request.CallerService ?? string.Empty,
JobSearchSessionId = request.JobSearchSessionId,
HttpStatusCode = statusCode,
Html = html,
Text = text,
DurationMs = sw.ElapsedMilliseconds,
Success = success,
ErrorMessage = errorMessage
};
_db.PageFetches.Add(entity);
await _db.SaveChangesAsync(ct);
return new FetchPageResponse
{
Url = finalUrl,
StatusCode = statusCode ?? 0,
Html = html,
Text = text,
Success = success,
Error = errorMessage
};
}
/// <summary>
/// Strips script/style blocks and all HTML tags from raw HTML, normalises whitespace,
/// and truncates to <see cref="PageFetcherSettings.MaxTextChars"/>.
/// </summary>
private string ExtractText(string html)
{
if (string.IsNullOrWhiteSpace(html)) return string.Empty;
var text = html;
text = Regex.Replace(text, "<script[\\s\\S]*?</script>", " ", RegexOptions.IgnoreCase);
text = Regex.Replace(text, "<style[\\s\\S]*?</style>", " ", RegexOptions.IgnoreCase);
text = Regex.Replace(text, "<[^>]+>", " ");
text = WebUtility.HtmlDecode(text);
text = string.Join(' ', text.Split((char[]?)null, StringSplitOptions.RemoveEmptyEntries)).Trim();
var max = Math.Max(4_000, _settings.MaxTextChars);
return text.Length <= max ? text : text[..max];
}
}
@@ -0,0 +1,49 @@
using Microsoft.Playwright;
namespace Api.Services;
/// <summary>
/// Singleton hosted service that owns the Playwright Chromium browser process for the lifetime of the application.
/// Launches the browser once at startup and exposes it for injection into <see cref="PageFetcherService"/>.
/// </summary>
public sealed class PlaywrightBrowserService : IHostedService, IAsyncDisposable
{
private IPlaywright? _playwright;
private IBrowser? _browser;
private readonly ILogger<PlaywrightBrowserService> _logger;
public PlaywrightBrowserService(ILogger<PlaywrightBrowserService> logger)
{
_logger = logger;
}
/// <summary>The running Chromium browser instance. Available after <see cref="StartAsync"/> completes.</summary>
public IBrowser Browser => _browser ?? throw new InvalidOperationException("Browser has not been started yet.");
/// <inheritdoc />
public async Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Launching Playwright Chromium browser...");
_playwright = await Playwright.CreateAsync();
_browser = await _playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
{
Headless = true,
Args = ["--no-sandbox", "--disable-setuid-sandbox", "--disable-dev-shm-usage"]
});
_logger.LogInformation("Playwright Chromium browser launched successfully.");
}
/// <inheritdoc />
public async Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Closing Playwright Chromium browser...");
if (_browser is not null) await _browser.CloseAsync();
}
/// <inheritdoc />
public async ValueTask DisposeAsync()
{
if (_browser is not null) await _browser.DisposeAsync();
_playwright?.Dispose();
}
}
+73
View File
@@ -0,0 +1,73 @@
{
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
"Override": {
"Microsoft.AspNetCore": "Warning",
"Microsoft.AspNetCore.Hosting": "Information",
"Microsoft.AspNetCore.Routing": "Warning",
"System.Net.Http.HttpClient": "Warning",
"PageFetcherApi": "Information"
}
},
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "File",
"Args": {
"path": "logs/page-fetcher-api-.log",
"rollingInterval": "Day",
"retainedFileCountLimit": 30,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
}
],
"Enrich": [
"FromLogContext",
"WithMachineName",
"WithEnvironmentName"
]
},
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"Microsoft.AspNetCore.Hosting": "Information",
"Microsoft.AspNetCore.Routing": "Warning",
"System.Net.Http.HttpClient": "Warning",
"PageFetcherApi": "Information"
}
},
"LogEnvironmentOnStartup": true,
"AllowedHosts": "*",
"KeyVault": {
"VaultUri": "",
"Enabled": false
},
"Database": {
"Host": "localhost",
"Port": 1433,
"Name": "MyAiDb",
"User": "sa",
"Password": "",
"TrustServerCertificate": true
},
"InternalApi": {
"ApiKey": "",
"RequireApiKey": true
},
"PageFetcher": {
"DefaultWaitFor": "networkidle",
"TimeoutSeconds": 30,
"MaxTextChars": 60000
}
}
@@ -0,0 +1,34 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
<RootNamespace>PageFetcherApi</RootNamespace>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<NoWarn>$(NoWarn);1591</NoWarn>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Playwright" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Serilog.AspNetCore" />
<PackageReference Include="Serilog.Enrichers.Environment" />
<PackageReference Include="Serilog.Sinks.Console" />
<PackageReference Include="Serilog.Sinks.File" />
<PackageReference Include="Serilog.Sinks.Email" />
<PackageReference Include="Swashbuckle.AspNetCore" />
<PackageReference Include="Swashbuckle.AspNetCore.Annotations" />
<PackageReference Include="DotNetEnv" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\page-fetcher-data\page-fetcher-data.csproj" />
<ProjectReference Include="..\page-fetcher-api-models\page-fetcher-api-models.csproj" />
<ProjectReference Include="..\common\common.csproj" />
<ProjectReference Include="..\..\Helpers\startup-helpers\startup-helpers.csproj" />
<ProjectReference Include="..\..\Helpers\common-helpers\common-helpers.csproj" />
</ItemGroup>
</Project>
@@ -0,0 +1,40 @@
using Shared.Data.Entities;
namespace PageFetcher.Data.Entities;
/// <summary>
/// Audit record of a single page-fetch operation performed by the page-fetcher-api.
/// Stores the full rendered HTML and extracted plain text for every URL fetched.
/// </summary>
public sealed class PageFetchEntity : BaseEntity
{
/// <summary>The URL that was requested.</summary>
public string Url { get; set; } = string.Empty;
/// <summary>Name of the service that requested the fetch (e.g. <c>cv-matcher-api</c>, <c>cv-search-job</c>).</summary>
public string CallerService { get; set; } = string.Empty;
/// <summary>HTTP status code returned by the remote server. <c>null</c> on network failure.</summary>
public int? HttpStatusCode { get; set; }
/// <summary>Full rendered HTML as returned by Playwright.</summary>
public string Html { get; set; } = string.Empty;
/// <summary>Plain text extracted from the HTML (script/style stripped, whitespace normalised).</summary>
public string Text { get; set; } = string.Empty;
/// <summary>Playwright round-trip time in milliseconds.</summary>
public long DurationMs { get; set; }
/// <summary><c>true</c> when the page was fetched successfully; <c>false</c> on timeout or network error.</summary>
public bool Success { get; set; }
/// <summary>Exception message when <see cref="Success"/> is <c>false</c>.</summary>
public string? ErrorMessage { get; set; }
/// <summary>
/// Optional reference to the <c>cvSearch.JobSearchSessions</c> row that triggered this fetch.
/// Null for fetches not originating from a job search session (e.g. direct CV-to-job matches).
/// </summary>
public string? JobSearchSessionId { get; set; }
}
@@ -0,0 +1,8 @@
namespace PageFetcher.Data;
/// <summary>Schema and migration-history table name constants for the pageFetcher EF schema.</summary>
public static class MigrationConstants
{
public const string SchemaName = "pageFetcher";
public const string MigrationTableName = "_Migrations";
}
@@ -0,0 +1,82 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using PageFetcher.Data;
#nullable disable
namespace PageFetcher.Data.Migrations
{
[DbContext(typeof(PageFetchDbContext))]
[Migration("20260608143523_InitialSchema")]
partial class InitialSchema
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("pageFetcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("PageFetcher.Data.Entities.PageFetchEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("CallerService")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<long>("DurationMs")
.HasColumnType("bigint");
b.Property<string>("ErrorMessage")
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.Property<string>("Html")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int?>("HttpStatusCode")
.HasColumnType("int");
b.Property<bool>("Success")
.HasColumnType("bit");
b.Property<string>("Text")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("Url")
.IsRequired()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.HasKey("Id");
b.HasIndex("CreatedAt");
b.HasIndex("Url");
b.ToTable("PageFetches", "pageFetcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,59 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace PageFetcher.Data.Migrations
{
/// <inheritdoc />
public partial class InitialSchema : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.EnsureSchema(
name: MigrationConstants.SchemaName);
migrationBuilder.CreateTable(
name: "PageFetches",
schema: MigrationConstants.SchemaName,
columns: table => new
{
Id = table.Column<string>(type: "nvarchar(64)", maxLength: 64, nullable: false),
Url = table.Column<string>(type: "nvarchar(2000)", maxLength: 2000, nullable: false),
CallerService = table.Column<string>(type: "nvarchar(64)", maxLength: 64, nullable: false),
HttpStatusCode = table.Column<int>(type: "int", nullable: true),
Html = table.Column<string>(type: "nvarchar(max)", nullable: false),
Text = table.Column<string>(type: "nvarchar(max)", nullable: false),
DurationMs = table.Column<long>(type: "bigint", nullable: false),
Success = table.Column<bool>(type: "bit", nullable: false),
ErrorMessage = table.Column<string>(type: "nvarchar(2000)", maxLength: 2000, nullable: true),
CreatedAt = table.Column<DateTime>(type: "datetime2", nullable: false, defaultValueSql: "SYSUTCDATETIME()")
},
constraints: table =>
{
table.PrimaryKey("PK_PageFetches", x => x.Id);
});
migrationBuilder.CreateIndex(
name: "IX_PageFetches_CreatedAt",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
column: "CreatedAt");
migrationBuilder.CreateIndex(
name: "IX_PageFetches_Url",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
column: "Url");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "PageFetches",
schema: MigrationConstants.SchemaName);
}
}
}
@@ -0,0 +1,88 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using PageFetcher.Data;
#nullable disable
namespace PageFetcher.Data.Migrations
{
[DbContext(typeof(PageFetchDbContext))]
[Migration("20260608165542_AddJobSearchSessionId")]
partial class AddJobSearchSessionId
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("pageFetcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("PageFetcher.Data.Entities.PageFetchEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("CallerService")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<long>("DurationMs")
.HasColumnType("bigint");
b.Property<string>("ErrorMessage")
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.Property<string>("Html")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int?>("HttpStatusCode")
.HasColumnType("int");
b.Property<string>("JobSearchSessionId")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<bool>("Success")
.HasColumnType("bit");
b.Property<string>("Text")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("Url")
.IsRequired()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.HasKey("Id");
b.HasIndex("CreatedAt");
b.HasIndex("JobSearchSessionId");
b.HasIndex("Url");
b.ToTable("PageFetches", "pageFetcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,43 @@
using PageFetcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace PageFetcher.Data.Migrations
{
/// <inheritdoc />
public partial class AddJobSearchSessionId : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
type: "nvarchar(64)",
maxLength: 64,
nullable: true);
migrationBuilder.CreateIndex(
name: "IX_PageFetches_JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
column: "JobSearchSessionId");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropIndex(
name: "IX_PageFetches_JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches");
migrationBuilder.DropColumn(
name: "JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches");
}
}
}
@@ -0,0 +1,85 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using PageFetcher.Data;
#nullable disable
namespace PageFetcher.Data.Migrations
{
[DbContext(typeof(PageFetchDbContext))]
partial class PageFetchDbContextModelSnapshot : ModelSnapshot
{
protected override void BuildModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("pageFetcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("PageFetcher.Data.Entities.PageFetchEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("CallerService")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<long>("DurationMs")
.HasColumnType("bigint");
b.Property<string>("ErrorMessage")
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.Property<string>("Html")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int?>("HttpStatusCode")
.HasColumnType("int");
b.Property<string>("JobSearchSessionId")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<bool>("Success")
.HasColumnType("bit");
b.Property<string>("Text")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("Url")
.IsRequired()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.HasKey("Id");
b.HasIndex("CreatedAt");
b.HasIndex("JobSearchSessionId");
b.HasIndex("Url");
b.ToTable("PageFetches", "pageFetcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,48 @@
using Microsoft.EntityFrameworkCore;
using PageFetcher.Data.Entities;
namespace PageFetcher.Data;
/// <summary>
/// EF Core DbContext for the <c>pageFetcher</c> schema.
/// Owns the <c>PageFetches</c> audit table.
/// </summary>
public sealed class PageFetchDbContext : DbContext
{
public const string SchemaName = MigrationConstants.SchemaName;
public const string MigrationTableName = MigrationConstants.MigrationTableName;
public PageFetchDbContext(DbContextOptions<PageFetchDbContext> options) : base(options) { }
public DbSet<PageFetchEntity> PageFetches => Set<PageFetchEntity>();
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseSqlServer(x => x.MigrationsHistoryTable(MigrationTableName, SchemaName));
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasDefaultSchema(SchemaName);
modelBuilder.Entity<PageFetchEntity>(entity =>
{
entity.ToTable("PageFetches");
entity.HasKey(x => x.Id);
entity.Property(x => x.Id).HasMaxLength(64);
entity.Property(x => x.Url).HasMaxLength(2000).IsRequired();
entity.Property(x => x.CallerService).HasMaxLength(64).IsRequired();
entity.Property(x => x.Html).IsRequired();
entity.Property(x => x.Text).IsRequired();
entity.Property(x => x.ErrorMessage).HasMaxLength(2000);
entity.Property(x => x.JobSearchSessionId).HasMaxLength(64);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.HasIndex(x => x.JobSearchSessionId);
entity.HasIndex(x => x.Url);
entity.HasIndex(x => x.CreatedAt);
});
}
}
@@ -0,0 +1,19 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AssemblyName>page-fetcher-data</AssemblyName>
<RootNamespace>PageFetcher.Data</RootNamespace>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\shared-data\shared-data.csproj" />
</ItemGroup>
</Project>
+1 -21
View File
@@ -2,8 +2,7 @@
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File",
"Serilog.Sinks.Email"
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
@@ -30,25 +29,6 @@
"retainedFileCountLimit": 30,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "Email",
"Args": {
"restrictedToMinimumLevel": "Error",
"fromEmail": "",
"toEmail": "",
"mailServer": "",
"networkCredential": {
"userName": "",
"password": ""
},
"port": 587,
"enableSsl": true,
"emailSubject": "[mihes.ro API] Error Alert",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}",
"batchPostingLimit": 10,
"period": "0.00:05:00"
}
}
],
"Enrich": [
+5 -1
View File
@@ -34,11 +34,15 @@ This applies to both the staging and production repos as appropriate.
- .NET 10, ASP.NET Core, Worker Service
- Entity Framework Core + SQL Server (multi-schema)
- Refit for typed HTTP clients between services
- Serilog (JSON structured logging, Console + File + Email sinks)
- Serilog — Compact JSON logs to stdout (+ optional email sink); see Observability
- MailKit for SMTP (used exclusively in `email-api`)
- Docker Compose for local and production deployment
- Watchtower for automatic container updates in production
## Observability (central stack on monitoring host 10.0.0.156)
- **Logs**: every service uses `ConfigureJsonSerilog(ServiceName, appVersion)` (startup-helpers) → Serilog **Compact JSON** to stdout, enriched `Application`/`Environment`/`AppVersion`. The host's Grafana **Alloy** agent ships stdout → **Loki**; view/query in Grafana. No file sink; optional email sink only if `SerilogEmail:*` is configured.
- **No app metrics/traces** — these are simple/minimal services, so (unlike easyDent) they don't expose Prometheus metrics or OTLP traces. Container/host metrics still come from the host's cAdvisor/node_exporter.
## Project taxonomy
| Category | Naming | Contains | EF dependency |
BIN
View File
Binary file not shown.
+2 -1
View File
@@ -18,11 +18,12 @@
<!-- Config -->
<PackageVersion Include="DotNetEnv" Version="3.2.0" />
<!-- HTTP / Refit -->
<PackageVersion Include="Refit.HttpClientFactory" Version="10.1.6" />
<PackageVersion Include="Refit.HttpClientFactory" Version="11.0.1" />
<!-- Serilog -->
<PackageVersion Include="Serilog.AspNetCore" Version="10.0.0" />
<PackageVersion Include="Serilog.Enrichers.Environment" Version="3.0.1" />
<PackageVersion Include="Serilog.Sinks.Console" Version="6.1.1" />
<PackageVersion Include="Serilog.Formatting.Compact" Version="3.0.0" />
<PackageVersion Include="Serilog.Sinks.Email" Version="4.2.1" />
<PackageVersion Include="Serilog.Sinks.File" Version="7.0.0" />
<!-- Swagger -->
+42 -8
View File
@@ -1,5 +1,7 @@
using System.Net;
using System.Reflection;
using Azure.Identity;
using MailKit.Security;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Diagnostics;
using Microsoft.AspNetCore.Http;
@@ -9,6 +11,7 @@ using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Serilog;
using Serilog.Events;
using Swashbuckle.AspNetCore.SwaggerGen;
using Swashbuckle.AspNetCore.Annotations;
@@ -36,11 +39,12 @@ public static class StartupExtensions
.ReadFrom.Configuration(context.Configuration)
.ReadFrom.Services(services)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithEnvironmentName()
.Enrich.WithProperty("Service", serviceName)
.Enrich.WithProperty("Application", serviceName)
.Enrich.WithProperty("Environment", context.HostingEnvironment.EnvironmentName)
.Enrich.WithProperty("AppVersion", appVersion)
.WriteTo.Console(new Serilog.Formatting.Json.JsonFormatter());
.WriteTo.Console(new Serilog.Formatting.Compact.CompactJsonFormatter());
AddEmailSinkIfConfigured(configuration, context.Configuration, serviceName);
});
}
@@ -52,14 +56,44 @@ public static class StartupExtensions
.ReadFrom.Configuration(builder.Configuration)
.ReadFrom.Services(services)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithEnvironmentName()
.Enrich.WithProperty("Service", serviceName)
.Enrich.WithProperty("Application", serviceName)
.Enrich.WithProperty("Environment", builder.Environment.EnvironmentName)
.Enrich.WithProperty("AppVersion", appVersion)
.WriteTo.Console(new Serilog.Formatting.Json.JsonFormatter());
.WriteTo.Console(new Serilog.Formatting.Compact.CompactJsonFormatter());
AddEmailSinkIfConfigured(configuration, builder.Configuration, serviceName);
});
}
private static void AddEmailSinkIfConfigured(LoggerConfiguration loggerConfig, IConfiguration appConfig, string serviceName)
{
var from = appConfig["SerilogEmail:From"];
var to = appConfig["SerilogEmail:To"];
var host = appConfig["SerilogEmail:Host"];
if (string.IsNullOrWhiteSpace(from) || string.IsNullOrWhiteSpace(to) || string.IsNullOrWhiteSpace(host))
return;
var port = appConfig.GetValue("SerilogEmail:Port", 587);
var userName = appConfig["SerilogEmail:UserName"];
var password = appConfig["SerilogEmail:Password"];
NetworkCredential? credentials = null;
if (!string.IsNullOrWhiteSpace(userName))
credentials = new NetworkCredential(userName, password);
loggerConfig.WriteTo.Email(
from: from,
to: to,
host: host,
port: port,
connectionSecurity: SecureSocketOptions.StartTls,
credentials: credentials,
subject: $"[myAi {serviceName}] Error Alert",
body: "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}",
restrictedToMinimumLevel: LogEventLevel.Error);
}
public static void AddAzureKeyVaultIfConfigured(this WebApplicationBuilder builder)
{
var keyVaultUri = builder.Configuration["KeyVault:VaultUri"];
@@ -17,6 +17,7 @@
<PackageReference Include="DotNetEnv" />
<PackageReference Include="Serilog.AspNetCore" />
<PackageReference Include="Serilog.Enrichers.Environment" />
<PackageReference Include="Serilog.Formatting.Compact" />
<PackageReference Include="Serilog.Sinks.Email" />
<PackageReference Include="Serilog.Sinks.File" />
<PackageReference Include="Swashbuckle.AspNetCore" />
+1 -21
View File
@@ -2,8 +2,7 @@
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File",
"Serilog.Sinks.Email"
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
@@ -31,25 +30,6 @@
"retainedFileCountLimit": 30,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "Email",
"Args": {
"restrictedToMinimumLevel": "Error",
"fromEmail": "",
"toEmail": "",
"mailServer": "",
"networkCredential": {
"userName": "",
"password": ""
},
"port": 587,
"enableSsl": true,
"emailSubject": "[mihes.ro CV cleanup job] Error Alert",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}{NewLine}{Message:lj}{NewLine}{Exception}",
"batchPostingLimit": 10,
"period": "0.00:05:00"
}
}
],
"Enrich": [
+2
View File
@@ -9,6 +9,7 @@ COPY Apis/cv-search-data/cv-search-data.csproj Apis/cv-search-data/
COPY Apis/cv-matcher-api-models/cv-matcher-api-models.csproj Apis/cv-matcher-api-models/
COPY Apis/email-data/email-data.csproj Apis/email-data/
COPY Apis/email-api-models/email-api-models.csproj Apis/email-api-models/
COPY Apis/page-fetcher-api-models/page-fetcher-api-models.csproj Apis/page-fetcher-api-models/
COPY Apis/common/common.csproj Apis/common/
COPY Apis/myai-data/myai-data.csproj Apis/myai-data/
COPY Apis/shared-data/shared-data.csproj Apis/shared-data/
@@ -22,6 +23,7 @@ COPY Apis/cv-search-data/ Apis/cv-search-data/
COPY Apis/cv-matcher-api-models/ Apis/cv-matcher-api-models/
COPY Apis/email-data/ Apis/email-data/
COPY Apis/email-api-models/ Apis/email-api-models/
COPY Apis/page-fetcher-api-models/ Apis/page-fetcher-api-models/
COPY Apis/common/ Apis/common/
COPY Apis/myai-data/ Apis/myai-data/
COPY Apis/shared-data/ Apis/shared-data/
+15 -2
View File
@@ -7,13 +7,14 @@ using Email.Data;
using Email.Data.Repositories;
using Email.Data.Repositories.Contracts;
using Email.Data.Services;
using EmailApi.Models.Clients;
using Email.Models.Clients;
using CvSearchJob.Tasks;
using JobScheduler.Scheduling;
using JobScheduler.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using PageFetcher.Models;
using Refit;
using Serilog;
using Common.Settings;
@@ -81,7 +82,19 @@ try
client.DefaultRequestHeaders.Add("X-Internal-Api-Key", key);
});
builder.Services.AddHttpClient<HtmlJobSearcher>();
builder.Services.AddRefitClient<IPageFetcherApiClient>()
.ConfigureHttpClient((sp, client) =>
{
var config = sp.GetRequiredService<Microsoft.Extensions.Configuration.IConfiguration>();
var baseUrl = config["PageFetcherApi:BaseUrl"] ?? string.Empty;
if (!string.IsNullOrWhiteSpace(baseUrl))
client.BaseAddress = new Uri(baseUrl.TrimEnd('/') + "/");
var key = config["PageFetcherApi:InternalApiKey"];
if (!string.IsNullOrWhiteSpace(key))
client.DefaultRequestHeaders.Add("X-Internal-Api-Key", key);
});
builder.Services.AddSingleton<HtmlJobSearcher>();
builder.Services.AddSingleton<CvSearchEmailSender>();
builder.Services.AddSingleton<CvSearchJobTask>();

Some files were not shown because too many files have changed in this diff Show More