49 Commits

Author SHA1 Message Date
claude 10bac5eb91 ci: clone via docker-git.easysoft.ro:3000 (LAN DNS; drop git.easysoft.ro:3000 dependency)
Build and Push Docker Images / build (push) Successful in 18s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 09:51:31 +03:00
claude 65ae4b42da myai: document MSSQL host as mssql.easysoft.ro (DNS name) in the env template
Deployed staging/prod envs now use Database__Host=mssql.easysoft.ro (LAN DNS -> the MSSQL
VM 10.0.0.240) instead of the raw IP, matching the infra DNS standardization. The
docker-compose default stays 'sqlserver' for local dev. (.env/.env.staging/.env.production
are gitignored; their deployed values were updated locally and apply on redeploy.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 17:01:06 +03:00
claude 5aaf848423 chore: remove stray CV.pdf test artifact from repo root (unused, 284KB)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 19:20:21 +03:00
claude 62654978af ci: re-enable BuildKit (runner now has buildx plugin)
Build and Push Docker Images / build (push) Successful in 43m53s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 18:09:35 +03:00
claude 7da084c174 ci: revert DOCKER_BUILDKIT (runner job env has no buildx plugin -> build failed)
Keep the cache win (no base-image nuke); back to the legacy builder which the job
context supports. BuildKit needs buildx installed in the runner before it can be used.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 18:02:29 +03:00
claude 27f4cfe21e ci: enable BuildKit + let its GC manage the layer cache
DOCKER_BUILDKIT=1 (explicit) so the restore layer is cached across builds; drop the
explicit 'docker builder prune' (it was wiping that cache) and rely on BuildKit's own
GC + 86GB headroom. Cleanup keeps dangling-image removal only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 17:58:00 +03:00
claude 903fbcd143 ci: preserve Docker build cache (faster runner)
Stop wiping the layer cache + base images after every build (the host has 86GB free).
Keep base images (prune dangling only) and ~2 weeks of build cache, so the cache-friendly
Dockerfiles (COPY *.csproj + restore before source) actually benefit -> warm rebuilds skip
restore and base-image pulls.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 17:36:59 +03:00
claude c5e1b7f687 ci: branch-driven deploys (staging/production branches), build the pushed commit
main = day-to-day work (no deploy). Merge into staging -> :staging, into
production -> :production; IMAGE_TAG = branch name. Also fixes the checkout to
build the PUSHED commit (git checkout $GITHUB_SHA) instead of always cloning
the default branch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 16:29:07 +03:00
claude 9b33876c11 docs: replace template README with a proper one (intro, run, deploy, logging)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 16:04:23 +03:00
claude 2d5572725d docs: add Observability section to CLAUDE.md (Compact JSON logs->Loki); gitignore ACCESS.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 15:46:46 +03:00
claude da1f90449e Merge observability/compact-json: standardize Serilog to Compact JSON (Application/Environment/AppVersion enrichment) 2026-06-18 10:59:13 +03:00
claude 2192c3f4c5 Logs: Compact JSON + aligned enrichment in shared StartupExtensions
CompactJsonFormatter in both ConfigureJsonSerilog overloads; rename Service->Application,
EnvironmentName->Environment (keep AppVersion). Applies to all myAi services.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:03:55 +03:00
claude 492859f17f ci: prune images + build cache after build (prevent runner disk exhaustion)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 22:51:31 +03:00
claude a3567ce8e9 Merge pull request 'Fix CV keyword extraction — derive from candidate CV, not matched job' (#55) from feature/fix-keyword-extraction-prompt into main
Build and Push Docker Images Staging / build (push) Successful in 2m23s
Merge PR #55: Fix CV keyword extraction — derive from candidate CV, not matched job
2026-06-09 13:40:10 +00:00
claude b52ef8ddff Fix CV keyword extraction to reflect candidate identity, not matched job
The AI prompt now instructs the LLM to derive keywords entirely from the
candidate's CV (seniority level, primary role title, core technologies they
emphasize) rather than from the job description being matched. This ensures
the job-board search keywords used by cv-search-job represent who the
candidate actually is, not a mirror of the job they happened to match against.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:38:02 +03:00
claude d2b12e39ec Merge PR #54: Fix hardcoded user-facing strings (Issue #53)
Build and Push Docker Images Staging / build (push) Successful in 46s
2026-06-08 22:34:28 +03:00
claude 1e8758796e Fix hardcoded user-facing strings — localize email fallbacks, API errors, AI parse messages
- Frontend: update extractApiError to check body.code first via i18n 'error.<code>' keys;
  add en/ro translations for cv_file_missing, captcha_verification_failed, request_cancelled
- email-data migration: seed 6 fallback template keys (match N/A, subject label, unknown IP,
  job search results empty states for keywords/providers/location)
- EmailApiEmailSender: replace "N/A", "Job", "Unknown" literals with template lookups
- CvSearchEmailSender: replace "none detected", "none", "-" literals with template lookups
- cv-matcher-data migration: seed parse-error.summary and parse-error.recommendation in AiPrompts
- CvMatcherService: look up localized parse-error messages from AiPrompts before calling ParseResult

Closes #53

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 22:32:03 +03:00
claude ef2793448a Fix: declare language before jobLabel in CvMatcherController
Build and Push Docker Images Staging / build (push) Successful in 8m20s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 22:10:22 +03:00
claude cbf06031e8 Merge pull request 'Fix language consistency in job search and match emails' (#52) from feature/language-consistency into main
Build and Push Docker Images Staging / build (push) Failing after 32s
Merge PR #52: Fix language consistency in job search and match emails
2026-06-08 19:08:28 +00:00
claude 90f540139a Fix language consistency in job search and match emails
1. Pass session.Language to MatchJobRequest in cv-search-job so the LLM
   uses the correct language-specific prompt for job titles and summaries.

2. Replace hardcoded "Manual job description" with a template-driven label
   (email.match.manual-job-label) seeded in English and Romanian, so the
   match email subject and Job row reflect the user's language.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 22:07:15 +03:00
claude 71d5ac8e06 Remove environment name prefix from email subjects
Build and Push Docker Images Staging / build (push) Successful in 1m26s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:47:21 +03:00
claude c2082d6729 Suppress environment prefix in email subjects on Production
[ENV_NAME] prefix is now only prepended in non-production environments
(Development, Staging, etc.). Production emails get a clean subject line.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:45:46 +03:00
claude 6f1d8992ab Merge pull request 'Link pageFetcher.PageFetches to cvSearch.JobSearchSessions' (#50) from feature/page-fetch-session-link into main
Build and Push Docker Images Staging / build (push) Successful in 17m39s
Merge PR #50: Link pageFetcher.PageFetches to cvSearch.JobSearchSessions
2026-06-08 16:57:07 +00:00
claude 2d9ffc9c2b Link pageFetcher.PageFetches to cvSearch.JobSearchSessions
Adds nullable JobSearchSessionId to PageFetchEntity and FetchPageRequest.
cv-search-job passes session.Id on every fetch so all Playwright page
loads for a job search session can be traced back to their session.
Includes index on JobSearchSessionId for efficient lookup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:56:13 +03:00
claude 9fbad722fc Merge pull request 'Add Email and ClientIpAddress audit fields to cvSearch.JobSearchSessions and JobSearchResults' (#48) from feature/job-search-audit-fields into main
Build and Push Docker Images Staging / build (push) Successful in 39s
Merge PR #48: Add Email and ClientIpAddress audit fields to cvSearch
2026-06-08 16:21:46 +00:00
claude 473c36d65f Store match-time ClientIpAddress on cvSearch.JobSearchTokens
Captures the IP when the user submits the CV match form and stores it on
the token, giving a full audit trail: token holds the match-site IP,
session holds the email link-click IP.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:20:02 +03:00
claude 292d19d5ed Populate JobText from fetched page content in JobSearchResults
Previously always stored empty string; now stores the full page text
returned by page-fetcher-api, which is already in scope at save time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:13:10 +03:00
claude d56729de42 Add Email and ClientIpAddress audit fields to cvSearch.JobSearchSessions and JobSearchResults
Captures client IP at job-search link-click time and threads it through to the session.
Both Email and ClientIpAddress are copied from session to each result row during processing.
Closes #47

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:11:50 +03:00
claude 79a3dec679 Merge pull request 'Add Email and ClientIpAddress audit fields to cvMatcher.Results' (#46) from feature/result-email-and-ip into main
Build and Push Docker Images Staging / build (push) Successful in 16m43s
Merge PR #46: Add Email and ClientIpAddress audit fields to cvMatcher.Results
2026-06-08 15:58:18 +00:00
claude 02d2b1e510 Add Email and ClientIpAddress audit fields to cvMatcher.Results
Threads the caller's email and client IP through the match pipeline so
every Results row records who triggered the match and from where.
Closes #45

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:56:36 +03:00
claude 3c3451b198 Merge branch 'main' into staging
Build and Push Docker Images Staging / build (push) Successful in 17m15s
2026-06-08 18:43:56 +03:00
claude a83f6f705f Remove UseHeadlessBrowser from JobProvider — all fetches now go via page-fetcher-api
page-fetcher-api always uses Playwright (networkidle by default), so the
per-provider flag that chose between headless and plain HTTP is obsolete.

- Removed from JobProviderEntity, CvSearchDbContext, JobProviderConfig, JobTokenService
- HtmlJobSearcher no longer passes WaitFor (uses page-fetcher-api default)
- EF migration drops the column from cvSearch.JobProviders

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:43:42 +03:00
claude b68cf942a8 Merge branch 'main' into staging
Build and Push Docker Images Staging / build (push) Successful in 28m0s
2026-06-08 18:37:00 +03:00
claude 61805e2fb5 Merge pull request 'feat: page-fetcher-api centralised Playwright page fetcher' (#44) from feature/page-fetcher-api into main
Merge feature/page-fetcher-api into main
2026-06-08 15:36:44 +00:00
claude dcfc50ff32 Fix Docker builds: upgrade Refit to 11.0.1, add page-fetcher-api-models to Dockerfiles
- Refit 10.1.6 signing certificate was revoked; upgraded to 11.0.1 in Directory.Packages.props
- cv-matcher-api/Dockerfile and cv-search-job/Dockerfile were missing COPY steps
  for page-fetcher-api-models (added in this feature branch)

All 8 images now build cleanly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:35:41 +03:00
claude b1ed1cb201 Rename EmailApi.Models.* namespace to Email.Models.* in email-api-models
Removes the spurious Api segment to match the pattern used by all other
models projects: CvMatcher.Models.*, Rag.Models.*, PageFetcher.Models.*.

Updated all consumers: email-api, api, cv-search-job.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:06:38 +03:00
claude e1f171168e Align email-api and page-fetcher-api namespaces to Api.* convention
Fixes inconsistency where email-api used EmailApi.* and page-fetcher-api
used PageFetcherApi.*, while cv-matcher-api and rag-api use the generic
Api.* namespace. All four API projects now follow the same pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:04:03 +03:00
claude ae2bc9b902 Move SmtpSettings and PageFetcherSettings into their respective models projects
Settings classes now live in the -models project alongside DTOs and client
interfaces, eliminating the Settings/ folder from both API projects.

- SmtpSettings: email-api/Settings/ → email-api-models/Settings/ (namespace EmailApi.Models.Settings)
- PageFetcherSettings: page-fetcher-api/Settings/ → page-fetcher-api-models/Settings/ (namespace PageFetcher.Models.Settings)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:00:44 +03:00
claude 30a8df431f Move PageFetcherSettings back to page-fetcher-api/Settings/, matching SmtpSettings pattern
Server-side-only settings (internal config not needed by callers) belong in
the API project itself, not in the models project. PageFetcherSettings
(DefaultWaitFor, TimeoutSeconds, MaxTextChars) mirrors SmtpSettings in
email-api/Settings/ — callers never reference these.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:56:21 +03:00
claude 95b0cfa0a9 Move PageFetcherSettings to page-fetcher-api-models, consistent with EmailApiSettings pattern
Settings class now lives in Apis/page-fetcher-api-models/Settings/ with
namespace PageFetcher.Models.Settings, matching how EmailApiSettings is
placed in email-api-models/Settings/.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:54:08 +03:00
claude 20b13647de Move PageFetcherSettings to Settings/ folder, consistent with email-api pattern
Settings classes belong in Settings/ with namespace PageFetcherApi.Settings,
not Services/. Matches the SmtpSettings placement in email-api.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:51:51 +03:00
claude df011f2a03 Fix PageFetcherApi BaseUrl default to use Docker service name, not container name
Use http://page-fetcher-api:8080 (the Compose service key) for Docker DNS
resolution, consistent with all other internal service URLs (rag-api,
email-api, cv-matcher-api).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:49:00 +03:00
claude 3414c61cea Commit 2026-06-08 17:47:17 +03:00
claude 898dd09d50 feat: add page-fetcher-api — centralised Playwright page fetcher
Introduces page-fetcher-api, a new internal ASP.NET Core service that
centralises all web-page fetching through a single Playwright (headless
Chromium) browser instance. All fetches are persisted to the pageFetcher
SQL schema for auditing.

New projects:
- Apis/page-fetcher-api-models: FetchPageRequest, FetchPageResponse, IPageFetcherApiClient
- Apis/page-fetcher-data: PageFetchDbContext, PageFetchEntity, InitialSchema migration (schema: pageFetcher)
- Apis/page-fetcher-api: PlaywrightBrowserService (singleton), PageFetcherService, PageController

Changes to existing services:
- cv-matcher-api: JobTextExtractor now calls IPageFetcherApiClient instead of HttpClient
- cv-search-job: HtmlJobSearcher uses IPageFetcherApiClient (removes inline Playwright);
  CvSearchJobTask fetches individual job pages and applies keyword pre-filter before
  LLM call; passes pre-fetched JobDescription to cv-matcher-api to skip re-fetch
- common: add PageFetcherApiSettings
- docker-compose.yml, build.yml: add new service + env vars for callers

Closes #43

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:43:56 +03:00
claude 4de6f1db45 Merge branch 'main' into staging
Build and Push Docker Images Staging / build (push) Successful in 7m25s
2026-06-08 17:30:49 +03:00
claude 1222a86eb7 Fix file:// URL bug in HtmlJobSearcher — skip non-HTTP(S) URLs
After resolving relative hrefs against the base search URL, some ejobs.ro
links were producing file:/// URIs (e.g. file:///user/locuri-de-munca/...).
These were sent to cv-matcher-api and rejected with HTTP 400, causing 0 matches.

Added a scheme guard after URI resolution to skip any URL that is not
http:// or https://, preventing malformed URLs from reaching the matcher.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 17:30:45 +03:00
claude 2e9069cbdb Fix file:// URL bug in HtmlJobSearcher — skip non-HTTP(S) URLs
Build and Push Docker Images Staging / build (push) Successful in 35s
After resolving relative hrefs against the base search URL, some ejobs.ro
links were producing file:/// URIs (e.g. file:///user/locuri-de-munca/...).
These were sent to cv-matcher-api and rejected with HTTP 400, causing 0 matches.

Added a scheme guard after URI resolution to skip any URL that is not
http:// or https://, preventing malformed URLs from reaching the matcher.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 16:57:52 +03:00
claude c89df975bd Add searched location to job search results email
Build and Push Docker Images Staging / build (push) Successful in 14m42s
Show the candidate's location in the scan summary block of the results email
alongside keywords and providers, for both en and ro templates.

- CvSearchEmailSender.SendResultsAsync accepts location and passes it to BuildScanSummary
- BuildScanSummary passes {{location}} to the template (falls back to '-' when absent)
- CvSearchJobTask passes session.Location to SendResultsAsync
- New migration AddLocationToScanSummaryTemplate updates both language variants of
  email.search-results.scan-summary to include a 'Location / Locație căutată' row

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 15:54:38 +03:00
claude 709c0ac4c3 Merge pull request 'Fix job search: location filtering, keyword quality, anchor filter bypass' (#42) from feature/job-search-location-keywords into main
Merge PR #42: Fix job search — location filtering, keyword quality, anchor filter bypass
2026-06-08 12:51:16 +00:00
96 changed files with 3485 additions and 213 deletions
+26 -5
View File
@@ -1,13 +1,19 @@
name: Build and Push Docker Images Staging
name: Build and Push Docker Images
# Branch-driven deploys — no yaml edits to switch environment:
# merge into `staging` -> tag :staging (staging Watchtower deploys)
# merge into `production` -> tag :production (production Watchtower deploys)
# `main` is the day-to-day work branch and deploys nothing.
on:
push:
branches:
- staging
- production
env:
GIT_HOST: git.easysoft.ro
GIT_HOST: docker-git.easysoft.ro
REGISTRY_HOST: registry.easysoft.ro
DOCKER_BUILDKIT: "1"
API_IMAGE: apps/myai-api
CV_MATCHER_API_IMAGE: apps/myai-cv-matcher-api
RAG_API_IMAGE: apps/myai-rag-api
@@ -15,18 +21,20 @@ env:
WEB_IMAGE: apps/myai-web
CV_CLEANUP_JOB_IMAGE: apps/myai-cv-cleanup-job
CV_SEARCH_JOB_IMAGE: apps/myai-cv-search-job
IMAGE_TAG: staging
PAGE_FETCHER_API_IMAGE: apps/myai-page-fetcher-api
IMAGE_TAG: ${{ github.ref_name }} # branch name == image tag (staging | production)
jobs:
build:
runs-on: host
steps:
- name: Checkout repository
- name: Checkout the pushed commit
env:
TOKEN: ${{ secrets.REPO_TOKEN }}
run: |
git clone "http://gelu:${TOKEN}@${GIT_HOST}:3000/${GITHUB_REPOSITORY}.git" .
git checkout "${{ github.sha }}"
- name: Login to registry
run: |
@@ -62,6 +70,10 @@ jobs:
run: |
docker build -f Jobs/cv-search-job/Dockerfile -t "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}" .
- name: Build Page Fetcher API image
run: |
docker build -f Apis/page-fetcher-api/Dockerfile -t "${REGISTRY_HOST}/${PAGE_FETCHER_API_IMAGE}:${IMAGE_TAG}" .
- name: Push API image
run: |
docker push "${REGISTRY_HOST}/${API_IMAGE}:${IMAGE_TAG}"
@@ -88,4 +100,13 @@ jobs:
- name: Push CV search job image
run: |
docker push "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}"
docker push "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}"
- name: Push Page Fetcher API image
run: |
docker push "${REGISTRY_HOST}/${PAGE_FETCHER_API_IMAGE}:${IMAGE_TAG}"
- name: Reclaim disk space (keep recent build cache)
if: always()
run: |
docker image prune -f # dangling only (keep base images)
+3
View File
@@ -376,3 +376,6 @@ files/
/docker-compose/.env.production
/docker-compose/.env.staging
# local infra access notes (secrets) — never commit
ACCESS.md
@@ -10,4 +10,6 @@ public sealed class JobMatchRequest
public string? CaptchaToken { get; set; }
/// <summary>ISO 639-1 language code for the match result (e.g. "en", "ro"). Defaults to "en".</summary>
public string? Language { get; set; }
/// <summary>Client IP address — set by the api layer from the HTTP context before forwarding. Not supplied by the browser.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -10,5 +10,5 @@ public interface IJobSearchApi
Task<CreateJobSearchTokenResponse> CreateTokenAsync([Body] CreateJobSearchTokenRequest request, CancellationToken ct);
[Post("/api/cv/job-search/token/{tokenId}/start")]
Task<StartJobSearchResponse> StartSearchAsync(string tokenId, CancellationToken ct);
Task<StartJobSearchResponse> StartSearchAsync(string tokenId, [Body] StartJobSearchRequest request, CancellationToken ct);
}
+6 -5
View File
@@ -163,17 +163,17 @@ public sealed class CvMatcherController : ControllerBase
return BadRequest(new ErrorResponse { Error = "Captcha verification failed.", Code = "captcha_verification_failed" });
}
request.ClientIpAddress = userIp;
_logger.LogInformation("Proxying job match request to cv-matcher-api. CvDocumentId={CvDocumentId}, HasJobUrl={HasJobUrl}, HasJobDescription={HasJobDescription}",
request.CvDocumentId,
!string.IsNullOrWhiteSpace(request.JobUrl),
!string.IsNullOrWhiteSpace(request.JobDescription));
var res = await _cvApi.MatchJob(request, ct);
var attachmentPath = TryGetCachedCvPath(request.CvDocumentId);
var language = NormalizeLanguage(request.Language);
var jobLabel = !string.IsNullOrWhiteSpace(request.JobUrl)
? request.JobUrl
: "Manual job description";
var language = NormalizeLanguage(request.Language);
: _emailSender.GetManualJobLabel(language);
string? jobSearchLink = null;
if (!string.IsNullOrWhiteSpace(request.Email) && !string.IsNullOrWhiteSpace(request.CvDocumentId))
@@ -181,7 +181,7 @@ public sealed class CvMatcherController : ControllerBase
try
{
var tokenResp = await _jobSearchApi.CreateTokenAsync(
new CreateJobSearchTokenRequest { CvDocumentId = request.CvDocumentId, Email = request.Email, Language = language, Keywords = res.Keywords, Location = res.Location },
new CreateJobSearchTokenRequest { CvDocumentId = request.CvDocumentId, Email = request.Email, Language = language, Keywords = res.Keywords, Location = res.Location, ClientIpAddress = userIp },
ct);
if (!string.IsNullOrWhiteSpace(tokenResp.TokenId))
{
@@ -244,7 +244,8 @@ public sealed class CvMatcherController : ControllerBase
{
try
{
var result = await _jobSearchApi.StartSearchAsync(t, ct);
var userIp = HttpContext.Connection.RemoteIpAddress?.ToString();
var result = await _jobSearchApi.StartSearchAsync(t, new StartJobSearchRequest { ClientIpAddress = userIp }, ct);
var lang = "en";
var (title, message) = result.Status switch
{
+2 -2
View File
@@ -5,8 +5,8 @@ using Email.Data;
using Email.Data.Repositories;
using Email.Data.Repositories.Contracts;
using Email.Data.Services;
using EmailApi.Models.Clients;
using EmailApi.Models.Settings;
using Email.Models.Clients;
using Email.Models.Settings;
using Microsoft.EntityFrameworkCore;
using Models.Settings;
using MyAi.Data;
@@ -61,5 +61,11 @@ namespace Api.Services.Contracts
/// <param name="expiryDays">Number of days until the job-search link expires (shown in the footer copy).</param>
/// <returns>Rendered HTML body string.</returns>
string BuildMatchEmailBody(string cvDocumentId, JobMatchResponse result, string? jobLabel, string language, string? jobSearchLink = null, int expiryDays = 7);
/// <summary>
/// Returns the localised label for a manually-entered job description (no URL provided).
/// </summary>
/// <param name="language">Two-letter language code.</param>
string GetManualJobLabel(string language);
}
}
+9 -6
View File
@@ -1,8 +1,8 @@
using Api.Services.Contracts;
using CvMatcher.Models.Responses;
using Email.Data.Services;
using EmailApi.Models.Clients;
using EmailApi.Models.Requests;
using Email.Models.Clients;
using Email.Models.Requests;
using Microsoft.Extensions.Options;
using Models.Requests;
using Models.Settings;
@@ -138,7 +138,7 @@ public sealed class EmailApiEmailSender : IEmailSender
</tr>
<tr style="background:#f8f9fa">
<td style="font-weight:600;border:1px solid #dee2e6;color:#495057">IP Address</td>
<td style="border:1px solid #dee2e6">{userIp ?? "Unknown"}</td>
<td style="border:1px solid #dee2e6">{userIp ?? _emailTemplates.Get("email.notification.unknown-ip", "en")}</td>
</tr>
</table>
""";
@@ -215,8 +215,8 @@ public sealed class EmailApiEmailSender : IEmailSender
// email.match.body is now stored as HTML in the database
var body = _emailTemplates.Render("email.match.body", language,
("cvDocumentId", cvDocumentId),
("jobLabel", jobLabel ?? "N/A"),
("jobUrl", result.JobUrl ?? "N/A"),
("jobLabel", jobLabel ?? _emailTemplates.Get("email.match.fallback-na", language)),
("jobUrl", result.JobUrl ?? _emailTemplates.Get("email.match.fallback-na", language)),
("score", result.Score.ToString()),
("summary", WebUtility.HtmlEncode(result.Summary ?? string.Empty)),
("strengths", strengths),
@@ -238,5 +238,8 @@ public sealed class EmailApiEmailSender : IEmailSender
public string BuildMatchEmailSubject(int score, string? jobLabel, string language) =>
_emailTemplates.Render("email.match.subject", language,
("score", score.ToString()),
("jobLabel", jobLabel ?? "Job"));
("jobLabel", jobLabel ?? _emailTemplates.Get("email.match.subject-fallback-label", language)));
public string GetManualJobLabel(string language) =>
_emailTemplates.Get("email.match.manual-job-label", language);
}
@@ -0,0 +1,11 @@
namespace Common.Settings;
/// <summary>
/// Connection settings for the internal page-fetcher-api service.
/// Bound from the <c>PageFetcherApi</c> configuration section.
/// </summary>
public sealed class PageFetcherApiSettings
{
public string BaseUrl { get; set; } = string.Empty;
public string InternalApiKey { get; set; } = string.Empty;
}
@@ -7,4 +7,6 @@ public sealed class CreateJobSearchTokenRequest
public string Language { get; set; } = "en";
public List<string> Keywords { get; set; } = [];
public string? Location { get; set; }
/// <summary>Client IP address forwarded by the api layer at CV match time. Null when not available.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -9,5 +9,7 @@
public string? Email { get; set; }
/// <summary>ISO 639-1 language code for the match result (e.g. "en", "ro"). Defaults to "en".</summary>
public string? Language { get; set; }
/// <summary>Client IP address forwarded by the api layer. Null when called from a background job.</summary>
public string? ClientIpAddress { get; set; }
}
}
@@ -0,0 +1,11 @@
namespace CvMatcher.Models.Requests;
/// <summary>
/// Request body sent by <c>api</c> when activating a one-time job-search link.
/// Carries the caller's IP address so it can be persisted on the session for auditing.
/// </summary>
public sealed class StartJobSearchRequest
{
/// <summary>Client IP address forwarded by the api layer. Null when not available.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -21,8 +21,6 @@ public sealed class JobProviderConfig
public string JobLinkContains { get; set; } = string.Empty;
public List<string> InitialKeywords { get; set; } = [];
public int MaxResults { get; set; } = 20;
/// <summary>When true the scraper uses a headless Chromium browser to render JS-heavy pages.</summary>
public bool UseHeadlessBrowser { get; set; }
/// <summary>
/// When false, the Stage 2 anchor-text keyword filter is skipped.
/// Set to false for providers whose search URL already filters by relevance server-side.
@@ -53,7 +53,7 @@ public sealed class JobSearchController : ControllerBase
if (string.IsNullOrWhiteSpace(request.CvDocumentId) || string.IsNullOrWhiteSpace(request.Email))
return BadRequest(new ErrorResponse { Error = "CvDocumentId and Email are required.", Code = "invalid_request" });
var tokenId = await _tokenService.CreateTokenAsync(request.CvDocumentId, request.Email, request.Language, request.Keywords, request.Location, ct);
var tokenId = await _tokenService.CreateTokenAsync(request.CvDocumentId, request.Email, request.Language, request.Keywords, request.Location, request.ClientIpAddress, ct);
return Ok(new CreateJobSearchTokenResponse { TokenId = tokenId });
}
catch (Exception ex)
@@ -81,11 +81,11 @@ public sealed class JobSearchController : ControllerBase
[SwaggerResponse(StatusCodes.Status500InternalServerError, "Session creation failed", typeof(ErrorResponse))]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(typeof(ErrorResponse), StatusCodes.Status500InternalServerError)]
public async Task<ActionResult<StartJobSearchResponse>> Start(string tokenId, CancellationToken ct)
public async Task<ActionResult<StartJobSearchResponse>> Start(string tokenId, [FromBody] StartJobSearchRequest? request, CancellationToken ct)
{
try
{
var status = await _tokenService.TriggerStartAsync(tokenId, ct);
var status = await _tokenService.TriggerStartAsync(tokenId, request?.ClientIpAddress, ct);
return Ok(new StartJobSearchResponse { Status = status });
}
catch (Exception ex)
+2
View File
@@ -8,6 +8,7 @@ COPY Apis/cv-search-data/cv-search-data.csproj Apis/cv-search-data/
COPY Apis/cv-matcher-data/cv-matcher-data.csproj Apis/cv-matcher-data/
COPY Apis/common/common.csproj Apis/common/
COPY Apis/cv-matcher-api-models/cv-matcher-api-models.csproj Apis/cv-matcher-api-models/
COPY Apis/page-fetcher-api-models/page-fetcher-api-models.csproj Apis/page-fetcher-api-models/
COPY Apis/myai-data/myai-data.csproj Apis/myai-data/
COPY Apis/shared-data/shared-data.csproj Apis/shared-data/
COPY Helpers/common-helpers/common-helpers.csproj Helpers/common-helpers/
@@ -20,6 +21,7 @@ COPY Apis/cv-search-data/ Apis/cv-search-data/
COPY Apis/cv-matcher-data/ Apis/cv-matcher-data/
COPY Apis/common/ Apis/common/
COPY Apis/cv-matcher-api-models/ Apis/cv-matcher-api-models/
COPY Apis/page-fetcher-api-models/ Apis/page-fetcher-api-models/
COPY Apis/myai-data/ Apis/myai-data/
COPY Apis/shared-data/ Apis/shared-data/
COPY Helpers/common-helpers/ Helpers/common-helpers/
+12 -1
View File
@@ -13,6 +13,7 @@ using Microsoft.EntityFrameworkCore;
using Refit;
using Serilog;
using Common.Settings;
using PageFetcher.Models;
using StartupHelpers;
using System.Reflection;
@@ -36,6 +37,16 @@ try
builder.Services.Configure<CvMatcher.Models.Settings.AiSettings>(builder.Configuration.GetSection("Ai"));
builder.Services.Configure<MatcherSettings>(builder.Configuration.GetSection("Matcher"));
builder.Services.Configure<JobSearchSettings>(builder.Configuration.GetSection("JobSearch"));
builder.Services.Configure<PageFetcherApiSettings>(builder.Configuration.GetSection("PageFetcherApi"));
builder.Services.AddRefitClient<IPageFetcherApiClient>()
.ConfigureHttpClient((sp, c) =>
{
var settings = sp.GetRequiredService<Microsoft.Extensions.Options.IOptions<PageFetcherApiSettings>>().Value;
c.BaseAddress = new Uri(settings.BaseUrl.TrimEnd('/') + "/");
if (!string.IsNullOrWhiteSpace(settings.InternalApiKey))
c.DefaultRequestHeaders.Add("X-Internal-Api-Key", settings.InternalApiKey);
});
builder.Services.AddRefitClient<IRefitRagApi>()
.ConfigureHttpClient((sp, c) =>
@@ -50,7 +61,7 @@ try
builder.Services.AddScoped<IRagApiClient, RagApiClient>();
builder.Services.AddHttpClient<IMatcherAiClient, MatcherAiClient>();
builder.Services.AddHttpClient<IJobTextExtractor, JobTextExtractor>();
builder.Services.AddScoped<IJobTextExtractor, JobTextExtractor>();
builder.Services.AddDbContext<CvMatcherDbContext>(options =>
{
@@ -19,16 +19,17 @@ public interface IJobTokenService
/// The generated token ID to embed in the one-click job search link,
/// or <c>null</c> when no job providers are currently enabled (link should be suppressed).
/// </returns>
Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, string? location, CancellationToken ct);
Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, string? location, string? clientIpAddress, CancellationToken ct);
/// <summary>
/// Validates the token and, if valid, marks it as used and creates a <c>Pending</c> job search session.
/// </summary>
/// <param name="tokenId">The token ID from the one-click link.</param>
/// <param name="clientIpAddress">Client IP address forwarded by the api layer. Null when not available.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>
/// One of the <c>StartJobSearchStatus</c> string constants:
/// <c>Started</c>, <c>AlreadyUsed</c>, <c>Expired</c>, or <c>NotFound</c>.
/// </returns>
Task<string> TriggerStartAsync(string tokenId, CancellationToken ct);
Task<string> TriggerStartAsync(string tokenId, string? clientIpAddress, CancellationToken ct);
}
@@ -77,7 +77,7 @@ public sealed class CvMatcherService : ICvMatcherService
{
var job = await _rag.GetDocumentAsync(result.DocumentId, ct);
if (job is null) continue;
jobs.Add(await ScorePairAsync(cv, job, result.MatchedChunks.Select(x => x.Text).ToArray(), request.Email, NormalizeLanguage(null), ct));
jobs.Add(await ScorePairAsync(cv, job, result.MatchedChunks.Select(x => x.Text).ToArray(), request.Email, null, NormalizeLanguage(null), ct));
}
return new FindJobsResponse { CvDocumentId = request.CvDocumentId, Jobs = jobs };
@@ -107,7 +107,7 @@ public sealed class CvMatcherService : ICvMatcherService
.FirstOrDefault(x => x.DocumentId == job.DocumentId)?
.MatchedChunks.Select(x => x.Text).ToArray() ?? [];
return await ScorePairAsync(cv, jobDocument, matchedChunks, request.Email, NormalizeLanguage(request.Language), ct);
return await ScorePairAsync(cv, jobDocument, matchedChunks, request.Email, request.ClientIpAddress, NormalizeLanguage(request.Language), ct);
}
/// <summary>
@@ -115,7 +115,7 @@ public sealed class CvMatcherService : ICvMatcherService
/// Returns a cached result immediately when the same (CV, job, language) triple has been scored before.
/// When no evidence chunks are available from the vector search, falls back to the raw job text.
/// </summary>
private async Task<JobMatchResponse> ScorePairAsync(RagDocumentDetails cv, RagDocumentDetails job, IReadOnlyList<string> evidenceChunks, string? email, string language, CancellationToken ct)
private async Task<JobMatchResponse> ScorePairAsync(RagDocumentDetails cv, RagDocumentDetails job, IReadOnlyList<string> evidenceChunks, string? email, string? clientIpAddress, string language, CancellationToken ct)
{
var cached = await _repository.GetMatchAsync(cv.Id, job.Id, language, ct);
if (cached is not null) return cached;
@@ -141,11 +141,13 @@ public sealed class CvMatcherService : ICvMatcherService
""";
var json = await _ai.CreateChatCompletionAsync(systemPrompt, userPrompt, 0.2m, ct);
var result = ParseResult(json);
var errorSummary = await _aiPrompts.GetAsync("parse-error.summary", language, ct);
var errorRec = await _aiPrompts.GetAsync("parse-error.recommendation", language, ct);
var result = ParseResult(json, errorSummary, errorRec);
result.JobDocumentId = job.Id;
result.JobUrl = job.SourceUrl;
result.Cached = false;
await _repository.SaveMatchAsync(cv.Id, job.Id, language, result, ct);
await _repository.SaveMatchAsync(cv.Id, job.Id, language, result, email, clientIpAddress, ct);
return result;
}
@@ -153,7 +155,10 @@ public sealed class CvMatcherService : ICvMatcherService
/// Deserialises the LLM's JSON output into a <see cref="JobMatchResponse"/>.
/// Returns a safe fallback response instead of throwing when the JSON cannot be parsed.
/// </summary>
private static JobMatchResponse ParseResult(string json)
private static JobMatchResponse ParseResult(
string json,
string? errorSummary = null,
string? errorRec = null)
{
try
{
@@ -168,8 +173,8 @@ public sealed class CvMatcherService : ICvMatcherService
return new JobMatchResponse
{
Score = 0,
Summary = "The AI response could not be parsed as structured JSON.",
Recommendations = ["Inspect the raw model output and tune the scoring prompt."]
Summary = errorSummary ?? "The AI response could not be parsed as structured JSON.",
Recommendations = [errorRec ?? "Inspect the raw model output and tune the scoring prompt."]
};
}
@@ -1,26 +1,23 @@
using System.Net;
using System.Text.RegularExpressions;
using CvMatcher.Models.Settings;
using Api.Services.Contracts;
using Microsoft.Extensions.Options;
using PageFetcher.Models;
namespace Api.Services;
/// <summary>
/// Extracts normalised plain text from a job posting, either from a pasted description or by
/// fetching and stripping the HTML of the job page URL.
/// fetching the job page text via <c>page-fetcher-api</c> (headless Chromium rendering).
/// </summary>
public sealed class JobTextExtractor : IJobTextExtractor
{
private readonly HttpClient _http;
private readonly IPageFetcherApiClient _pageFetcher;
private readonly MatcherSettings _settings;
public JobTextExtractor(HttpClient http, IOptions<MatcherSettings> options)
public JobTextExtractor(IPageFetcherApiClient pageFetcher, IOptions<MatcherSettings> options)
{
_http = http;
_pageFetcher = pageFetcher;
_settings = options.Value;
_http.Timeout = TimeSpan.FromSeconds(25);
_http.DefaultRequestHeaders.UserAgent.ParseAdd("MyAi.ro CV Matcher/1.0");
}
/// <inheritdoc />
@@ -31,15 +28,18 @@ public sealed class JobTextExtractor : IJobTextExtractor
if (string.IsNullOrWhiteSpace(jobUrl)) return string.Empty;
if (!Uri.TryCreate(jobUrl, UriKind.Absolute, out var uri) || uri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException("Invalid job URL.");
}
var html = await _http.GetStringAsync(uri, ct);
html = Regex.Replace(html, "<script[\\s\\S]*?</script>", " ", RegexOptions.IgnoreCase);
html = Regex.Replace(html, "<style[\\s\\S]*?</style>", " ", RegexOptions.IgnoreCase);
html = Regex.Replace(html, "<[^>]+>", " ");
return Limit(Normalize(WebUtility.HtmlDecode(html)));
var response = await _pageFetcher.FetchAsync(new FetchPageRequest
{
Url = jobUrl,
CallerService = "cv-matcher-api"
}, ct);
if (!response.Success)
throw new InvalidOperationException($"Failed to fetch job page: {response.Error}");
return Limit(Normalize(response.Text));
}
/// <summary>Truncates text to the configured maximum character count.</summary>
@@ -34,7 +34,7 @@ public sealed class JobTokenService : IJobTokenService
}
/// <inheritdoc />
public async Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, string? location, CancellationToken ct)
public async Task<string?> CreateTokenAsync(string cvDocumentId, string email, string language, IReadOnlyList<string> keywords, string? location, string? clientIpAddress, CancellationToken ct)
{
var hasEnabledProviders = await _db.JobProviders.AnyAsync(p => p.Enabled, ct);
if (!hasEnabledProviders)
@@ -51,6 +51,7 @@ public sealed class JobTokenService : IJobTokenService
Language = language,
Keywords = string.Join(",", keywords),
Location = location,
ClientIpAddress = clientIpAddress,
ExpiresAt = DateTime.UtcNow.AddDays(_settings.TokenExpiryDays),
Used = false,
CreatedAt = DateTime.UtcNow
@@ -63,7 +64,7 @@ public sealed class JobTokenService : IJobTokenService
}
/// <inheritdoc />
public async Task<string> TriggerStartAsync(string tokenId, CancellationToken ct)
public async Task<string> TriggerStartAsync(string tokenId, string? clientIpAddress, CancellationToken ct)
{
var token = await _db.JobSearchTokens.FirstOrDefaultAsync(x => x.Id == tokenId, ct);
if (token is null) return StartJobSearchStatus.NotFound;
@@ -94,6 +95,7 @@ public sealed class JobTokenService : IJobTokenService
Status = JobSearchStatus.Pending,
Keywords = keywords,
Location = token.Location,
ClientIpAddress = clientIpAddress,
ProviderConfigJson = providerConfigJson,
CreatedAt = DateTime.UtcNow
};
@@ -128,7 +130,6 @@ public sealed class JobTokenService : IJobTokenService
JobLinkContains = entity.JobLinkContains,
InitialKeywords = keywords,
MaxResults = entity.MaxResults,
UseHeadlessBrowser = entity.UseHeadlessBrowser,
RequireKeywordInAnchor = entity.RequireKeywordInAnchor
};
}
@@ -82,6 +82,7 @@
<ProjectReference Include="..\cv-search-data\cv-search-data.csproj" />
<ProjectReference Include="..\cv-matcher-data\cv-matcher-data.csproj" />
<ProjectReference Include="..\common\common.csproj" />
<ProjectReference Include="..\page-fetcher-api-models\page-fetcher-api-models.csproj" />
<ProjectReference Include="..\..\Helpers\startup-helpers\startup-helpers.csproj" />
</ItemGroup>
</Project>
@@ -36,6 +36,8 @@ public sealed class CvMatcherDbContext : DbContext
entity.Property(x => x.JobDocumentId).HasMaxLength(64).IsRequired();
entity.Property(x => x.ResultJson).IsRequired();
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.Property(x => x.Email).HasMaxLength(256);
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.HasIndex(x => new { x.CvDocumentId, x.JobDocumentId, x.Language }).IsUnique();
});
@@ -9,4 +9,6 @@ public sealed class CvMatchResultEntity : BaseEntity
public string Language { get; set; } = "en";
public string ResultJson { get; set; } = string.Empty;
public int Score { get; set; }
public string? Email { get; set; }
public string? ClientIpAddress { get; set; }
}
@@ -0,0 +1,138 @@
// <auto-generated />
using System;
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvMatcher.Data.Migrations
{
[DbContext(typeof(CvMatcherDbContext))]
[Migration("20260608155310_AddEmailAndIpToResults")]
partial class AddEmailAndIpToResults
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvMatcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvMatcher.Data.Entities.AiPromptEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("AiPrompts", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Language")
.IsRequired()
.HasColumnType("nvarchar(450)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("CvDocumentId", "JobDocumentId", "Language")
.IsUnique();
b.ToTable("Results", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatcherChatCacheEntity", b =>
{
b.Property<string>("CacheKey")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Model")
.IsRequired()
.HasMaxLength(120)
.HasColumnType("nvarchar(120)");
b.Property<string>("ResponseText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<decimal>("Temperature")
.HasColumnType("decimal(4,2)");
b.HasKey("CacheKey");
b.ToTable("ChatCache", "cvMatcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,45 @@
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvMatcher.Data.Migrations
{
/// <inheritdoc />
public partial class AddEmailAndIpToResults : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "Results",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
migrationBuilder.AddColumn<string>(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "Results",
type: "nvarchar(256)",
maxLength: 256,
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "Results");
migrationBuilder.DropColumn(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "Results");
}
}
}
@@ -0,0 +1,138 @@
// <auto-generated />
using System;
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvMatcher.Data.Migrations
{
[DbContext(typeof(CvMatcherDbContext))]
[Migration("20260608193046_AddParseErrorPrompts")]
partial class AddParseErrorPrompts
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvMatcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvMatcher.Data.Entities.AiPromptEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("AiPrompts", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Language")
.IsRequired()
.HasColumnType("nvarchar(450)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("CvDocumentId", "JobDocumentId", "Language")
.IsUnique();
b.ToTable("Results", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatcherChatCacheEntity", b =>
{
b.Property<string>("CacheKey")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Model")
.IsRequired()
.HasMaxLength(120)
.HasColumnType("nvarchar(120)");
b.Property<string>("ResponseText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<decimal>("Temperature")
.HasColumnType("decimal(4,2)");
b.HasKey("CacheKey");
b.ToTable("ChatCache", "cvMatcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,67 @@
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvMatcher.Data.Migrations
{
/// <inheritdoc />
public partial class AddParseErrorPrompts : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.summary", "en", "The AI response could not be parsed. Please try again.", "Summary shown in match email when the AI returns an unparseable response"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.summary", "ro", "Răspunsul AI nu a putut fi interpretat. Vă rugăm să încercați din nou.", "Sumar afișat în emailul de potrivire când AI returnează un răspuns care nu poate fi interpretat"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.recommendation", "en", "If the problem persists, try a different job link or description.", "Recommendation shown in match email when the AI returns an unparseable response"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
columns: ["Key", "Language", "Value", "Description"],
values: ["parse-error.recommendation", "ro", "Dacă problema persistă, încercați un alt link sau descriere de job.", "Recomandare afișată în emailul de potrivire când AI returnează un răspuns care nu poate fi interpretat"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.summary", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.summary", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.recommendation", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["parse-error.recommendation", "ro"]);
}
}
}
@@ -0,0 +1,138 @@
// <auto-generated />
using System;
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvMatcher.Data.Migrations
{
[DbContext(typeof(CvMatcherDbContext))]
[Migration("20260609133623_FixKeywordExtractionPrompt")]
partial class FixKeywordExtractionPrompt
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvMatcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvMatcher.Data.Entities.AiPromptEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("AiPrompts", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Language")
.IsRequired()
.HasColumnType("nvarchar(450)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("CvDocumentId", "JobDocumentId", "Language")
.IsUnique();
b.ToTable("Results", "cvMatcher");
});
modelBuilder.Entity("CvMatcher.Data.Entities.CvMatcherChatCacheEntity", b =>
{
b.Property<string>("CacheKey")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Model")
.IsRequired()
.HasMaxLength(120)
.HasColumnType("nvarchar(120)");
b.Property<string>("ResponseText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<decimal>("Temperature")
.HasColumnType("decimal(4,2)");
b.HasKey("CacheKey");
b.ToTable("ChatCache", "cvMatcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,93 @@
using CvMatcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvMatcher.Data.Migrations
{
/// <inheritdoc />
public partial class FixKeywordExtractionPrompt : Migration
{
// Full prompt values — only the 'keywords' instruction changes vs. the previous migration.
// Stored in full so Down() can restore the previous version exactly.
private const string EnNew =
"You are a strict CV-to-job matching engine. Return JSON only. Score realistically from 0 to 100. Penalize missing required skills. Do not invent experience. Use concise business language. All text fields in the JSON response must be in English.\n" +
"JSON shape: {\"score\":number,\"summary\":\"one-line summary in English\",\"strengths\":[\"strength 1 in English\"],\"gaps\":[\"gap 1 in English\"],\"recommendations\":[\"recommendation 1 in English\"],\"evidence\":[\"evidence 1 in English\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"City, Country\"}.\n" +
"For 'keywords': extract 2-4 job-board search terms that represent the candidate's professional identity as shown in their CV — their seniority level and primary role title (e.g. 'Software Architect', 'Engineering Manager', 'Senior .NET Developer') plus 1-2 core technologies they genuinely emphasize throughout the CV. Derive these entirely from the CV — do not use the job title or job technologies unless they independently match the candidate's actual positioning. Avoid generic terms like 'developer', 'engineer', 'cloud', or 'leadership'.\n" +
"For 'location': extract the candidate's city and country from the CV (e.g. 'Cluj-Napoca, Romania'). Use an empty string if not found.";
private const string EnPrev =
"You are a strict CV-to-job matching engine. Return JSON only. Score realistically from 0 to 100. Penalize missing required skills. Do not invent experience. Use concise business language. All text fields in the JSON response must be in English.\n" +
"JSON shape: {\"score\":number,\"summary\":\"one-line summary in English\",\"strengths\":[\"strength 1 in English\"],\"gaps\":[\"gap 1 in English\"],\"recommendations\":[\"recommendation 1 in English\"],\"evidence\":[\"evidence 1 in English\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"City, Country\"}.\n" +
"For 'keywords': extract 2-4 short, concrete terms a recruiter would search for on a job board — the candidate's primary role title and key technologies (e.g. 'Senior .NET Developer', 'C#', 'Azure'). Avoid abstract concepts like 'leadership', 'cloud', or 'microservices'.\n" +
"For 'location': extract the candidate's city and country from the CV (e.g. 'Cluj-Napoca, Romania'). Use an empty string if not found.";
private const string RoNew =
"Ești un motor strict de potrivire CV-job. Returnează doar JSON. Punctează realist între 0 și 100. Penalizează abilitățile lipsă necesare. Nu inventa experiență. Folosește limbaj profesional concis. Toate câmpurile text din răspunsul JSON trebuie să fie în limba română.\n" +
"JSON shape: {\"score\":number,\"summary\":\"rezumat pe o linie în română\",\"strengths\":[\"punct forte 1 în română\"],\"gaps\":[\"lipsă 1 în română\"],\"recommendations\":[\"recomandare 1 în română\"],\"evidence\":[\"dovadă 1 în română\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"Oraș, Țară\"}.\n" +
"Pentru 'keywords': extrage 2-4 termeni de căutare pe site-uri de joburi care reprezintă identitatea profesională a candidatului conform CV-ului — nivelul de senioritate și titlul principal de rol (ex. 'Software Architect', 'Engineering Manager', 'Senior .NET Developer') și 1-2 tehnologii de bază pe care candidatul le evidențiază cu adevărat în CV. Derivă aceștia exclusiv din CV — nu folosi titlul jobului sau tehnologiile din job dacă nu corespund poziționării reale a candidatului. Evită termeni generici precum 'developer', 'engineer', 'cloud' sau 'leadership'.\n" +
"Pentru 'location': extrage orașul și țara candidatului din CV (ex. 'Cluj-Napoca, România'). Folosește string gol dacă nu se găsește.";
private const string RoPrev =
"Ești un motor strict de potrivire CV-job. Returnează doar JSON. Punctează realist între 0 și 100. Penalizează abilitățile lipsă necesare. Nu inventa experiență. Folosește limbaj profesional concis. Toate câmpurile text din răspunsul JSON trebuie să fie în limba română.\n" +
"JSON shape: {\"score\":number,\"summary\":\"rezumat pe o linie în română\",\"strengths\":[\"punct forte 1 în română\"],\"gaps\":[\"lipsă 1 în română\"],\"recommendations\":[\"recomandare 1 în română\"],\"evidence\":[\"dovadă 1 în română\"],\"keywords\":[\"Senior .NET Developer\",\"C#\",\"Azure\"],\"location\":\"Oraș, Țară\"}.\n" +
"Pentru 'keywords': extrage 2-4 termeni scurți și concreți pe care un recrutor i-ar căuta pe un site de joburi — titlul principal al rolului și tehnologiile cheie (ex. 'Senior .NET Developer', 'C#', 'Azure'). Evită concepte abstracte precum 'leadership', 'cloud' sau 'microservicii'.\n" +
"Pentru 'location': extrage orașul și țara candidatului din CV (ex. 'Cluj-Napoca, România'). Folosește string gol dacă nu se găsește.";
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
// Update English prompt: keywords must now be derived from the CV only,
// not influenced by the job description being matched against.
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "en"],
columns: ["Value", "Description"],
values: [
EnNew,
"System prompt for CV-to-job matching in English. Keywords represent the candidate's CV identity (seniority + role + core tech), not the job being matched."
]);
// Update Romanian prompt: same improvement.
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "ro"],
columns: ["Value", "Description"],
values: [
RoNew,
"System prompt pentru potrivire CV-job în română. Cuvintele cheie reprezintă identitatea CV-ului candidatului (senioritate + rol + tehnologii cheie), nu jobul cu care se face potrivirea."
]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "en"],
columns: ["Value", "Description"],
values: [
EnPrev,
"System prompt for CV-to-job matching in English. Extracts job-board-friendly keywords (role title + key tech) and candidate location."
]);
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "AiPrompts",
keyColumns: ["Key", "Language"],
keyValues: ["ai.cv-match.system-prompt", "ro"],
columns: ["Value", "Description"],
values: [
RoPrev,
"System prompt pentru potrivire CV-job în limba română. Extrage cuvinte cheie prietenoase pentru site-uri de joburi (titlu rol + tehnologii cheie) și locația candidatului."
]);
}
}
}
@@ -60,6 +60,10 @@ namespace CvMatcher.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
@@ -70,6 +74,10 @@ namespace CvMatcher.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobDocumentId")
.IsRequired()
.HasMaxLength(64)
@@ -6,7 +6,7 @@ public interface IMatcherRepository
{
Task InitializeAsync(CancellationToken ct);
Task<JobMatchResponse?> GetMatchAsync(string cvDocumentId, string jobDocumentId, string language, CancellationToken ct);
Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, CancellationToken ct);
Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, string? email, string? clientIpAddress, CancellationToken ct);
Task<string?> GetChatCompletionAsync(string cacheKey, CancellationToken ct);
Task SaveChatCompletionAsync(string cacheKey, string model, decimal temperature, string responseText, CancellationToken ct);
}
@@ -40,7 +40,7 @@ public sealed class EfMatcherRepository : IMatcherRepository
return result;
}
public async Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, CancellationToken ct)
public async Task SaveMatchAsync(string cvDocumentId, string jobDocumentId, string language, JobMatchResponse response, string? email, string? clientIpAddress, CancellationToken ct)
{
var exists = await _db.CvMatchResults.AnyAsync(
x => x.CvDocumentId == cvDocumentId && x.JobDocumentId == jobDocumentId && x.Language == language,
@@ -58,6 +58,8 @@ public sealed class EfMatcherRepository : IMatcherRepository
Language = language,
ResultJson = JsonSerializer.Serialize(response, new JsonSerializerOptions(JsonSerializerDefaults.Web)),
Score = response.Score,
Email = email,
ClientIpAddress = clientIpAddress,
CreatedAt = DateTime.UtcNow
});
@@ -36,6 +36,7 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.Language).HasMaxLength(8).HasDefaultValue("en").IsRequired();
entity.Property(x => x.Keywords).HasMaxLength(1000).HasDefaultValue(string.Empty);
entity.Property(x => x.Used).HasDefaultValue(false);
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
});
@@ -51,6 +52,7 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.Keywords).HasMaxLength(1000);
entity.Property(x => x.ProviderConfigJson).IsRequired(false);
entity.Property(x => x.Language).HasMaxLength(8).HasDefaultValue("en").IsRequired();
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.HasIndex(x => x.Status);
});
@@ -64,6 +66,8 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.ProviderName).HasMaxLength(128);
entity.Property(x => x.JobUrl).HasMaxLength(2048);
entity.Property(x => x.JobTitle).HasMaxLength(512);
entity.Property(x => x.Email).HasMaxLength(256);
entity.Property(x => x.ClientIpAddress).HasMaxLength(45);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.HasIndex(x => x.SessionId);
});
@@ -79,7 +83,6 @@ public sealed class CvSearchDbContext : DbContext
entity.Property(x => x.InitialKeywordsJson).HasMaxLength(2000).HasDefaultValue("[]").IsRequired();
entity.Property(x => x.MaxResults).HasDefaultValue(20);
entity.Property(x => x.DisplayOrder).HasDefaultValue(0);
entity.Property(x => x.UseHeadlessBrowser).HasDefaultValue(false);
});
}
}
@@ -31,9 +31,6 @@ public sealed class JobProviderEntity
/// <summary>Controls display ordering in future admin UIs.</summary>
public int DisplayOrder { get; set; }
/// <summary>When true, the scraper renders the page with headless Chromium instead of a plain HTTP GET.</summary>
public bool UseHeadlessBrowser { get; set; }
/// <summary>
/// When false, the Stage 2 anchor-text keyword filter is skipped.
/// Set to false for providers whose search URL already filters by relevance server-side (ejobs.ro, bestjobs.eu).
@@ -11,4 +11,8 @@ public sealed class JobSearchResultEntity : BaseEntity
public string JobText { get; set; } = string.Empty;
public int Score { get; set; }
public string ResultJson { get; set; } = string.Empty;
/// <summary>Email address of the user who triggered the search. Copied from the parent session.</summary>
public string? Email { get; set; }
/// <summary>Client IP address at link-click time. Copied from the parent session.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -10,6 +10,8 @@ public sealed class JobSearchSessionEntity : BaseEntity
public string Status { get; set; } = JobSearchStatus.Pending;
public string Keywords { get; set; } = string.Empty;
public string? Location { get; set; }
/// <summary>Client IP address captured when the user clicked the one-time job-search link. Null for sessions created before this field was added.</summary>
public string? ClientIpAddress { get; set; }
public string? ProviderConfigJson { get; set; }
public string Language { get; set; } = "en";
}
@@ -11,4 +11,6 @@ public sealed class JobSearchTokenEntity : BaseEntity
public bool Used { get; set; }
public string Keywords { get; set; } = string.Empty;
public string? Location { get; set; }
/// <summary>Client IP address captured when the user submitted the CV match request. Null for tokens created before this field was added.</summary>
public string? ClientIpAddress { get; set; }
}
@@ -0,0 +1,238 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608154221_RemoveUseHeadlessBrowser")]
partial class RemoveUseHeadlessBrowser
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,32 @@
using CvSearch.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class RemoveUseHeadlessBrowser : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "UseHeadlessBrowser",
schema: MigrationConstants.SchemaName,
table: "JobProviders");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<bool>(
name: "UseHeadlessBrowser",
schema: MigrationConstants.SchemaName,
table: "JobProviders",
type: "bit",
nullable: false,
defaultValue: false);
}
}
}
@@ -0,0 +1,250 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608161102_AddEmailIpToSessionAndResults")]
partial class AddEmailIpToSessionAndResults
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,58 @@
using CvSearch.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class AddEmailIpToSessionAndResults : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchSessions",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
migrationBuilder.AddColumn<string>(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults",
type: "nvarchar(256)",
maxLength: 256,
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchSessions");
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults");
migrationBuilder.DropColumn(
name: "Email",
schema: MigrationConstants.SchemaName,
table: "JobSearchResults");
}
}
}
@@ -0,0 +1,254 @@
// <auto-generated />
using System;
using CvSearch.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace CvSearch.Data.Migrations
{
[DbContext(typeof(CvSearchDbContext))]
[Migration("20260608161930_AddClientIpToJobSearchTokens")]
partial class AddClientIpToJobSearchTokens
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("cvSearch")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("CvSearch.Data.Entities.JobProviderEntity", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("int");
SqlServerPropertyBuilderExtensions.UseIdentityColumn(b.Property<int>("Id"));
b.Property<int>("DisplayOrder")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(0);
b.Property<bool>("Enabled")
.HasColumnType("bit");
b.Property<string>("InitialKeywordsJson")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)")
.HasDefaultValue("[]");
b.Property<string>("JobLinkContains")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<int>("MaxResults")
.ValueGeneratedOnAdd()
.HasColumnType("int")
.HasDefaultValue(20);
b.Property<string>("Name")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<bool>("RequireKeywordInAnchor")
.HasColumnType("bit");
b.Property<string>("SearchUrlTemplate")
.IsRequired()
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchResultEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("JobTitle")
.IsRequired()
.HasMaxLength(512)
.HasColumnType("nvarchar(512)");
b.Property<string>("JobUrl")
.IsRequired()
.HasMaxLength(2048)
.HasColumnType("nvarchar(2048)");
b.Property<string>("ProviderName")
.IsRequired()
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("ResultJson")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int>("Score")
.HasColumnType("int");
b.Property<string>("SessionId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("SessionId");
b.ToTable("JobSearchResults", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchSessionEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("Keywords")
.IsRequired()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<string>("ProviderConfigJson")
.HasColumnType("nvarchar(max)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("nvarchar(32)");
b.Property<string>("TokenId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.HasKey("Id");
b.HasIndex("Status");
b.ToTable("JobSearchSessions", "cvSearch");
});
modelBuilder.Entity("CvSearch.Data.Entities.JobSearchTokenEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("CvDocumentId")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("Email")
.IsRequired()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("datetime2");
b.Property<string>("Keywords")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(1000)
.HasColumnType("nvarchar(1000)")
.HasDefaultValue("");
b.Property<string>("Language")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(8)
.HasColumnType("nvarchar(8)")
.HasDefaultValue("en");
b.Property<string>("Location")
.HasColumnType("nvarchar(max)");
b.Property<bool>("Used")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobSearchTokens", "cvSearch");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,32 @@
using CvSearch.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace CvSearch.Data.Migrations
{
/// <inheritdoc />
public partial class AddClientIpToJobSearchTokens : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchTokens",
type: "nvarchar(45)",
maxLength: 45,
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ClientIpAddress",
schema: MigrationConstants.SchemaName,
table: "JobSearchTokens");
}
}
}
@@ -69,11 +69,6 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.Property<bool>("UseHeadlessBrowser")
.ValueGeneratedOnAdd()
.HasColumnType("bit")
.HasDefaultValue(false);
b.HasKey("Id");
b.ToTable("JobProviders", "cvSearch");
@@ -85,11 +80,19 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<string>("JobText")
.IsRequired()
.HasColumnType("nvarchar(max)");
@@ -134,6 +137,10 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
@@ -190,6 +197,10 @@ namespace CvSearch.Data.Migrations
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("ClientIpAddress")
.HasMaxLength(45)
.HasColumnType("nvarchar(45)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
@@ -1,7 +1,7 @@
using EmailApi.Models.Requests;
using Email.Models.Requests;
using Refit;
namespace EmailApi.Models.Clients;
namespace Email.Models.Clients;
public interface IEmailApiClient
{
@@ -1,4 +1,4 @@
namespace EmailApi.Models.Requests;
namespace Email.Models.Requests;
public sealed class SendEmailRequest
{
@@ -1,4 +1,4 @@
namespace EmailApi.Models.Settings;
namespace Email.Models.Settings;
public sealed class EmailApiSettings
{
@@ -1,4 +1,4 @@
namespace Models.Settings;
namespace Email.Models.Settings;
public sealed class SmtpSettings
{
@@ -1,9 +1,9 @@
using EmailApi.Models.Requests;
using EmailApi.Services;
using Api.Services;
using Email.Models.Requests;
using Microsoft.AspNetCore.Mvc;
using Swashbuckle.AspNetCore.Annotations;
namespace EmailApi.Controllers;
namespace Api.Controllers;
/// <summary>
/// Internal email relay. Accepts an HTML body fragment from trusted callers
+2 -1
View File
@@ -3,8 +3,9 @@ using Email.Data;
using Email.Data.Repositories;
using Email.Data.Repositories.Contracts;
using Email.Data.Services;
using EmailApi.Services;
using Api.Services;
using Microsoft.EntityFrameworkCore;
using Email.Models.Settings;
using Models.Settings;
using Serilog;
using StartupHelpers;
@@ -1,12 +1,13 @@
using Email.Data.Services;
using EmailApi.Models.Requests;
using Email.Models.Requests;
using MailKit.Net.Smtp;
using MailKit.Security;
using Microsoft.Extensions.Options;
using MimeKit;
using Email.Models.Settings;
using Models.Settings;
namespace EmailApi.Services;
namespace Api.Services;
/// <summary>
/// Wraps an HTML body fragment in the branded HTML shell and sends the resulting email via SMTP using MailKit.
@@ -57,7 +58,7 @@ public sealed class SmtpEmailDispatcher
if (!string.IsNullOrWhiteSpace(req.ReplyTo))
msg.ReplyTo.Add(MailboxAddress.Parse(req.ReplyTo));
msg.Subject = $"[{_environmentName}] {req.Subject}".Trim();
msg.Subject = req.Subject.Trim();
var shellStart = _templates.Get("email.html-shell.start", "*");
var shellEnd = _templates.Get("email.html-shell.end", "*");
@@ -0,0 +1,69 @@
// <auto-generated />
using System;
using Email.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace Email.Data.Migrations
{
[DbContext(typeof(EmailDbContext))]
[Migration("20260608125339_AddLocationToScanSummaryTemplate")]
partial class AddLocationToScanSummaryTemplate
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("email")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("Email.Data.Entities.EmailTemplateEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<string>("OperatorCopy")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("Templates", "email");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,80 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace Email.Data.Migrations
{
/// <inheritdoc />
public partial class AddLocationToScanSummaryTemplate : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "en"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Keywords used:</strong>&nbsp;{{keywordsHtml}}</div>
<div style=""margin-bottom: 8px;""><strong>Location:</strong>&nbsp;{{location}}</div>
<div><strong>Providers scanned:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "ro"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Cuvinte cheie folosite:</strong>&nbsp;{{keywordsHtml}}</div>
<div style=""margin-bottom: 8px;""><strong>Locație căutată:</strong>&nbsp;{{location}}</div>
<div><strong>Furnizori scanați:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "en"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Keywords used:</strong>&nbsp;{{keywordsHtml}}</div>
<div><strong>Providers scanned:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
migrationBuilder.UpdateData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.scan-summary", "ro"],
columns: ["Value"],
values: [@"<table width=""100%"" cellpadding=""0"" cellspacing=""0"" border=""0"" style=""width: 100%; margin-bottom: 18px; border: 1px solid #dee2e6; border-collapse: collapse;"">
<tr>
<td bgcolor=""#f8f9fa"" style=""background-color: #f8f9fa; padding: 14px 16px; font-size: 13px; color: #495057;"">
<div style=""margin-bottom: 8px;""><strong>Cuvinte cheie folosite:</strong>&nbsp;{{keywordsHtml}}</div>
<div><strong>Furnizori scanați:</strong>&nbsp;{{providers}}</div>
</td>
</tr>
</table>"]);
}
}
}
@@ -0,0 +1,69 @@
// <auto-generated />
using System;
using Email.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace Email.Data.Migrations
{
[DbContext(typeof(EmailDbContext))]
[Migration("20260608190611_AddManualJobLabelTemplate")]
partial class AddManualJobLabelTemplate
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("email")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("Email.Data.Entities.EmailTemplateEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<string>("OperatorCopy")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("Templates", "email");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,43 @@
using Email.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace Email.Data.Migrations
{
/// <inheritdoc />
public partial class AddManualJobLabelTemplate : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.manual-job-label", "en", "Manual job description", "Label used in the match email subject and body when no job URL was provided"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.manual-job-label", "ro", "Descriere manuală a jobului", "Etichetă folosită în subiectul și corpul emailului când nu a fost furnizat un URL de job"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.manual-job-label", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.manual-job-label", "ro"]);
}
}
}
@@ -0,0 +1,69 @@
// <auto-generated />
using System;
using Email.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
#nullable disable
namespace Email.Data.Migrations
{
[DbContext(typeof(EmailDbContext))]
[Migration("20260608192938_AddFallbackStringTemplates")]
partial class AddFallbackStringTemplates
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("email")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("Email.Data.Entities.EmailTemplateEntity", b =>
{
b.Property<string>("Key")
.HasMaxLength(128)
.HasColumnType("nvarchar(128)");
b.Property<string>("Language")
.HasMaxLength(8)
.HasColumnType("nvarchar(8)");
b.Property<string>("Description")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(500)
.HasColumnType("nvarchar(500)")
.HasDefaultValue("");
b.Property<string>("OperatorCopy")
.IsRequired()
.ValueGeneratedOnAdd()
.HasMaxLength(256)
.HasColumnType("nvarchar(256)")
.HasDefaultValue("");
b.Property<DateTime>("UpdatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<string>("Value")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.HasKey("Key", "Language");
b.ToTable("Templates", "email");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,163 @@
using Email.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace Email.Data.Migrations
{
/// <inheritdoc />
public partial class AddFallbackStringTemplates : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.fallback-na", "en", "N/A", "Fallback when a match email field (job label or URL) has no value"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.fallback-na", "ro", "N/A", "Fallback când un câmp al emailului de potrivire (etichetă job sau URL) nu are valoare"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.subject-fallback-label", "en", "Job", "Fallback job label used in match email subject when no specific label is available"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.match.subject-fallback-label", "ro", "Job", "Etichetă fallback pentru subiectul emailului de potrivire când nu există o etichetă specifică"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.notification.unknown-ip", "en", "Unknown", "Fallback IP address label in operator notification emails"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.notification.unknown-ip", "ro", "Necunoscut", "Etichetă fallback pentru adresa IP în emailurile de notificare operator"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.keywords-empty", "en", "none detected", "Text shown in job search results email when no CV keywords were extracted"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.keywords-empty", "ro", "niciunul detectat", "Text afișat în emailul cu rezultatele căutării când nu au fost extrase cuvinte cheie din CV"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.providers-empty", "en", "none", "Text shown in job search results email when no providers were searched"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.providers-empty", "ro", "niciunul", "Text afișat în emailul cu rezultatele căutării când nu au fost căutați furnizori"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.location-empty", "en", "-", "Fallback location display in job search results email scan summary"]);
migrationBuilder.InsertData(
schema: MigrationConstants.SchemaName,
table: "Templates",
columns: ["Key", "Language", "Value", "Description"],
values: ["email.search-results.location-empty", "ro", "-", "Afișaj fallback pentru locație în sumarului de scanare al emailului cu rezultatele căutării"]);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.fallback-na", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.fallback-na", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.subject-fallback-label", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.match.subject-fallback-label", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.notification.unknown-ip", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.notification.unknown-ip", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.keywords-empty", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.keywords-empty", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.providers-empty", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.providers-empty", "ro"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.location-empty", "en"]);
migrationBuilder.DeleteData(
schema: MigrationConstants.SchemaName,
table: "Templates",
keyColumns: ["Key", "Language"],
keyValues: ["email.search-results.location-empty", "ro"]);
}
}
}
@@ -0,0 +1,26 @@
namespace PageFetcher.Models;
/// <summary>
/// Request to fetch a web page via the page-fetcher-api.
/// </summary>
public sealed class FetchPageRequest
{
/// <summary>Absolute HTTP or HTTPS URL to fetch.</summary>
public string Url { get; set; } = string.Empty;
/// <summary>
/// Playwright wait condition. Accepted values: <c>networkidle</c> (default), <c>domcontentloaded</c>, <c>load</c>.
/// </summary>
public string WaitFor { get; set; } = "networkidle";
/// <summary>
/// Identifies the calling service for audit purposes (e.g. <c>cv-matcher-api</c>, <c>cv-search-job</c>).
/// </summary>
public string CallerService { get; set; } = string.Empty;
/// <summary>
/// Optional reference to the job search session that triggered this fetch.
/// Stored on <c>pageFetcher.PageFetches</c> for cross-schema audit queries.
/// </summary>
public string? JobSearchSessionId { get; set; }
}
@@ -0,0 +1,25 @@
namespace PageFetcher.Models;
/// <summary>
/// Result of a page fetch operation.
/// </summary>
public sealed class FetchPageResponse
{
/// <summary>Final URL after any redirects.</summary>
public string Url { get; set; } = string.Empty;
/// <summary>HTTP status code returned by the page. <c>0</c> on network failure.</summary>
public int StatusCode { get; set; }
/// <summary>Full rendered HTML as returned by Playwright.</summary>
public string Html { get; set; } = string.Empty;
/// <summary>Plain text extracted from the HTML (script/style stripped, whitespace normalised).</summary>
public string Text { get; set; } = string.Empty;
/// <summary>Whether the fetch succeeded. <c>false</c> on timeout or network error.</summary>
public bool Success { get; set; }
/// <summary>Exception message when <see cref="Success"/> is <c>false</c>.</summary>
public string? Error { get; set; }
}
@@ -0,0 +1,16 @@
using Refit;
namespace PageFetcher.Models;
/// <summary>
/// Refit client for the internal page-fetcher-api service.
/// All calls require the <c>X-Internal-Api-Key</c> header, configured at registration time.
/// </summary>
public interface IPageFetcherApiClient
{
/// <summary>
/// Fetches a web page via headless Chromium and returns the rendered HTML and extracted plain text.
/// </summary>
[Post("/api/page/fetch")]
Task<FetchPageResponse> FetchAsync([Body] FetchPageRequest request, CancellationToken ct = default);
}
@@ -0,0 +1,17 @@
namespace PageFetcher.Models.Settings;
/// <summary>
/// Runtime settings for the page-fetcher service.
/// Bound from the <c>PageFetcher</c> configuration section.
/// </summary>
public sealed class PageFetcherSettings
{
/// <summary>Default Playwright wait condition (<c>networkidle</c>, <c>load</c>, <c>domcontentloaded</c>).</summary>
public string DefaultWaitFor { get; set; } = "networkidle";
/// <summary>Page navigation timeout in seconds.</summary>
public int TimeoutSeconds { get; set; } = 30;
/// <summary>Maximum characters stored/returned in the extracted text field.</summary>
public int MaxTextChars { get; set; } = 60_000;
}
@@ -0,0 +1,12 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AssemblyName>page-fetcher-api-models</AssemblyName>
<RootNamespace>PageFetcher.Models</RootNamespace>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Refit.HttpClientFactory" />
</ItemGroup>
</Project>
@@ -0,0 +1,47 @@
using Api.Services;
using Microsoft.AspNetCore.Mvc;
using PageFetcher.Models;
using Swashbuckle.AspNetCore.Annotations;
namespace Api.Controllers;
/// <summary>
/// Handles page-fetch requests: navigates to the URL via Playwright and returns rendered HTML and extracted text.
/// </summary>
[ApiController]
[Route("api/page")]
public sealed class PageController : ControllerBase
{
private readonly PageFetcherService _service;
private readonly ILogger<PageController> _logger;
public PageController(PageFetcherService service, ILogger<PageController> logger)
{
_service = service;
_logger = logger;
}
/// <summary>
/// Fetches a web page via headless Chromium.
/// Returns rendered HTML and extracted plain text.
/// </summary>
[HttpPost("fetch")]
[SwaggerOperation(Summary = "Fetch a web page", Description = "Navigates to the given URL using Playwright, returns rendered HTML and stripped plain text.")]
[SwaggerResponse(StatusCodes.Status200OK, "Page fetched successfully", typeof(FetchPageResponse))]
[SwaggerResponse(StatusCodes.Status400BadRequest, "Invalid or non-HTTP(S) URL")]
public async Task<ActionResult<FetchPageResponse>> Fetch([FromBody] FetchPageRequest request, CancellationToken ct)
{
if (string.IsNullOrWhiteSpace(request.Url))
return BadRequest(new { Error = "Url is required." });
if (!Uri.TryCreate(request.Url, UriKind.Absolute, out var uri) ||
(uri.Scheme != Uri.UriSchemeHttp && uri.Scheme != Uri.UriSchemeHttps))
return BadRequest(new { Error = "Url must be an absolute HTTP or HTTPS URL." });
_logger.LogInformation("Fetch request: {Url} | caller={Caller} | waitFor={WaitFor}",
request.Url, request.CallerService, request.WaitFor);
var result = await _service.FetchAsync(request, ct);
return Ok(result);
}
}
+50
View File
@@ -0,0 +1,50 @@
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY Directory.Packages.props ./
COPY Apis/page-fetcher-api/page-fetcher-api.csproj Apis/page-fetcher-api/
COPY Apis/page-fetcher-data/page-fetcher-data.csproj Apis/page-fetcher-data/
COPY Apis/page-fetcher-api-models/page-fetcher-api-models.csproj Apis/page-fetcher-api-models/
COPY Apis/common/common.csproj Apis/common/
COPY Apis/shared-data/shared-data.csproj Apis/shared-data/
COPY Helpers/startup-helpers/startup-helpers.csproj Helpers/startup-helpers/
COPY Helpers/common-helpers/common-helpers.csproj Helpers/common-helpers/
RUN dotnet restore Apis/page-fetcher-api/page-fetcher-api.csproj
COPY Apis/page-fetcher-api/ Apis/page-fetcher-api/
COPY Apis/page-fetcher-data/ Apis/page-fetcher-data/
COPY Apis/page-fetcher-api-models/ Apis/page-fetcher-api-models/
COPY Apis/common/ Apis/common/
COPY Apis/shared-data/ Apis/shared-data/
COPY Helpers/startup-helpers/ Helpers/startup-helpers/
COPY Helpers/common-helpers/ Helpers/common-helpers/
RUN dotnet publish Apis/page-fetcher-api/page-fetcher-api.csproj -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false
# Download Playwright Chromium browser in the build stage.
# Node.js is only needed here to run npx — it is not copied to the final image.
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
RUN apt-get update && apt-get install -y --no-install-recommends nodejs npm \
&& npx --yes playwright@1.60.0 install chromium \
&& rm -rf /var/lib/apt/lists/*
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS final
WORKDIR /app
# System libraries required by Chromium on Debian bookworm
RUN apt-get update && apt-get install -y --no-install-recommends \
libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 \
libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 \
libgbm1 libasound2t64 libpango-1.0-0 libcairo2 libatspi2.0-0 \
libwayland-client0 libx11-xcb1 libx11-6 libxcb1 libxext6 \
&& rm -rf /var/lib/apt/lists/*
# Copy the Playwright Chromium browser from the build stage
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
COPY --from=build /ms-playwright /ms-playwright
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "page-fetcher-api.dll"]
+75
View File
@@ -0,0 +1,75 @@
using System.Reflection;
using Microsoft.EntityFrameworkCore;
using PageFetcher.Data;
using Api.Services;
using PageFetcher.Models.Settings;
using Serilog;
using StartupHelpers;
StartupExtensions.LoadDotEnvFile();
const string ServiceName = "page-fetcher-api";
var appVersion = StartupExtensions.GetApplicationVersion(Assembly.GetExecutingAssembly());
try
{
var builder = WebApplication.CreateBuilder(args);
builder.ConfigureJsonSerilog(ServiceName, appVersion);
Log.Information("Starting {Service} version {AppVersion}", ServiceName, appVersion);
builder.AddAzureKeyVaultIfConfigured();
builder.Services.Configure<PageFetcherSettings>(builder.Configuration.GetSection("PageFetcher"));
builder.Services.AddDbContext<PageFetchDbContext>(options =>
{
var connectionString = builder.Services.GetConfiguredDbConnectionString(builder.Configuration);
options.UseSqlServer(connectionString, sql =>
{
sql.MigrationsHistoryTable(PageFetchDbContext.MigrationTableName, PageFetchDbContext.SchemaName);
sql.MigrationsAssembly("page-fetcher-data");
});
});
// Playwright browser: singleton hosted service, shared across all requests
builder.Services.AddSingleton<PlaywrightBrowserService>();
builder.Services.AddHostedService(sp => sp.GetRequiredService<PlaywrightBrowserService>());
builder.Services.AddScoped<PageFetcherService>();
builder.Services.AddControllers();
builder.Services.AddSwaggerWithXmlComments(Assembly.GetExecutingAssembly(), "Page Fetcher API");
var app = builder.Build();
app.LogStartupDiagnostics(ServiceName);
app.UseDefaultSerilogRequestLogging();
app.UseJsonExceptionHandler(ServiceName);
app.UseInternalApiKeyProtection();
app.UseSwaggerInDevelopment("Page Fetcher API", "PageFetcherAPI");
app.UseRouting();
app.UseAuthorization();
app.MapControllers();
Log.Information("Running EF Core migrations if any");
using (var scope = app.Services.CreateScope())
{
var db = scope.ServiceProvider.GetRequiredService<PageFetchDbContext>();
db.Database.Migrate();
}
Log.Information("{Service} startup complete. Listening for requests...", ServiceName);
app.Run();
}
catch (Exception ex)
{
Log.Fatal(ex, "{Service} terminated unexpectedly", ServiceName);
}
finally
{
Log.Information("Shutting down {Service}", ServiceName);
Log.CloseAndFlush();
}
@@ -0,0 +1,12 @@
{
"profiles": {
"page-fetcher-api": {
"commandName": "Project",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"applicationUrl": "https://localhost:50268;http://localhost:50269"
}
}
}
@@ -0,0 +1,145 @@
using System.Diagnostics;
using System.Net;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Options;
using Microsoft.Playwright;
using PageFetcher.Data;
using PageFetcher.Data.Entities;
using PageFetcher.Models;
using PageFetcher.Models.Settings;
namespace Api.Services;
/// <summary>
/// Fetches a web page via Playwright, extracts plain text, persists the result to the database,
/// and returns a <see cref="FetchPageResponse"/>.
/// </summary>
public sealed class PageFetcherService
{
private readonly PlaywrightBrowserService _browserService;
private readonly PageFetchDbContext _db;
private readonly PageFetcherSettings _settings;
private readonly ILogger<PageFetcherService> _logger;
public PageFetcherService(
PlaywrightBrowserService browserService,
PageFetchDbContext db,
IOptions<PageFetcherSettings> settings,
ILogger<PageFetcherService> logger)
{
_browserService = browserService;
_db = db;
_settings = settings.Value;
_logger = logger;
}
/// <summary>
/// Fetches the page at <paramref name="request.Url"/> using Playwright, saves the fetch record,
/// and returns the HTML and extracted text.
/// Returns a failed response (with <see cref="FetchPageResponse.Success"/> = false) rather than throwing
/// on network or navigation errors.
/// </summary>
public async Task<FetchPageResponse> FetchAsync(FetchPageRequest request, CancellationToken ct)
{
var sw = Stopwatch.StartNew();
string html = string.Empty;
string text = string.Empty;
int? statusCode = null;
bool success = false;
string? errorMessage = null;
string finalUrl = request.Url;
try
{
var page = await _browserService.Browser.NewPageAsync();
await using var _ = page.ConfigureAwait(false);
var waitUntil = request.WaitFor?.ToLowerInvariant() switch
{
"load" => WaitUntilState.Load,
"domcontentloaded" => WaitUntilState.DOMContentLoaded,
_ => WaitUntilState.NetworkIdle
};
IResponse? response;
try
{
response = await page.GotoAsync(request.Url, new PageGotoOptions
{
WaitUntil = waitUntil,
Timeout = _settings.TimeoutSeconds * 1_000
});
}
catch (TimeoutException)
{
_logger.LogWarning("Playwright NetworkIdle timeout for {Url}, using partial content", request.Url);
response = null;
}
statusCode = response?.Status;
finalUrl = page.Url;
html = await page.ContentAsync();
text = ExtractText(html);
success = true;
_logger.LogInformation("Fetched {Url} → HTTP {Status} | HTML {HtmlLen} chars | text {TextLen} chars | {DurationMs} ms",
request.Url, statusCode?.ToString() ?? "timeout", html.Length, text.Length, sw.ElapsedMilliseconds);
}
catch (Exception ex)
{
errorMessage = ex.Message;
_logger.LogError(ex, "Failed to fetch {Url}", request.Url);
}
finally
{
sw.Stop();
}
// Persist fetch record
var entity = new PageFetchEntity
{
Id = Guid.NewGuid().ToString("N"),
Url = request.Url,
CallerService = request.CallerService ?? string.Empty,
JobSearchSessionId = request.JobSearchSessionId,
HttpStatusCode = statusCode,
Html = html,
Text = text,
DurationMs = sw.ElapsedMilliseconds,
Success = success,
ErrorMessage = errorMessage
};
_db.PageFetches.Add(entity);
await _db.SaveChangesAsync(ct);
return new FetchPageResponse
{
Url = finalUrl,
StatusCode = statusCode ?? 0,
Html = html,
Text = text,
Success = success,
Error = errorMessage
};
}
/// <summary>
/// Strips script/style blocks and all HTML tags from raw HTML, normalises whitespace,
/// and truncates to <see cref="PageFetcherSettings.MaxTextChars"/>.
/// </summary>
private string ExtractText(string html)
{
if (string.IsNullOrWhiteSpace(html)) return string.Empty;
var text = html;
text = Regex.Replace(text, "<script[\\s\\S]*?</script>", " ", RegexOptions.IgnoreCase);
text = Regex.Replace(text, "<style[\\s\\S]*?</style>", " ", RegexOptions.IgnoreCase);
text = Regex.Replace(text, "<[^>]+>", " ");
text = WebUtility.HtmlDecode(text);
text = string.Join(' ', text.Split((char[]?)null, StringSplitOptions.RemoveEmptyEntries)).Trim();
var max = Math.Max(4_000, _settings.MaxTextChars);
return text.Length <= max ? text : text[..max];
}
}
@@ -0,0 +1,49 @@
using Microsoft.Playwright;
namespace Api.Services;
/// <summary>
/// Singleton hosted service that owns the Playwright Chromium browser process for the lifetime of the application.
/// Launches the browser once at startup and exposes it for injection into <see cref="PageFetcherService"/>.
/// </summary>
public sealed class PlaywrightBrowserService : IHostedService, IAsyncDisposable
{
private IPlaywright? _playwright;
private IBrowser? _browser;
private readonly ILogger<PlaywrightBrowserService> _logger;
public PlaywrightBrowserService(ILogger<PlaywrightBrowserService> logger)
{
_logger = logger;
}
/// <summary>The running Chromium browser instance. Available after <see cref="StartAsync"/> completes.</summary>
public IBrowser Browser => _browser ?? throw new InvalidOperationException("Browser has not been started yet.");
/// <inheritdoc />
public async Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Launching Playwright Chromium browser...");
_playwright = await Playwright.CreateAsync();
_browser = await _playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
{
Headless = true,
Args = ["--no-sandbox", "--disable-setuid-sandbox", "--disable-dev-shm-usage"]
});
_logger.LogInformation("Playwright Chromium browser launched successfully.");
}
/// <inheritdoc />
public async Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Closing Playwright Chromium browser...");
if (_browser is not null) await _browser.CloseAsync();
}
/// <inheritdoc />
public async ValueTask DisposeAsync()
{
if (_browser is not null) await _browser.DisposeAsync();
_playwright?.Dispose();
}
}
+73
View File
@@ -0,0 +1,73 @@
{
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
"Override": {
"Microsoft.AspNetCore": "Warning",
"Microsoft.AspNetCore.Hosting": "Information",
"Microsoft.AspNetCore.Routing": "Warning",
"System.Net.Http.HttpClient": "Warning",
"PageFetcherApi": "Information"
}
},
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
},
{
"Name": "File",
"Args": {
"path": "logs/page-fetcher-api-.log",
"rollingInterval": "Day",
"retainedFileCountLimit": 30,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {SourceContext}: {Message:lj}{NewLine}{Exception}"
}
}
],
"Enrich": [
"FromLogContext",
"WithMachineName",
"WithEnvironmentName"
]
},
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"Microsoft.AspNetCore.Hosting": "Information",
"Microsoft.AspNetCore.Routing": "Warning",
"System.Net.Http.HttpClient": "Warning",
"PageFetcherApi": "Information"
}
},
"LogEnvironmentOnStartup": true,
"AllowedHosts": "*",
"KeyVault": {
"VaultUri": "",
"Enabled": false
},
"Database": {
"Host": "localhost",
"Port": 1433,
"Name": "MyAiDb",
"User": "sa",
"Password": "",
"TrustServerCertificate": true
},
"InternalApi": {
"ApiKey": "",
"RequireApiKey": true
},
"PageFetcher": {
"DefaultWaitFor": "networkidle",
"TimeoutSeconds": 30,
"MaxTextChars": 60000
}
}
@@ -0,0 +1,34 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
<RootNamespace>PageFetcherApi</RootNamespace>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<NoWarn>$(NoWarn);1591</NoWarn>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Playwright" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Serilog.AspNetCore" />
<PackageReference Include="Serilog.Enrichers.Environment" />
<PackageReference Include="Serilog.Sinks.Console" />
<PackageReference Include="Serilog.Sinks.File" />
<PackageReference Include="Serilog.Sinks.Email" />
<PackageReference Include="Swashbuckle.AspNetCore" />
<PackageReference Include="Swashbuckle.AspNetCore.Annotations" />
<PackageReference Include="DotNetEnv" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\page-fetcher-data\page-fetcher-data.csproj" />
<ProjectReference Include="..\page-fetcher-api-models\page-fetcher-api-models.csproj" />
<ProjectReference Include="..\common\common.csproj" />
<ProjectReference Include="..\..\Helpers\startup-helpers\startup-helpers.csproj" />
<ProjectReference Include="..\..\Helpers\common-helpers\common-helpers.csproj" />
</ItemGroup>
</Project>
@@ -0,0 +1,40 @@
using Shared.Data.Entities;
namespace PageFetcher.Data.Entities;
/// <summary>
/// Audit record of a single page-fetch operation performed by the page-fetcher-api.
/// Stores the full rendered HTML and extracted plain text for every URL fetched.
/// </summary>
public sealed class PageFetchEntity : BaseEntity
{
/// <summary>The URL that was requested.</summary>
public string Url { get; set; } = string.Empty;
/// <summary>Name of the service that requested the fetch (e.g. <c>cv-matcher-api</c>, <c>cv-search-job</c>).</summary>
public string CallerService { get; set; } = string.Empty;
/// <summary>HTTP status code returned by the remote server. <c>null</c> on network failure.</summary>
public int? HttpStatusCode { get; set; }
/// <summary>Full rendered HTML as returned by Playwright.</summary>
public string Html { get; set; } = string.Empty;
/// <summary>Plain text extracted from the HTML (script/style stripped, whitespace normalised).</summary>
public string Text { get; set; } = string.Empty;
/// <summary>Playwright round-trip time in milliseconds.</summary>
public long DurationMs { get; set; }
/// <summary><c>true</c> when the page was fetched successfully; <c>false</c> on timeout or network error.</summary>
public bool Success { get; set; }
/// <summary>Exception message when <see cref="Success"/> is <c>false</c>.</summary>
public string? ErrorMessage { get; set; }
/// <summary>
/// Optional reference to the <c>cvSearch.JobSearchSessions</c> row that triggered this fetch.
/// Null for fetches not originating from a job search session (e.g. direct CV-to-job matches).
/// </summary>
public string? JobSearchSessionId { get; set; }
}
@@ -0,0 +1,8 @@
namespace PageFetcher.Data;
/// <summary>Schema and migration-history table name constants for the pageFetcher EF schema.</summary>
public static class MigrationConstants
{
public const string SchemaName = "pageFetcher";
public const string MigrationTableName = "_Migrations";
}
@@ -0,0 +1,82 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using PageFetcher.Data;
#nullable disable
namespace PageFetcher.Data.Migrations
{
[DbContext(typeof(PageFetchDbContext))]
[Migration("20260608143523_InitialSchema")]
partial class InitialSchema
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("pageFetcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("PageFetcher.Data.Entities.PageFetchEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("CallerService")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<long>("DurationMs")
.HasColumnType("bigint");
b.Property<string>("ErrorMessage")
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.Property<string>("Html")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int?>("HttpStatusCode")
.HasColumnType("int");
b.Property<bool>("Success")
.HasColumnType("bit");
b.Property<string>("Text")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("Url")
.IsRequired()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.HasKey("Id");
b.HasIndex("CreatedAt");
b.HasIndex("Url");
b.ToTable("PageFetches", "pageFetcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,59 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace PageFetcher.Data.Migrations
{
/// <inheritdoc />
public partial class InitialSchema : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.EnsureSchema(
name: MigrationConstants.SchemaName);
migrationBuilder.CreateTable(
name: "PageFetches",
schema: MigrationConstants.SchemaName,
columns: table => new
{
Id = table.Column<string>(type: "nvarchar(64)", maxLength: 64, nullable: false),
Url = table.Column<string>(type: "nvarchar(2000)", maxLength: 2000, nullable: false),
CallerService = table.Column<string>(type: "nvarchar(64)", maxLength: 64, nullable: false),
HttpStatusCode = table.Column<int>(type: "int", nullable: true),
Html = table.Column<string>(type: "nvarchar(max)", nullable: false),
Text = table.Column<string>(type: "nvarchar(max)", nullable: false),
DurationMs = table.Column<long>(type: "bigint", nullable: false),
Success = table.Column<bool>(type: "bit", nullable: false),
ErrorMessage = table.Column<string>(type: "nvarchar(2000)", maxLength: 2000, nullable: true),
CreatedAt = table.Column<DateTime>(type: "datetime2", nullable: false, defaultValueSql: "SYSUTCDATETIME()")
},
constraints: table =>
{
table.PrimaryKey("PK_PageFetches", x => x.Id);
});
migrationBuilder.CreateIndex(
name: "IX_PageFetches_CreatedAt",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
column: "CreatedAt");
migrationBuilder.CreateIndex(
name: "IX_PageFetches_Url",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
column: "Url");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "PageFetches",
schema: MigrationConstants.SchemaName);
}
}
}
@@ -0,0 +1,88 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using PageFetcher.Data;
#nullable disable
namespace PageFetcher.Data.Migrations
{
[DbContext(typeof(PageFetchDbContext))]
[Migration("20260608165542_AddJobSearchSessionId")]
partial class AddJobSearchSessionId
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("pageFetcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("PageFetcher.Data.Entities.PageFetchEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("CallerService")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<long>("DurationMs")
.HasColumnType("bigint");
b.Property<string>("ErrorMessage")
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.Property<string>("Html")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int?>("HttpStatusCode")
.HasColumnType("int");
b.Property<string>("JobSearchSessionId")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<bool>("Success")
.HasColumnType("bit");
b.Property<string>("Text")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("Url")
.IsRequired()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.HasKey("Id");
b.HasIndex("CreatedAt");
b.HasIndex("JobSearchSessionId");
b.HasIndex("Url");
b.ToTable("PageFetches", "pageFetcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,43 @@
using PageFetcher.Data;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace PageFetcher.Data.Migrations
{
/// <inheritdoc />
public partial class AddJobSearchSessionId : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
type: "nvarchar(64)",
maxLength: 64,
nullable: true);
migrationBuilder.CreateIndex(
name: "IX_PageFetches_JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches",
column: "JobSearchSessionId");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropIndex(
name: "IX_PageFetches_JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches");
migrationBuilder.DropColumn(
name: "JobSearchSessionId",
schema: MigrationConstants.SchemaName,
table: "PageFetches");
}
}
}
@@ -0,0 +1,85 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using PageFetcher.Data;
#nullable disable
namespace PageFetcher.Data.Migrations
{
[DbContext(typeof(PageFetchDbContext))]
partial class PageFetchDbContextModelSnapshot : ModelSnapshot
{
protected override void BuildModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasDefaultSchema("pageFetcher")
.HasAnnotation("ProductVersion", "10.0.7")
.HasAnnotation("Relational:MaxIdentifierLength", 128);
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
modelBuilder.Entity("PageFetcher.Data.Entities.PageFetchEntity", b =>
{
b.Property<string>("Id")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<string>("CallerService")
.IsRequired()
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<DateTime>("CreatedAt")
.ValueGeneratedOnAdd()
.HasColumnType("datetime2")
.HasDefaultValueSql("SYSUTCDATETIME()");
b.Property<long>("DurationMs")
.HasColumnType("bigint");
b.Property<string>("ErrorMessage")
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.Property<string>("Html")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<int?>("HttpStatusCode")
.HasColumnType("int");
b.Property<string>("JobSearchSessionId")
.HasMaxLength(64)
.HasColumnType("nvarchar(64)");
b.Property<bool>("Success")
.HasColumnType("bit");
b.Property<string>("Text")
.IsRequired()
.HasColumnType("nvarchar(max)");
b.Property<string>("Url")
.IsRequired()
.HasMaxLength(2000)
.HasColumnType("nvarchar(2000)");
b.HasKey("Id");
b.HasIndex("CreatedAt");
b.HasIndex("JobSearchSessionId");
b.HasIndex("Url");
b.ToTable("PageFetches", "pageFetcher");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,48 @@
using Microsoft.EntityFrameworkCore;
using PageFetcher.Data.Entities;
namespace PageFetcher.Data;
/// <summary>
/// EF Core DbContext for the <c>pageFetcher</c> schema.
/// Owns the <c>PageFetches</c> audit table.
/// </summary>
public sealed class PageFetchDbContext : DbContext
{
public const string SchemaName = MigrationConstants.SchemaName;
public const string MigrationTableName = MigrationConstants.MigrationTableName;
public PageFetchDbContext(DbContextOptions<PageFetchDbContext> options) : base(options) { }
public DbSet<PageFetchEntity> PageFetches => Set<PageFetchEntity>();
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseSqlServer(x => x.MigrationsHistoryTable(MigrationTableName, SchemaName));
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasDefaultSchema(SchemaName);
modelBuilder.Entity<PageFetchEntity>(entity =>
{
entity.ToTable("PageFetches");
entity.HasKey(x => x.Id);
entity.Property(x => x.Id).HasMaxLength(64);
entity.Property(x => x.Url).HasMaxLength(2000).IsRequired();
entity.Property(x => x.CallerService).HasMaxLength(64).IsRequired();
entity.Property(x => x.Html).IsRequired();
entity.Property(x => x.Text).IsRequired();
entity.Property(x => x.ErrorMessage).HasMaxLength(2000);
entity.Property(x => x.JobSearchSessionId).HasMaxLength(64);
entity.Property(x => x.CreatedAt).HasDefaultValueSql("SYSUTCDATETIME()");
entity.HasIndex(x => x.JobSearchSessionId);
entity.HasIndex(x => x.Url);
entity.HasIndex(x => x.CreatedAt);
});
}
}
@@ -0,0 +1,19 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AssemblyName>page-fetcher-data</AssemblyName>
<RootNamespace>PageFetcher.Data</RootNamespace>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\shared-data\shared-data.csproj" />
</ItemGroup>
</Project>
+5 -1
View File
@@ -34,11 +34,15 @@ This applies to both the staging and production repos as appropriate.
- .NET 10, ASP.NET Core, Worker Service
- Entity Framework Core + SQL Server (multi-schema)
- Refit for typed HTTP clients between services
- Serilog (JSON structured logging, Console + File + Email sinks)
- Serilog — Compact JSON logs to stdout (+ optional email sink); see Observability
- MailKit for SMTP (used exclusively in `email-api`)
- Docker Compose for local and production deployment
- Watchtower for automatic container updates in production
## Observability (central stack on monitoring host 10.0.0.156)
- **Logs**: every service uses `ConfigureJsonSerilog(ServiceName, appVersion)` (startup-helpers) → Serilog **Compact JSON** to stdout, enriched `Application`/`Environment`/`AppVersion`. The host's Grafana **Alloy** agent ships stdout → **Loki**; view/query in Grafana. No file sink; optional email sink only if `SerilogEmail:*` is configured.
- **No app metrics/traces** — these are simple/minimal services, so (unlike easyDent) they don't expose Prometheus metrics or OTLP traces. Container/host metrics still come from the host's cAdvisor/node_exporter.
## Project taxonomy
| Category | Naming | Contains | EF dependency |
BIN
View File
Binary file not shown.
+2 -1
View File
@@ -18,11 +18,12 @@
<!-- Config -->
<PackageVersion Include="DotNetEnv" Version="3.2.0" />
<!-- HTTP / Refit -->
<PackageVersion Include="Refit.HttpClientFactory" Version="10.1.6" />
<PackageVersion Include="Refit.HttpClientFactory" Version="11.0.1" />
<!-- Serilog -->
<PackageVersion Include="Serilog.AspNetCore" Version="10.0.0" />
<PackageVersion Include="Serilog.Enrichers.Environment" Version="3.0.1" />
<PackageVersion Include="Serilog.Sinks.Console" Version="6.1.1" />
<PackageVersion Include="Serilog.Formatting.Compact" Version="3.0.0" />
<PackageVersion Include="Serilog.Sinks.Email" Version="4.2.1" />
<PackageVersion Include="Serilog.Sinks.File" Version="7.0.0" />
<!-- Swagger -->
+6 -8
View File
@@ -39,11 +39,10 @@ public static class StartupExtensions
.ReadFrom.Configuration(context.Configuration)
.ReadFrom.Services(services)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithEnvironmentName()
.Enrich.WithProperty("Service", serviceName)
.Enrich.WithProperty("Application", serviceName)
.Enrich.WithProperty("Environment", context.HostingEnvironment.EnvironmentName)
.Enrich.WithProperty("AppVersion", appVersion)
.WriteTo.Console(new Serilog.Formatting.Json.JsonFormatter());
.WriteTo.Console(new Serilog.Formatting.Compact.CompactJsonFormatter());
AddEmailSinkIfConfigured(configuration, context.Configuration, serviceName);
});
@@ -57,11 +56,10 @@ public static class StartupExtensions
.ReadFrom.Configuration(builder.Configuration)
.ReadFrom.Services(services)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithEnvironmentName()
.Enrich.WithProperty("Service", serviceName)
.Enrich.WithProperty("Application", serviceName)
.Enrich.WithProperty("Environment", builder.Environment.EnvironmentName)
.Enrich.WithProperty("AppVersion", appVersion)
.WriteTo.Console(new Serilog.Formatting.Json.JsonFormatter());
.WriteTo.Console(new Serilog.Formatting.Compact.CompactJsonFormatter());
AddEmailSinkIfConfigured(configuration, builder.Configuration, serviceName);
});
@@ -17,6 +17,7 @@
<PackageReference Include="DotNetEnv" />
<PackageReference Include="Serilog.AspNetCore" />
<PackageReference Include="Serilog.Enrichers.Environment" />
<PackageReference Include="Serilog.Formatting.Compact" />
<PackageReference Include="Serilog.Sinks.Email" />
<PackageReference Include="Serilog.Sinks.File" />
<PackageReference Include="Swashbuckle.AspNetCore" />
+2
View File
@@ -9,6 +9,7 @@ COPY Apis/cv-search-data/cv-search-data.csproj Apis/cv-search-data/
COPY Apis/cv-matcher-api-models/cv-matcher-api-models.csproj Apis/cv-matcher-api-models/
COPY Apis/email-data/email-data.csproj Apis/email-data/
COPY Apis/email-api-models/email-api-models.csproj Apis/email-api-models/
COPY Apis/page-fetcher-api-models/page-fetcher-api-models.csproj Apis/page-fetcher-api-models/
COPY Apis/common/common.csproj Apis/common/
COPY Apis/myai-data/myai-data.csproj Apis/myai-data/
COPY Apis/shared-data/shared-data.csproj Apis/shared-data/
@@ -22,6 +23,7 @@ COPY Apis/cv-search-data/ Apis/cv-search-data/
COPY Apis/cv-matcher-api-models/ Apis/cv-matcher-api-models/
COPY Apis/email-data/ Apis/email-data/
COPY Apis/email-api-models/ Apis/email-api-models/
COPY Apis/page-fetcher-api-models/ Apis/page-fetcher-api-models/
COPY Apis/common/ Apis/common/
COPY Apis/myai-data/ Apis/myai-data/
COPY Apis/shared-data/ Apis/shared-data/
+15 -2
View File
@@ -7,13 +7,14 @@ using Email.Data;
using Email.Data.Repositories;
using Email.Data.Repositories.Contracts;
using Email.Data.Services;
using EmailApi.Models.Clients;
using Email.Models.Clients;
using CvSearchJob.Tasks;
using JobScheduler.Scheduling;
using JobScheduler.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using PageFetcher.Models;
using Refit;
using Serilog;
using Common.Settings;
@@ -81,7 +82,19 @@ try
client.DefaultRequestHeaders.Add("X-Internal-Api-Key", key);
});
builder.Services.AddHttpClient<HtmlJobSearcher>();
builder.Services.AddRefitClient<IPageFetcherApiClient>()
.ConfigureHttpClient((sp, client) =>
{
var config = sp.GetRequiredService<Microsoft.Extensions.Configuration.IConfiguration>();
var baseUrl = config["PageFetcherApi:BaseUrl"] ?? string.Empty;
if (!string.IsNullOrWhiteSpace(baseUrl))
client.BaseAddress = new Uri(baseUrl.TrimEnd('/') + "/");
var key = config["PageFetcherApi:InternalApiKey"];
if (!string.IsNullOrWhiteSpace(key))
client.DefaultRequestHeaders.Add("X-Internal-Api-Key", key);
});
builder.Services.AddSingleton<HtmlJobSearcher>();
builder.Services.AddSingleton<CvSearchEmailSender>();
builder.Services.AddSingleton<CvSearchJobTask>();
@@ -1,8 +1,8 @@
using CvMatcher.Models.Responses;
using CvSearch.Data.Entities;
using Email.Data.Services;
using EmailApi.Models.Clients;
using EmailApi.Models.Requests;
using Email.Models.Clients;
using Email.Models.Requests;
using Microsoft.Extensions.Logging;
namespace CvSearchJob.Services;
@@ -46,6 +46,7 @@ public sealed class CvSearchEmailSender
IReadOnlyList<string> keywords,
IReadOnlyList<string> providerNames,
string language,
string? location,
CancellationToken ct)
{
var operatorCopy = _emailTemplates.GetOperatorCopy("email.search-results.subject", language);
@@ -58,7 +59,7 @@ public sealed class CvSearchEmailSender
if (recipients.Count == 0) return;
var htmlBody = BuildBody(results, keywords, providerNames, language);
var htmlBody = BuildBody(results, keywords, providerNames, language, location);
var subject = _emailTemplates.Render("email.search-results.subject", language,
("count", results.Count.ToString()));
@@ -87,9 +88,9 @@ public sealed class CvSearchEmailSender
/// Returns the empty-results template when no results are present.
/// Prepends a scan summary block showing the keywords and providers used.
/// </summary>
private string BuildBody(IReadOnlyList<JobSearchResultEntity> results, IReadOnlyList<string> keywords, IReadOnlyList<string> providerNames, string language)
private string BuildBody(IReadOnlyList<JobSearchResultEntity> results, IReadOnlyList<string> keywords, IReadOnlyList<string> providerNames, string language, string? location)
{
var scanSummary = BuildScanSummary(keywords, providerNames, language);
var scanSummary = BuildScanSummary(keywords, providerNames, language, location);
if (results.Count == 0)
return scanSummary + _emailTemplates.Get("email.search-results.empty", language);
@@ -121,20 +122,25 @@ public sealed class CvSearchEmailSender
/// Renders the scan summary block via template, passing keyword tags and provider list as data.
/// Keyword tags are built here because they are variable-count inline elements, not structural HTML.
/// </summary>
private string BuildScanSummary(IReadOnlyList<string> keywords, IReadOnlyList<string> providerNames, string language)
private string BuildScanSummary(IReadOnlyList<string> keywords, IReadOnlyList<string> providerNames, string language, string? location)
{
var keywordsHtml = keywords.Count > 0
? string.Join(" ", keywords.Select(k =>
$"<span style=\"display:inline-block;background:#e9ecef;padding:2px 8px;margin:2px 2px 2px 0;font-size:12px;\">{k}</span>"))
: "<span style=\"color:#6c757d;font-size:12px;font-style:italic;\">none detected</span>";
: $"<span style=\"color:#6c757d;font-size:12px;font-style:italic;\">{_emailTemplates.Get("email.search-results.keywords-empty", language)}</span>";
var providers = providerNames.Count > 0
? string.Join(", ", providerNames)
: "none";
: _emailTemplates.Get("email.search-results.providers-empty", language);
var locationDisplay = string.IsNullOrWhiteSpace(location)
? _emailTemplates.Get("email.search-results.location-empty", language)
: location;
return _emailTemplates.Render("email.search-results.scan-summary", language,
("keywordsHtml", keywordsHtml),
("providers", providers));
("providers", providers),
("location", locationDisplay));
}
/// <summary>
+35 -81
View File
@@ -1,36 +1,39 @@
using System.Text.RegularExpressions;
using System.Web;
using CvMatcher.Models.Settings;
using Microsoft.Playwright;
using PageFetcher.Models;
using Microsoft.Extensions.Logging;
namespace CvSearchJob.Services;
/// <summary>
/// Config-driven HTML scraper that fetches a provider's job listing page and extracts matching job URLs.
/// Uses a two-stage anchor filter: href must contain the provider's link pattern, and anchor text must
/// contain at least one CV keyword.
/// Supports both plain HTTP GET (default) and headless Chromium rendering for JS-heavy SPAs.
/// A URL and its anchor text as scraped from a job listing search-results page.
/// </summary>
public sealed record JobCandidate(string Url, string Title);
/// <summary>
/// Config-driven HTML scraper that fetches a provider's job listing page via <c>page-fetcher-api</c>
/// and extracts matching job URL candidates.
/// Uses a two-stage anchor filter: href must contain the provider's link pattern, and (optionally)
/// anchor text must contain at least one CV keyword.
/// </summary>
public sealed class HtmlJobSearcher
{
private readonly HttpClient _http;
private readonly IPageFetcherApiClient _pageFetcher;
private readonly ILogger<HtmlJobSearcher> _logger;
public HtmlJobSearcher(HttpClient http, ILogger<HtmlJobSearcher> logger)
public HtmlJobSearcher(IPageFetcherApiClient pageFetcher, ILogger<HtmlJobSearcher> logger)
{
_http = http;
_pageFetcher = pageFetcher;
_logger = logger;
_http.Timeout = TimeSpan.FromSeconds(20);
_http.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (compatible; MyAi.ro CV-Search/1.0)");
}
/// <summary>
/// Fetches the provider's search result page for the combined initial + CV keywords, parses all anchor
/// tags, applies the two-stage filter, and returns up to <see cref="JobProviderConfig.MaxResults"/> absolute URLs.
/// Returns an empty list when the HTTP request fails rather than throwing.
/// Fetches the provider's search result page, parses all anchor tags, applies the two-stage filter,
/// and returns up to <see cref="JobProviderConfig.MaxResults"/> candidates (URL + title).
/// Returns an empty list when the page fetch fails rather than throwing.
/// </summary>
public async Task<IReadOnlyList<string>> SearchJobUrlsAsync(
public async Task<IReadOnlyList<JobCandidate>> SearchJobUrlsAsync(
JobProviderConfig provider,
IReadOnlyList<string> cvKeywords,
string? location,
@@ -61,24 +64,28 @@ public sealed class HtmlJobSearcher
.Replace("{location-slug}", locationSlug);
_logger.LogInformation(
"Provider {Provider}: fetching {Url} [{Mode}] | CV keywords: [{Keywords}] | Location: {Location}",
"Provider {Provider}: fetching {Url} | CV keywords: [{Keywords}] | Location: {Location}",
provider.Name, searchUrl,
provider.UseHeadlessBrowser ? "headless" : "http",
string.Join(", ", cvKeywords),
location ?? "(none)");
string? html;
if (provider.UseHeadlessBrowser)
html = await FetchWithPlaywrightAsync(provider.Name, searchUrl, ct);
else
html = await FetchWithHttpAsync(provider.Name, searchUrl, ct);
var fetchResponse = await _pageFetcher.FetchAsync(new FetchPageRequest
{
Url = searchUrl,
CallerService = "cv-search-job"
}, ct);
if (html is null) return [];
if (!fetchResponse.Success || string.IsNullOrWhiteSpace(fetchResponse.Html))
{
_logger.LogWarning("Provider {Provider}: page fetch failed — {Error}", provider.Name, fetchResponse.Error);
return [];
}
var html = fetchResponse.Html;
_logger.LogInformation("Provider {Provider}: received {Length} chars of HTML", provider.Name, html.Length);
var baseUri = new Uri(searchUrl);
var results = new List<string>();
var results = new List<JobCandidate>();
var seen = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var anchorPattern = new Regex(@"<a[^>]+href=[""']([^""']+)[""'][^>]*>(.*?)</a>",
@@ -117,9 +124,13 @@ public sealed class HtmlJobSearcher
continue;
}
// Skip non-HTTP(S) URLs (e.g. file:// or javascript: that can appear in scraped HTML)
if (absoluteUri.Scheme != Uri.UriSchemeHttp && absoluteUri.Scheme != Uri.UriSchemeHttps)
continue;
var url = absoluteUri.GetLeftPart(UriPartial.Path);
if (seen.Add(url))
results.Add(url);
results.Add(new JobCandidate(url, anchorText));
}
_logger.LogInformation(
@@ -128,61 +139,4 @@ public sealed class HtmlJobSearcher
return results;
}
private async Task<string?> FetchWithHttpAsync(string providerName, string url, CancellationToken ct)
{
try
{
return await _http.GetStringAsync(url, ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Provider {Provider}: HTTP fetch failed for {Url}", providerName, url);
return null;
}
}
private async Task<string?> FetchWithPlaywrightAsync(string providerName, string url, CancellationToken ct)
{
try
{
using var playwright = await Playwright.CreateAsync();
await using var browser = await playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
{
Headless = true,
Args = ["--no-sandbox", "--disable-setuid-sandbox", "--disable-dev-shm-usage"]
});
var page = await browser.NewPageAsync();
IResponse? response;
try
{
response = await page.GotoAsync(url, new PageGotoOptions
{
WaitUntil = WaitUntilState.NetworkIdle,
Timeout = 30_000
});
}
catch (TimeoutException)
{
// NetworkIdle timed out — use whatever content rendered so far
_logger.LogWarning("Provider {Provider}: Playwright NetworkIdle timeout for {Url}, using partial content", providerName, url);
return await page.ContentAsync();
}
if (response is null || response.Status >= 400)
{
_logger.LogWarning("Provider {Provider}: Playwright got HTTP {Status} for {Url}", providerName, response?.Status, url);
return null;
}
return await page.ContentAsync();
}
catch (Exception ex)
{
_logger.LogError(ex, "Provider {Provider}: Playwright fetch failed for {Url}", providerName, url);
return null;
}
}
}
+52 -12
View File
@@ -11,6 +11,7 @@ using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using PageFetcher.Models;
namespace CvSearchJob.Tasks;
@@ -24,6 +25,7 @@ public sealed class CvSearchJobTask : IJobTask
private readonly JobSearchSettings _settings;
private readonly HtmlJobSearcher _searcher;
private readonly ICvMatcherInternalApi _matcherApi;
private readonly IPageFetcherApiClient _pageFetcher;
private readonly CvSearchEmailSender _emailSender;
private readonly ILogger<CvSearchJobTask> _logger;
@@ -34,6 +36,7 @@ public sealed class CvSearchJobTask : IJobTask
IOptions<JobSearchSettings> settings,
HtmlJobSearcher searcher,
ICvMatcherInternalApi matcherApi,
IPageFetcherApiClient pageFetcher,
CvSearchEmailSender emailSender,
ILogger<CvSearchJobTask> logger)
{
@@ -41,6 +44,7 @@ public sealed class CvSearchJobTask : IJobTask
_settings = settings.Value;
_searcher = searcher;
_matcherApi = matcherApi;
_pageFetcher = pageFetcher;
_emailSender = emailSender;
_logger = logger;
}
@@ -111,6 +115,7 @@ public sealed class CvSearchJobTask : IJobTask
cvKeywords,
providers.Select(p => p.Name).ToList(),
pending.Language,
pending.Location,
cancellationToken);
_logger.LogInformation("Session {SessionId} done. {Count} results sent.", pending.Id, results.Count);
@@ -125,7 +130,8 @@ public sealed class CvSearchJobTask : IJobTask
/// <summary>
/// Runs the full search pipeline for a session: scrapes all providers, deduplicates URLs,
/// scores each candidate via the matcher API, and persists results that meet the minimum score threshold.
/// fetches each individual job page via page-fetcher-api, applies a keyword pre-filter,
/// scores passing candidates via the matcher API, and persists results that meet the minimum score threshold.
/// </summary>
private async Task<List<JobSearchResultEntity>> RunSearchAsync(
JobSearchSessionEntity session,
@@ -137,32 +143,64 @@ public sealed class CvSearchJobTask : IJobTask
if (cvKeywords.Count == 0)
_logger.LogWarning("Session {SessionId}: keyword list is empty — scraper will rely on provider InitialKeywords only", session.Id);
var jobUrls = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var jobCandidates = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase); // url → title
foreach (var provider in providers)
{
var urls = await _searcher.SearchJobUrlsAsync(provider, cvKeywords, session.Location, ct);
_logger.LogInformation("Session {SessionId}: provider {Provider} returned {Count} URLs", session.Id, provider.Name, urls.Count);
foreach (var url in urls) jobUrls.Add(url);
var candidates = await _searcher.SearchJobUrlsAsync(provider, cvKeywords, session.Location, ct);
_logger.LogInformation("Session {SessionId}: provider {Provider} returned {Count} candidates", session.Id, provider.Name, candidates.Count);
foreach (var c in candidates)
jobCandidates.TryAdd(c.Url, c.Title);
}
var candidates = jobUrls.Take(_settings.MaxJobsToMatch).ToList();
var deduped = jobCandidates.Take(_settings.MaxJobsToMatch).ToList();
_logger.LogInformation(
"Session {SessionId}: {Total} unique URLs across all providers, scoring {Scoring} (cap={Cap})",
session.Id, jobUrls.Count, candidates.Count, _settings.MaxJobsToMatch);
"Session {SessionId}: {Total} unique URLs across all providers, processing up to {Cap}",
session.Id, jobCandidates.Count, deduped.Count);
var results = new List<JobSearchResultEntity>();
foreach (var url in candidates)
foreach (var (url, title) in deduped)
{
try
{
// Fetch individual job page text via page-fetcher-api
var fetchResponse = await _pageFetcher.FetchAsync(new FetchPageRequest
{
Url = url,
WaitFor = "domcontentloaded",
CallerService = "cv-search-job",
JobSearchSessionId = session.Id
}, ct);
if (!fetchResponse.Success || string.IsNullOrWhiteSpace(fetchResponse.Text))
{
_logger.LogWarning("Session {SessionId}: fetch failed for {Url} — {Error}", session.Id, url, fetchResponse.Error);
continue;
}
var jobText = fetchResponse.Text;
// Keyword pre-filter: skip LLM call if no CV keyword appears in the job page text
if (cvKeywords.Count > 0 &&
!cvKeywords.Any(k => jobText.Contains(k, StringComparison.OrdinalIgnoreCase)))
{
_logger.LogInformation(
"Session {SessionId}: pre-filter skip | {Url} | no CV keyword found in job text",
session.Id, url);
continue;
}
var matchRequest = new MatchJobRequest
{
CvDocumentId = session.CvDocumentId,
JobUrl = url,
// Pre-fetched text passed directly so cv-matcher-api skips re-fetching the page
JobDescription = jobText,
// User already gave GDPR consent when they clicked the one-time job search link
GdprConsent = true
GdprConsent = true,
// Propagate language so the LLM uses the correct language-specific prompt
Language = session.Language
};
var matchResult = await _matcherApi.MatchJobAsync(matchRequest, ct);
@@ -181,10 +219,12 @@ public sealed class CvSearchJobTask : IJobTask
SessionId = session.Id,
ProviderName = GuessProvider(url, providers),
JobUrl = url,
JobTitle = matchResult.Summary.Split('.').FirstOrDefault()?.Trim() ?? "Job",
JobText = string.Empty,
JobTitle = matchResult.Summary.Split('.').FirstOrDefault()?.Trim() ?? title,
JobText = jobText,
Score = matchResult.Score,
ResultJson = JsonSerializer.Serialize(matchResult, new JsonSerializerOptions(JsonSerializerDefaults.Web)),
Email = session.Email,
ClientIpAddress = session.ClientIpAddress,
CreatedAt = DateTime.UtcNow
};
+1 -1
View File
@@ -13,7 +13,6 @@
<PackageReference Include="Microsoft.Extensions.Hosting" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" />
<PackageReference Include="Refit.HttpClientFactory" />
<PackageReference Include="Microsoft.Playwright" />
</ItemGroup>
<ItemGroup>
@@ -26,6 +25,7 @@
<ProjectReference Include="..\..\Apis\cv-search-data\cv-search-data.csproj" />
<ProjectReference Include="..\..\Apis\common\common.csproj" />
<ProjectReference Include="..\..\Apis\email-data\email-data.csproj" />
<ProjectReference Include="..\..\Apis\page-fetcher-api-models\page-fetcher-api-models.csproj" />
<ProjectReference Include="..\..\Helpers\startup-helpers\startup-helpers.csproj" />
<ProjectReference Include="..\job-scheduler\job-scheduler.csproj" />
</ItemGroup>
+23 -16
View File
@@ -1,20 +1,27 @@
# Introduction
TODO: Give a short introduction of your project. Let this section explain the objectives or the motivation behind this project.
# myAi
# Getting Started
TODO: Guide users through getting your code up and running on their own system. In this section you can talk about:
1. Installation process
2. Software dependencies
3. Latest releases
4. API references
The **myai.ro** platform — a set of .NET microservices (CV matching, RAG, email, CV search, page
fetching, …) behind a web frontend + API. Part of the easySoft platform.
# Build and Test
TODO: Describe and show how to build your code and run the tests.
## Layout
Multiple services (`*-api`, `*-job`) + `web`, sharing a common bootstrap in
`startup-helpers/` (Serilog, Swagger, `.env`/Key Vault loading, middleware). See **CLAUDE.md**
for the full service map, dependency chain, and conventions.
# Contribute
TODO: Explain how other users and developers can contribute to make your code better.
## Run locally
```bash
docker compose up --build # or run individual services with: dotnet run --project <svc>
```
If you want to learn more about creating good readme files then refer the following [guidelines](https://docs.microsoft.com/en-us/azure/devops/repos/git/create-a-readme?view=azure-devops). You can also seek inspiration from the below readme files:
- [ASP.NET Core](https://github.com/aspnet/Home)
- [Visual Studio Code](https://github.com/Microsoft/vscode)
- [Chakra Core](https://github.com/Microsoft/ChakraCore)
## Deploy
CI builds `registry.easysoft.ro/apps/myai-*:{staging,production}`; Watchtower rolls them out to
the **staging (`10.0.0.183`)** + **production (`10.0.0.248`)** Portainer stacks. Edge Caddy serves
**myai.ro** (prod) / **myai.easysoft.ro** (staging).
## Logging
Every service: `ConfigureJsonSerilog(name, version)` → Serilog **Compact JSON** to stdout → Grafana
**Alloy****Loki**. No app metrics/traces (simple services).
---
See **CLAUDE.md** for the detailed solution guide and **ACCESS.md** (local, gitignored) for
infrastructure access.
+4 -2
View File
@@ -49,8 +49,10 @@ Ai__Ollama__ChatModel=llama2
Ai__Ollama__EmbeddingModel=embedding-model
Ai__Ollama__TimeoutSeconds=30
# Database (shared) - maps to Database:Host etc. used by apps
Database__Host=sqlserver
# Database (shared) - maps to Database:Host etc. used by apps.
# Deployed (staging/prod) uses the LAN DNS name (resolves to the MSSQL VM 10.0.0.240);
# for local dev leave it unset to use the docker-compose 'sqlserver' service default.
Database__Host=mssql.easysoft.ro
Database__Port=1433
Database__Name=MyAiDb
Database__User=sa
+38
View File
@@ -72,6 +72,9 @@ services:
- RagApi__BaseUrl=${RagApi__BaseUrl:-http://rag-api:8080}
- RagApi__InternalApiKey=${RagApi__InternalApiKey:-}
- PageFetcherApi__BaseUrl=${PageFetcherApi__BaseUrl:-http://page-fetcher-api:8080}
- PageFetcherApi__InternalApiKey=${PageFetcherApi__InternalApiKey:-}
- Ai__Provider=${Ai__Provider:-OpenAI}
- Ai__OpenAI__ApiKey=${Ai__OpenAI__ApiKey:-}
- Ai__OpenAI__ChatModel=${Ai__OpenAI__ChatModel:-gpt-4o-mini}
@@ -266,6 +269,9 @@ services:
- EmailApi__BaseUrl=${EmailApi__BaseUrl:-http://email-api:8080}
- EmailApi__InternalApiKey=${EmailApi__InternalApiKey:-}
- PageFetcherApi__BaseUrl=${PageFetcherApi__BaseUrl:-http://page-fetcher-api:8080}
- PageFetcherApi__InternalApiKey=${PageFetcherApi__InternalApiKey:-}
- FileStorage__Path=${FileStorage__Path:-Files}
- JobSearch__Enabled=${JobSearch__Enabled:-true}
@@ -293,6 +299,38 @@ services:
labels:
- "com.centurylinklabs.watchtower.enable=true"
page-fetcher-api:
image: registry.easysoft.ro/apps/myai-page-fetcher-api:${IMAGE_TAG:-staging}
container_name: myai-page-fetcher-api
environment:
- ASPNETCORE_ENVIRONMENT=${ASPNETCORE_ENVIRONMENT:-Staging}
- ASPNETCORE_URLS=${ASPNETCORE_URLS:-http://+:8080}
- APP_ENVIRONMENT_NAME=${APP_ENVIRONMENT_NAME:-myai.staging}
- Database__Host=${Database__Host:-sqlserver}
- Database__Port=${Database__Port:-1433}
- Database__Name=${Database__Name:-MyAiDb}
- Database__User=${Database__User:-sa}
- Database__Password=${Database__Password:-}
- Database__TrustServerCertificate=${Database__TrustServerCertificate:-true}
- InternalApi__ApiKey=${PageFetcherApi__InternalApiKey:-}
- InternalApi__RequireApiKey=true
- SerilogEmail__From=${SerilogEmail__From:-}
- SerilogEmail__To=${SerilogEmail__To:-}
- SerilogEmail__Host=${SerilogEmail__Host:-}
- SerilogEmail__Port=${SerilogEmail__Port:-587}
- SerilogEmail__UserName=${SerilogEmail__UserName:-}
- SerilogEmail__Password=${SerilogEmail__Password:-}
volumes:
- ${LOGS_PATH:-/opt/myai/logs}/page-fetcher-api:/app/logs
networks:
- myai-network
restart: unless-stopped
labels:
- "com.centurylinklabs.watchtower.enable=true"
web:
image: registry.easysoft.ro/apps/myai-web:${IMAGE_TAG:-staging}
container_name: myai-web
+45
View File
@@ -63,6 +63,12 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "email-api", "Apis\email-api
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "email-data", "Apis\email-data\email-data.csproj", "{C1D2E3F4-A5B6-4789-CDEF-012345678ABC}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "page-fetcher-api-models", "Apis\page-fetcher-api-models\page-fetcher-api-models.csproj", "{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "page-fetcher-data", "Apis\page-fetcher-data\page-fetcher-data.csproj", "{06F803CD-329D-40C2-B62D-0F14E137D3C7}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "page-fetcher-api", "Apis\page-fetcher-api\page-fetcher-api.csproj", "{FC5A722A-7B12-459E-AB9F-0A724797783E}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@@ -357,6 +363,42 @@ Global
{C1D2E3F4-A5B6-4789-CDEF-012345678ABC}.Release|x64.Build.0 = Release|Any CPU
{C1D2E3F4-A5B6-4789-CDEF-012345678ABC}.Release|x86.ActiveCfg = Release|Any CPU
{C1D2E3F4-A5B6-4789-CDEF-012345678ABC}.Release|x86.Build.0 = Release|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Debug|Any CPU.Build.0 = Debug|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Debug|x64.ActiveCfg = Debug|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Debug|x64.Build.0 = Debug|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Debug|x86.ActiveCfg = Debug|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Debug|x86.Build.0 = Debug|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Release|Any CPU.ActiveCfg = Release|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Release|Any CPU.Build.0 = Release|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Release|x64.ActiveCfg = Release|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Release|x64.Build.0 = Release|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Release|x86.ActiveCfg = Release|Any CPU
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B}.Release|x86.Build.0 = Release|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Debug|Any CPU.Build.0 = Debug|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Debug|x64.ActiveCfg = Debug|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Debug|x64.Build.0 = Debug|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Debug|x86.ActiveCfg = Debug|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Debug|x86.Build.0 = Debug|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Release|Any CPU.ActiveCfg = Release|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Release|Any CPU.Build.0 = Release|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Release|x64.ActiveCfg = Release|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Release|x64.Build.0 = Release|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Release|x86.ActiveCfg = Release|Any CPU
{06F803CD-329D-40C2-B62D-0F14E137D3C7}.Release|x86.Build.0 = Release|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Debug|x64.ActiveCfg = Debug|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Debug|x64.Build.0 = Debug|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Debug|x86.ActiveCfg = Debug|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Debug|x86.Build.0 = Debug|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Release|Any CPU.Build.0 = Release|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Release|x64.ActiveCfg = Release|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Release|x64.Build.0 = Release|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Release|x86.ActiveCfg = Release|Any CPU
{FC5A722A-7B12-459E-AB9F-0A724797783E}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
@@ -385,6 +427,9 @@ Global
{BE44B4EB-9AB9-4D81-A9BF-5CF2832BEEE5} = {A9B8C7D6-E5F4-4321-ABCD-FEDCBA987654}
{434119EA-2FFC-4433-9B8E-1E6D94006413} = {0FE6558F-2157-47F2-A835-558416CE0E2B}
{C1D2E3F4-A5B6-4789-CDEF-012345678ABC} = {D4E5F6A7-B8C9-4012-3456-789ABCDEF012}
{4F1A669E-C8AF-428F-87E7-3E0A213DD20B} = {A9B8C7D6-E5F4-4321-ABCD-FEDCBA987654}
{06F803CD-329D-40C2-B62D-0F14E137D3C7} = {D4E5F6A7-B8C9-4012-3456-789ABCDEF012}
{FC5A722A-7B12-459E-AB9F-0A724797783E} = {0FE6558F-2157-47F2-A835-558416CE0E2B}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {6246A67B-299E-4E64-8DBE-1A66771E7C67}
+8 -2
View File
@@ -115,7 +115,10 @@
"cv.noItems": "No items yet.",
"cv.strengths": "Strengths",
"cv.gaps": "Gaps",
"cv.evidence": "Supporting CV excerpts"
"cv.evidence": "Supporting CV excerpts",
"error.cv_file_missing": "Missing CV PDF.",
"error.captcha_verification_failed": "Captcha verification failed.",
"error.request_cancelled": "Request was cancelled."
},
ro: {
"brand.subtitle": "prezentare inginerie AI",
@@ -224,7 +227,10 @@
"cv.noItems": "Niciun element.",
"cv.strengths": "Puncte forte",
"cv.gaps": "Lipsuri",
"cv.evidence": "Fragmente relevante din CV"
"cv.evidence": "Fragmente relevante din CV",
"error.cv_file_missing": "Fișierul CV PDF lipsește.",
"error.captcha_verification_failed": "Verificarea captcha a eșuat.",
"error.request_cancelled": "Cererea a fost anulată."
}
};
})();
+12 -2
View File
@@ -52,6 +52,7 @@ function isValidEmail(value) {
*
* Rules:
* - 429 (rate limit) → return rateLimitKey translation
* - 4xx with known error code → look up 'error.<code>' in i18n dictionary first
* - 4xx with error body → return server's error message (intentional feedback)
* - 5xx or no body → return fallbackKey translation
*
@@ -65,8 +66,17 @@ function extractApiError(body, status, fallbackKey, rateLimitKey) {
if (status === 429) {
return window.MyAi.t(rateLimitKey || 'form.rateLimited');
}
var msg = body && (body.error || body.Error || body.title);
return (status >= 400 && status < 500 && msg) ? msg : window.MyAi.t(fallbackKey);
if (status >= 400 && status < 500) {
// Prefer i18n translation keyed on the machine-readable error code
if (body && body.code) {
var codeKey = 'error.' + body.code;
var translated = window.MyAi.t(codeKey);
if (translated !== codeKey) return translated;
}
var msg = body && (body.error || body.Error || body.title);
if (msg) return msg;
}
return window.MyAi.t(fallbackKey);
}
// Expose helpers on window.MyAi for use by other scripts