feat: add page-fetcher-api — centralised Playwright page fetcher
Introduces page-fetcher-api, a new internal ASP.NET Core service that centralises all web-page fetching through a single Playwright (headless Chromium) browser instance. All fetches are persisted to the pageFetcher SQL schema for auditing. New projects: - Apis/page-fetcher-api-models: FetchPageRequest, FetchPageResponse, IPageFetcherApiClient - Apis/page-fetcher-data: PageFetchDbContext, PageFetchEntity, InitialSchema migration (schema: pageFetcher) - Apis/page-fetcher-api: PlaywrightBrowserService (singleton), PageFetcherService, PageController Changes to existing services: - cv-matcher-api: JobTextExtractor now calls IPageFetcherApiClient instead of HttpClient - cv-search-job: HtmlJobSearcher uses IPageFetcherApiClient (removes inline Playwright); CvSearchJobTask fetches individual job pages and applies keyword pre-filter before LLM call; passes pre-fetched JobDescription to cv-matcher-api to skip re-fetch - common: add PageFetcherApiSettings - docker-compose.yml, build.yml: add new service + env vars for callers Closes #43 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -15,6 +15,7 @@ env:
|
||||
WEB_IMAGE: apps/myai-web
|
||||
CV_CLEANUP_JOB_IMAGE: apps/myai-cv-cleanup-job
|
||||
CV_SEARCH_JOB_IMAGE: apps/myai-cv-search-job
|
||||
PAGE_FETCHER_API_IMAGE: apps/myai-page-fetcher-api
|
||||
IMAGE_TAG: staging
|
||||
|
||||
jobs:
|
||||
@@ -62,6 +63,10 @@ jobs:
|
||||
run: |
|
||||
docker build -f Jobs/cv-search-job/Dockerfile -t "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}" .
|
||||
|
||||
- name: Build Page Fetcher API image
|
||||
run: |
|
||||
docker build -f Apis/page-fetcher-api/Dockerfile -t "${REGISTRY_HOST}/${PAGE_FETCHER_API_IMAGE}:${IMAGE_TAG}" .
|
||||
|
||||
- name: Push API image
|
||||
run: |
|
||||
docker push "${REGISTRY_HOST}/${API_IMAGE}:${IMAGE_TAG}"
|
||||
@@ -88,4 +93,8 @@ jobs:
|
||||
|
||||
- name: Push CV search job image
|
||||
run: |
|
||||
docker push "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}"
|
||||
docker push "${REGISTRY_HOST}/${CV_SEARCH_JOB_IMAGE}:${IMAGE_TAG}"
|
||||
|
||||
- name: Push Page Fetcher API image
|
||||
run: |
|
||||
docker push "${REGISTRY_HOST}/${PAGE_FETCHER_API_IMAGE}:${IMAGE_TAG}"
|
||||
Reference in New Issue
Block a user