Initial department sync now requires InitialSyncLocationSiteId and can
run across all active departments when department IDs are empty. Employee
selection filters by location site, reports inactive/missing departments,
and writes per-department outcomes to a dedicated DepartmentalSync biz log.
Add PowerShell harnesses to exercise EmployeePhotoFaceProcessor against
real HRMS portal photos without touching a device: run_photo_processor
processes given employee ids and writes source/processed JPEGs, and
run_photo_soak repeatedly processes real and synthetic images in one
process to check stability and memory. Both target 32-bit PowerShell to
match the service's x86 build.
Wire the initial department sync to the new photo processor: downloaded
JPEGs are face-cropped/normalized before FaceDataRecord, and photos with
no detectable face are skipped and logged as FACE_NOT_DETECTED.
Add an optional InitialSyncEmployeeIds allow-list so only selected
employees within the chosen departments are enrolled (empty = whole
department), filtered directly in the SQL query with parameters.
Emit business entries to logs/user_sync_logs for provisioned users
("added successfully", "face enrolled successfully", readable failure
reasons) so onboarded users appear alongside the existing deletion
entries instead of only in internal_logs.
Add EmployeePhotoFaceProcessor: a managed pipeline that detects the
largest face in an HRMS portal photo, crops it with margin in a 3:4
frame, normalizes to Hikvision's 480x640 enrollment size (capped
upscale), applies percentile contrast/gamma correction plus an unsharp
mask, and encodes JPEG within a size budget.
Face detection uses Accord.Vision's managed Haar cascade (added as a
package reference) so it works on the existing net48/x86 target with no
native binaries. Returns an explicit NoFace status so callers can skip
enrollment for photos without a detectable face.
Add initial department-sync and employee-photo source settings, plus department-based user-sync filters. Load, normalize, and apply the new configuration values.
Add a configurable initial-sync flow that creates department employees on target Hikvision devices and uploads portal JPEG photos.
Support department-based employee filtering for existing user/template sync.
Discover and retry available Hikvision face libraries, then verify face enrollment after upload.
Extend service configuration and scheduled sync handling for the new onboarding flow.
Business-log “Saved successfully” only when a template is newly inserted; skip already-present templates in biz logs. Pass hikvision-service as updatedBy on user upserts.
Show machine context in user-sync logs and clear is_deleted after device add
Include Machine ID/IP on add/update/remove business logs. After a successful device user create, upsert the DB row so is_deleted returns to 0 with updated_by set.
After loading serviceconfig.json, apply DeviceSettings.config values onto the existing config properties without changing runtime behavior or property names.
Extend FileLogger with Biz/Ops/Diag writers and service start/stop blocks; add EnableTemplateDeviceToDbSync / EnableTemplateDbToDeviceSync so only the enabled template direction is logged and run.
1) Immediate startup attendance run
2) Shared real fetch cycle path with skip-reason logs
3) last_sync_date read/write from DB
4) Preserve cursor on status updates
5) IP-based machine matching
6) zero-record cursor advance to window end
7) CSV write lock/file-sharing fix
This change set turns the service into a configurable, operations-friendly pipeline with independent jobs for attendance sync, template fetching, and user sync, while hardening data safety and observability for DS-K1T642MFW deployments.
What was implemented:
1) ISAPI-first user info + face URL handling
- Added direct HTTP ISAPI user search with Digest auth:
- POST /ISAPI/AccessControl/UserInfo/Search?format=json
- Added source switch to prefer HTTP user info while keeping SDK/STDXML fallback.
- Added face image download via ISAPI using configured device IP even when faceURL host differs.
- Preserved existing HCNetSDK functionality (no regression/removal).
2) Multi-device user sync orchestration
- Added sync config model and policies:
- EnableUserSync, SyncIntervalMinutes, SourceDeviceId, TargetDeviceIds
- UserSyncFaceCacheDirectory, SyncPolicies (update fields, upload policy, delete policy, retries, face lib settings)
- Implemented RunUserFaceSyncCycle with focused helpers:
- user search (paged), compare, create/modify/delete users, upload faces, verify user existence
- Added retries with exponential backoff for transient HTTP failures.
- Added status tracking enum and cycle summaries.
- Added one-shot CLI/test mode trigger:
- --user-sync-once
- Scheduler now runs first cycle immediately on startup, then at configured interval.
3) Scheduled template fetch job (independent)
- Added EnableTemplateFetch + TemplateFetchIntervalHours.
- Implemented periodic template export scheduler loop.
- Runs per device session and writes output under date/device directories.
- Scheduler now runs first cycle immediately on startup, then at configured interval.
4) Attendance job controls + dynamic historical fetch pagination
- Added EnableAttendanceSync + AttendanceSyncIntervalMinutes.
- Attendance scheduler now respects dedicated attendance flags/intervals.
- Refactored STDXML ISAPI ACS fetch to support pagination loop with:
- responseStatusStrg handling (MORE/END)
- dynamic batch size growth (30 -> 50 -> 100)
- searchResultPosition advancement by actual returned rows
- loop safety for empty-page anomalies
5) Safe cleanup of old device ACS records after confirmed DB insert
- Implemented device cleanup call:
- PUT /ISAPI/AccessControl/AcsEvent/StorageCfg?format=json
- mode=time, checkTime=last synced event time
- Cleanup executes only when:
- historical fetch parsed records
- DB persistence enabled
- all parsed records inserted successfully
- If DB insert is incomplete/failed:
- cleanup is skipped
- last-sync is not advanced
- next cycle retries same window
- Added explicit skip log:
- “Cleanup skipped: DB insert not confirmed; delete not executed.”
6) Logging architecture overhaul
- Structured logs into daily files per concern:
- internal deep logs
- attendance_logs
- template_fetching_logs
- user_sync_logs
- Moved internal root to:
- C:\Users\Public\HikvisionAttendanceService\internal_logs
(no longer under logs/internal_logs)
- Added job cycle banners:
- SERVICE STARTED / JOB / started at
- SERVICE ENDED / finished at
- Added attendance-specific operational lines:
- MACHINE <name> has <n> records !
- [AttendanceInsert] machine_id= ip= emp_no= checktime= rows=
- Enhanced SDK failure logs to multi-line readable format with:
- operation, device context, translated reason, original call+error code
- preserved raw error context for troubleshooting
7) Template logging improvements (operational + internal split)
- Main template job log now includes full post-run summary:
- totals, fetched vs not fetched, per-user lists for face and fingerprint
- Moved detailed per-user template status to internal debug location:
- C:\Users\Public\HikvisionAttendanceService\internal_logs\template_internal_logs\face
- C:\Users\Public\HikvisionAttendanceService\internal_logs\template_internal_logs\finger
- Added headings in internal files and “no templates to fetch” cases.
8) Config/path standardization for production
- Standardized runtime/log output root to:
- C:\Users\Public\HikvisionAttendanceService
- Updated defaults and active serviceconfig fields accordingly
(LogDirectory, AttendanceTextFilePath, HrExportPath, face cache path).
- Ensured last-sync markers and CSV/text outputs follow this root.
9) Test and build coverage
- Added/updated unit tests for:
- user JSON parsing
- retry helper/backoff behavior
- dynamic historical batch thresholds
- Added integration placeholder test for hardware runs.
- Verified successful build/test runs after each major refactor.
Outcome:
- Service now supports independent, flag-driven operations for attendance, templates, and user sync.
- Logging is production-readable and audit-friendly.
- Historical sync is safer and more scalable.
- Device cleanup is guarded by DB-success semantics to prevent data loss.
