1) Immediate startup attendance run
2) Shared real fetch cycle path with skip-reason logs
3) last_sync_date read/write from DB
4) Preserve cursor on status updates
5) IP-based machine matching
6) zero-record cursor advance to window end
7) CSV write lock/file-sharing fix
This change set turns the service into a configurable, operations-friendly pipeline with independent jobs for attendance sync, template fetching, and user sync, while hardening data safety and observability for DS-K1T642MFW deployments.
What was implemented:
1) ISAPI-first user info + face URL handling
- Added direct HTTP ISAPI user search with Digest auth:
- POST /ISAPI/AccessControl/UserInfo/Search?format=json
- Added source switch to prefer HTTP user info while keeping SDK/STDXML fallback.
- Added face image download via ISAPI using configured device IP even when faceURL host differs.
- Preserved existing HCNetSDK functionality (no regression/removal).
2) Multi-device user sync orchestration
- Added sync config model and policies:
- EnableUserSync, SyncIntervalMinutes, SourceDeviceId, TargetDeviceIds
- UserSyncFaceCacheDirectory, SyncPolicies (update fields, upload policy, delete policy, retries, face lib settings)
- Implemented RunUserFaceSyncCycle with focused helpers:
- user search (paged), compare, create/modify/delete users, upload faces, verify user existence
- Added retries with exponential backoff for transient HTTP failures.
- Added status tracking enum and cycle summaries.
- Added one-shot CLI/test mode trigger:
- --user-sync-once
- Scheduler now runs first cycle immediately on startup, then at configured interval.
3) Scheduled template fetch job (independent)
- Added EnableTemplateFetch + TemplateFetchIntervalHours.
- Implemented periodic template export scheduler loop.
- Runs per device session and writes output under date/device directories.
- Scheduler now runs first cycle immediately on startup, then at configured interval.
4) Attendance job controls + dynamic historical fetch pagination
- Added EnableAttendanceSync + AttendanceSyncIntervalMinutes.
- Attendance scheduler now respects dedicated attendance flags/intervals.
- Refactored STDXML ISAPI ACS fetch to support pagination loop with:
- responseStatusStrg handling (MORE/END)
- dynamic batch size growth (30 -> 50 -> 100)
- searchResultPosition advancement by actual returned rows
- loop safety for empty-page anomalies
5) Safe cleanup of old device ACS records after confirmed DB insert
- Implemented device cleanup call:
- PUT /ISAPI/AccessControl/AcsEvent/StorageCfg?format=json
- mode=time, checkTime=last synced event time
- Cleanup executes only when:
- historical fetch parsed records
- DB persistence enabled
- all parsed records inserted successfully
- If DB insert is incomplete/failed:
- cleanup is skipped
- last-sync is not advanced
- next cycle retries same window
- Added explicit skip log:
- “Cleanup skipped: DB insert not confirmed; delete not executed.”
6) Logging architecture overhaul
- Structured logs into daily files per concern:
- internal deep logs
- attendance_logs
- template_fetching_logs
- user_sync_logs
- Moved internal root to:
- C:\Users\Public\HikvisionAttendanceService\internal_logs
(no longer under logs/internal_logs)
- Added job cycle banners:
- SERVICE STARTED / JOB / started at
- SERVICE ENDED / finished at
- Added attendance-specific operational lines:
- MACHINE <name> has <n> records !
- [AttendanceInsert] machine_id= ip= emp_no= checktime= rows=
- Enhanced SDK failure logs to multi-line readable format with:
- operation, device context, translated reason, original call+error code
- preserved raw error context for troubleshooting
7) Template logging improvements (operational + internal split)
- Main template job log now includes full post-run summary:
- totals, fetched vs not fetched, per-user lists for face and fingerprint
- Moved detailed per-user template status to internal debug location:
- C:\Users\Public\HikvisionAttendanceService\internal_logs\template_internal_logs\face
- C:\Users\Public\HikvisionAttendanceService\internal_logs\template_internal_logs\finger
- Added headings in internal files and “no templates to fetch” cases.
8) Config/path standardization for production
- Standardized runtime/log output root to:
- C:\Users\Public\HikvisionAttendanceService
- Updated defaults and active serviceconfig fields accordingly
(LogDirectory, AttendanceTextFilePath, HrExportPath, face cache path).
- Ensured last-sync markers and CSV/text outputs follow this root.
9) Test and build coverage
- Added/updated unit tests for:
- user JSON parsing
- retry helper/backoff behavior
- dynamic historical batch thresholds
- Added integration placeholder test for hardware runs.
- Verified successful build/test runs after each major refactor.
Outcome:
- Service now supports independent, flag-driven operations for attendance, templates, and user sync.
- Logging is production-readable and audit-friendly.
- Historical sync is safer and more scalable.
- Device cleanup is guarded by DB-success semantics to prevent data loss.
SYED MUSTUFA AHMED NAQVI