Feature #1398 » redmine-1398-backend-implementation-plan.md
Redmine #1398 Backend Implementation Plan
Status: Awaiting Ken approval
Date: 2026-07-31
Requirement baseline: Redmine #1397 attachments #1102 and #1103
Scope:
java/ems_branch,java/ems_hq, schema patch, automated tests, AI knowledge base, and E2E catalog. No React source changes.
1. Goal
Implement the approved DriverState, long-term off-peak charging, semantic command API, and persisted RemoteStart/RemoteStop lifecycle across Branch and HQ.
The implementation must preserve these boundaries:
- Branch owns business state, authorization, operation lifecycle, queue source, and off-peak scheduling.
- HQ derives the trusted Branch user identity, forwards commands, and filters targeted WebSocket results.
- Shared DriverState is connector-wide; authorization remains authoritative in the Branch command API.
- Terminal operation results are transient events and are not replayed in WebSocket initial snapshots.
- Existing Branch admin
/api/tx/charge/startand/api/tx/charge/stopremain becausereact/branchstill uses them. - Old HQ LIFF
/api/connector/start,/stop,/dequeue, and/offpeak-switchare removed after the coordinated #1398/#1399 deployment.
2. Database And Migration
Create:
java/ems_branch/patch/patch_20260731_redmine_1398_driver_state_operations.sql
2.1 e_remote_transaction_log
Keep the existing command-level status column for OCPP request/response compatibility. Add a separate business lifecycle:
| Column | Type | Purpose |
|---|---|---|
source |
VARCHAR(20) |
MANUAL or OFF_PEAK
|
requested_by_user_id |
VARCHAR(64) |
Authenticated Branch user who caused the operation |
operation_status |
VARCHAR(24) |
SENT, ACCEPTED, STARTED, STARTED_LATE, REJECTED, TIMED_OUT, CANCELLED, FAILED, STOPPED
|
accepted_at |
DATETIME |
Time of accepted OCPP CALLRESULT |
deadline_at |
DATETIME |
Business deadline |
grace_until |
DATETIME |
RemoteStart reconciliation grace end; null for RemoteStop |
completed_at |
DATETIME |
Terminal lifecycle timestamp |
failure_code |
VARCHAR(64) |
Stable failure/reason code |
Use the existing row ID as public operationId. Continue using ocpp_unique_id only for OCPP correlation and fill the existing transaction_id after StartTransaction correlation.
Add ordinary indexes for:
- Connector + operation type + lifecycle status + request time.
- Transaction + operation type + lifecycle status.
- Lifecycle status + deadline/grace scan.
Do not add UNIQUE, FOREIGN KEY, CHECK, or cascade constraints. The command service serializes connector operations with a pessimistic connector row lock and verifies that no applicable active operation exists before insert.
Existing rows keep the new lifecycle fields null and remain historical only.
2.2 e_connector_priority
Add:
| Column | Type | Purpose |
|---|---|---|
source |
VARCHAR(20) |
MANUAL or OFF_PEAK
|
requested_by_user_id |
VARCHAR(64) |
Original driver whose IdTag must be used when dispatching |
Add ordinary indexes for connector + source and charge group + lifecycle status + priority.
Legacy rows with null source are not treated as authoritative LIFF queue requests. Existing runtime reconciliation may classify an unambiguous row; ambiguous stale rows are ignored or cleaned without dispatching under an invented user.
2.3 e_connectors
Add:
| Column | Type | Purpose |
|---|---|---|
off_peak_enabled_by_user_id |
VARCHAR(64) |
Driver who enabled the long-term preference and receives scheduler operation results |
off_peak_suppressed_window_start |
DATETIME |
Start of the suppressed off-peak window |
off_peak_suppressed_window_end |
DATETIME |
End of the suppressed off-peak window |
off_peak_suppression_reason |
VARCHAR(64) |
Rejected, transaction ended, or driver stopped for this window |
Window start and end are persisted instead of a process-memory flag so restart does not cause a repeated RemoteStart in the same window.
Migration policy for existing enable_off_peak=true rows:
- If exactly one currently linked Branch user exists, backfill that user as
off_peak_enabled_by_user_id. - If multiple or zero linked users exist, preserve
enable_off_peak=truebut leave actor null. Scheduler pauses dispatch and DriverState exposes a reconfirmation-required reason until a legal user performs OFF then ON. - Never guess the enabled actor from set ordering.
2.4 API Registration
- Add Branch SS3A API entry for
POST /api/cp/hq/connector/commands. - Disable the old Branch HQ API entries for
/cp/hq/connector/offpeak-switchand/cp/hq/connector/dequeue. - Keep
A1401/A1402and/api/tx/charge/start|stopfor the Branch admin UI.
3. Domain Model
Existing files to modify
java/ems_branch/src/main/java/com/sylksoft/ems/branch/model/RemoteTransactionLog.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/model/ConnectorPriority.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/model/Connector.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/model/enums/ConnectorType.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/model/enums/OffPeakStatusReason.java
New enum/converter files
model/enums/ConnectorCommandAction.javamodel/enums/ChargingRequestSource.javamodel/enums/RemoteOperationStatus.javamodel/enums/OffPeakSuppressionReason.java- Matching JPA converters under
model/converter/.
Align Branch's duplicate ConnectorType.maxWaitTime values with the unchanged Charge Point values:
- Fortune: 30 seconds.
- CloudLink/Tesla/default: 10 minutes.
- Unknown: 24 hours.
No java/charge_point change is planned.
4. DriverState Contract
Existing files to modify
java/ems_branch/src/main/java/com/sylksoft/ems/branch/model/dto/DriverConnectorPresentationDto.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/service/ConnectorDriverPresentationService.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/api/cp/ConnectorForHqService.java
New DTO files
model/dto/ConnectorActionDto.javamodel/dto/ActiveConnectorOperationDto.javamodel/dto/ConnectorOperationResultDto.java
DriverConnectorPresentationDto will contain:
codetitledescriptionreasonactions[]
Each action contains:
codeenabledlabeldisabledMessagerequiresConfirmation
Remove driverState.start and driverState.stop; no compatibility fields.
activeOperation and operationResult remain siblings of driverState, not fields inside it:
- Initial REST/WS snapshot may include only unresolved
activeOperation. - MQ update may include a terminal
operationResultonly during the transition event. - Initial snapshot never queries or replays terminal results.
State precedence
ConnectorDriverPresentationService evaluates persisted evidence in this order:
- Active transaction + offline ->
OFFLINE_CHARGING. - Active STOP operation ->
FINISHING. - Active transaction ->
CHARGING. - Active START operation ->
PREPARING. - Offline without transaction ->
OFFLINE_UNAVAILABLE. - Faulted ->
FAULTED. - Applicable queue row ->
QUEUED. - Long-term off-peak enabled without applicable queue ->
OFF_PEAK_WAITING. - Raw
AVAILABLEor vendor-only rawPREPARINGwithout an operation ->READY. - Otherwise
UNAVAILABLEorUNKNOWN.
Internal queue states such as BLOCKED, PAUSED, NOT_PLUGGED, and ELIGIBLE become reason values and never public DriverState codes.
5. Semantic Command API
Branch
Modify:
java/ems_branch/src/main/java/com/sylksoft/ems/branch/api/cp/ConnectorHqController.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/repository/ConnectorRepository.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/api/error/ErrorCode.javajava/ems_branch/src/main/java/com/sylksoft/ems/branch/api/error/ErrorMessageResolver.java
Add:
api/cp/ConnectorCommandService.javaapi/cp/dto/ConnectorCommandRequest.javaapi/cp/dto/ConnectorCommandResponse.javaapi/cp/dto/ConnectorCommandErrorResponse.java
Endpoint:
POST /api/cp/hq/connector/commands
Internal request:
{
"connectorId": "CP001-001",
"action": "START_CHARGING",
"userId": "U2607310001"
}
The endpoint performs latest-state authorization and guard checks under a connector row lock.
Response rules:
- Immediate RemoteStart/RemoteStop dispatch:
202 Accepted,operationId,operationStatus=SENT, latest DriverState. -
START_CHARGINGresulting in a MANUAL queue without OCPP dispatch:200 OK, no operationId, latestQUEUEDstate. - Cancellation and off-peak preference commands:
200 OK, no operationId, latest DriverState. - Authorization failure: structured
403. - State/configuration conflict: structured
409. - Error response includes stable error code/message, actual
enableOffPeak, and latest DriverState.
Authoritative rules include:
- PUBLIC stop: transaction owner only.
- PRIVATE stop: any currently linked resident; preserve transaction owner and persist actual requester in the STOP operation.
- PRIVATE cancellation/off-peak disable: any currently linked resident.
- Off-peak enable: PRIVATE only, valid Charge Group, tariff/off-peak range, IdTag, and user relation.
- PREPARING/CHARGING/FINISHING prevents off-peak switch changes.
- No overlapping active START for one connector or active STOP for one transaction.
HQ
Modify:
java/ems_hq/src/main/java/com/sylksoft/ems/hq/api/connector/ConnectorController.javajava/ems_hq/src/main/java/com/sylksoft/ems/hq/api/connector/ConnectorService.javajava/ems_hq/src/main/java/com/sylksoft/ems/hq/api/connector/ConnectorBranchApiHandler.java
Add:
java/ems_hq/src/main/java/com/sylksoft/ems/hq/api/connector/dto/ConnectorCommandRequest.javajava/ems_hq/src/main/java/com/sylksoft/ems/hq/api/connector/dto/ConnectorCommandResponse.java
Public endpoint:
POST /api/connector/commands
HQ accepts only buildingId, connectorId, and semantic action. It resolves UserBuilding.branchUserId from the LINE session and never accepts userId from LIFF.
HQ preserves Branch HTTP status and structured body. It does not convert Branch 403/409 into HTTP 200 or overwrite business failure as success.
The existing Branch connector-info call used by HQ WebSocket initial fetch is changed to require the Branch user ID and verify that the user is legally related to the requested connector. A building membership alone is not enough to subscribe to an unrelated PRIVATE connector.
Remove these HQ methods after the new command endpoint is present:
/api/connector/start/api/connector/stop/api/connector/dequeue/api/connector/offpeak-switch
Remove the old Branch HQ-only off-peak/dequeue controller and service methods. Keep Branch admin transaction endpoints unchanged.
6. Remote Operation Lifecycle
Files to modify
service/charging/RemoteChargingSupport.javaservice/charging/ChargingOrchestrator.javaservice/RemoteTransactionLogService.javarepository/RemoteTransactionLogRepository.javaocpp/handler/RemoteStartTransactionResponseHandler.javaocpp/handler/RemoteStopTransactionResponseHandler.javaocpp/handler/StartTransactionHandler.javaocpp/handler/StopTransactionHandler.java
New files
service/RemoteOperationLifecycleService.javaschedule/RemoteOperationTimeoutTask.java
Start lifecycle
- Validate, lock connector, reject overlapping active START.
- Persist operation before OCPP dispatch.
- On successful send, return
202/SENT. - For SENT with no CALLRESULT, use a fallback deadline from request time so restart or a lost response cannot leave PREPARING forever.
- On Accepted, set
accepted_at, recalculatedeadline_at = accepted_at + ConnectorType.maxWaitTime, andgrace_until = deadline_at + 30 seconds. - On Rejected/CALLERROR/send failure, persist terminal result and publish once.
- On StartTransaction, correlate by connector + IdTag + latest applicable operation, fill transaction ID, transition to STARTED, and publish targeted success.
- If StartTransaction arrives after TIMED_OUT, record STARTED_LATE for audit/reconciliation without replaying the timeout modal; actual DriverState becomes CHARGING.
- AVAILABLE/FAULTED after the wait boundary and without a transaction can terminate early instead of waiting through grace.
Stop lifecycle
- Validate and lock connector/transaction; reject overlapping active STOP.
- Persist operation, set
deadline_at = request_time + configured 30 seconds, then dispatch. - On Accepted remain FINISHING; on Rejected/CALLERROR/send failure publish terminal result once.
- On StopTransaction, correlate by transaction ID, transition to STOPPED, complete the transaction, and publish targeted success.
- At deadline, re-read actual transaction:
- ACTIVE -> TIMED_OUT, DriverState CHARGING, retry allowed.
- Completed -> reconcile as STOPPED.
- Late StopTransaction completes transaction reconciliation but does not change/reopen a terminal TIMED_OUT operation or replay a modal.
Restart behavior
There is no in-memory operation source of truth. DriverState and timeout scans query persisted active operations, so a Branch restart naturally restores PREPARING/FINISHING and continues deadline handling.
Configuration
Modify EmsProperties and all Branch profile YAML files:
ems.branch.remoteOperation.stopDeadlineSeconds = 30ems.branch.remoteOperation.reconciliationGraceSeconds = 30ems.branch.remoteOperation.scanFixedDelayMs = 5000
RemoteStart wait remains vendor-defined by the existing Charge Point values; only Branch's duplicate enum is aligned.
7. Queue And Long-Term Off-Peak
Modify:
service/charging/ChargingOrchestrator.javaservice/charging/OffPeakEnqueuePolicy.javaservice/charging/QueueStateMachine.javaschedule/OffPeakChargingTask.javarepository/ConnectorPriorityRepository.javaapi/cp/ConnectorService.javaapi/cp/ConnectorForHqService.javautil/TariffUtil.java
Rules:
- ENABLE_OFF_PEAK persists preference and enabled actor immediately, publishes
OFF_PEAK_WAITING, and does not create Priority before the applicable window. - Scheduler creates OFF_PEAK Priority only inside the window, using enabled actor as requester.
- MANUAL queue stores the initiating user and dispatches with that user's IdTag.
- Dispatch never selects the first connector owner as a substitute requester.
- Queue lookup and dispatch are source-aware; disabling off-peak cannot delete a MANUAL row.
- CANCEL_MANUAL_START deletes only MANUAL queue state.
- DISABLE_OFF_PEAK clears preference, enabled actor, suppression, and OFF_PEAK queue only.
- Current-window suppression blocks additional OFF_PEAK dispatch only for the persisted window.
- Rejected start, confirmed driver stop, or any completed off-peak transaction suppresses the current window.
- OFF -> ON clears suppression and starts a fresh scheduling decision.
- Losing the enabled actor's connector relation disables off-peak and removes its queue.
- Invalidated IdTag/tariff pauses automatic dispatch while preserving long-term preference and reason.
- PRIVATE -> PUBLIC disables off-peak and clears OFF_PEAK state.
All queue/off-peak writes publish Connector events only after transaction commit.
8. RabbitMQ And WebSocket
Branch files
Modify:
mq/model/ConnectorEventData.javamq/publisher/ConnectorEventPublisher.java
Add:
mq/model/ConnectorOperationResultEvent.java
Branch connector events contain:
- Shared connector data and DriverState.
- Optional unresolved
activeOperation. - Optional terminal
operationResultwith internaltargetUserId.
Publisher supports a normal shared update and a transition update carrying one terminal operation result. Terminal publication occurs only when the lifecycle transition succeeds for the first time.
HQ files
Modify:
mq/model/ConnectorEventData.javamq/subscriber/ConnectorEventSubscriber.javawebsocket/LineIdHandshakeInterceptor.javawebsocket/ConnectorSessionRegistry.javawebsocket/ConnectorWebSocketHandler.javawebsocket/dto/ConnectorUpdateMessage.java
Add:
websocket/dto/ConnectorOperationResultMessage.java
Handshake resolves and stores branchUserId together with lineId/buildingId/connectorId.
Before registering the session, HQ verifies connector visibility through the user-aware Branch connector-info call. PUBLIC visibility follows the existing Branch user/connector listing rule; PRIVATE visibility requires the current connector relation.
For each MQ update:
- Every legal connector session receives the same shared DriverState.
- Only sessions whose stored Branch user ID equals
operationResult.targetUserIdreceive the public operation result. - HQ strips targetUserId before serialization to the browser.
- Non-target sessions receive the same update without operationResult.
Initial WebSocket data continues to use HQ -> Branch REST internally, but only includes unresolved activeOperation and never a terminal result. LIFF itself does not need a separate detail REST call.
If authorization or initial Branch fetch fails, HQ closes and unregisters the WebSocket session instead of leaving an open connection with no initial state. This gives #1399 a reliable failure signal for its one reconnect attempt and return-to-list modal flow.
9. Automated Tests
Branch tests to update
service/ConnectorDriverPresentationServiceTest.javaservice/ChargingOrchestratorDispatchOffPeakGuardTest.javaapi/cp/ConnectorServiceUpdateOffPeakTest.javaapi/cp/ConnectorE2EIntegrationTest.javamq/publisher/ConnectorEventPublisherTest.java
Branch tests to add
api/cp/ConnectorCommandServiceTest.javaapi/cp/ConnectorCommandIntegrationTest.javaservice/RemoteOperationLifecycleServiceTest.javaschedule/RemoteOperationTimeoutTaskTest.javaschedule/OffPeakChargingTaskTest.javaservice/ChargingOrchestratorQueueSourceTest.javaocpp/handler/RemoteStartTransactionResponseHandlerTest.javaocpp/handler/RemoteStopTransactionResponseHandlerTest.javaocpp/handler/StartTransactionOperationCorrelationTest.javaocpp/handler/StopTransactionOperationCorrelationTest.java
Update integration-test schema/data fixtures for all new columns and command scenarios.
Required automated scenarios include:
- Raw Fortune PREPARING without active operation -> READY.
- SENT/ACCEPTED START -> PREPARING; active STOP -> FINISHING.
- Start/Stop success, rejection, send failure, timeout, restart, duplicate request, and late event.
- PUBLIC owner-only stop and PRIVATE linked-resident stop.
- Manual queue requester preservation and source-specific cancellation.
- Immediate OFF_PEAK_WAITING, in-window QUEUED, current-window suppression, and next-window recovery.
- Existing ambiguous off-peak actor pauses without invented ownership.
- HTTP 202/200/403/409 contract.
- MQ shared state plus targeted result.
HQ tests to update
websocket/ConnectorWebSocketHandlerTest.java
HQ tests to add
api/connector/ConnectorServiceCommandTest.javaapi/connector/ConnectorBranchApiHandlerCommandTest.javamq/subscriber/ConnectorEventSubscriberTest.javawebsocket/LineIdHandshakeInterceptorTest.javawebsocket/ConnectorSessionRegistryTargetingTest.java
Required HQ scenarios:
- Never trust userId from LIFF.
- Preserve Branch 202/200/403/409.
- Target session receives result; non-target session receives shared state only.
- targetUserId is never serialized to the browser.
- Initial snapshot includes active operation only and never terminal result.
- Building membership without connector visibility cannot subscribe to an unrelated PRIVATE connector.
- Initial fetch failure closes/unregisters the session instead of leaving a stale empty connection.
10. E2E Catalog Impact
Classification: Add + Update. No silent deletion.
Existing cases to update
-
CHG-007: PREPARING cancellation is operation-backed; CloudLink supports CANCEL_START and Fortune reports disabled capability. -
CHG-009,CHG-010,CHG-018: persisted start lifecycle, deadline/grace, restart, and safe fallback. -
CHG-020: STOP returns 202 and completes only on StopTransaction. -
CHG-021: narrow to PUBLIC non-owner rejection. -
CHG-023: replace old HTTP-200/body-failure expectation with structured 403/409 passthrough and no false success. -
CHG-024,CHG-025,CHG-032: actions-only contract, initial/event consistency, operationId, and targeted result. -
ROT-001,ROT-003,ROT-005,ROT-011,ROT-019: OFF_PEAK_WAITING, source/requester, suppression, and transactional publication. -
UI-002: semantic actions and DriverState codes. -
REL-002,REL-003,REL-007,REL-009: additive migration, restart recovery, cross-service payload, and rollback compatibility.
New cases to add
-
CHG-033P0, private connector: another currently linked resident may stop while original transaction owner remains unchanged and actual requester is audited. -
CHG-034P0, RemoteStart lifecycle: 202/SENT -> targeted terminal result; duplicate command blocked; late StartTransaction reconciled. -
CHG-035P0, RemoteStop lifecycle: 30-second timeout, explicit retry, and late StopTransaction without duplicate modal. -
ROT-021P1: off-peak ON immediately produces OFF_PEAK_WAITING; Priority creation changes it to QUEUED. -
ROT-022P0: completed/rejected/stopped off-peak transaction suppresses only the current window and prevents repeated RemoteStart. -
ROT-023P1: MANUAL and OFF_PEAK queues retain requester/source and cancellation affects only the requested source. -
ROT-024P1: enabled actor relation, IdTag, tariff, or PRIVATE area changes converge according to the long-term preference rules. -
UI-010P1: shared WebSocket state reaches all legal sessions while operationResult reaches only the target and initial snapshot does not replay it.
Release pack
Use Charging/OCPP/DB Major Release:
- All applicable P0 and P1.
- Affected P2 for OCPP correlation, restart, schema rollback, and WebSocket races.
- CloudLink physical flow is mandatory.
- Fortune cases are Conditional but mandatory when a Fortune test site/device is available.
- P0 requires 100% PASS; Skip is not Pass.
11. Documentation
After implementation and tests:
- Update
ai/02-backend-services.md. - Update
ai/03-database.md. - Update
ai/04-frontend.mdfor the backend contract consumed by #1399. - Update
ai/06-domain-glossary.md. - Update
documents/E2E_TEST_CASE_INVENTORY.md. - Validate with
python3 .claude/skills/maintain-ems-e2e-catalog/scripts/validate_catalog.py. - Produce the run-specific report under
test_report/after deployment and actual E2E execution.
12. Verification Commands
cd java/ems_branch
mvn test
mvn clean package
cd ../ems_hq
mvn test
mvn clean package
cd ../..
python3 .claude/skills/maintain-ems-e2e-catalog/scripts/validate_catalog.py
Schema migration is verified against an isolated integration database before any site deployment. Real RemoteStart/RemoteStop, relay state, StartTransaction/StopTransaction, RabbitMQ, and LIFF WebSocket evidence remain deployment-stage E2E work and cannot be claimed from unit tests alone.
13. Plan Gate
No business implementation starts until Ken approves this plan.
Approval includes these explicit choices:
- Existing ambiguous
enable_off_peak=truerows remain enabled but paused until a legal user reconfirms OFF -> ON; only single-linked-user rows are backfilled. - Branch admin
/api/tx/charge/start|stopremain; only old HQ LIFF endpoints and old Branch HQ off-peak/dequeue endpoints are removed. - Connector row locking and ordinary indexes enforce one active operation; no DB UNIQUE constraint is added.
- A START command that only creates a MANUAL queue returns 200 without operationId; only actual RemoteStart/RemoteStop dispatch returns 202 with operationId.