Compare commits

...

15 Commits

Author SHA1 Message Date
yu
407aa385d2 docs: V2 补记对面事实一致性门禁合规要求(补丁须同提交更新事实文件+文档区块) 2026-11-06 13:32:34 +08:00
yu
a01667bdb3 feat(engine): NodeState 补 waiting_user/unknown 两态+StepStatus 映射表;node.finished 增检查点负载(A++ 字段定稿);补 sm-crypto 类型声明 2026-11-06 13:19:29 +08:00
yu
4615fcab10 docs: 账面盘点——包依赖3项核实销账,剩余 NodeState 两态+A++ 字段设计 2026-11-06 13:08:58 +08:00
yu
2bcda3a558 fix(catalog): build-catalog 白名单归一化 platformTag,清洗上游 3 条数字脏值(重派生 win32) 2026-11-06 13:06:54 +08:00
yu
4b5c73b752 feat(word): Word_Save/Word_SaveAs 真实现(COM 落盘),替换路径回显占位 2026-11-06 11:52:56 +08:00
yu
b18d5b1f0b docs: bun 1.4.4 本机冒烟通过(engine 37/37, exceljs/pdf 无差异),R6 部分缓解 2026-11-06 11:40:36 +08:00
yu
d99be8f4ab docs: ADR-R2 补记 A++ 检查点协议(两侧改动清单+落地节奏);标定阶段1已完成项 2026-11-06 11:36:10 +08:00
yu
1f804971f4 feat(engine): 补交 engine.execute 实现(A1 漏加文件) 2026-11-06 11:09:46 +08:00
yu
7fa1c4038c docs: 补记工作日志(迁移阶段1四项) 2026-11-06 11:09:14 +08:00
yu
9bb5a44b95 chore: rpa-cli 显式注册+engine.execute;V2 现状表同步阶段1修复状态 2026-11-06 11:09:12 +08:00
yu
5e8fc9686f fix(catalog): 修复重生成遗留的三类数据回归(迁移阶段1-T4)
- 昨日重生成用的对面旧版源数据缺 select 选项:从重生成前快照收割 782 个
  select/radio 字段的 options 进 overrides(含默认值落在选项内的校验)
- 移除实现不读的假字段:Excel_Cs_InsertColumnV3/InsertRowV3 的 direction
- 默认值修复:Time 系列 offset_unit_num 改数字 0(实现按 num 读取)、
  Mail_DoGetMailMessage_v2:attachment 恢复旧选项、Window_SizeWindow_v1:window_size
  恢复 text widget、Mobile_TouchSlideV1 无实现清空默认值
- app-system-tests 期望更新:12 号锁屏流程因假必填清理不再阻塞(有意变更)
- catalog-properties 13 项 + app-system 全过
2026-11-06 10:53:56 +08:00
yu
1a20dd72d1 feat: 视觉能力去 Electron 化,经 HostBridge.screen 注入(迁移阶段1-A9)
- HostBridge 新增可选 screen 服务(listDisplays/primaryDisplay/dipToScreenPoint/
  displayMatching/captureDisplay),vision 3 处、cv-vision 2 处动态 import(electron)
  全部改为 requireScreenService();未注入时抛稳定的 host_unavailable
- demo 宿主 src/main/index.ts 用 desktopCapturer/screen 实现 screen 服务
- check-deps.mjs 升级:新增动态 import()/require() 扫描(旧正则实测 0 命中的盲区),
  合成用例验证命中,升级后真实源码 0 违规
- 包依赖闭包:补声明 @cantoo/pdf-lib、@napi-rs/canvas、pdfjs-dist
- cv-vision 测试从 vi.mock(electron) 改为宿主桥注入,11/11 通过
2026-11-06 10:53:42 +08:00
yu
1da00827ac feat(engine): createEngine 注入参数 + 消除模块级 driver 捕获(迁移阶段1-A1)
- createEngine(options) 支持 driver 注入,新增 engine.setDriver;
  缺省保持按平台自动选择的行为
- desktop/common-final/system-extended/window-extended 四文件模块加载时
  捕获 defaultEngine.driver 改为惰性 getDriver():非 win32 import 不再抛错
  (此前 darwin 上 import 即炸),调用时才触发 platform_unsupported
- engine-driver.test 补注入/隔离/后置替换 4 例
2026-11-06 10:20:29 +08:00
yu
9b76dc6987 feat(engine): 取消贯通/超时真中止/重试门控/图校验(迁移阶段1-A3)
- 节点尝试级 AbortController:run 取消与节点超时汇聚到同一信号,
  handler 收到的 ctx.signal 即尝试信号,其内部 sleep/HTTP/子进程可被真正中止
- 超时中止底层后 handler 先抛 cancelled 的竞态归一为 timeout
- executeNode 取消映射 NodeState=cancelled:不适用 skipOnError、不进异常分支
- 重试门控:CapabilityMeta.idempotent===false 时无视 retryCount(防重复副作用)
- 新增 validateFlowGraph:重复ID/悬空边/自环/顶层环/孤儿节点/不可达节点,
  阻断性错误在 runFlow 内统一收口为 validation_error
- 新增 11 项测试;既有 error-route/integration 回归通过
2026-11-06 10:13:38 +08:00
yu
d972a18bda docs: V2 拍板 ADR-R2(方案A+两补丁),6 项 ADR 全部闭环 2026-11-06 10:01:51 +08:00
31 changed files with 15560 additions and 251 deletions

View File

@ -286,3 +286,16 @@
- 用户确认两项关键决策:①桌面点击逻辑改写进对面 C# 原生进程(PowerShell 通道仅限 demo);②按能力域拆分部署(纯 Node 域→backend plugin-worker,桌面/视觉/弹窗域→用户电脑 Electron 宿主)
- V2 文档已更新:§0 结论、§10 拆"已拍板/待评审"、A6 改哪侧、阶段 0 范围、风险 R1/R2 降级为已缓解
- 剩余 4 ADR(R1/R2/R4/R5)+ 3 Q 留待与对面对评审
## ADR-R2 拍板(11-06 10:00)
- 方案 A + 两补丁:RPA 流程=单能力 rpa.flow.execute;节点事件走 event-bus 旁路;人审/高危节点升级对面审批。B 触发条件留档(仅当产品需逐步控制)。不用 B 的三层理由(双账本/控制流节点无副作用/审批重试粒度)已写入 §10。
- 至此 6 项 ADR 全部拍板,仅剩 Q1/Q2 待对面确认。6-阶段计划可开工,阶段 1(RPA 侧收口)不依赖对面。
## 迁移阶段1开工(11-06 10:03-11:10)
- A3(9b76dc6):节点尝试级 AbortController(run取消+节点超时汇聚),超时真中止底层;取消映射 NodeState=cancelled(不适用 skipOnError/不进异常分支);CapabilityMeta.idempotent 门控重试;graph-validator.ts(重复ID/悬空边/自环/环/孤儿/不可达)runFlow 前置校验;11 新测试
- A1(1da0082):createEngine({driver})+setDriver;4 个能力文件模块级 driver 捕获改惰性 getDriver()(非 win32 import 不再炸)
- A9(1a20dd7):HostBridge.screen 服务(listDisplays/primaryDisplay/dipToScreenPoint/displayMatching/captureDisplay),vision/cv-vision 5 处动态 electron import 清零;demo 宿主实现 screen 服务;check-deps 升级覆盖动态 import(实测合成命中1/真实0);包依赖补 3 项
- T4(5e8fc96):修复昨日 catalog 重生成遗留回归——对面旧版源数据缺 select 选项,从旧快照收割 782 个 options 进 overrides;移除 InsertColumn/RowV3 假字段 direction;Time 系列 offset_unit_num 默认改 0(实现按 num 读);Mail attachment 恢复选项;app-system-tests 12 号流程期望更新(假必填清理后不再阻塞)
- rpa-cli 重写:显式 5 域注册 + engine.execute 语义真实入口
- 全量回归:390 过;唯一失败是 mouse-moveto-smoke 真桌面测试被用户鼠标干扰的环境抖动(两次全量失败用例不同=铁证),要求独占桌面
- 教训: harvest overrides 时只对齐了 required/label,漏了 options——重生成前必须全字段 diff 而不是抽查

View File

@ -12,17 +12,18 @@
1. **宿主底座已大变,V1 的"空壳契约"叙事大部分失效。** V1 说对面 7 个契约包每个只有 1-9 个声明文件、windows-runner 是 0 源文件空壳。实测(2026-11-06):7 个包中 5 个已有实质实现(workflow-engine 25 文件、capability-gateway 321 行、browser-plugin 已接 Playwright、windows-plugin 新增原生观察适配器、runtime-governance 有 readiness 投影),windows-runner 有真实 C# 原生实现(.NET 10 + FlaUI UIA3,observe-only)。**仍无实现的只剩两处:computer-control 的 Runner、windows-plugin 的 WindowsRunner.click/type。**
→ 迁入的本质从 V1 的"拿实现去填契约"变为**"接底座 + 给已有实现让路"**:不重复造 workflow 状态机、不重复造 Playwright 浏览器、不重复造 UIA 观察。
2. **RPA 侧此轮整改完成了一部分**:平台标记单一源(PLATFORM_GROUPS 34 域 + 守卫测试)、驱动按平台自动选择(platform_unsupported)、catalog platformTag 687 项首次生效、5 个真桌面测试补平台守卫、`driver/platform.ts` 收拢。**但接入前 P0 大部分未解决**:`createEngine()` 仍无注入参数、Registry 仍是进程级全局、超时不中止底层操作、取消被标为 failed、5 处动态 `import("electron")` 残留、包依赖缺 3 项。
2. **RPA 侧接入前 P0 整改已于 2026-11-06 完成(迁移阶段 1 收口)**:节点尝试级 AbortController(超时即真中止底层操作)、取消映射 cancelled 不走 error route、重试改 `idempotent` 元数据门控、图校验器执行前拒非法流程图、`createEngine(options)` 注入参数(driver/hostBridge/subflowSource/registry)、视觉能力 5 处动态 `import("electron")` 清零改走 HostBridge.screen 注入、check-deps 覆盖动态 import、catalog 脏值修复、rpa-cli 显式注册重写。★ 标记明细见 §2/§7/§8。**接入前本仓库代码活已全部完成(2026-11-06)**:①NodeState 补 `waiting_user`/`unknown` 两态 + `toStepStatus` 全量映射表(与对面 StepStatus 8 态同构,含契约测试);②A++ 事件负载字段定稿——`node.finished` 增 `checkpointId`/`resumable`/`idempotencyKey`(幂等键=runId:nodeId 跨重入稳定,非幂等能力不作为续跑锚点),契约测试覆盖。附带:补 sm-crypto 类型声明(typecheck 归零)。**对接面工作全部在对面仓库**:rpa-adapter、subtree 迁包、垂直切片;本方仅待对面答复 Q1/Q2 与 rpa.browser 备案。PowerShell 编码差异验证留待阶段 2 迁包后(bun 本机冒烟已于 2026-11-06 完成:bun 1.4.4,engine 37/37 全绿,exceljs/pdf 行为无差异;对面锁 1.4.3 为 patch 级差异)。
3. **迁入位置与名称维持 V1 结论**:`code/backend/packages/rpa-core` → `@agent-worker/rpa-core`,backend workspace(bun@1.4.3,workspaces 含 `packages/*`)放进去即生效,backend 全家统一 `@agent-worker/*` 前缀、`workspace:*` 源码消费。
4. **两条硬边界不变且加严**(用户红线):
- 所有带副作用的能力必须经 capability-gateway(授权→租约→幂等→执行→审计),**不得绕行**;
- rpa-core **不得依赖 Electron 主进程**(当前仍有 5 处动态 import,是接入阻塞项)。
5. **6 项 ADR 已拍板 5 项(2026-11-06)**,仅 R2(集成形态)带问题清单去对面对评审,详见 §10。已定基调:
5. **6 项 ADR 全部拍板(2026-11-06)**,仅剩 2 个小问题(Q1/Q2)需对面口头确认,详见 §10。已定基调:
- **R3 桌面执行**:C# 路线——五域执行逻辑移植进 windows-runner 原生进程,PowerShell 通道仅限本 demo。
- **R6 部署拓扑**:按能力域拆分——纯 Node 域进 backend plugin-worker,桌面/视觉/弹窗域装用户电脑 Electron 宿主,同一包按分组加载。
- **R1 能力清单**:全局一份,权限隔离全交对面网关按租户管。
- **R4 浏览器**:按场景分工——RPA 场景用咱们 CDP 接管(复用登录态/真实页面),对面 Playwright 留给 AI 智能体;白名单与证据共用对面一套。
- **R5 消费方式**:源码 git subtree 并入对面仓库统一编译,Bun 锁 1.4.3。
- **R2 集成形态(含 A++ 补记)**:方案 A——整个 RPA 流程注册为 `rpa.flow.execute` 单能力,事件流旁路+人审升级两补丁;事件流再升级为检查点协议(崩溃后带 resumeToken 断点续跑),权威账本始终单 step。两侧改动清单见 §7-A5。
---
@ -35,34 +36,34 @@
---
## 2. RPA 侧现状事实清单(2026-11-06 实测)
## 2. RPA 侧现状事实清单(2026-11-06 实测;★=阶段 1 已于当日修复)
| # | 事项 | 状态 | 证据 |
|---|---|---|---|
| 1 | Driver 惰性 getter + 非 win32 抛 `platform_unsupported` | 已实现 | engine.ts:47-61;error-codes.ts:43;engine-driver.test.ts |
| 2 | **Driver 可注入** | **未实现** | `createEngine()` 无参数(engine.ts:64-66);desktop.ts:9 等在模块加载时捕获 `defaultEngine.driver` |
| 3 | Registry 全局 Map、last-wins、runtime 读全局 getHandler | 属实(未改) | registry.ts:36-45;runtime.ts:260 |
| 4 | PLATFORM_GROUPS 单一源(34 域全 win32)+ buildDomainPlatformMap 真实实现 + 守卫测试 | 已实现 | register-all.ts:16-50,98-127;platform-groups.test.ts |
| 5 | withTimeout 监听 signal 但无 AbortController——超时不中止底层 | 属实(未修) | runtime.ts:33-56 |
| 6 | NodeState 类型含 cancelled,但 executeNode catch 把取消标成 failed | 属实(未修) | types.ts:142;runtime.ts:334-339 |
| 7 | 重试上限 100、黑名单式 isRetryable | 属实(未修) | runtime.ts:58-60,276 |
| 8 | Graph Validator | 未实现 | 全仓无 validateFlowGraph |
| 9 | 动态 `import("electron")` 5 处(vision.ts×3、cv-vision.ts×2) | 属实(未修) | vision.ts:18,66,432;cv-vision.ts:51,213 |
| 10 | check-deps.mjs 对 `import("electron")` 零匹配(盲区实测) | 属实 | check-deps.mjs:20 正则要求 `import\s+` |
| 11 | 包依赖缺 3 项:@napi-rs/canvas、pdfjs-dist、@cantoo/pdf-lib | 属实(未修) | rpa-core/package.json:17-32 对比 pdf.ts/vision.ts import |
| 12 | main/exports 直指 src 源码、private | 属实(决策项) | rpa-core/package.json:4-11 |
| 13 | rpa-cli 副作用全量 import + 全局 getHandler | 属实(未改) | scripts/rpa-cli.ts:25,53-59 |
| 14 | CDP transport 单命令无超时/无 AbortSignal/pending 仅断连清理 | 属实(未修) | transport.ts:96-118 |
| 15 | 两段式元素定位(当前页优先,失败才跟前台)| 已实现 | session.ts:296-307,518-523 |
| 16 | locate() 不支持 xpath / Shadow DOM(与 withElement() 不一致)| 属实(未修) | session.ts:465-517 对比 252-287 |
| 17 | HostBridge/SubflowSource 模块级单例 | 属实(未改) | host-bridge.ts:16-24,61-69 |
| 18 | AI ChatCompletionOptions 无 signal 参数 | 属实(未修) | ai-service.ts:39-48 |
| 19 | 桌面拾取无 AbortSignal(热键+轮询+deadline 自有机制)| 属实(未修) | desktop-pick.ts 全文零 signal 命中 |
| 20 | catalog platformTag:win32=687 / null=326 / **数字 1=3(脏值未清洗)** | 部分实现 | build-catalog.mjs:245-246 对 truthy 旧值透传 |
| 21 | catalog overrides(fields/removeFields/addFields) | 已实现 | catalog-overrides.mjs:6-24 |
| 1 | Driver 惰性 getter + 非 win32 抛 `platform_unsupported` | 已实现 | engine.ts;error-codes.ts:43;engine-driver.test.ts |
| 2 | **Driver 可注入** | ★已实现 | `createEngine({driver})` + `engine.setDriver()`;desktop/common-final/system-extended/window-extended 的模块级捕获改惰性 `getDriver()` |
| 3 | Registry 全局 Map、last-wins、runtime 读全局 getHandler | 属实(ADR-R1 短期形态固化;新增 `engine.execute` 语义真实入口) | registry.ts;engine.ts execute() |
| 4 | PLATFORM_GROUPS 单一源(34 域全 win32)+ buildDomainPlatformMap 真实实现 + 守卫测试 | 已实现 | register-all.ts;platform-groups.test.ts |
| 5 | withTimeout 监听 signal 但无 AbortController——超时不中止底层 | ★已修复 | 尝试级 AbortController 汇聚 run 取消+节点超时,handler 收到的 ctx.signal 即尝试信号 |
| 6 | NodeState 类型含 cancelled,但 executeNode catch 把取消标成 failed | ★已修复 | 取消映射 cancelled(不适用 skipOnError、不进异常分支);runtime.cancel.test.ts |
| 7 | 重试上限 100、黑名单式 isRetryable | ★已门控 | CapabilityMeta.idempotent===false 时无视 retryCount |
| 8 | Graph Validator | ★已实现 | engine/graph-validator.ts(重复ID/悬空边/自环/环/孤儿/不可达),runFlow 前置校验 |
| 9 | 动态 `import("electron")` 5 处(vision.ts×3、cv-vision.ts×2) | ★已修复 | HostBridge.screen 服务注入(requireScreenService),Core 恢复零 Electron |
| 10 | check-deps.mjs 对 `import("electron")` 零匹配(盲区实测) | ★已修复 | 新增动态 import()/require() 扫描;合成用例命中=1,真实源码 0 违规 |
| 11 | 包依赖缺 3 项:@napi-rs/canvas、pdfjs-dist、@cantoo/pdf-lib | ★已修复 | rpa-core/package.json dependencies 补齐 |
| 12 | main/exports 直指 src 源码、private | 属实(决策项,与对面源码消费一致) | rpa-core/package.json |
| 13 | rpa-cli 副作用全量 import + 全局 getHandler | ★已重写 | 显式按域注册(5 域)+ engine.execute |
| 14 | CDP transport 单命令无超时/无 AbortSignal/pending 仅断连清理 | 属实(未修,A7 迁移前修) | transport.ts:96-118 |
| 15 | 两段式元素定位(当前页优先,失败才跟前台)| 已实现 | session.ts |
| 16 | locate() 不支持 xpath / Shadow DOM(与 withElement() 不一致)| 属实(未修,A7 迁移前修) | session.ts |
| 17 | HostBridge/SubflowSource 模块级单例 | 属实(ADR-R1 范畴;screen 服务已并入 HostBridge) | host-bridge.ts |
| 18 | AI ChatCompletionOptions 无 signal 参数 | 属实(未修,P2) | ai-service.ts |
| 19 | 桌面拾取无 AbortSignal(热键+轮询+deadline 自有机制)| 属实(未修,P1) | desktop-pick.ts |
| 20 | catalog platformTag:win32=687 / null=326 / 数字 1=3(脏值未清洗) | ✅ 已清洗(2026-11-06) | build-catalog.mjs 增白名单归一化(非 win32/darwin/linux 字符串一律重派生),3 条脏值重派生为 win32,全字段 diff 仅 3 条变化 |
| 21 | catalog overrides(fields/removeFields/addFields) | 已实现(+782 个 options 收割修复重生成回归) | catalog-overrides.mjs |
| 22 | 能力 719 个 / 目录组件 1016(vendor 997 + native 16 + trigger 3)/ fields 4299 / 34 域 | 实测 | platform-groups.test.ts;catalog.json counts |
| 23 | 平台守卫测试 5 文件 + driver/platform.ts 唯一入口 | 已实现 | capabilities/*.test.ts;driver/platform.ts |
| 24 | Driver 五域方法均无 AbortSignal 参数 | 属实(未修) | driver/types.ts:87-119 |
| 24 | Driver 五域方法均无 AbortSignal 参数 | 属实(未修;运行时取消已能中止监听 signal 的能力内部操作) | driver/types.ts |
---
@ -85,6 +86,8 @@
对面安全模型要点(直接影响 RPA 桌面路线):windows-plugin 策略校验只允许 `*.exe` 进程名、不允许路径;native-observer.ts 设计约束明言"不执行任意 PowerShell";桌面交互 Runner 留宿主、不进 Linux 容器。
**对面协作门禁(2026-11-06 新增,本项目补丁必须遵守)**:对面已建立代码与文档事实一致性门禁——`code/tooling/document-facts.mjs` + `项目资料/开发方案/当前事实.json`(factRevision 2026-11-06.1)。规则:代码、迁移、合同、端口、运行事实的任何变化,必须与事实文件、README 机器生成区块**同一提交**;CI 的 `document-facts --check` / `ci-gate.mjs` 会重扫源码输入并阻断漂移。**因此本项目未来合入对面仓库的任何补丁(含 C# actuate 补丁包、subtree 迁包)须:①同一提交内更新 `当前事实.json` 与相关 README 区块;②提交前跑 `node code/tooling/document-facts.mjs --check` 与 `node code/tooling/ci-gate.mjs --mode fast`。** 另:对面 backend/README Windows 行已明确下一步为"click/type、视觉/OCR 或随本轮客户端交付"——R3 C# actuate 路线与其路线图一致,可据此作为提案依据。
---
## 4. 原指南(V1)内容分类
@ -94,7 +97,7 @@
| 包现状描述(719 能力、34 域、宿主桥 4 方法、Driver 五域) | **保留**(微调) | 仍准确;宿主桥维持 4 方法不变 |
| "完全不依赖 Electron、check:deps 天天检查" | **过时/不实** | 5 处动态 import 逃过护栏;改为"静态依赖已清、动态残留待清",护栏必须升级 |
| 第 -1 步对面契约包全部空壳的叙事 | **过时** | 按 §3 改写:5/7 已实现,"填契约"变"接底座" |
| 第 0 步三件作业(bun 冒烟 / catalog 重生成 / 注册方式决策) | **部分保留** | catalog 重生成已完成(platformTag 687 生效,3 个脏值待清洗);bun 冒烟仍未做;注册方式决策并入 ADR-R1 |
| 第 0 步三件作业(bun 冒烟 / catalog 重生成 / 注册方式决策) | ✅ 全部完成 | catalog 重生成完成且脏值已清洗(platformTag win32=690 / null=326,数字脏值归零);bun 1.4.4 本机冒烟 37/37;注册方式决策并入 ADR-R1 |
| 第 1 步搬包(git subtree → code/backend/packages/rpa-core、改名 @agent-worker/rpa-core、demo 侧不双活) | **保留** | 双侧命名与 workspace 布局核实无误 |
| 第 2 步 A 类契约对齐表 | **需要修改** | capability-gateway/workflow-engine 从"对照后改对面"变"按已实现接口写 adapter";新增 plugin-contract/registry/worker 三层对接 |
| 第 2 步 B 类宿主注入表(浏览器=最大变数) | **需要修改** | 浏览器变数已收敛:对面有 Playwright 实现;变数改为"双路径取舍"(ADR-R4) |
@ -152,7 +155,7 @@
| RPA 侧(packages/rpa-core) | Agent-Employee 侧 | 关系 | 现状 |
|---|---|---|---|
| `engine/runtime.ts` 流程执行 | `workflow-engine` WorkflowEngine/StepRun | RPA=执行算法候选,AE=权威状态 | 集成形态待 ADR-R2 |
| `engine/runtime.ts` 流程执行 | `workflow-engine` WorkflowEngine/StepRun | RPA=执行算法候选,AE=权威状态 | 已拍板方案 A(ADR-R2) |
| `engine/registry.ts` 全局注册表 | `plugin-registry` + `capability-gateway` CapabilityRef | ref↔componentId 映射层(rpa-adapter) | 映射待建 |
| 719 能力 handler | `plugin-contract` CapabilityDefinition + PermissionScope 13 项 | 能力清单生成 + 权限映射 | 待建(脚本生成) |
| `RunContext.signal` | `CapabilityExecutionContext.signal` | 均 AbortSignal,直通 | 兼容 |
@ -180,7 +183,7 @@
- **兼容性**:部分。平台错误语义(platform_unsupported)已就绪;注入面缺失。
- **问题**:宿主无法注入测试替身或 computer-control 适配器;能力文件模块级捕获引擎状态(excelHandles/ftpClient/ocrInstance/driver),多会话共享。
- **改哪侧**:RPA。
- **方案**:`createEngine(options?: { driver?, hostBridge?, subflowSource?, registry? })`,全部可选、缺省保持现行为;能力文件改为经 RunContext/engine 参数取 driver 与资源表(模块级捕获改为首用时向 engine 取);幂等句柄表(excelHandles/ftpClient/ocrInstance)迁入 engine 实例字段(部分已是)。
- **方案(★ 已实现,2026-11-06 阶段 1-A1)**:`createEngine(options?: { driver?, hostBridge?, subflowSource?, registry? })`,全部可选、缺省保持现行为;能力文件改为经 RunContext/engine 参数取 driver 与资源表(模块级捕获改为首用时向 engine 取);幂等句柄表(excelHandles/ftpClient/ocrInstance)迁入 engine 实例字段(部分已是)。
- **验收**:FakeDriver 注入测试覆盖五域;两个 Engine 各注入不同 FakeDriver 并行执行互不串用;非 win32 上 import 全部 34 域不抛错、调用桌面能力时才抛 platform_unsupported。
### A2 Registry 注册机制与实例隔离
@ -200,7 +203,7 @@
- **兼容性**:不兼容。对面以幂等键 + 租约纪元管理副作用安全;RPA 超时/取消/重试语义与它冲突。
- **问题**:超时后底层 PowerShell/HTTP/CDP 仍执行(重复副作用);取消状态不一致;非幂等能力可被重试 100 次。**这是接入的第一阻塞项。**
- **改哪侧**:RPA。
- **方案**:①节点尝试创建独立 AbortController,合并 Run 级 signal 与节点 timeout,贯通 handler→driver→PowerShell 子进程/HTTP/CDP(driver 五域方法加可选 `signal` 参数);②executeNode catch 优先映射 cancelled → `state.status="cancelled"`,不走 skipOnError/error route/retry;③重试改元数据驱动:CapabilityMeta 增 `idempotent`/`sideEffect`/`retryPolicy`,非幂等默认 0 次重试;④新增 `validateFlowGraph`(ID 唯一、边合法、入口/出口基数、环、悬空边)。
- **方案(★ 已实现,2026-11-06 阶段 1-A3)**:①节点尝试创建独立 AbortController,合并 Run 级 signal 与节点 timeout,贯通 handler→driver→PowerShell 子进程/HTTP/CDP(driver 五域方法加可选 `signal` 参数);②executeNode catch 优先映射 cancelled → `state.status="cancelled"`,不走 skipOnError/error route/retry;③重试改元数据驱动:CapabilityMeta 增 `idempotent`/`sideEffect`/`retryPolicy`,非幂等默认 0 次重试;④新增 `validateFlowGraph`(ID 唯一、边合法、入口/出口基数、环、悬空边)。
- **验收**:构造 5 秒 Fake Handler + 100ms 超时,断言 handler 收到 abort 并停止、无并发重入;执行中取消后 Run 与 NodeState 均为 cancelled;对 File 写入/点击类能力验证"超时不自动重试";非法图执行前得到稳定错误码。
### A4 Capability Gateway(能力注册与权限治理)
@ -220,8 +223,12 @@
- **兼容性**:状态映射大体可行;RPA 缺 `waiting_user`/`unknown` 两态;对面 StepStatus 无 RPA 的 `idle`。
- **问题**:两套状态权威冲突风险;集成形态二选一未定(整体作为一个 capability vs 节点级映射为 step)。
- **改哪侧**:adapter + 双侧小改。
- **方案**:**推荐 MVP 方案 A**——整个 RPA 流程注册为**一个** capability(`rpa.flow.execute`),workflow-engine 的一个 step 调它,RunResult→ActionResult、节点级事件→event-bus 事件流、human-dialog→requestApproval 挂对面 approval 流、取消→signal 直通;方案 B(RPA 节点=step 1:1 映射)等方案 A 验收后按需评估。NodeState 增 `waiting_user`/`unknown` 两态(unknown 用于崩溃恢复对齐)。
- **验收**:状态映射契约测试(含 waiting_user/unknown/cancelled 全分支);一次含审批节点、一次执行中取消、一次进程崩溃恢复的端到端演练;对面 Run 状态终态与 RPA RunResult 永远一致。
- **方案**:**已拍板(ADR-R2)方案 A + 两补丁**——整个 RPA 流程注册为**一个** capability(`rpa.flow.execute`),workflow-engine 的一个 step 调它,RunResult→ActionResult、节点级事件→event-bus 事件流(旁路,不进权威账本)、human-dialog→requestApproval 挂对面 approval 流、取消→signal 直通。补丁 1:节点事件流保透明;补丁 2:人审/高危节点照常升级对面审批。方案 B(节点=step 1:1)的触发条件留档:仅当对面产品确认需要"逐步控制 RPA"(单步执行/任意步回退)时局部启用,届时事件流已备好数据不返工。NodeState 增 `waiting_user`/`unknown` 两态(unknown 用于崩溃恢复对齐)。
- **A++ 检查点补记(2026-11-06,本方负责人确认)**:在方案 A 骨架内将节点事件流升级为检查点协议(决策全文见 §10-R2)。
- **RPA 侧改动**:①事件负载增 `nodeId`/`status`/`checkpointId`/`idempotencyKey`/`resumable` 字段;②`rpa.flow.execute` 接受可选 `resumeToken`,重入时从最近检查点续跑(幂等句柄表已迁 engine 实例字段,地基现成);③幂等键生成与传递贯通(复用 A3 的 `idempotent` 门控,仅补 key 生成与透传)。
- **对面侧改动**:①workflow-engine 对 `rpa.flow.execute` 所在 step 的重试/恢复策略允许带 resumeToken 重入(不新建 step、不改租约语义);②event-bus 消费端识别检查点事件并入审计链;③崩溃恢复时把 resumeToken 经 adapter 回传(复用 workflow-recovery 的 unknown→恢复路径)。
- **落地节奏**:阶段 2 只做事件负载字段设计定稿(payload 预留,近零成本);阶段 3 垂直切片做最小验证(崩溃→重入续跑);阶段 4/5 规模化时强制执行。
- **验收**:状态映射契约测试(含 waiting_user/unknown/cancelled 全分支);一次含审批节点、一次执行中取消、一次进程崩溃恢复的端到端演练;对面 Run 状态终态与 RPA RunResult 永远一致;崩溃重入演练——杀掉执行进程后带 resumeToken 续跑,已完成节点不重复执行、幂等键防重复副作用。
### A6 Computer Control 与 Windows 桌面执行
@ -245,10 +252,10 @@
### A8 平台适配(Windows / Linux / macOS)
- **RPA 位置**:`register-all.ts` PLATFORM_GROUPS(34 域全 win32)、`driver/platform.ts`、`engine/engine.ts:47-61`、5 个 skipIf 测试;catalog 687 win32 / 326 null / **3 个数字 1 脏值**。
- **RPA 位置**:`register-all.ts` PLATFORM_GROUPS(34 域全 win32)、`driver/platform.ts`、`engine/engine.ts:47-61`、5 个 skipIf 测试;catalog **win32=690 / null=326,数字脏值已归零(2026-11-06 白名单归一化)**。
- **AE 位置**:`plugin-contract` Platform 类型;windows-runner 明确"留在宿主、不进 Linux 容器"(README)。
- **兼容性**:兼容(两边都是 win32-first 务实路线)。
- **问题**:①catalog 3 个 `platformTag: 1` 脏值透传(build-catalog.mjs:246 对 truthy 不清洗);②对面 CapabilityDefinition 未见 platform 字段(**待确认**:插件清单如何表达平台限制)。
- **问题**:①~~catalog 3 个 `platformTag: 1` 脏值透传~~ **已修复(2026-11-06)**:build-catalog.mjs 白名单归一化,上游 shizai 的数字 1 重派生为 win32(三条均为包内真实现,语义正确);②对面 CapabilityDefinition 未见 platform 字段(**待确认**:插件清单如何表达平台限制)。
- **改哪侧**:RPA(清洗);对面(清单平台字段,待确认后)。
- **方案**:build-catalog 归一化——源值不在 `win32|darwin|linux|null` 白名单的一律重派生;三端扩展节奏沿用 V1 第三节(Linux xdotool/AT-SPI、Mac AX/CGEvent/TCC、Office 分端实现),驱动实现统一走 ADR-R3 的原生进程通道。
- **验收**:catalog 全量过 schema 校验(1016/1016);非 win32 上 Core 可加载、桌面能力明确报 platform_unsupported;Mac/Linux CI 跑纯逻辑子集。
@ -260,7 +267,7 @@
- **兼容性**:不兼容。视觉能力在对面 backend 进程必然失败。
- **问题**:Core 不是 Electron-free;护栏查不出动态 import;单包安装缺 3 个依赖。
- **改哪侧**:RPA。
- **方案**:①抽 `ScreenCaptureHost`/`DisplayCoordinateService` 接口并入 HostBridge 注入面(弹窗/DPI×2/AI 配置 4 方法 → 增至 6 方法,或独立 host services 包——实现期定,倾向并入 HostBridge 减少 API 面);vision/cv-vision 只依赖接口,宿主未注入时抛 `host_service_unavailable`;②check-deps 升级为覆盖 `import()`/`require()`/`createRequire()`(先补正则,中期 AST);③补齐 3 个依赖声明(pdf.js 与 canvas 为原生/大件,评估 optionalDependencies + 能力级降级);④消费方式锁定 **Bun 源码消费**(对面全家 workspace:* 源码依赖,与现状一致),锁 bun@1.4.3 做 CI 冒烟。
- **方案(★ 已实现,2026-11-06 阶段 1-A9)**:①抽 `ScreenCaptureHost`/`DisplayCoordinateService` 接口并入 HostBridge 注入面(弹窗/DPI×2/AI 配置 4 方法 → 增至 6 方法,或独立 host services 包——实现期定,倾向并入 HostBridge 减少 API 面);vision/cv-vision 只依赖接口,宿主未注入时抛 `host_service_unavailable`;②check-deps 升级为覆盖 `import()`/`require()`/`createRequire()`(先补正则,中期 AST);③补齐 3 个依赖声明(pdf.js 与 canvas 为原生/大件,评估 optionalDependencies + 能力级降级);④消费方式锁定 **Bun 源码消费**(对面全家 workspace:* 源码依赖,与现状一致),锁 bun@1.4.3 做 CI 冒烟。
- **验收**:无可解析 electron 的纯净 Node/Bun 环境 Core 全量 import 成功;视觉能力未注入 Host 时返回 host_service_unavailable、注入 Fake Host 后可测;隔离目录仅按包声明装依赖跑通 PDF/OCR 代表能力。
### A10 插件生命周期、异常隔离与资源清理
@ -281,10 +288,10 @@
| 阶段 | 内容 | 前置 | 验收门 |
|---|---|---|---|
| **0 事实确认**(双侧,1 次评审会) | 评审 R2(带方案 A/B 对比与三项待答问题)+ 3 项 Q;对面确认 CapabilityDefinition 平台字段、utilityExtended 对应 scope;同步已拍板 5 项结论(R1/R3/R4/R5/R6)给对面并预估 C# actuate 扩展量 | 无 | R2 与 Q 有书面结论 |
| **1 RPA 侧收口**(本仓库) | A3 四件(取消贯通/状态映射/重试元数据/Graph Validator)+ A1 注入参数 + A9(Electron 残留、check-deps 升级、依赖闭包)+ A2 的 ADR 固化 + catalog 脏值清洗 + rpa-cli 重写(显式注册、经 engine 取 handler) | 无 | 审查报告 §7 第一/二阶段清单全绿;纯 Node 环境 Core 全量加载 |
| **2 迁入与 adapter 骨架** | git subtree 迁包改名 @agent-worker/rpa-core;bun@1.4.3 test 冒烟(exceljs/pdf/PowerShell 编码);新建 @agent-worker/rpa-adapter(CapabilityRunner + ref 映射 + 状态映射表);demo 侧冻结 | 阶段 1 + ADR-R1/R2 | bun test 纯逻辑全绿;未经 gateway 不可达 handler;719 映射完整性测试 |
| **3 垂直切片** | 一个文件能力 + 一个 HTTP 能力走通:workflow-engine create/start → gateway authorize → adapter → rpa-core handler → 事件/证据回传 → cancel 中途取消;方案 A(rpa.flow.execute 整体注册) | 阶段 2 | 取消/审批/审计全链有证据;对面 Run 终态与 RunResult 一致 |
| **0 事实确认**(双侧,1 次评审会) | 向对面同步 6 项已拍板结论(R1-R6)并预估 C# actuate 扩展量;确认 2 个小问题(Q1 platform 字段 / Q2 utilityExtended 权限归属);确认事件流接口与 rpa.browser 清单备案 | 无 | Q 有书面结论 |
| **1 RPA 侧收口**(本仓库)✅ **已完成(2026-11-06,提交 9b76dc6/1da0082/1a20dd7/5e8fc96)** | A3 四件(取消贯通/状态映射/重试元数据/Graph Validator)+ A1 注入参数 + A9(Electron 残留、check-deps 升级、依赖闭包)+ A2 的 ADR 固化 + catalog 脏值清洗 + rpa-cli 重写(显式注册、经 engine 取 handler) | 无 | 审查报告 §7 第一/二阶段清单全绿;纯 Node 环境 Core 全量加载 |
| **2 迁入与 adapter 骨架** | git subtree 迁包改名 @agent-worker/rpa-core;bun@1.4.3 test 冒烟(exceljs/pdf/PowerShell 编码);新建 @agent-worker/rpa-adapter(CapabilityRunner + ref 映射 + 状态映射表);A++ 事件负载字段设计定稿(nodeId/检查点/幂等键,见 §7-A5);demo 侧冻结 | 阶段 1 + ADR-R1/R2 | bun test 纯逻辑全绿;未经 gateway 不可达 handler;719 映射完整性测试 |
| **3 垂直切片** | 一个文件能力 + 一个 HTTP 能力走通:workflow-engine create/start → gateway authorize → adapter → rpa-core handler → 事件/证据回传 → cancel 中途取消;方案 A(rpa.flow.execute 整体注册)+ A++ 检查点续跑最小验证(崩溃→resumeToken 重入,见 §7-A5) | 阶段 2 | 取消/审批/审计全链有证据;对面 Run 终态与 RunResult 一致 |
| **4 浏览器与桌面双 ADR 落地** | ADR-R4(已拍板):RPA 浏览器能力修完 transport/resolver,接对面白名单与证据体系后作为 RPA 场景主力 runner;ADR-R3:windows-runner C# 扩展 actuate(RPA 五域逻辑移植) | 阶段 3 | §7-A6/A7 验收条目 |
| **5 能力规模化** | 34 域按风险分批:files/data/http/ftp → office/excel/word(COM 仅 win32)→ desktop/utility 高影响(末批+审批);CapabilityDefinition 与权限映射随批生成 | 阶段 4 | 每批过 gateway 全量冒烟 + 审批矩阵抽查 |
| **6 平台扩展** | V1 第三节三端计划原样生效(Linux/macOS 驱动按 ADR-R3 原生通道) | 阶段 5 | 三平台 CI 矩阵;产品按 platformTag 置灰 |
@ -304,9 +311,9 @@
| R3 | Registry 全局单例在多租户 backend 的隔离缺陷 | 高 | 短期按 ADR-R1 限定部署模型;长期 engine-scoped registry 列入 backlog |
| R4 | 浏览器双路径重复投资 | 低(已缓解) | R4 已拍板按场景分工(RPA=CDP 接管,AI 智能体=Playwright),白名单与证据共用一套,无重复投资 |
| R5 | 719 能力的治理映射与元数据补齐工作量失控 | 中 | 脚本从 catalog 生成初稿 + 人工只审高风险域;分批(阶段 5) |
| R6 | Bun 与 Node 行为差异(exceljs/pdf-lib/PowerShell 编码) | 中 | 阶段 2 bun 冒烟前置,差异清单化 |
| R6 | Bun 与 Node 行为差异(exceljs/pdf-lib/PowerShell 编码) | 中(部分缓解) | exceljs/pdf 已于 2026-11-06 本机 bun 1.4.4 冒烟通过(37/37);PowerShell 编码差异留待阶段 2 迁包后清单化 |
| R7 | demo 侧与 backend 双活跃漂移 | 中 | git subtree + "迁移后 demo 冻结"纪律(V1 决策 3) |
| R8 | 两套状态机语义漂移(NodeState vs StepStatus) | 中 | 映射表 + 契约测试常驻 CI |
| R8 | 两套状态机语义漂移(NodeState vs StepStatus) | 中 | 映射表 + 契约测试常驻 CI;A++ 检查点协议(ADR-R2 补记)统一崩溃恢复语义 |
| R9 | catalog 源数据(shizai)上游已删,重生成受阻 | 低 | 已从对面历史恢复至 temp/catalog-source;计划快照入库 |
| R10 | 对面底座继续快速演进,本文行号/接口过期 | 中 | 本文标注核查日期;联调前对关键接口做增量 diff |
@ -336,18 +343,16 @@
| R4 | 浏览器双路径 | **按场景分工**:RPA 场景走咱们的 CDP 接管路线(复用用户登录态/真实页面——RPA 核心价值);对面 PlaywrightBrowserRunner 留给 AI 智能体场景(无头自管浏览器)。约束:①咱们 CDP 路线必须接对面 BrowserContextPolicy 白名单与证据体系,不自建;②迁移前修完 transport 超时与定位器不统一两缺陷;③需对面在插件清单给咱们的 runner 备案(rpa.browser) |
| R5 | 包消费方式 | **源码并入 + 锁版本**:git subtree 进对面仓库统一编译,Bun 锁 1.4.3(与对面全家 workspace:* 源码消费现状一致) |
| R6 | 部署拓扑:rpa-core 各域跑在哪 | **按能力域拆分部署**:纯 Node 域→backend plugin-worker;桌面/视觉/弹窗域→用户电脑 Electron 宿主;同一包按 PLATFORM_GROUPS 分组加载,不维护两份代码 |
| R2 | 集成形态:RPA 流程=单个 capability(方案 A)还是节点=step 映射(方案 B) | **方案 A + 两补丁**:RPA 流程=`rpa.flow.execute` 单能力,对面一个 step 一次租约一次结算;节点级事件走 event-bus 旁路(保透明、不进权威账本);人审/高危节点照常升级对面审批。**不用 B 的理由**(记录在案):双账本一致性(崩溃窗口两边分歧、破坏对面 recovery 的单账本假设)、控制流节点无副作用无法映射 CapabilityRef、审批/重试粒度与业务语义错位。**B 的触发条件**:对面产品确认需要"逐步控制 RPA"(单步执行/任意步回退)时局部启用,事件流已备好数据不返工。**A++ 补记(2026-11-06 二次拍板)**:方案 A 骨架不变,节点事件流从"透明度日志"升级为**检查点协议**(nodeId+状态+幂等键+可恢复标记),崩溃后 `rpa.flow.execute` 带 resumeToken 重入、从最近检查点续跑,权威账本仍为单 step、无双账本;B 的隐藏触发条件(长流程断点续跑,RPA 桌面场景高发)由此覆盖而无需节点=step。业界对照:UiPath Orchestrator=Job 黑盒+日志旁路(≈A);Camunda/Zeebe External Task 的"单引擎假设"不适用于双引擎集成;AWS Step Functions callback≈补丁 2。两侧改动清单与落地节奏见 §7-A5 |
### 待对面对评审确认(不猜测)
| ADR | 问题 | 评审要点 |
|---|---|---|
| R2 | 集成形态:RPA 流程=单个 capability(方案 A)还是节点=step 映射(方案 B)? | 本方需对面补充信息后定:①对面产品上是否要求逐步审批/逐步可见(决定 A 是否够用);②对面 workflow-engine 的 step 粒度是否有上限或性能顾虑(719 能力×节点数映射成 step 的量级);③对面对 RPA 执行中途取消/暂停的交互预期。方案对比:A=对面一个 step 跑完一条 RPA 流程(工作量小、状态映射简单、对面只见"开始/结束"+事件流);B=每个节点一个 step(全程透明可逐步审批,工作量数倍、两套状态机对表复杂)。本方倾向 A 起步、协议预留节点级事件上报。 |
### 待对面对确认(仅剩小问题,不猜测)
| Q# | 问题 | 评审要点 |
|---|---|---|
| Q1 | CapabilityDefinition 是否需要 platform 字段(对面清单未见) | 建议对面补字段或约定由 catalog platformTag 承载 |
| Q2 | utilityExtended(命令执行)对应哪个 PermissionScope | 13 项中 desktop.process 是候选,需对面确认语义(backend 无桌面,进程执行属哪域) |
| Q3 | human-dialog 弹窗宿主落点 | 已被 R6 顺带回答:用户电脑 Electron 宿主窗口;仅弹窗样式/交互细节与对面 UI 对齐 |
> 已回答:Q3(human-dialog 弹窗落点)已由 R6 回答——用户电脑 Electron 宿主窗口;仅样式/交互细节与对面 UI 对齐。
---

View File

@ -15,7 +15,9 @@
"check:deps": "node scripts/check-deps.mjs"
},
"dependencies": {
"@cantoo/pdf-lib": "^2.11.1",
"@gutenye/ocr-node": "^1.4.9",
"@napi-rs/canvas": "^1.1.0",
"adm-zip": "^0.5.16",
"basic-ftp": "^6.2.3",
"exceljs": "^4.4.0",
@ -25,6 +27,7 @@
"mysql2": "^3.24.6",
"nodemailer": "^6.9.16",
"pdf-lib": "^1.17.1",
"pdfjs-dist": "^4.10.38",
"pg": "^8.23.1",
"pngjs": "^7.0.0",
"sm-crypto": "^0.5.8",

View File

@ -18,6 +18,10 @@ const pkgRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..")
const srcRoot = path.join(pkgRoot, "src");
const IMPORT_RE = /(?:import|export)\s+(?:[\s\S]*?from\s*)?["']([^"']+)["']/g;
// 动态形式:import("x") / require("x") / createRequire 后的 require 调用。
// 旧正则要求 import 后跟空白,`import("electron")` 完全不匹配(实测 0 命中,
// vision/cv-vision 曾靠这个盲区逃过检查——迁移阶段1-A9 修复)。
const DYNAMIC_IMPORT_RE = /(?:\bimport\s*\(|\brequire\s*\()\s*["']([^"']+)["']/g;
async function walk(dir) {
const entries = await readdir(dir, { withFileTypes: true });
@ -53,6 +57,17 @@ for (const file of await walk(srcRoot)) {
}
}
}
for (const match of source.matchAll(DYNAMIC_IMPORT_RE)) {
const spec = match[1];
const rel = path.relative(pkgRoot, file).replaceAll("\\", "/");
if (spec === "electron" || spec.startsWith("electron/")) {
offenders.push(`${rel}: 禁止动态 import("${spec}")(宿主能力走 HostBridge 注入)`);
continue;
}
if (spec.includes("src/main")) {
offenders.push(`${rel}: 禁止动态引用宿主模块 "${spec}"`);
}
}
}
if (offenders.length) {

View File

@ -8,7 +8,7 @@ import { powershell, psString } from "../driver/windows/powershell";
import { defaultEngine } from "../engine/engine";
// 窗口/屏幕查询统一走驱动(任务 3.3)
const driver = defaultEngine.driver;
const getDriver = () => defaultEngine.driver;
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) { const value = args[key]; if (value !== undefined && value !== null && value !== "") return String(value); }
@ -24,21 +24,21 @@ register("Window_GetRect", delegate("Window_GetWindowRect"));
register("Window_MaximumSize", (ctx, node, args, component) => delegate("Window_SetWindowState")(ctx, node, { ...args, WindowType: "maximize" }, component));
register(["Keyboard_DriveSend", "Keyboard_DriveSend_V1", "Keyboard_DriveSend_V2"], delegate("Keyboard_Send_V1"));
register("Window_GetScreenSize", async () => driver.screen.size());
register("Window_GetScreenSize", async () => getDriver().screen.size());
register("Window_WindowList", async (_ctx, _node, args) => {
const title = text(args, ["match_title", "select_title"]); const regex = String(args.is_regex ?? "0") === "1";
return driver.window.list({ title, regex });
return getDriver().window.list({ title, regex });
});
register("Window_SetWindowShow", async (_ctx, node, args) => {
const title = text(args, ["窗口对象", "window_obj", "title"]); if (!title || title === "窗口对象") throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
const show = !text(args, ["WindowType"], "show").toLowerCase().includes("hide");
await driver.window.setState(title, show ? 5 : 0);
await getDriver().window.setState(title, show ? 5 : 0);
return show;
});
register("Window_WaitWindow", async (_ctx, node, args) => {
const title = text(args, ["match_title", "select_title", "window_obj", "target_window"]); if (!title) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
const expected = text(args, ["switchType"], "0") !== "1"; const deadline = Date.now() + Math.max(0, Number(args.second ?? 5)) * 1000;
do { const exists = await driver.window.exists(title); if (exists === expected) return true; await new Promise((resolve) => setTimeout(resolve, 200)); } while (Date.now() <= deadline);
do { const exists = await getDriver().window.exists(title); if (exists === expected) return true; await new Promise((resolve) => setTimeout(resolve, 200)); } while (Date.now() <= deadline);
return false;
});

View File

@ -10,30 +10,50 @@ const detectMock = vi.hoisted(() => vi.fn(async () => ({
{ text: "确认", box: [[100, 200], [130, 200], [130, 210], [100, 210]] }
]
})));
// desktopCapturer 模拟: 默认无画面(走 GDI 回退), 多屏用例按请求宽度提供对应显示器画面
// 屏幕服务模拟: 默认无画面(走 GDI 回退), 多屏用例按显示器 id/请求宽度提供对应画面
const electronState = vi.hoisted(() => ({ sources: [] as Array<{ display_id: string; thumbnail: any }> }));
vi.mock("../driver/windows/powershell", () => ({ powershell: powershellMock }));
vi.mock("@gutenye/ocr-node", () => ({
default: { create: vi.fn(async () => ({ detect: detectMock })) }
}));
vi.mock("electron", () => ({
desktopCapturer: {
getSources: vi.fn(async (opts: { thumbnailSize: { width: number } }) =>
electronState.sources.filter(s => s.thumbnail.getSize().width === opts.thumbnailSize.width))
},
screen: {
getAllDisplays: vi.fn(() => [
{ id: 1, bounds: { x: 0, y: 0, width: 1920, height: 1080 }, scaleFactor: 1 },
{ id: 2, bounds: { x: 1920, y: 0, width: 1280, height: 720 }, scaleFactor: 2 }
]),
dipToScreenPoint: vi.fn((p: { x: number; y: number }) => ({ x: p.x, y: p.y }))
}
}));
import { getHandler } from "../engine/registry";
import { setHostBridge, type HostDisplay } from "../engine/host-bridge";
import "./cv-vision";
const displayOf = (id: string, x: number, width: number, height: number, scaleFactor: number): HostDisplay => ({
id,
bounds: { x, y: 0, width, height },
scaleFactor
});
const displays: HostDisplay[] = [
displayOf("1", 0, 1920, 1080, 1),
displayOf("2", 1920, 1280, 720, 2)
];
setHostBridge({
createFormWindow: () => { throw new Error("not used in cv-vision tests"); },
dipToScreenPoint: (point) => ({ x: point.x, y: point.y }),
screenToDipPoint: (point) => ({ x: point.x, y: point.y }),
loadAiConfig: () => ({ baseUrl: "", apiKey: "", model: "", visionModel: "" }),
screen: {
listDisplays: () => displays,
primaryDisplay: () => displays[0],
dipToScreenPoint: (point) => ({ x: point.x, y: point.y }),
displayMatching: () => displays[0],
captureDisplay: async (displayId: string, wantWidth: number, wantHeight: number) => {
// 模拟真实 captureDisplay 的对位规则,但不做 sources[0] 兜底:
// 保持"主屏无画面 → 跳过该屏"的旧多屏语义
const source = electronState.sources.find((item) => item.display_id === displayId && item.thumbnail.getSize().width === wantWidth)
|| electronState.sources.find((item) => item.thumbnail.getSize().width === wantWidth);
if (!source || source.thumbnail.isEmpty()) throw new Error(`无法获取显示器 ${displayId} 的屏幕截图`);
void wantHeight;
return source.thumbnail.toPNG();
}
}
});
describe("CV vision capabilities", () => {
beforeEach(() => {
powershellMock.mockClear();

View File

@ -1,5 +1,6 @@
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { requireScreenService } from "../engine/host-bridge";
import { powershell } from "../driver/windows/powershell";
import fs from "node:fs";
import os from "node:os";
@ -44,30 +45,27 @@ async function gdiGrabScreen(imagePath: string): Promise<void> {
/** 一次屏幕截图: 图片左上角在虚拟桌面上的绝对 DIP 坐标 + 物理/DIP 缩放比 */
interface ScreenShot { imagePath: string; dipX: number; dipY: number; scale: number }
/** desktopCapturer 逐显示器截屏(每屏原生分辨率), 不可用时回退 GDI 全虚拟桌面单图 */
/** 宿主屏幕服务逐显示器截屏(每屏原生分辨率), 不可用时回退 GDI 全虚拟桌面单图 */
async function grabScreenAll(baseDir: string): Promise<ScreenShot[]> {
fs.mkdirSync(baseDir, { recursive: true });
try {
const { desktopCapturer, screen } = await import("electron");
if (desktopCapturer && screen?.getAllDisplays) {
const shots: ScreenShot[] = [];
const displays = screen.getAllDisplays();
for (const display of displays) {
const scale = display.scaleFactor || 1;
const sources = await desktopCapturer.getSources({
types: ["screen"],
thumbnailSize: { width: Math.round(display.bounds.width * scale), height: Math.round(display.bounds.height * scale) }
});
const source = sources.find(item => item.display_id === String(display.id))
|| sources.find(item => item.thumbnail.getSize().width === Math.round(display.bounds.width * scale))
|| sources[0];
if (!source || source.thumbnail.isEmpty()) continue;
const screenHost = requireScreenService();
const shots: ScreenShot[] = [];
for (const display of screenHost.listDisplays()) {
const scale = display.scaleFactor || 1;
try {
const pngBytes = await screenHost.captureDisplay(
display.id,
Math.round(display.bounds.width * scale),
Math.round(display.bounds.height * scale)
);
if (!pngBytes?.length) continue;
const imagePath = path.join(baseDir, `rpa-cv-screen-${display.id}-${Date.now()}.png`);
fs.writeFileSync(imagePath, source.thumbnail.toPNG());
fs.writeFileSync(imagePath, Buffer.from(pngBytes));
shots.push({ imagePath, dipX: display.bounds.x, dipY: display.bounds.y, scale });
}
if (shots.length) return shots;
} catch {}
}
if (shots.length) return shots;
} catch {}
const imagePath = path.join(baseDir, `rpa-cv-screen-${Date.now()}.png`);
await gdiGrabScreen(imagePath);
@ -207,14 +205,11 @@ function coordinateScript(x: number, y: number, action: "hover" | "click" | "dou
return lines.join("\n");
}
/** DIP → 物理像素(SetCursorPos 在 DPI 感知进程里用物理坐标); 无 Electron API 时按原值(缩放 100% 等价) */
/** DIP → 物理像素(SetCursorPos 在 DPI 感知进程里用物理坐标); 无宿主屏幕服务时按原值(缩放 100% 等价) */
async function toPhysical(x: number, y: number): Promise<{ x: number; y: number }> {
try {
const { screen } = await import("electron");
if (screen && typeof screen.dipToScreenPoint === "function") {
const p = screen.dipToScreenPoint({ x: Math.round(x), y: Math.round(y) });
return { x: p.x, y: p.y };
}
const point = requireScreenService().dipToScreenPoint({ x: Math.round(x), y: Math.round(y) });
return { x: point.x, y: point.y };
} catch {}
return { x: Math.round(x), y: Math.round(y) };
}

View File

@ -6,7 +6,9 @@ import { powershell, psString } from "../driver/windows/powershell";
// 人为拟真设置本体挂在引擎实例上(任务 1.3),此处委托 defaultEngine。
const manualOperation: ManualOperationSettings = defaultEngine.manualOperation;
// 桌面自动化驱动(任务 3.3):user32 注入原语统一走 WindowsDriver
const driver = defaultEngine.driver;
// 惰性获取驱动(迁移阶段1-A1):非 win32 上 import 本模块不再抛错,
// 调用桌面能力时才触发 platform_unsupported;也使注入 driver 的引擎可被测试/宿主使用
const getDriver = () => defaultEngine.driver;
function jitter(value: number, amount = 3): number {
return Math.max(0, Math.round(value + (Math.random() * amount * 2 - amount)));
@ -63,7 +65,7 @@ const keyboardSend: CapabilityHandler = async (_ctx, node, args) => {
const isTextSend = node.componentId === "Keyboard_Send" || node.componentId === "Keyboard_Send_V1";
// 指定目标窗口时先激活再输入,内容才会进到目标窗口而不是运行时的焦点窗口
const targetTitle = text(args, ["window_title", "target_window", "window_obj", "match_title", "select_title"]);
await driver.keyboard.send({ keys, intervalMs, mode: mode === "2" ? "clipboard" : mode === "1" ? "manual" : "api", textMode: isTextSend, activateTitle: targetTitle || undefined });
await getDriver().keyboard.send({ keys, intervalMs, mode: mode === "2" ? "clipboard" : mode === "1" ? "manual" : "api", textMode: isTextSend, activateTitle: targetTitle || undefined });
await manualDelay();
return keys;
};
@ -88,47 +90,47 @@ const disableManualOperation: CapabilityHandler = async () => {
const mouseClick: CapabilityHandler = async (_ctx, node, args) => {
const [x, y] = manualJitter(point(args.position ?? args.point ?? args.location ?? "[0,0]"), false, true);
const button = text(args, ["side", "button"], "0").toLowerCase();
await driver.mouse.click({ x, y }, { button: button.includes("right") || button === "1" ? "right" : "left" });
await getDriver().mouse.click({ x, y }, { button: button.includes("right") || button === "1" ? "right" : "left" });
return { x, y };
};
const mouseMove: CapabilityHandler = async (_ctx, node, args) => {
const [x, y] = manualJitter(point(args.position ?? args.point ?? args.location ?? "[0,0]"), true, false);
await driver.mouse.moveTo({ x, y });
await getDriver().mouse.moveTo({ x, y });
return { x, y };
};
const mouseDrag: CapabilityHandler = async (_ctx, node, args) => {
const [sx, sy] = point(args.Start ?? args.start ?? "[0,0]");
const [ex, ey] = point(args.End ?? args.end ?? "[0,0]");
await driver.mouse.drag({ x: sx, y: sy }, { x: ex, y: ey }, { holdMs: 80, releaseDelayMs: 80 });
await getDriver().mouse.drag({ x: sx, y: sy }, { x: ex, y: ey }, { holdMs: 80, releaseDelayMs: 80 });
return { start: [sx, sy], end: [ex, ey] };
};
const mouseRoll: CapabilityHandler = async (_ctx, node, args) => {
const amount = number(args, ["RollNum", "rollNum", "amount"], 1) * 120;
await driver.mouse.scroll(amount);
await getDriver().mouse.scroll(amount);
return amount;
};
const setActiveWindow: CapabilityHandler = async (_ctx, node, args) => {
const selector = text(args, ["window_obj", "match_title", "select_title", "title", "target_window", "window_title"]);
if (!selector) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
return driver.window.focus(selector);
return getDriver().window.focus(selector);
};
const getActiveWindow: CapabilityHandler = async () => driver.window.foregroundTitle();
const getActiveWindow: CapabilityHandler = async () => getDriver().window.foregroundTitle();
const windowExists: CapabilityHandler = async (_ctx, node, args) => {
const selector = text(args, ["window_obj", "match_title", "select_title", "title", "target_window", "window_title"]);
if (!selector) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
return driver.window.exists(selector);
return getDriver().window.exists(selector);
};
const closeWindow: CapabilityHandler = async (_ctx, node, args) => {
const selector = text(args, ["window_obj", "match_title", "select_title", "title", "target_window", "window_title"]);
if (!selector) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
return driver.window.close(selector);
return getDriver().window.close(selector);
};
const runApp: CapabilityHandler = async (_ctx, node, args) => {

View File

@ -33,7 +33,7 @@ function selector(args: Record<string, unknown>): string {
}
// 窗口矩形/状态操作统一走驱动(任务 3.3)
const driver = defaultEngine.driver;
const getDriver = () => defaultEngine.driver;
register("System_GetEnv", (_ctx, node, args) => {
const name = required(args, ["name"], "环境变量名称", node.id, node.componentId);
@ -104,14 +104,14 @@ register("Clipboard_ClearClipboard", async () => {
register("Window_GetWindowRect", async (_ctx, node, args) => {
const title = required({ title: selector(args) }, ["title"], "窗口标题", node.id, node.componentId);
return driver.window.findByTitle(title);
return getDriver().window.findByTitle(title);
});
register(["Window_SetWindowState", "Window_SetWindowState_v1"], async (_ctx, node, args) => {
const title = required({ title: selector(args) }, ["title"], "窗口标题", node.id, node.componentId);
const state = value(args, ["WindowType", "switchType"], "maximize").toLowerCase();
const command = state.includes("min") ? 6 : state.includes("max") ? 3 : 9;
await driver.window.setState(title, command);
await getDriver().window.setState(title, command);
return state;
});
@ -124,6 +124,6 @@ register("Window_SizeWindow_v1", async (_ctx, node, args) => {
}
const [width, height] = Array.isArray(size) ? size.map(Number) : [0, 0];
if (!(width > 0 && height > 0)) throw new RuntimeError("validation_error", "窗口尺寸无效", { nodeId: node.id, componentId: node.componentId });
await driver.window.setSize(title, width, height);
await getDriver().window.setSize(title, width, height);
return { width, height };
});

View File

@ -3,6 +3,7 @@ import { promises as fs } from "node:fs";
import { PNG } from "pngjs";
import { register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import { requireScreenService, type HostDisplay } from "../engine/host-bridge";
import { powershell, psString } from "../driver/windows/powershell";
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
@ -15,28 +16,20 @@ function text(args: Record<string, unknown>, keys: string[], fallback = ""): str
* 逐屏截图后按各自物理原点拼到一张画布上——目标在副屏也能搜到。
* 返回 originX/originY = 拼接画布左上角对应的屏幕物理坐标(副屏在主屏左侧时为负),用于结果坐标平移。 */
async function captureVirtualScreenPng(): Promise<{ png: PNG; originX: number; originY: number }> {
const { desktopCapturer, screen } = await import("electron");
const displays = screen.getAllDisplays();
const screenHost = requireScreenService();
const displays = screenHost.listDisplays();
if (displays.length === 0) throw new RuntimeError("desktop_error", "无法获取屏幕信息");
// 每屏按自己的物理分辨率截图(display_id 匹配 → 缩略图尺寸匹配 → 按物理原点排序对位)
const captures: { display: Electron.Display; origin: { x: number; y: number }; png: PNG }[] = [];
// 每屏按自己的物理分辨率截图(宿主按 displayId 对位 → 按物理原点排序对位)
const captures: { display: HostDisplay; origin: { x: number; y: number }; png: PNG }[] = [];
for (const display of displays) {
const scale = display.scaleFactor || 1;
const wantW = Math.round(display.bounds.width * scale);
const wantH = Math.round(display.bounds.height * scale);
const sources = await desktopCapturer.getSources({
types: ["screen"],
thumbnailSize: { width: wantW, height: wantH }
});
const source = sources.find(item => item.display_id === String(display.id))
|| sources.find(item => item.thumbnail.getSize().width === wantW)
|| sources[0];
if (!source || source.thumbnail.isEmpty()) {
throw new RuntimeError("desktop_error", "无法获取屏幕截图");
}
const pngBytes = await screenHost.captureDisplay(display.id, wantW, wantH);
if (!pngBytes?.length) throw new RuntimeError("desktop_error", "无法获取屏幕截图");
// 显示器 DIP 原点的物理坐标 = 该屏物理原点(屏内线性映射: 物理 = (DIP-原点)*缩放 + 物理原点)
const origin = screen.dipToScreenPoint({ x: display.bounds.x, y: display.bounds.y });
captures.push({ display, origin, png: PNG.sync.read(source.thumbnail.toPNG()) });
const origin = screenHost.dipToScreenPoint({ x: display.bounds.x, y: display.bounds.y });
captures.push({ display, origin, png: PNG.sync.read(Buffer.from(pngBytes)) });
}
const originX = Math.min(...captures.map(c => c.origin.x));
const originY = Math.min(...captures.map(c => c.origin.y));
@ -63,9 +56,9 @@ async function captureVirtualScreenPng(): Promise<{ png: PNG; originX: number; o
/** DIP 区域 → 物理像素区域:左上角经 dipToScreenPoint 换算(天然支持副屏/负坐标),宽高乘所在屏缩放比 */
async function dipRegionToPhysical(region: { x: number; y: number; width: number; height: number }): Promise<{ x: number; y: number; width: number; height: number }> {
const { screen } = await import("electron");
const topLeft = screen.dipToScreenPoint({ x: Math.round(region.x), y: Math.round(region.y) });
const display = screen.getDisplayMatching({
const screenHost = requireScreenService();
const topLeft = screenHost.dipToScreenPoint({ x: Math.round(region.x), y: Math.round(region.y) });
const display = screenHost.displayMatching({
x: region.x, y: region.y,
width: Math.max(1, Math.round(region.width)), height: Math.max(1, Math.round(region.height))
});
@ -429,7 +422,7 @@ async function locate(args: Record<string, unknown>, evidenceDir: string, nodeId
const rawAccuracy = Number(text(args, ["accuracy"], "0.8"));
const similarity = Math.max(0.05, Math.min(1, Number.isFinite(rawAccuracy) && rawAccuracy > 0 ? rawAccuracy : 0.8));
// 原生 1:1 匹配失败时, 按缩放比对模板放大/缩小重试——兼容旧版 DIP 截图存的模板与任意来源的图片
const scale = (await import("electron")).screen.getPrimaryDisplay().scaleFactor || 1;
const scale = requireScreenService().primaryDisplay().scaleFactor || 1;
const result = await findImage(template, screenshot, 1 - similarity, scale !== 1 ? [scale, 1 / scale] : []);
if (!result) throw new RuntimeError("not_found", `屏幕中未找到目标图像(要求相似度 ${(similarity * 100).toFixed(0)}%,未达到阈值)`, { nodeId, recommendation: "重新框选模板(框小一点、只框目标图案),或适当调低相似度(如 0.8);目标在副屏时确认其清晰可见" });
return { ...result, x: result.x + offsetX, y: result.y + offsetY };

View File

@ -4,7 +4,7 @@ import { getHostBridge } from "../engine/host-bridge";
import { defaultEngine } from "../engine/engine";
// 桌面自动化驱动(任务 3.3):窗口矩形/鼠标注入原语统一走 WindowsDriver
const driver = defaultEngine.driver;
const getDriver = () => defaultEngine.driver;
function value(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) {
@ -77,17 +77,17 @@ function physicalToDip(x: number, y: number): [number, number] {
return [Math.round(x), Math.round(y)];
}
const getWindow: CapabilityHandler = async (_ctx, node, args) => driver.window.findByTitle(requiredTitle(args, node.id, node.componentId));
const getWindow: CapabilityHandler = async (_ctx, node, args) => getDriver().window.findByTitle(requiredTitle(args, node.id, node.componentId));
register(["Window_Get", "Window_GetWindow", "Window_GetWindow_v1"], getWindow);
register("Window_ActiveWindow_v1", async (_ctx, node, args) => {
const title = requiredTitle(args, node.id, node.componentId);
const maximize = [true, 1, "1", "true", "True"].includes(args.isMax as never);
return driver.window.activate(title, { maximize });
return getDriver().window.activate(title, { maximize });
});
register(["Window_GetWindowMessage", "Window_GetWindowMessage_v1"], async (_ctx, node, args) => {
const object = await driver.window.findByTitle(requiredTitle(args, node.id, node.componentId));
const object = await getDriver().window.findByTitle(requiredTitle(args, node.id, node.componentId));
const type = value(args, ["switchType"], "all").toLowerCase();
if (type === "title") return object.title;
if (type === "pid") return object.pid;
@ -98,7 +98,7 @@ register(["Window_GetWindowMessage", "Window_GetWindowMessage_v1"], async (_ctx,
register(["Window_SetPos", "Window_MoveWindow", "Window_MoveWindow_v1", "Window_SizeWindow"], async (_ctx, node, args) => {
const title = requiredTitle(args, node.id, node.componentId);
const current = await driver.window.findByTitle(title);
const current = await getDriver().window.findByTitle(title);
const moveOnly = node.componentId.includes("MoveWindow");
const sizeOnly = node.componentId === "Window_SizeWindow";
const position = args.location !== undefined ? parsePoint(args.location, "窗口位置", node.id, node.componentId) : [Number(args.x ?? current.x), Number(args.y ?? current.y)] as [number, number];
@ -109,7 +109,7 @@ register(["Window_SetPos", "Window_MoveWindow", "Window_MoveWindow_v1", "Window_
const width = moveOnly ? Number(current.width) : size[0];
const height = moveOnly ? Number(current.height) : size[1];
if (!(width > 0 && height > 0)) throw new RuntimeError("validation_error", "窗口尺寸必须大于 0", { nodeId: node.id, componentId: node.componentId });
return driver.window.setPos(title, { x, y, width, height });
return getDriver().window.setPos(title, { x, y, width, height });
});
const moveMouse: CapabilityHandler = async (_ctx, node, args) => {
@ -122,7 +122,7 @@ const moveMouse: CapabilityHandler = async (_ctx, node, args) => {
let dipTarget: [number, number];
if (relative) {
// 相对激活窗口左上角:窗口矩形是物理像素,换回 DIP 再叠加偏移,最后统一转物理
const { x: wx, y: wy } = await driver.window.foregroundPosition();
const { x: wx, y: wy } = await getDriver().window.foregroundPosition();
if (!Number.isFinite(wx) || !Number.isFinite(wy)) throw new RuntimeError("browser_error", "无法获取激活窗口位置", { nodeId: node.id, componentId: node.componentId });
const [dipWx, dipWy] = physicalToDip(wx, wy);
dipTarget = [dipWx + rawX, dipWy + rawY];
@ -131,10 +131,10 @@ const moveMouse: CapabilityHandler = async (_ctx, node, args) => {
}
const [tx, ty] = dipToPhysical(dipTarget[0], dipTarget[1]);
if (moveType === "2") {
await driver.mouse.moveTo({ x: tx, y: ty });
await getDriver().mouse.moveTo({ x: tx, y: ty });
} else {
// 平滑移动:从当前光标位置出发,按速度插值(路径全程物理像素,与目标同一坐标系)
await driver.mouse.moveSmooth({ x: tx, y: ty }, { ease: moveType === "0", speed });
await getDriver().mouse.moveSmooth({ x: tx, y: ty }, { ease: moveType === "0", speed });
}
return { x: Math.round(dipTarget[0]), y: Math.round(dipTarget[1]) };
};
@ -147,7 +147,7 @@ const dragMouse: CapabilityHandler = async (_ctx, node, args) => {
const [epx, epy] = dipToPhysical(ex, ey);
const right = ["1", "right"].includes(value(args, ["side"], "0").toLowerCase());
// 旧实现按下后停 80ms、拖到终点立即松开(与桌面版 Window_Mouse 不同,松开前不停顿)
await driver.mouse.drag({ x: spx, y: spy }, { x: epx, y: epy }, { button: right ? "right" : "left", holdMs: 80, releaseDelayMs: 0 });
await getDriver().mouse.drag({ x: spx, y: spy }, { x: epx, y: epy }, { button: right ? "right" : "left", holdMs: 80, releaseDelayMs: 0 });
return { start: [Math.round(sx), Math.round(sy)], end: [Math.round(ex), Math.round(ey)] };
};
register(["Mouse_DragToV1", "Mouse_DragToV2", "Mouse_DragToNew"], dragMouse);
@ -159,6 +159,6 @@ register(["Mouse_ClickCombine", "Mouse_ClickCombine_v1", "Mouse_ClickCombine_v2"
const right = value(args, ["click_key"], "left").toLowerCase() === "right";
const doubleClick = value(args, ["click_time"], "click").toLowerCase().includes("double");
// 旧实现每次 down-up 之后停 80ms
await driver.mouse.click({ x: px, y: py }, { button: right ? "right" : "left", double: doubleClick, settleMs: 80 });
await getDriver().mouse.click({ x: px, y: py }, { button: right ? "right" : "left", double: doubleClick, settleMs: 80 });
return { x: Math.round(x), y: Math.round(y), button: right ? "right" : "left", clicks: doubleClick ? 2 : 1 };
});

View File

@ -56,4 +56,27 @@ describe("Word extended capabilities", () => {
await run("Word_WriteTable", { objWord: handle, data: [["name", "age"], ["Alice", "30"]] });
await expect(run("Word_ReadTable", { objWord: handle, find_type: "content", table_content: "Alice" })).resolves.toEqual([["name", "age"], ["Alice", "30"]]);
}, 180000);
it("saves and saves-as a Word document through real COM", async () => {
if (process.platform !== "win32") return;
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-word-saveas-")); directories.push(directory);
const handle = await run("Word_New_V1", { filePath: directory, name: "source.docx" });
await run("Word_Write", { objWord: handle, text: "另存前内容" });
await expect(run("Word_Save", { objWord: handle })).resolves.toEqual({ path: path.join(directory, "source.docx") });
await expect(run("Word_Save", { objWord: { path: path.join(directory, "missing.docx") } })).rejects.toThrow("Word文件不存在");
const targetDir = path.join(directory, "out");
const saved = await run("Word_SaveAs", { objWord: handle, fileAddr: targetDir, filename: "renamed" }) as { path: string };
expect(saved.path).toBe(path.join(targetDir, "renamed.docx"));
await expect(readFile(saved.path)).resolves.toBeTruthy();
const kept = await run("Word_SaveAs", { objWord: saved, fileAddr: directory }) as { path: string };
expect(kept.path).toBe(path.join(directory, "renamed.docx"));
await expect(readFile(kept.path)).resolves.toBeTruthy();
const relative = await run("Word_SaveAs", { objWord: kept, fileAddr: "sub", filename: "rel-target" }) as { path: string };
expect(relative.path).toBe(path.join(directory, "sub", "rel-target.docx"));
await expect(readFile(relative.path)).resolves.toBeTruthy();
}, 180000);
});

View File

@ -1,4 +1,5 @@
import path from "node:path";
import { access, mkdir } from "node:fs/promises";
import { register } from "../engine/registry";
import { powershell, psString } from "../driver/windows/powershell";
@ -41,4 +42,31 @@ register("Word_Write", async (_ctx, _node, args) => {
return { path: filePath } satisfies WordHandle;
});
register(["Word_Save", "Word_SaveAs"], async (_ctx, _node, args) => ({ path: handlePath(args) } satisfies WordHandle));
async function ensureFile(value: string): Promise<string> {
try {
await access(value);
} catch {
throw new Error(`Word文件不存在: ${value}`);
}
return value;
}
register("Word_Save", async (_ctx, _node, args) => {
const filePath = await ensureFile(ensure(handlePath(args), "Word文件路径"));
// 打开即存:架构上每次操作独立开合 Word 实例,Save 负责兜底刷新并证明文件真实可写
await powershell(wordScript(`$doc = $word.Documents.Open(${psString(filePath)}); $doc.Save()`));
return { path: filePath } satisfies WordHandle;
});
register("Word_SaveAs", async (_ctx, _node, args) => {
const previousPath = await ensureFile(ensure(handlePath(args), "Word文件路径"));
const rawDir = text(args, ["fileAddr", "path", "folder", "savePath"]);
const rawName = text(args, ["filename", "name"]);
// 目录空缺沿用原目录;目录相对时基于原文档目录解析;文件名空缺沿用原名(与 catalog 语义一致)
const dir = rawDir ? path.resolve(path.dirname(previousPath), rawDir) : path.dirname(previousPath);
const target = path.resolve(dir, rawName || path.basename(previousPath));
const withExt = path.extname(target) ? target : `${target}.docx`;
await mkdir(path.dirname(withExt), { recursive: true });
await powershell(wordScript(`$doc = $word.Documents.Open(${psString(previousPath)}); $doc.SaveAs2(${psString(withExt)}); ${psString(withExt)}`));
return { path: withExt } satisfies WordHandle;
});

View File

@ -0,0 +1,112 @@
import { describe, expect, it } from "vitest";
import type { CatalogComponent, FlowDefinition, FlowNode, RunEvent } from "../types";
import { register } from "./registry";
import { NODE_STATUS_TO_STEP_STATUS, toStepStatus } from "./types";
import { runFlow } from "./runtime";
type EmittedEvent = Omit<RunEvent, "runId">;
type RunFlowOptions = Parameters<typeof runFlow>[1];
function makeFlow(partial: Partial<FlowDefinition> & { nodes: FlowNode[] }): FlowDefinition {
return {
schemaVersion: 1,
id: "flow_checkpoint",
name: "检查点测试流程",
variables: [],
nodes: partial.nodes,
edges: partial.edges || [],
meta: { createdAt: "2026-01-01T00:00:00Z", updatedAt: "2026-01-01T00:00:00Z", appVersion: "test" }
};
}
function node(id: string, componentId: string): FlowNode {
return { id, componentId, position: { x: 0, y: 0 }, values: {}, options: {} };
}
const fakeComponent = (id: string): CatalogComponent =>
({
id,
name: id,
version: "1.0.0",
tags: [],
platformTag: null,
icon: "",
risk: "low",
fields: [],
output: { label: "", key: "", suggestedName: "", valueTypes: [], fields: [] },
slots: [],
functionCalls: []
}) as unknown as CatalogComponent;
const baseOptions = (emit: (event: EmittedEvent) => void): RunFlowOptions => ({
runId: "run_cp",
evidenceDir: ".",
signal: new AbortController().signal,
emit,
log: () => {},
componentOf: (n: FlowNode) => fakeComponent(n.componentId),
variables: {}
});
register("Cp_Idempotent", async () => "ok", { idempotent: true });
register("Cp_NonIdempotent", async () => "ok", { idempotent: false });
register("Cp_DefaultMeta", async () => "ok");
describe("状态映射(NodeState ↔ 对面 StepStatus)", () => {
it("8 个状态全分支映射,唯一非同名是 idle→pending", () => {
expect(toStepStatus("idle")).toBe("pending");
for (const status of ["running", "waiting_user", "succeeded", "failed", "cancelled", "unknown", "skipped"] as const) {
expect(toStepStatus(status)).toBe(status);
}
});
it("映射表为全量双射(无遗漏键)", () => {
expect(Object.keys(NODE_STATUS_TO_STEP_STATUS).sort()).toEqual(
["cancelled", "failed", "idle", "running", "skipped", "succeeded", "unknown", "waiting_user"]
);
});
});
describe("A++ 检查点事件负载", () => {
it("node.finished 携带 checkpointId/resumable/幂等键,非幂等不带键", async () => {
const events: EmittedEvent[] = [];
const flow = makeFlow({
nodes: [node("first", "Cp_Idempotent"), node("second", "Cp_NonIdempotent"), node("third", "Cp_DefaultMeta")],
edges: [
{ id: "e1", source: "first", target: "second" },
{ id: "e2", source: "second", target: "third" }
]
});
const result = await runFlow(flow, baseOptions((event) => events.push(event)));
expect(result.status).toBe("succeeded");
const finished = events.filter((e) => e.type === "node.finished" && e.status === "succeeded");
expect(finished).toHaveLength(3);
const first = finished.find((e) => e.nodeId === "first")!;
expect(first.checkpointId).toBe("first");
expect(first.resumable).toBe(true);
expect(first.idempotencyKey).toBe("run_cp:first");
const second = finished.find((e) => e.nodeId === "second")!;
expect(second.checkpointId).toBe("second");
expect(second.resumable).toBe(false);
expect(second.idempotencyKey).toBeUndefined();
const third = finished.find((e) => e.nodeId === "third")!;
expect(third.resumable).toBe(true);
expect(third.idempotencyKey).toBe("run_cp:third");
});
it("幂等键由 runId+nodeId 派生,跨重入稳定(重入去重的基石)", async () => {
const flow = makeFlow({ nodes: [node("only", "Cp_Idempotent")] });
const runOnce = async (runId: string): Promise<string | undefined> => {
const list: EmittedEvent[] = [];
await runFlow(flow, { ...baseOptions((event) => list.push(event)), runId });
return list.find((e) => e.type === "node.finished")?.idempotencyKey;
};
expect(await runOnce("run_a")).toBe("run_a:only");
expect(await runOnce("run_a")).toBe("run_a:only");
expect(await runOnce("run_b")).toBe("run_b:only");
});
});

View File

@ -1,4 +1,5 @@
import { describe, expect, it } from "vitest";
import type { AutomationDriver } from "../driver/types";
import { createEngine } from "./engine";
import { WindowsDriver } from "../driver/windows/windows-driver";
@ -20,3 +21,50 @@ describe("driver 平台自动选择(迁移指南五件小事 #2)", () => {
}
});
});
describe("createEngine 注入(迁移阶段1-A1)", () => {
const fakeDriver = (): AutomationDriver =>
({
mouse: {},
keyboard: {},
window: {},
screen: { size: async () => ({ x: 0, y: 0, width: 800, height: 600 }) },
element: {}
}) as unknown as AutomationDriver;
it("createEngine 可注入自定义 driver", () => {
const driver = fakeDriver();
const engine = createEngine({ driver });
expect(engine.driver).toBe(driver);
});
it("注入后不触发平台自动选择(darwin 上也可用)", () => {
const driver = fakeDriver();
const engine = createEngine({ driver });
const original = process.platform;
Object.defineProperty(process, "platform", { value: "darwin", configurable: true });
try {
expect(engine.driver).toBe(driver);
expect(engine.driver.screen.size).toBeTruthy();
} finally {
Object.defineProperty(process, "platform", { value: original, configurable: true });
}
});
it("两个引擎注入不同 driver 互不影响", () => {
const driverA = fakeDriver();
const driverB = fakeDriver();
const engineA = createEngine({ driver: driverA });
const engineB = createEngine({ driver: driverB });
expect(engineA.driver).toBe(driverA);
expect(engineB.driver).toBe(driverB);
expect(engineA.driver).not.toBe(engineB.driver);
});
it("setDriver 可后置替换", () => {
const engine = createEngine();
const driver = fakeDriver();
engine.setDriver(driver);
expect(engine.driver).toBe(driver);
});
});

View File

@ -2,7 +2,9 @@ import type { Client } from "basic-ftp";
import type { Workbook as ExcelJSWorkbook } from "exceljs";
import { WindowsDriver } from "../driver/windows/windows-driver";
import type { AutomationDriver } from "../driver/types";
import { RuntimeError } from "./types";
import type { CatalogComponent, FlowNode } from "../types";
import { getHandler } from "./registry";
import { RuntimeError, type RunContext } from "./types";
/** 人为拟真操作的开关与随机参数(desktop 能力的进程级设置)。 */
export interface ManualOperationSettings {
@ -59,10 +61,40 @@ export class RpaEngine {
}
return this._driver;
}
/** 注入驱动实现(测试替身或宿主适配器);注入后不再走平台自动选择 */
setDriver(driver: AutomationDriver): void {
this._driver = driver;
}
/**
* 单能力直接调用(ADR-R1 语义真实化):CLI/宿主脚本不再裸摸全局 getHandler,
* 而是经引擎入口调用。注意 handler 仍解析自进程注册表(ADR-R1 短期形态)。
*/
async execute(
componentId: string,
args: Record<string, unknown>,
ctx: RunContext,
node?: FlowNode,
component?: CatalogComponent
): Promise<unknown> {
const handler = getHandler(componentId);
if (!handler) {
throw new RuntimeError("capability_not_implemented", `能力 ${componentId} 未注册`, { componentId });
}
const resolvedNode: FlowNode = node ?? { id: "direct", componentId, position: { x: 0, y: 0 }, values: {}, options: {} };
return handler(ctx, resolvedNode, args, component as CatalogComponent);
}
}
export function createEngine(): RpaEngine {
return new RpaEngine();
export interface EngineOptions {
/** 注入桌面自动化驱动(测试替身或宿主适配器);缺省按 process.platform 自动选择 */
driver?: AutomationDriver;
}
export function createEngine(options?: EngineOptions): RpaEngine {
const engine = new RpaEngine();
if (options?.driver) engine.setDriver(options.driver);
return engine;
}
/** 能力模块内部使用的默认引擎实例(与全局注册表配套的进程级单例)。 */

View File

@ -0,0 +1,117 @@
import type { FlowDefinition } from "../types";
/**
* 流程图静态校验(任务:审查报告 P0-03)。
* 在执行前拦截结构性错误,避免运行期以诡异方式失败。
* 只做纯结构检查,不读目录/注册表(组件是否存在由运行时校验并给出友好错误)。
*/
export interface FlowIssue {
severity: "error" | "warning";
code: string;
nodeId?: string;
edgeId?: string;
message: string;
}
const CONTROL_FLOW_COMPONENTS = new Set([
"logic_control_if_else",
"logic_control_count_for",
"logic_control_array_for_v3",
"logic_control_dictionary_for_v1",
"logic_control_while",
"infinite_loop",
"logic_control_try_catch"
]);
export function validateFlowGraph(flow: FlowDefinition): FlowIssue[] {
const issues: FlowIssue[] = [];
const nodeIds = new Set<string>();
for (const node of flow.nodes) {
if (nodeIds.has(node.id)) {
issues.push({ severity: "error", code: "duplicate_node_id", nodeId: node.id, message: `节点 ID 重复:${node.id}` });
}
nodeIds.add(node.id);
if (node.parentId && !flow.nodes.some((candidate) => candidate.id === node.parentId)) {
issues.push({
severity: "error",
code: "orphan_child",
nodeId: node.id,
message: `节点「${node.label || node.id}」挂在不存在的父节点 ${node.parentId} 下`
});
}
}
for (const edge of flow.edges) {
if (!nodeIds.has(edge.source)) {
issues.push({ severity: "error", code: "dangling_edge", edgeId: edge.id, message: `连线 ${edge.id} 的起点 ${edge.source} 不存在于画布` });
}
if (!nodeIds.has(edge.target)) {
issues.push({ severity: "error", code: "dangling_edge", edgeId: edge.id, message: `连线 ${edge.id} 的终点 ${edge.target} 不存在于画布` });
}
if (edge.source === edge.target && nodeIds.has(edge.source)) {
issues.push({
severity: "error",
code: "self_loop",
edgeId: edge.id,
nodeId: edge.source,
message: `节点「${edge.source}」连接到自己,执行时会立即跳过`
});
}
}
// 顶层图环检测:执行引擎从起点沿边游走且不重访,环意味着存在永远不会执行的分支或意图不明的回边
const top = flow.nodes.filter((node) => !node.parentId);
const topIds = new Set(top.map((node) => node.id));
const edges = flow.edges.filter((edge) => topIds.has(edge.source) && topIds.has(edge.target));
const adjacency = new Map<string, string[]>();
for (const edge of edges) {
if (edge.source === edge.target) continue;
adjacency.set(edge.source, [...(adjacency.get(edge.source) || []), edge.target]);
}
const incoming = new Set(edges.map((edge) => edge.target));
const start =
top.find((node) => !incoming.has(node.id) && node.componentId.startsWith("Trigger.")) ||
top.find((node) => !incoming.has(node.id)) ||
top[0];
if (start) {
const state = new Map<string, "visiting" | "done">();
const reachable = new Set<string>();
const visit = (nodeId: string, path: string[]): void => {
if (state.get(nodeId) === "done") return;
if (state.get(nodeId) === "visiting") {
const cycleStart = path.indexOf(nodeId);
const cycle = [...path.slice(cycleStart), nodeId].join(" → ");
issues.push({ severity: "error", code: "cycle", nodeId, message: `顶层流程存在环,环上的节点执行顺序不明确:${cycle}` });
return;
}
state.set(nodeId, "visiting");
reachable.add(nodeId);
for (const next of adjacency.get(nodeId) || []) visit(next, [...path, nodeId]);
state.set(nodeId, "done");
};
visit(start.id, [start.id]);
for (const node of top) {
if (!reachable.has(node.id)) {
const knownContainer = CONTROL_FLOW_COMPONENTS.has(node.componentId);
if (!knownContainer && !flow.edges.some((edge) => edge.target === node.id)) {
issues.push({
severity: "warning",
code: "unreachable_node",
nodeId: node.id,
message: `节点「${node.label || node.id}」没有从起点可达的连线,执行时不会运行`
});
}
}
}
}
return issues;
}
export function hasBlockingIssues(issues: FlowIssue[]): boolean {
return issues.some((issue) => issue.severity === "error");
}

View File

@ -1,4 +1,5 @@
import type { FlowDefinition } from "../types";
import { RuntimeError } from "./types";
/**
* 宿主桥(任务 2.1 正式化)。
@ -46,7 +47,34 @@ export interface HostAiConfig {
visionModel: string;
}
/** 宿主能力桥最小面(决策点 4):dialog 窗口工厂 / DPI 坐标换算 / 配置读取 */
/** 显示器信息(视觉能力的屏幕枚举用;id 为宿主侧稳定标识字符串) */
export interface HostDisplay {
id: string;
bounds: { x: number; y: number; width: number; height: number };
scaleFactor: number;
}
/**
* 屏幕服务(迁移阶段1-A9):视觉能力原本经动态 import("electron") 拿
* desktopCapturer/screen,现改为宿主注入,Core 回归零 Electron 依赖。
*/
export interface HostScreenService {
/** 全部显示器 */
listDisplays(): HostDisplay[];
/** 主显示器 */
primaryDisplay(): HostDisplay;
/** DIP 屏幕坐标 → 物理像素 */
dipToScreenPoint(point: HostPoint): HostPoint;
/** 与该 DIP 矩形相交面积最大的显示器 */
displayMatching(rect: { x: number; y: number; width: number; height: number }): HostDisplay;
/**
* 抓取指定显示器当前画面,像素尺寸尽量接近 want 值,返回 PNG 字节。
* 无画面/抓取失败时抛错(调用方决定回退策略)。
*/
captureDisplay(displayId: string, wantWidth: number, wantHeight: number): Promise<Uint8Array>;
}
/** 宿主能力桥最小面(决策点 4):dialog 窗口工厂 / DPI 坐标换算 / 配置读取 / 屏幕服务 */
export interface HostBridge {
/** 创建人机交互表单窗口(human-dialog 用) */
createFormWindow(options: { title: string }): HostDialogWindow;
@ -56,6 +84,8 @@ export interface HostBridge {
screenToDipPoint(point: HostPoint): HostPoint;
/** 读取 AI 服务配置(ai-service 用) */
loadAiConfig(): HostAiConfig;
/** 屏幕服务(vision/cv-vision 截屏与多屏几何);未注入时视觉能力报 host_unavailable */
screen?: HostScreenService;
}
let hostBridge: HostBridge | undefined;
@ -67,3 +97,14 @@ export function setHostBridge(bridge: HostBridge): void {
export function getHostBridge(): HostBridge | undefined {
return hostBridge;
}
/** 视觉能力取屏幕服务的统一入口:未注入宿主桥/屏幕服务时抛稳定错误码 */
export function requireScreenService(): HostScreenService {
const bridge = getHostBridge();
if (!bridge?.screen) {
throw new RuntimeError("host_unavailable", "宿主未注入屏幕服务(HostBridge.screen),截屏/视觉能力不可用", {
recommendation: "宿主启动时需调用 setHostBridge 并提供 screen 服务(参考 demo 的 src/main/index.ts)"
});
}
return bridge.screen;
}

View File

@ -29,6 +29,12 @@ export interface CapabilityMeta {
errorCodes?: string[];
/** 执行中是否需要回传截图(Agent 侧视觉判断用) */
needsScreenshot?: boolean;
/**
* 能力是否可安全重试(无副作用或副作用可重复)。
* false 时运行时强制忽略节点的 retryCount(重试门控,防重复副作用);
* 缺省表示未知,遵循流程显式配置。
*/
idempotent?: boolean;
/** 平台限制;省略表示跨平台 */
platforms?: CapabilityPlatform[];
}

View File

@ -0,0 +1,204 @@
import { describe, expect, it } from "vitest";
import type { CatalogComponent, FlowDefinition, FlowNode } from "../types";
import { validateFlowGraph, hasBlockingIssues } from "./graph-validator";
import { register } from "./registry";
import { RuntimeError } from "./types";
import { runFlow } from "./runtime";
type RunFlowOptions = Parameters<typeof runFlow>[1];
function makeFlow(partial: Partial<FlowDefinition> & { nodes: FlowNode[] }): FlowDefinition {
return {
schemaVersion: 1,
id: "flow_test",
name: "测试流程",
variables: [],
nodes: partial.nodes,
edges: partial.edges || [],
meta: { createdAt: "2026-01-01T00:00:00Z", updatedAt: "2026-01-01T00:00:00Z", appVersion: "test" }
};
}
function node(id: string, componentId: string, overrides: Partial<FlowNode> = {}): FlowNode {
return {
id,
componentId,
position: { x: 0, y: 0 },
values: {},
options: {},
...overrides
};
}
const fakeComponent = (id: string): CatalogComponent =>
({
id,
name: id,
version: "1.0.0",
tags: [],
platformTag: null,
icon: "",
risk: "low",
fields: [],
output: { label: "", key: "", suggestedName: "", valueTypes: [], fields: [] },
slots: [],
functionCalls: []
}) as unknown as CatalogComponent;
const baseOptions = (signal: AbortSignal): RunFlowOptions => ({
runId: "run_test",
evidenceDir: ".",
signal,
emit: () => {},
log: () => {},
componentOf: (n: FlowNode) => fakeComponent(n.componentId),
variables: {}
});
const hangUntilAborted = (ctx: { signal: AbortSignal }): Promise<never> =>
new Promise((_, reject) => {
const onAbort = () => reject(new RuntimeError("cancelled", "运行已取消"));
if (ctx.signal.aborted) return onAbort();
ctx.signal.addEventListener("abort", onAbort, { once: true });
});
register("Test_HangHandler", (ctx) => hangUntilAborted(ctx));
register("Test_SlowHandler", async (ctx) => {
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(resolve, 5000);
const onAbort = () => {
clearTimeout(timer);
reject(new RuntimeError("cancelled", "运行已取消"));
};
if (ctx.signal.aborted) return onAbort();
ctx.signal.addEventListener("abort", onAbort, { once: true });
});
return "done";
});
let nonIdempotentCalls = 0;
register("Test_NonIdempotent", async () => {
nonIdempotentCalls += 1;
throw new Error("boom");
}, { idempotent: false });
let idempotentCalls = 0;
register("Test_Idempotent", async () => {
idempotentCalls += 1;
if (idempotentCalls < 3) throw new Error("transient");
return "ok";
}, { idempotent: true });
describe("validateFlowGraph", () => {
it("正常图无问题", () => {
const flow = makeFlow({
nodes: [node("a", "Trigger.Manual"), node("b", "File_Read")],
edges: [{ id: "e1", source: "a", target: "b" }]
});
expect(hasBlockingIssues(validateFlowGraph(flow))).toBe(false);
});
it("重复节点 ID 记为错误", () => {
const flow = makeFlow({ nodes: [node("a", "Trigger.Manual"), node("a", "File_Read")] });
const issues = validateFlowGraph(flow);
expect(issues.some((issue) => issue.code === "duplicate_node_id" && issue.severity === "error")).toBe(true);
});
it("悬空边记为错误", () => {
const flow = makeFlow({
nodes: [node("a", "Trigger.Manual")],
edges: [{ id: "e1", source: "a", target: "ghost" }]
});
expect(validateFlowGraph(flow).some((issue) => issue.code === "dangling_edge")).toBe(true);
});
it("自环记为错误", () => {
const flow = makeFlow({
nodes: [node("a", "Trigger.Manual")],
edges: [{ id: "e1", source: "a", target: "a" }]
});
expect(validateFlowGraph(flow).some((issue) => issue.code === "self_loop")).toBe(true);
});
it("顶层环记为错误", () => {
const flow = makeFlow({
nodes: [node("a", "Trigger.Manual"), node("b", "File_Read"), node("c", "File_Write")],
edges: [
{ id: "e1", source: "a", target: "b" },
{ id: "e2", source: "b", target: "c" },
{ id: "e3", source: "c", target: "b" }
]
});
expect(validateFlowGraph(flow).some((issue) => issue.code === "cycle" && issue.severity === "error")).toBe(true);
});
it("孤儿子节点与不可达节点", () => {
const flow = makeFlow({
nodes: [node("a", "Trigger.Manual"), { ...node("x", "File_Read"), parentId: "ghost" }, node("y", "Http_Get")]
});
const issues = validateFlowGraph(flow);
expect(issues.some((issue) => issue.code === "orphan_child")).toBe(true);
expect(issues.some((issue) => issue.code === "unreachable_node")).toBe(true);
});
});
describe("runtime 取消/超时/重试门控", () => {
it("超时会中止监听尝试信号的 handler", async () => {
const flow = makeFlow({
nodes: [{ ...node("slow", "Test_SlowHandler"), options: { timeoutMs: 120 } }]
});
const controller = new AbortController();
const result = await runFlow(flow, baseOptions(controller.signal));
expect(result.status).toBe("failed");
expect(result.error?.code).toBe("timeout");
expect(result.error?.reason).toContain("执行超时");
});
it("运行取消时节点状态为 cancelled 且不走 error route", async () => {
const flow = makeFlow({
nodes: [node("n1", "Test_HangHandler"), node("err", "File_Write")],
edges: [{ id: "e1", source: "n1", target: "err", sourceHandle: "error" }]
});
const controller = new AbortController();
setTimeout(() => controller.abort(), 50);
const result = await runFlow(flow, baseOptions(controller.signal));
expect(result.status).toBe("cancelled");
const nodeState = result.nodes.find((item) => item.nodeId === "n1");
expect(nodeState?.status).toBe("cancelled");
// error 边存在,但取消不应路由到 err 节点
expect(result.nodes.some((item) => item.nodeId === "err")).toBe(false);
});
it("非幂等能力忽略 retryCount", async () => {
nonIdempotentCalls = 0;
const flow = makeFlow({
nodes: [{ ...node("r", "Test_NonIdempotent"), options: { retryCount: 3, retryIntervalMs: 1000 } }]
});
const controller = new AbortController();
const result = await runFlow(flow, baseOptions(controller.signal));
expect(result.status).toBe("failed");
expect(nonIdempotentCalls).toBe(1);
});
it("幂等能力遵循 retryCount 重试后成功", async () => {
idempotentCalls = 0;
const flow = makeFlow({
nodes: [{ ...node("r", "Test_Idempotent"), options: { retryCount: 3, retryIntervalMs: 1000, timeoutMs: 5000 } }]
});
const controller = new AbortController();
const result = await runFlow(flow, baseOptions(controller.signal));
expect(result.status).toBe("succeeded");
expect(idempotentCalls).toBe(3);
});
it("流程图校验失败返回 validation_error 且不执行任何节点", async () => {
const flow = makeFlow({ nodes: [node("a", "Trigger.Manual"), node("a", "File_Read")] });
const controller = new AbortController();
const result = await runFlow(flow, baseOptions(controller.signal));
expect(result.status).toBe("failed");
expect(result.error?.code).toBe("validation_error");
expect(result.error?.reason).toContain("流程图校验未通过");
expect(result.nodes).toHaveLength(0);
});
});

View File

@ -1,6 +1,7 @@
import type { FlowDefinition, FlowNode, RunError } from "../types";
import { evaluate, evaluateCondition, interpolate, toNumber } from "./expression";
import { getHandler } from "./registry";
import { hasBlockingIssues, validateFlowGraph } from "./graph-validator";
import { getCapabilityMeta, getHandler } from "./registry";
import {
asRuntimeError,
BreakSignal,
@ -138,6 +139,17 @@ export async function runFlow(
options.emit({ type: "run.started", startedAt });
try {
const issues = validateFlowGraph(flow);
for (const issue of issues.filter((item) => item.severity === "warning")) {
options.log(issue.message, "warn");
}
if (hasBlockingIssues(issues)) {
const summary = issues.filter((issue) => issue.severity === "error").map((item) => item.message).join(";");
throw new RuntimeError("validation_error", `流程图校验未通过:${summary}`, {
recommendation: "修复流程图结构后再运行",
detail: JSON.stringify(issues.filter((issue) => issue.severity === "error"))
});
}
await executeTopLevelGraph(flow, ctx, states);
} catch (error) {
if (error instanceof StopSignal) {
@ -202,6 +214,9 @@ async function executeTopLevelGraph(flow: FlowDefinition, ctx: RunContext, state
try {
await executeNode(node, ctx, states);
} catch (error) {
if (error instanceof BreakSignal || error instanceof ContinueSignal || error instanceof StopSignal) throw error;
// 取消不进异常分支:中止运行时直接上抛,由 runFlow 统一收口为 cancelled
if (asRuntimeError(error).code === "cancelled") throw error;
const errorEdge = edges.find((edge) => edge.source === node.id && edge.sourceHandle === "error");
const target = errorEdge ? topById.get(errorEdge.target) : undefined;
if (!target) throw error;
@ -273,23 +288,48 @@ async function executeNode(node: FlowNode, ctx: RunContext, states: Map<string,
const delayBefore = Math.max(0, Number(node.options?.delayBeforeMs || 0));
const delayAfter = Math.max(0, Number(node.options?.delayAfterMs || 0));
const timeoutMs = Math.max(0, Number(node.options?.timeoutMs || 30000));
const retries = Math.max(0, Math.min(100, Number(node.options?.retryCount || 0)));
const effectiveTimeout = timeoutMs > 0 ? timeoutMs : 30000;
const meta = getCapabilityMeta(node.componentId) || getCapabilityMeta(component.functionName);
let retries = Math.max(0, Math.min(100, Number(node.options?.retryCount || 0)));
// 重试门控:实现方声明非幂等的能力,无视流程里的重试配置(防重复副作用)
if (retries > 0 && meta?.idempotent === false) {
retries = 0;
ctx.log(`能力「${component.name}」为非幂等操作,已忽略重试设置(retryCount)`, "warn", node.id);
}
const interval = Math.max(1000, Number(node.options?.retryIntervalMs || 1000));
await sleep(delayBefore, ctx.signal);
for (let attempt = 1; attempt <= retries + 1; attempt += 1) {
state.attempt = attempt;
// 尝试级取消源:run 取消与节点超时都汇聚到它,真正中止底层操作
const attemptCtl = new AbortController();
const onRunAbort = () => attemptCtl.abort();
if (ctx.signal.aborted) attemptCtl.abort();
else ctx.signal.addEventListener("abort", onRunAbort, { once: true });
let timeoutError: RuntimeError | undefined;
const timeoutTimer = setTimeout(() => {
timeoutError = new RuntimeError("timeout", `${component.name} 执行超时(${effectiveTimeout}ms)`, {
recommendation: "检查目标是否可达,或调大该节点的超时时间",
nodeId: node.id,
componentId: component.id
});
attemptCtl.abort(timeoutError);
}, effectiveTimeout);
// handler 收到的 ctx.signal 是尝试级信号:它内部的 sleep/HTTP/子进程监听它即可被中止
const attemptCtx: RunContext = { ...ctx, signal: attemptCtl.signal };
try {
result = await withTimeout(
Promise.resolve(handler(ctx, node, args, component)),
timeoutMs,
ctx.signal,
Promise.resolve(handler(attemptCtx, node, args, component)),
effectiveTimeout,
attemptCtl.signal,
component.name
);
break;
} catch (error) {
if (error instanceof BreakSignal || error instanceof ContinueSignal || error instanceof StopSignal) throw error;
const runtimeError = asRuntimeError(error, node, component.id);
// 超时中止底层后,handler 可能先抛出 cancelled;若中止原因是超时则归一为 timeout
if (attemptCtl.signal.aborted && timeoutError && runtimeError.code === "cancelled") throw timeoutError;
if (attempt > retries || !isRetryable(runtimeError.code)) {
throw new RuntimeError(runtimeError.code, runtimeError.reason, {
recommendation: runtimeError.recommendation,
@ -304,6 +344,9 @@ async function executeNode(node: FlowNode, ctx: RunContext, states: Map<string,
node.id
);
await sleep(interval, ctx.signal);
} finally {
clearTimeout(timeoutTimer);
ctx.signal.removeEventListener("abort", onRunAbort);
}
}
if (node.options?.outputVar) ctx.vars[node.options.outputVar] = result;
@ -314,13 +357,20 @@ async function executeNode(node: FlowNode, ctx: RunContext, states: Map<string,
state.output = result;
state.finishedAt = new Date().toISOString();
state.durationMs = Date.now() - startMs;
// A++ 检查点协议:成功节点即续跑锚点;checkpointId 取节点 id(跨 attempt 稳定);
// 幂等键同样跨 attempt 稳定,供带 resumeToken 重入时去重;非幂等能力不作为锚点
const checkpointMeta = getCapabilityMeta(node.componentId) || getCapabilityMeta(component.functionName);
const resumable = checkpointMeta?.idempotent !== false;
ctx.emit({
type: "node.finished",
nodeId: node.id,
status: "succeeded",
output: result,
durationMs: state.durationMs,
evidence: state.evidence
evidence: state.evidence,
checkpointId: node.id,
resumable,
...(resumable ? { idempotencyKey: `${ctx.runId}:${node.id}` } : {})
});
return result;
} catch (error) {
@ -332,6 +382,22 @@ async function executeNode(node: FlowNode, ctx: RunContext, states: Map<string,
throw error;
}
const runtimeError = asRuntimeError(error, node, component.id);
// 取消语义:用户中止运行时节点状态为 cancelled,不适用 skipOnError,也不进异常分支
if (runtimeError.code === "cancelled" || ctx.signal.aborted) {
state.error = runtimeError;
state.status = "cancelled";
state.finishedAt = new Date().toISOString();
state.durationMs = Date.now() - startMs;
ctx.emit({
type: "node.finished",
nodeId: node.id,
status: "cancelled",
error: runtimeError,
durationMs: state.durationMs,
evidence: []
});
throw error;
}
state.error = runtimeError;
state.finishedAt = new Date().toISOString();
state.durationMs = Date.now() - startMs;

View File

@ -139,7 +139,8 @@ export type CapabilityHandler = (
export interface NodeState {
nodeId: string;
componentId: string;
status: "idle" | "running" | "succeeded" | "failed" | "skipped" | "cancelled";
// waiting_user:节点挂起等待人审(对面 approval 流);unknown:崩溃后结果不可知(对面 workflow-recovery 对齐)
status: "idle" | "running" | "succeeded" | "failed" | "skipped" | "cancelled" | "waiting_user" | "unknown";
startedAt?: string;
finishedAt?: string;
durationMs?: number;
@ -149,6 +150,24 @@ export interface NodeState {
attempt: number;
}
// 与对面 workflow-engine StepStatus(8 态)同构;唯一非同名映射是 idle↔pending
export type WorkflowStepStatus = "pending" | "running" | "waiting_user" | "succeeded" | "failed" | "cancelled" | "unknown" | "skipped";
export const NODE_STATUS_TO_STEP_STATUS: Record<NodeState["status"], WorkflowStepStatus> = {
idle: "pending",
running: "running",
waiting_user: "waiting_user",
succeeded: "succeeded",
failed: "failed",
cancelled: "cancelled",
unknown: "unknown",
skipped: "skipped"
};
export function toStepStatus(status: NodeState["status"]): WorkflowStepStatus {
return NODE_STATUS_TO_STEP_STATUS[status];
}
export interface RunResult {
runId: string;
status: "succeeded" | "failed" | "cancelled";

View File

@ -181,6 +181,10 @@ export interface RunEvent {
durationMs?: number;
evidence?: string[];
attempt?: number;
// A++ 检查点协议负载(ADR-R2 补记):node.finished 成功时由 runtime 填充
checkpointId?: string;
idempotencyKey?: string;
resumable?: boolean;
startedAt?: string;
nodeLabel?: string;
total?: number;

View File

@ -0,0 +1,24 @@
// sm-crypto 无官方类型声明;仅声明本仓库用到的 SM4 解密 API
declare module "sm-crypto" {
export interface Sm4Options {
padding?: "pkcs#7" | "none";
mode?: "cbc";
iv?: string;
}
export const sm4: {
decrypt(
cipherText: string,
key: string,
options?: Sm4Options
): string;
encrypt(plainText: string, key: string, options?: Sm4Options): string;
};
export const sm2: {
doEncrypt(msgString: string, publicKey: string, cipherMode?: 0 | 1): string;
doDecrypt(
cipherText: string,
privateKey: string,
cipherMode?: 0 | 1
): string;
};
}

View File

@ -1,5 +1,5 @@
{
"generatedAt": "2026-11-05T09:52:08.862Z",
"generatedAt": "2026-11-06T03:06:19.728Z",
"count": 909,
"ids": [
"App_GetPid",

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@ -75,7 +75,9 @@ describe("应用与系统测试流程包", () => {
expect(issues.every(issue => issue.code === "required"), flow.name).toBe(true);
if (issues.length) blocked.push(flow.id);
}
expect(blocked).toEqual(["test_appsystem_08", "test_appsystem_12"]);
// 12 号(锁屏/解锁)曾经因 Screenlock 的 6 个假必填而阻塞;
// 2026-11-05 参数审计确认实现不读这些字段,已改选填并清空,阻塞随之解除(有意变更)
expect(blocked).toEqual(["test_appsystem_08"]);
const pidOnly = buildSuite()[0].flow;
const killV1 = pidOnly.nodes.find(n => n.componentId === "App_KillV1");
killV1.values.p_name = "";

View File

@ -243,7 +243,9 @@ function normalizeVendorComponent(component, vendorMap) {
// 任务 3.4(决策点 7)+ 五件小事 #1:platformTag 从包内 PLATFORM_GROUPS 单一源派生
// (能力 id 命中包内注册 → win32;未实现的能力与宿主原生节点 → null 跨平台占位)。
const platforms = domainPlatformMap.get(component.id) || component.platforms;
const platformTag = component.platformTag || (platforms?.includes("win32") ? "win32" : null);
// A8 归一化:源值仅接受 win32|darwin|linux 白名单字符串,其余(含上游 shizai 的数字 1 脏值)一律重派生
const sourcePlatform = ["win32", "darwin", "linux"].includes(component.platformTag) ? component.platformTag : null;
const platformTag = sourcePlatform || (platforms?.includes("win32") ? "win32" : null);
return {
id: component.id,
name: component.name,

View File

@ -1,6 +1,7 @@
/**
* rpa-core CLI host(任务 5.1):Node 直接拉起 createEngine() + WindowsDriver,
* 不经过 Electron,验证包的零宿主可运行性。
* rpa-core CLI host(任务 5.1;迁移阶段1-T4 重写):
* Node/Bun 直接拉起 createEngine(),按需显式注册 5 个能力域(不再副作用全量注册),
* 能力调用统一走 engine.execute(ADR-R1 语义真实化),验证包的零宿主可运行性。
*
* 跑通 5 个代表能力(全部只读或落在临时目录,可重复执行):
* 1. 桌面(PowerShell user32):Window_GetActiveWindow —— 读前台窗口标题
@ -17,12 +18,9 @@ import path from "node:path";
import {
createEngine,
capabilityCount,
getHandler,
registerAll,
type CapabilityHandler,
type RunContext
} from "@rpa/core";
import "@rpa/core/capabilities";
const ctx = {
runId: "rpa-cli",
@ -37,26 +35,27 @@ const ctx = {
} as unknown as RunContext;
const node = (componentId: string) => ({ id: "rpa-cli", componentId, position: { x: 0, y: 0 }, values: {}, options: {} });
async function run(name: string, handler: CapabilityHandler, args: Record<string, unknown>): Promise<void> {
async function run(engine: import("@rpa/core").RpaEngine, name: string, args: Record<string, unknown>): Promise<void> {
process.stdout.write(`[${name}] `);
const result = await handler(ctx, node(name), args, undefined as never);
const result = await engine.execute(name, args, ctx, node(name));
console.log("OK ->", typeof result === "string" ? result.slice(0, 120) : JSON.stringify(result)?.slice(0, 200));
}
async function main(): Promise<void> {
const engine = createEngine();
await registerAll(engine);
console.log(`rpa-core CLI host:引擎已创建,注册能力 ${capabilityCount()} 个\n`);
// 显式注册本 CLI 用到的 5 个能力域(迁移后 Agent-Employee 侧同样按域注册)
await registerAll(engine, ["desktop", "commonFinal", "excelCommon", "files", "http"]);
console.log(`rpa-core CLI host:引擎已创建(注入 driver 可用 createEngine({ driver })),注册能力 ${capabilityCount()} 个\n`);
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "rpa-cli-"));
try {
await run("Window_GetActiveWindow", getHandler("Window_GetActiveWindow")!, {});
await run("Window_GetScreenSize", getHandler("Window_GetScreenSize")!, {});
await run("Excel_Cs_NewV5", getHandler("Excel_Cs_NewV5")!, { filePath: tmpDir, name: "rpa-cli-demo.xlsx" });
await run(engine, "Window_GetActiveWindow", {});
await run(engine, "Window_GetScreenSize", {});
await run(engine, "Excel_Cs_NewV5", { filePath: tmpDir, name: "rpa-cli-demo.xlsx" });
const file = path.join(tmpDir, "hello.txt");
await run("File_Write", getHandler("File_Write")!, { path: file, content: "hello from rpa-core CLI" });
await run("File_Read", getHandler("File_Read")!, { path: file });
await run("Http_Get", getHandler("Http_Get")!, { url: "https://www.baidu.com", timeout: 15 });
await run(engine, "File_Write", { path: file, content: "hello from rpa-core CLI" });
await run(engine, "File_Read", { path: file });
await run(engine, "Http_Get", { url: "https://www.baidu.com", timeout: 15 });
console.log("\n5 项代表能力全部通过 ✓");
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });

View File

@ -1,4 +1,4 @@
import { app, BrowserWindow, dialog, ipcMain, screen, shell } from "electron";
import { app, BrowserWindow, desktopCapturer, dialog, ipcMain, screen, shell } from "electron";
import path from "node:path";
import type { AppInfo, AppSettings, FlowDefinition, FlowSaveOptions, FlowSaveResult, FlowSummary, FlowsImportResult } from "../shared/types";
import {
@ -24,7 +24,38 @@ import { logMain } from "./logger";
// 子流程仓库在宿主(flows.ts),包内能力经 host-bridge 取用(阶段 2 正式化 HostBridge)
setSubflowSource({ listFlows: async () => listFlows(), getFlow: async (id) => getFlow(id) });
// 宿主能力桥(决策点 4 最小面):人机交互窗口工厂 / DPI 坐标换算 / AI 配置读取
// 宿主屏幕服务(迁移阶段1-A9):视觉能力经宿主桥拿 desktopCapturer/screen,Core 零 Electron 依赖
const hostScreenService = {
listDisplays: () =>
screen.getAllDisplays().map((display) => ({
id: String(display.id),
bounds: display.bounds,
scaleFactor: display.scaleFactor || 1
})),
primaryDisplay: () => {
const display = screen.getPrimaryDisplay();
return { id: String(display.id), bounds: display.bounds, scaleFactor: display.scaleFactor || 1 };
},
dipToScreenPoint: (point: { x: number; y: number }) => screen.dipToScreenPoint(point),
displayMatching: (rect: { x: number; y: number; width: number; height: number }) => {
const display = screen.getDisplayMatching(rect);
return { id: String(display.id), bounds: display.bounds, scaleFactor: display.scaleFactor || 1 };
},
captureDisplay: async (displayId: string, wantWidth: number, wantHeight: number): Promise<Uint8Array> => {
const sources = await desktopCapturer.getSources({
types: ["screen"],
thumbnailSize: { width: wantWidth, height: wantHeight }
});
const source =
sources.find((item) => item.display_id === displayId) ||
sources.find((item) => item.thumbnail.getSize().width === wantWidth) ||
sources[0];
if (!source || source.thumbnail.isEmpty()) throw new Error(`无法获取显示器 ${displayId} 的屏幕截图`);
return source.thumbnail.toPNG();
}
};
// 宿主能力桥(决策点 4 最小面):人机交互窗口工厂 / DPI 坐标换算 / AI 配置读取 / 屏幕服务
setHostBridge({
createFormWindow: ({ title }) => {
const win = new BrowserWindow({
@ -46,7 +77,8 @@ setHostBridge({
loadAiConfig: () => {
const ai = loadSettings().aiConfig;
return { baseUrl: ai.baseUrl ?? "", apiKey: ai.apiKey ?? "", model: ai.model ?? "", visionModel: ai.visionModel ?? "" };
}
},
screen: hostScreenService
});
let mainWindow: BrowserWindow | null = null;