diff --git a/docs/prompt-graphs.md b/docs/prompt-graphs.md new file mode 100644 index 0000000000..43ab24b456 --- /dev/null +++ b/docs/prompt-graphs.md @@ -0,0 +1,124 @@ +# Generic-engine prompt graphs + +AI Personas can store a declarative prompt graph and send it to the engine at +invocation. The graph is configuration: no deployment or feature flag is +needed once it is stored on the persona. + +The ready-to-store workshop example is +[`prompt-graphs/workshop-design.authoring.json`](./prompt-graphs/workshop-design.authoring.json). +It is the server-authoring conversion of the Virtual Contributor's +[`workshop-design.json`](https://github.com/alkem-io/virtual-contributor/blob/develop/docs/prompt-graphs/workshop-design.json). +For engine semantics and validation errors, see the Virtual Contributor +[prompt-graphs README](https://github.com/alkem-io/virtual-contributor/blob/develop/docs/prompt-graphs/README.md). + +## Engines that receive a stored graph + +| Engine | Stored graph attached at invocation? | Reason | +| --- | --- | --- | +| `expert` | Yes | The expert plugin executes prompt graphs. If no graph is stored, it receives its existing expert default graph. | +| `generic-openai` | Yes | The generic plugin executes declarative prompt graphs. It receives no fallback graph when none is stored. | +| `guidance` | No | Its plugin has no prompt-graph execution path. | +| `openai-assistant` | No | Its plugin has no prompt-graph execution path. | +| `libra-flow` | No | Its consumer does not execute prompt graphs. | +| `community-manager` | No | It has no active prompt-graph queue consumer. | + +Only `expert` and `generic-openai` are an invocation allowlist. Storage is +engine-independent, so a graph may be staged before an engine switch, but it +will not be sent for the excluded engines. + +## Activate the workshop graph + +1. Ensure the target persona uses `generic-openai` and its Virtual Contributor + has a body of knowledge. The supplied graph contains `retrieve` nodes. +2. Copy the complete JSON payload from + [`prompt-graphs/workshop-design.authoring.json`](./prompt-graphs/workshop-design.authoring.json) + into `promptGraph` in the following admin mutation variables. +3. Run the mutation with an actor that has `UPDATE` on the AI Persona. + +```graphql +mutation UpdatePersonaPromptGraph($aiPersonaData: UpdateAiPersonaInput!) { + aiServerUpdateAiPersona(aiPersonaData: $aiPersonaData) { + id + engine + promptGraph { + nodes { + name + type + } + edges { + from + to + on + map + default + } + } + } +} +``` + +```json +{ + "aiPersonaData": { + "ID": "", + "engine": "GENERIC_OPENAI", + "promptGraph": "" + } +} +``` + +The `promptGraph` value above is an object, not a JSON-encoded string. It is +shown as a placeholder to keep this guide readable; use the fixture unchanged. + +Updates merge only the supplied top-level `promptGraph` keys, so omitted keys +such as `start` and `end` are retained from a previously stored graph. For a +clean replacement, first update the persona with `promptGraph: null`, then +send the complete replacement graph in a subsequent update. + +### Retrieval precondition + +`retrieve` nodes require the Virtual Contributor invocation to carry a body +of knowledge. Operators must verify that pairing at activation time; a +misconfigured retrieve-bearing graph fails every invocation until it is +corrected. The server deliberately does not validate the pairing when the +persona is updated: `bodyOfKnowledgeID` is supplied per invocation, not stored +on the persona. If it is absent, the engine fails at parse time and the member +receives the standard error response at invocation. This is expected behavior, +not a server-side validation failure. + +### External-provider egress check + +Before activating a retrieve-bearing graph, verify that the body of knowledge's +data classification and the external model provider's processing basis permit +the egress. Retrieval can send up to 95,000 characters of body-of-knowledge +content to the external model provider in later prompts. + +## Authoring another engine payload + +The GraphQL input is deliberately a storage shape that matches the engine +payload. Convert a Virtual Contributor payload as follows: + +1. Add `"system": false` to every node. `system` is required by the server + authoring API even though the generic engine does not read it. +2. Keep typed-node fields verbatim: `type`, `source`, `collection_template`, + `query_template`, `n_results`, `max_context_chars`, and `output_key`. +3. Keep conditional edge fields verbatim: `on`, `map`, and `default`. + `map` is a JSON object of routing-value to node-name; its keys are matched + case-insensitively by the engine. +4. Convert JSON-Schema dictionary `state.properties` and node + `output.properties` to GraphQL's list form. For example, + `{"role":{"type":["string","null"]}}` becomes + `[{"name":"role","type":"string","optional":true}]`. +5. Convert dictionary-form `required: ["name"]` to `optional: false` on the + named list entries, then drop `required`. Convert nested arrays of objects + recursively using the data point's `items` struct. +6. Omit optional fields instead of passing `null`. Explicit nulls on retrieve + or conditional fields are stored but fail the engine parser with a named + error. The engine, not the server, is the validation authority for graph + semantics. + +The read surface returns stored graphs for any engine. A generic persona with +no stored graph reads back `null`. Separately, the existing field resolver +returns the expert default graph for `expert` and `libra-flow` personas without +a stored graph; that pre-existing read-surface asymmetry is out of scope and +does not change the invocation allowlist above. diff --git a/docs/prompt-graphs/workshop-design.authoring.json b/docs/prompt-graphs/workshop-design.authoring.json new file mode 100644 index 0000000000..06f612b4b4 --- /dev/null +++ b/docs/prompt-graphs/workshop-design.authoring.json @@ -0,0 +1,263 @@ +{ + "state": { + "type": "object", + "properties": [ + { + "name": "role", + "type": "string", + "optional": true + }, + { + "name": "duration", + "type": "string", + "optional": true + }, + { + "name": "workshop_type", + "type": "string", + "optional": true + }, + { + "name": "purpose", + "type": "string", + "optional": true + }, + { + "name": "audience_size", + "type": "integer", + "optional": true + }, + { + "name": "question", + "type": "string", + "optional": true + }, + { + "name": "complete", + "type": "boolean", + "optional": true + }, + { + "name": "action", + "type": "string", + "optional": true + }, + { + "name": "current", + "type": "string", + "optional": true + }, + { + "name": "knowledge_docs", + "type": "string", + "optional": true + }, + { + "name": "result", + "type": "string", + "optional": true + }, + { + "name": "conversation", + "type": "string", + "optional": true + }, + { + "name": "current_question", + "type": "string", + "optional": true + }, + { + "name": "bok_id", + "type": "string", + "optional": true + } + ] + }, + "nodes": [ + { + "name": "check_input", + "type": "llm", + "input_variables": ["conversation"], + "prompt": "You are a conversation analyser extracting key information points from a conversation between a human and an assistant. The conversation below is about a workshop creation and you are looking for the following data points: role, duration, workshop_type, purpose, audience_size.\n\nReply according to the output format instructions below, trying to fill in as many fields as possible.\n\nIf one or more data points are missing, generate a question asking for them. Ask for no more than 2 data points in a single question.\n\nIf the conversation is missing, assume all key points are missing.\n\nConversation:\n{conversation}\n\nOutput format instructions:\n{format_instructions}", + "output": { + "type": "object", + "properties": [ + { + "name": "role", + "type": "string", + "description": "The role of the workshop creator/facilitator in the organisation.", + "optional": false + }, + { + "name": "duration", + "type": "string", + "description": "The duration of the workshop.", + "optional": false + }, + { + "name": "workshop_type", + "type": "string", + "description": "The type of the workshop - in-person, online, etc.", + "optional": false + }, + { + "name": "purpose", + "type": "string", + "description": "The purpose/goal of the workshop.", + "optional": false + }, + { + "name": "audience_size", + "type": "integer", + "description": "The size of the audience of the workshop.", + "optional": false + }, + { + "name": "question", + "type": "string", + "description": "A question asking the workshop creator to provide a missing key variable.", + "optional": false + }, + { + "name": "complete", + "type": "boolean", + "description": "True when all five data points are present.", + "optional": false + } + ] + }, + "system": false + }, + { + "name": "ask", + "type": "echo", + "source": "question", + "system": false + }, + { + "name": "analyse_last_message", + "type": "llm", + "input_variables": ["conversation"], + "prompt": "You are a conversation analyser. You are provided with a conversation between a human and an assistant about a workshop design.\n\nBased on the last human message, determine if the user wants to refine an already generated workshop design ('refine') or wants a new one generated ('generate').\n\nConversation:\n{conversation}\n\nOutput format instructions:\n{format_instructions}", + "output": { + "type": "object", + "properties": [ + { + "name": "action", + "type": "string", + "description": "Either 'refine' or 'generate'.", + "optional": false + } + ] + }, + "system": false + }, + { + "name": "extract", + "type": "llm", + "input_variables": ["conversation"], + "prompt": "You are a conversation analyser. Below you are provided with a conversation between a human and an assistant about a workshop design, formatted like:\n```\nhuman:\nmessage content\nassistant:\nmessage content\n...\n```\nOne of the messages contains a workshop design. Find it and return the contents of the message without any additional information or alterations.\n\nNever explain your reasoning or provide any additional information outside of the workshop design.\n\nConversation:\n{conversation}\n\nOutput format instructions:\n{format_instructions}", + "output": { + "type": "object", + "properties": [ + { + "name": "current", + "type": "string", + "description": "The current iteration of the workshop design found in the conversation.", + "optional": false + } + ] + }, + "system": false + }, + { + "name": "retrieve_refine", + "type": "retrieve", + "collection_template": "{bok_id}-knowledge", + "query_template": "I need information about Liberating Structures suitable for a {workshop_type} for {audience_size} people held by {role} for {duration} in order to {purpose}.", + "n_results": 10, + "max_context_chars": 95000, + "output_key": "knowledge_docs", + "system": false + }, + { + "name": "refine", + "type": "llm", + "input_variables": ["current", "current_question", "knowledge_docs"], + "prompt": "You are a renowned expert in Liberating Structures.\n\nYou are provided with a workshop design based on Liberating Structures and a human request to alter it.\n\nBased on the context provided below, apply the changes the user requested to the current workshop design.\n\nNever explain your reasoning or provide any additional information outside of the workshop design.\n\nThe design format is Markdown. Preserve the formatting, indentation and structure of the design. Add nice indentation to the design and do not remove any empty lines.\n\nWorkshop design:\n{current}\n\nHuman request:\n{current_question}\n\nContext:\n{knowledge_docs}", + "system": false + }, + { + "name": "retrieve_generate", + "type": "retrieve", + "collection_template": "{bok_id}-knowledge", + "query_template": "I need information about Liberating Structures suitable for a {workshop_type} for {audience_size} people held by {role} for {duration} in order to {purpose}.", + "n_results": 10, + "max_context_chars": 95000, + "output_key": "knowledge_docs", + "system": false + }, + { + "name": "generate", + "type": "llm", + "input_variables": [ + "role", + "duration", + "workshop_type", + "purpose", + "audience_size", + "knowledge_docs" + ], + "prompt": "Here is information about Liberating Structures that are suitable for the scenario described below. Base your answer ONLY on that and ignore your previous knowledge about Liberating Structures.\n\nLiberating Structures information:\n\n{knowledge_docs}\n\nDesign a workshop for a {workshop_type} for {audience_size} people, held by a {role}, for {duration}, in order to {purpose}, using only the Liberating Structures information provided above.\n\nNever explain your reasoning or provide any additional information outside of the workshop design. The design format is Markdown.", + "system": false + } + ], + "edges": [ + { + "from": "START", + "to": "check_input" + }, + { + "from": "check_input", + "on": "complete", + "map": { + "true": "analyse_last_message", + "false": "ask" + } + }, + { + "from": "ask", + "to": "END" + }, + { + "from": "analyse_last_message", + "on": "action", + "map": { + "refine": "extract", + "generate": "retrieve_generate" + }, + "default": "retrieve_generate" + }, + { + "from": "extract", + "to": "retrieve_refine" + }, + { + "from": "retrieve_refine", + "to": "refine" + }, + { + "from": "refine", + "to": "END" + }, + { + "from": "retrieve_generate", + "to": "generate" + }, + { + "from": "generate", + "to": "END" + } + ] +} diff --git a/schema.graphql b/schema.graphql index 7b2783635b..f87fe72dcd 100644 --- a/schema.graphql +++ b/schema.graphql @@ -6,6 +6,11 @@ scalar DateTime """An Emoji.""" scalar Emoji +""" +The `JSON` scalar type represents JSON values as specified by [ECMA-404](http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf). +""" +scalar JSON + """ A representation of a Lifecycle Definition, based on XState. It is serialized JSON. """ @@ -5350,16 +5355,54 @@ type PromptGraphDefinitionNode { } type PromptGraphEdge { + """ + Conditional routing fallback target — validated by the engine at parse time. + """ + default: String from: String + """ + Conditional routing-value to node-name map; keys match case-insensitively in the engine. + """ + map: JSON + """ + Conditional routing state field — validated by the engine at parse time. + """ + on: String to: String } type PromptGraphNode { + """ + Retrieve collection template — validated by the engine contract at parse time. + """ + collection_template: String input_variables: [String!] + """ + Retrieve context character cap — validated by the engine contract at parse time. + """ + max_context_chars: Int + """ + Retrieve result count — validated by the engine contract at parse time. + """ + n_results: Int name: String! output: PromptGraphDataStruct + """ + Retrieve output state key — validated by the engine contract at parse time. + """ + output_key: String prompt: String + """ + Retrieve query template — validated by the engine contract at parse time. + """ + query_template: String + """Echo node state key — validated by the engine contract at parse time.""" + source: String system: Boolean! + """ + Node type: llm (default) | retrieve | echo — validated by the engine at parse time. + """ + type: String } type PruneInAppNotificationAdminResult { @@ -8792,7 +8835,19 @@ input PromptGraphDataStructInput { } input PromptGraphEdgeInput { + """ + Conditional routing fallback target — validated by the engine at parse time. + """ + default: String from: String + """ + Conditional routing-value to node-name map; keys match case-insensitively in the engine. + """ + map: JSON + """ + Conditional routing state field — validated by the engine at parse time. + """ + on: String to: String } @@ -8805,11 +8860,37 @@ input PromptGraphInput { } input PromptGraphNodeInput { + """ + Retrieve collection template — validated by the engine contract at parse time. + """ + collection_template: String input_variables: [String!] + """ + Retrieve context character cap — validated by the engine contract at parse time. + """ + max_context_chars: Int + """ + Retrieve result count — validated by the engine contract at parse time. + """ + n_results: Int name: String! output: PromptGraphDataStructInput + """ + Retrieve output state key — validated by the engine contract at parse time. + """ + output_key: String prompt: String + """ + Retrieve query template — validated by the engine contract at parse time. + """ + query_template: String + """Echo node state key — validated by the engine contract at parse time.""" + source: String system: Boolean! + """ + Node type: llm (default) | retrieve | echo — validated by the engine at parse time. + """ + type: String } input RefreshVirtualContributorBodyOfKnowledgeInput { diff --git a/src/services/ai-server/ai-persona/ai.persona.module.ts b/src/services/ai-server/ai-persona/ai.persona.module.ts index e966ce1b55..e96bba4a3e 100644 --- a/src/services/ai-server/ai-persona/ai.persona.module.ts +++ b/src/services/ai-server/ai-persona/ai.persona.module.ts @@ -2,6 +2,7 @@ import { AuthorizationModule } from '@core/authorization/authorization.module'; import { AuthorizationPolicyModule } from '@domain/common/authorization-policy/authorization.policy.module'; import { Module } from '@nestjs/common'; import { TypeOrmModule } from '@nestjs/typeorm'; +import { PlatformOperationsAuditModule } from '@src/platform-admin/platform-operations-audit/platform.operations.audit.module'; import { AiPersonaEngineAdapterModule } from '../ai-persona-engine-adapter/ai.persona.engine.adapter.module'; import { AiPersona } from './ai.persona.entity'; import { AiPersonaExternalConfigResolverFields } from './ai.persona.external.config.resolver.fields'; @@ -14,6 +15,7 @@ import { AiPersonaAuthorizationService } from './ai.persona.service.authorizatio imports: [ AuthorizationPolicyModule, AuthorizationModule, + PlatformOperationsAuditModule, TypeOrmModule.forFeature([AiPersona]), AiPersonaEngineAdapterModule, ], diff --git a/src/services/ai-server/ai-persona/ai.persona.resolver.mutations.audit.spec.ts b/src/services/ai-server/ai-persona/ai.persona.resolver.mutations.audit.spec.ts new file mode 100644 index 0000000000..2cd9a60bd8 --- /dev/null +++ b/src/services/ai-server/ai-persona/ai.persona.resolver.mutations.audit.spec.ts @@ -0,0 +1,89 @@ +import { AuthorizationService } from '@core/authorization/authorization.service'; +import { Test, TestingModule } from '@nestjs/testing'; +import { PlatformOperationsAuditService } from '@src/platform-admin/platform-operations-audit/platform.operations.audit.service'; +import { defaultMockerFactory } from '@test/utils/default.mocker.factory'; +import { type Mock } from 'vitest'; +import { AiPersonaResolverMutations } from './ai.persona.resolver.mutations'; +import { AiPersonaService } from './ai.persona.service'; + +describe('AiPersonaResolverMutations audit trail', () => { + let resolver: AiPersonaResolverMutations; + let aiPersonaService: Record; + let platformOperationsAuditService: Record; + + beforeEach(async () => { + const module: TestingModule = await Test.createTestingModule({ + providers: [AiPersonaResolverMutations], + }) + .useMocker(defaultMockerFactory) + .compile(); + + resolver = module.get(AiPersonaResolverMutations); + aiPersonaService = module.get(AiPersonaService) as unknown as Record< + string, + Mock + >; + platformOperationsAuditService = module.get( + PlatformOperationsAuditService + ) as unknown as Record; + ( + module.get(AuthorizationService) as unknown as Record + ).grantAccessOrFail.mockReturnValue(undefined); + platformOperationsAuditService.recordOperation.mockResolvedValue(undefined); + }); + + it('records the safe success details for a prompt-graph activation', async () => { + const actorContext = { actorID: 'user-1' } as any; + const updateData = { + ID: 'persona-1', + promptGraph: { nodes: [{ prompt: 'must not be audited' }] }, + }; + const persona = { id: 'persona-1', engine: 'generic-openai' }; + aiPersonaService.getAiPersonaOrFail.mockResolvedValue(persona); + aiPersonaService.updateAiPersona.mockResolvedValue(persona); + + await resolver.aiServerUpdateAiPersona(actorContext, updateData as any); + + expect(platformOperationsAuditService.recordOperation).toHaveBeenCalledWith( + { + actorID: 'user-1', + action: 'aiServerUpdateAiPersona', + target: { + aiPersonaID: 'persona-1', + engine: 'generic-openai', + promptGraphChanged: true, + }, + outcome: 'success', + } + ); + }); + + it('records a failure without raw graph data or error details', async () => { + const actorContext = { actorID: 'user-1' } as any; + const updateData = { + ID: 'persona-1', + promptGraph: { nodes: [{ prompt: 'must not be audited' }] }, + }; + const persona = { id: 'persona-1', engine: 'generic-openai' }; + const updateError = new Error('update failed'); + aiPersonaService.getAiPersonaOrFail.mockResolvedValue(persona); + aiPersonaService.updateAiPersona.mockRejectedValue(updateError); + + await expect( + resolver.aiServerUpdateAiPersona(actorContext, updateData as any) + ).rejects.toThrow(updateError); + + expect(platformOperationsAuditService.recordOperation).toHaveBeenCalledWith( + { + actorID: 'user-1', + action: 'aiServerUpdateAiPersona', + target: { + aiPersonaID: 'persona-1', + engine: 'generic-openai', + promptGraphChanged: true, + }, + outcome: 'failure', + } + ); + }); +}); diff --git a/src/services/ai-server/ai-persona/ai.persona.resolver.mutations.ts b/src/services/ai-server/ai-persona/ai.persona.resolver.mutations.ts index da6a4e6623..307b072f54 100644 --- a/src/services/ai-server/ai-persona/ai.persona.resolver.mutations.ts +++ b/src/services/ai-server/ai-persona/ai.persona.resolver.mutations.ts @@ -4,6 +4,7 @@ import { AuthorizationService } from '@core/authorization/authorization.service' import { Args, Mutation, Resolver } from '@nestjs/graphql'; import { InstrumentResolver } from '@src/apm/decorators'; import { CurrentActor } from '@src/common/decorators'; +import { PlatformOperationsAuditService } from '@src/platform-admin/platform-operations-audit/platform.operations.audit.service'; import { IAiPersona } from './ai.persona.interface'; import { AiPersonaService } from './ai.persona.service'; import { DeleteAiPersonaInput, UpdateAiPersonaInput } from './dto'; @@ -13,7 +14,8 @@ import { DeleteAiPersonaInput, UpdateAiPersonaInput } from './dto'; export class AiPersonaResolverMutations { constructor( private aiPersonaService: AiPersonaService, - private authorizationService: AuthorizationService + private authorizationService: AuthorizationService, + private platformOperationsAuditService: PlatformOperationsAuditService ) {} @Mutation(() => IAiPersona, { @@ -24,17 +26,46 @@ export class AiPersonaResolverMutations { @Args('aiPersonaData') aiPersonaServiceData: UpdateAiPersonaInput ): Promise { - const aiPersonaService = await this.aiPersonaService.getAiPersonaOrFail( - aiPersonaServiceData.ID - ); - this.authorizationService.grantAccessOrFail( - actorContext, - aiPersonaService.authorization, - AuthorizationPrivilege.UPDATE, - `orgUpdate: ${aiPersonaService.id}` - ); + const auditTarget: { + aiPersonaID: string; + engine?: string; + promptGraphChanged: boolean; + } = { + aiPersonaID: aiPersonaServiceData.ID, + promptGraphChanged: aiPersonaServiceData.promptGraph !== undefined, + }; + + try { + const aiPersona = await this.aiPersonaService.getAiPersonaOrFail( + aiPersonaServiceData.ID + ); + auditTarget.engine = aiPersona.engine; + this.authorizationService.grantAccessOrFail( + actorContext, + aiPersona.authorization, + AuthorizationPrivilege.UPDATE, + `orgUpdate: ${aiPersona.id}` + ); - return await this.aiPersonaService.updateAiPersona(aiPersonaServiceData); + const result = + await this.aiPersonaService.updateAiPersona(aiPersonaServiceData); + auditTarget.engine = result.engine; + await this.platformOperationsAuditService.recordOperation({ + actorID: actorContext.actorID, + action: 'aiServerUpdateAiPersona', + target: auditTarget, + outcome: 'success', + }); + return result; + } catch (error) { + await this.platformOperationsAuditService.recordOperation({ + actorID: actorContext.actorID, + action: 'aiServerUpdateAiPersona', + target: auditTarget, + outcome: 'failure', + }); + throw error; + } } @Mutation(() => IAiPersona, { diff --git a/src/services/ai-server/ai-persona/ai.persona.service.spec.ts b/src/services/ai-server/ai-persona/ai.persona.service.spec.ts index 9b7ea35071..5f4b5f55dd 100644 --- a/src/services/ai-server/ai-persona/ai.persona.service.spec.ts +++ b/src/services/ai-server/ai-persona/ai.persona.service.spec.ts @@ -10,8 +10,11 @@ import { MockWinstonProvider } from '@test/mocks/winston.provider.mock'; import { defaultMockerFactory } from '@test/utils/default.mocker.factory'; import { repositoryProviderMockFactory } from '@test/utils/repository.provider.mock.factory'; import { type Mock, vi } from 'vitest'; +import graphJson from '../prompt-graph/config/prompt.graph.expert.json'; +import { PromptGraph } from '../prompt-graph/dto/prompt.graph.dto'; import { AiPersona } from './ai.persona.entity'; import { AiPersonaService } from './ai.persona.service'; +import { PromptGraphTransformer } from './transformers/prompt.graph.transformer'; describe('AiPersonaService', () => { let service: AiPersonaService; @@ -227,6 +230,117 @@ describe('AiPersonaService', () => { ); }); + it('T-roundtrip-unit: preserves typed-node and conditional-edge fields without materializing omitted optionals', async () => { + const workshopGraph = { + nodes: [ + { + name: 'retrieve', + system: false, + type: 'retrieve', + collection_template: '{bok_id}-knowledge', + query_template: 'information about {topic}', + n_results: 10, + max_context_chars: 20000, + output_key: 'knowledge_docs', + }, + { + name: 'echo', + system: false, + type: 'echo', + source: 'question', + }, + { + name: 'plain-llm', + system: false, + prompt: 'Answer {question}', + }, + ], + edges: [ + { + from: 'retrieve', + on: 'route', + map: { answer: 'echo', retry: 'plain-llm' }, + default: 'plain-llm', + }, + ], + state: { + type: 'object', + properties: [{ name: 'question', type: 'string', optional: false }], + }, + }; + + const expectedGraph = { + nodes: [ + { + name: 'retrieve', + system: false, + type: 'retrieve', + collection_template: '{bok_id}-knowledge', + query_template: 'information about {topic}', + n_results: 10, + max_context_chars: 20000, + output_key: 'knowledge_docs', + }, + { + name: 'echo', + system: false, + type: 'echo', + source: 'question', + }, + { + name: 'plain-llm', + system: false, + prompt: 'Answer {question}', + }, + ], + edges: [ + { + from: 'retrieve', + on: 'route', + map: { answer: 'echo', retry: 'plain-llm' }, + default: 'plain-llm', + }, + ], + state: { + type: 'object', + properties: [{ name: 'question', type: 'string', optional: false }], + }, + }; + + const persistedGraph = JSON.parse( + JSON.stringify(PromptGraphTransformer.to(workshopGraph as PromptGraph)) + ); + const hydratedGraph = PromptGraphTransformer.from(persistedGraph); + + expect(hydratedGraph).toEqual(expectedGraph); + expect('n_results' in hydratedGraph!.nodes![2]).toBe(false); + + await service.updateAiPersona({ + ID: 'persona-1', + promptGraph: workshopGraph, + } as any); + + const savedPersona = aiPersonaRepository.save.mock.calls[0][0]; + expect(savedPersona.promptGraph).toEqual(expectedGraph); + expect('n_results' in savedPersona.promptGraph.nodes[2]).toBe(false); + }); + + it('preserves a plain LLM-only graph byte-for-byte', async () => { + const plainGraph = { + nodes: [{ name: 'answer', system: false, prompt: 'Answer {question}' }], + edges: [{ from: 'START', to: 'answer' }], + }; + + await service.updateAiPersona({ + ID: 'persona-1', + promptGraph: plainGraph, + } as any); + + expect(aiPersonaRepository.save.mock.calls[0][0].promptGraph).toEqual( + plainGraph + ); + }); + it('should throw EntityNotFoundException when persona does not exist', async () => { aiPersonaRepository.findOne.mockResolvedValue(null); @@ -432,6 +546,37 @@ describe('AiPersonaService', () => { ); }); + it('T-inv-expert-cases: preserves the expert default graph and joins array prompts', async () => { + const persona = { + id: 'persona-1', + engine: AiPersonaEngine.EXPERT, + prompt: ['test'], + externalConfig: {}, + promptGraph: undefined, + }; + aiPersonaRepository.findOne.mockResolvedValue(persona); + aiPersonaEngineAdapter.invoke.mockResolvedValue(undefined); + + await service.invoke( + { + aiPersonaID: 'persona-1', + message: 'Hello', + displayName: 'VC', + externalMetadata: {}, + resultHandler: { action: 'none' }, + } as any, + [] + ); + + const adapterInput = aiPersonaEngineAdapter.invoke.mock.calls[0][0]; + const defaultCheckInput = adapterInput.promptGraph.nodes.find( + (node: { name: string }) => node.name === 'check_input' + ); + expect(defaultCheckInput.prompt).toBe( + (graphJson.nodes[0].prompt as string[]).join('\n') + ); + }); + it('should skip default promptGraph when invocationInput.promptGraph is provided', async () => { const persona = { id: 'persona-1', @@ -463,6 +608,122 @@ describe('AiPersonaService', () => { ); }); + it('T-inv-generic-stored: forwards the stored graph by reference for GENERIC_OPENAI', async () => { + const storedGraph = { + nodes: [{ name: 'answer', system: false, prompt: 'Answer {question}' }], + edges: [{ from: 'START', to: 'answer' }], + }; + aiPersonaRepository.findOne.mockResolvedValue({ + id: 'persona-1', + engine: AiPersonaEngine.GENERIC_OPENAI, + prompt: ['test'], + externalConfig: {}, + promptGraph: storedGraph, + }); + aiPersonaEngineAdapter.invoke.mockResolvedValue(undefined); + + await service.invoke( + { + aiPersonaID: 'persona-1', + message: 'Hello', + displayName: 'VC', + externalMetadata: {}, + resultHandler: { action: 'none' }, + } as any, + [] + ); + + const adapterInput = aiPersonaEngineAdapter.invoke.mock.calls[0][0]; + expect(adapterInput.promptGraph).toBe(storedGraph); + }); + + it('T-inv-generic-none: omits promptGraph for a GENERIC_OPENAI persona without one stored', async () => { + aiPersonaRepository.findOne.mockResolvedValue({ + id: 'persona-1', + engine: AiPersonaEngine.GENERIC_OPENAI, + prompt: ['test'], + externalConfig: {}, + promptGraph: undefined, + }); + aiPersonaEngineAdapter.invoke.mockResolvedValue(undefined); + + await service.invoke( + { + aiPersonaID: 'persona-1', + message: 'Hello', + displayName: 'VC', + externalMetadata: {}, + resultHandler: { action: 'none' }, + } as any, + [] + ); + + const adapterInput = aiPersonaEngineAdapter.invoke.mock.calls[0][0]; + expect(aiPersonaEngineAdapter.invoke).toHaveBeenCalledWith( + expect.not.objectContaining({ promptGraph: expect.anything() }) + ); + expect('promptGraph' in adapterInput).toBe(false); + }); + + it('T-inv-generic-override: suppresses stored graph when the internal override is present', async () => { + aiPersonaRepository.findOne.mockResolvedValue({ + id: 'persona-1', + engine: AiPersonaEngine.GENERIC_OPENAI, + prompt: ['test'], + externalConfig: {}, + promptGraph: { nodes: [], edges: [] }, + }); + aiPersonaEngineAdapter.invoke.mockResolvedValue(undefined); + + await service.invoke( + { + aiPersonaID: 'persona-1', + message: 'Hello', + displayName: 'VC', + externalMetadata: {}, + resultHandler: { action: 'none' }, + promptGraph: { nodes: [], edges: [] }, + } as any, + [] + ); + + const adapterInput = aiPersonaEngineAdapter.invoke.mock.calls[0][0]; + expect(aiPersonaEngineAdapter.invoke).toHaveBeenCalledWith( + expect.not.objectContaining({ promptGraph: expect.anything() }) + ); + expect('promptGraph' in adapterInput).toBe(false); + }); + + it.each([ + AiPersonaEngine.GUIDANCE, + AiPersonaEngine.OPENAI_ASSISTANT, + AiPersonaEngine.LIBRA_FLOW, + AiPersonaEngine.COMMUNITY_MANAGER, + ])('T-inv-exclusion-cases: omits promptGraph for %s even when one is stored', async engine => { + aiPersonaRepository.findOne.mockResolvedValue({ + id: 'persona-1', + engine, + prompt: ['test'], + externalConfig: {}, + promptGraph: { nodes: [], edges: [] }, + }); + aiPersonaEngineAdapter.invoke.mockResolvedValue(undefined); + + await service.invoke( + { + aiPersonaID: 'persona-1', + message: 'Hello', + displayName: 'VC', + externalMetadata: {}, + resultHandler: { action: 'none' }, + } as any, + [] + ); + + const adapterInput = aiPersonaEngineAdapter.invoke.mock.calls[0][0]; + expect('promptGraph' in adapterInput).toBe(false); + }); + it('should decrypt external config for the engine input', async () => { const persona = { id: 'persona-1', diff --git a/src/services/ai-server/ai-persona/ai.persona.service.ts b/src/services/ai-server/ai-persona/ai.persona.service.ts index df284d260f..05fbfc8183 100644 --- a/src/services/ai-server/ai-persona/ai.persona.service.ts +++ b/src/services/ai-server/ai-persona/ai.persona.service.ts @@ -24,6 +24,12 @@ import { } from './dto'; import { IExternalConfig } from './dto/external.config'; +// Engines whose plugin executes Input.promptGraph (engine-invocation contract). +const PROMPT_GRAPH_ENGINES: AiPersonaEngine[] = [ + AiPersonaEngine.EXPERT, + AiPersonaEngine.GENERIC_OPENAI, +]; + @Injectable() export class AiPersonaService { constructor( @@ -193,11 +199,11 @@ export class AiPersonaService { }; if ( - input.engine === AiPersonaEngine.EXPERT && + PROMPT_GRAPH_ENGINES.includes(input.engine) && !invocationInput.promptGraph ) { let invocationGraph = aiPersona.promptGraph; - if (!invocationGraph) { + if (!invocationGraph && input.engine === AiPersonaEngine.EXPERT) { // Deep-clone the imported graphJson so we don't mutate the module-level object const processedGraph = JSON.parse(JSON.stringify(graphJson)); // For each node, if prompt is an array, concatenate it into a single string with new lines diff --git a/src/services/ai-server/ai-persona/dto/ai.persona.dto.update.ts b/src/services/ai-server/ai-persona/dto/ai.persona.dto.update.ts index 4ed64a4d80..583866c619 100644 --- a/src/services/ai-server/ai-persona/dto/ai.persona.dto.update.ts +++ b/src/services/ai-server/ai-persona/dto/ai.persona.dto.update.ts @@ -2,6 +2,8 @@ import { AiPersonaEngine } from '@common/enums/ai.persona.engine'; import { UpdateBaseAlkemioInput } from '@domain/common/entity/base-entity'; import { Field, InputType } from '@nestjs/graphql'; import { PromptGraph } from '@services/ai-server/prompt-graph/dto/prompt.graph.dto'; +import { Type } from 'class-transformer'; +import { ValidateNested } from 'class-validator'; import { IExternalConfig } from './external.config'; @InputType() @@ -16,5 +18,7 @@ export class UpdateAiPersonaInput extends UpdateBaseAlkemioInput { externalConfig?: IExternalConfig; @Field(() => PromptGraph, { nullable: true }) + @ValidateNested() + @Type(() => PromptGraph) promptGraph?: PromptGraph | null; } diff --git a/src/services/ai-server/prompt-graph/dto/prompt.graph.dto.ts b/src/services/ai-server/prompt-graph/dto/prompt.graph.dto.ts index a51f42a0ba..561fd04eb0 100644 --- a/src/services/ai-server/prompt-graph/dto/prompt.graph.dto.ts +++ b/src/services/ai-server/prompt-graph/dto/prompt.graph.dto.ts @@ -1,4 +1,6 @@ import { Field, InputType, ObjectType } from '@nestjs/graphql'; +import { Type } from 'class-transformer'; +import { ValidateNested } from 'class-validator'; import { PromptGraphDataStruct } from './prompt.graph.data.struct.dto'; import { PromptGraphEdge } from './prompt.graph.edge.dto'; import { PromptGraphNode } from './prompt.graph.node.dto'; @@ -7,9 +9,13 @@ import { PromptGraphNode } from './prompt.graph.node.dto'; @ObjectType() export class PromptGraph { @Field(() => [PromptGraphNode], { nullable: true }) + @ValidateNested({ each: true }) + @Type(() => PromptGraphNode) nodes?: PromptGraphNode[]; @Field(() => [PromptGraphEdge], { nullable: true }) + @ValidateNested({ each: true }) + @Type(() => PromptGraphEdge) edges?: PromptGraphEdge[]; @Field({ nullable: true }) diff --git a/src/services/ai-server/prompt-graph/dto/prompt.graph.edge.dto.spec.ts b/src/services/ai-server/prompt-graph/dto/prompt.graph.edge.dto.spec.ts new file mode 100644 index 0000000000..50e3a110df --- /dev/null +++ b/src/services/ai-server/prompt-graph/dto/prompt.graph.edge.dto.spec.ts @@ -0,0 +1,62 @@ +import { UpdateAiPersonaInput } from '@services/ai-server/ai-persona/dto'; +import { plainToInstance } from 'class-transformer'; +import { validate } from 'class-validator'; +import { PromptGraphEdge } from './prompt.graph.edge.dto'; + +describe('PromptGraphEdge map validation', () => { + const validateMap = async (map: unknown) => { + const edge = new PromptGraphEdge(); + edge.map = map as Record; + return validate(edge); + }; + + it('accepts a string-to-string routing map', async () => { + await expect( + validateMap({ answer: 'answer-node', retry: 'retry-node' }) + ).resolves.toEqual([]); + }); + + it('rejects a non-object routing map', async () => { + await expect(validateMap(['answer', 'answer-node'])).resolves.toEqual([ + expect.objectContaining({ property: 'map' }), + ]); + }); + + it('rejects a routing map with non-string values', async () => { + await expect(validateMap({ answer: 1 })).resolves.toEqual([ + expect.objectContaining({ property: 'map' }), + ]); + }); + + it('applies map validation through the persona update DTO', async () => { + const input = plainToInstance(UpdateAiPersonaInput, { + ID: 'persona-1', + promptGraph: { edges: [{ map: ['answer', 'answer-node'] }] }, + }); + + const errors = await validate(input); + + expect(errors[0].children?.[0].children?.[0].children).toEqual([ + expect.objectContaining({ property: 'map' }), + ]); + }); + + it('rejects routing maps that exceed entry or field-size caps', async () => { + const overEntryCap = Object.fromEntries( + Array.from({ length: 101 }, (_, index) => [ + `route-${index}`, + `node-${index}`, + ]) + ); + + await expect(validateMap(overEntryCap)).resolves.toEqual([ + expect.objectContaining({ property: 'map' }), + ]); + await expect(validateMap({ ['x'.repeat(129)]: 'target' })).resolves.toEqual( + [expect.objectContaining({ property: 'map' })] + ); + await expect(validateMap({ route: 'x'.repeat(129) })).resolves.toEqual([ + expect.objectContaining({ property: 'map' }), + ]); + }); +}); diff --git a/src/services/ai-server/prompt-graph/dto/prompt.graph.edge.dto.ts b/src/services/ai-server/prompt-graph/dto/prompt.graph.edge.dto.ts index 62e9a7925b..c9b2fb859f 100644 --- a/src/services/ai-server/prompt-graph/dto/prompt.graph.edge.dto.ts +++ b/src/services/ai-server/prompt-graph/dto/prompt.graph.edge.dto.ts @@ -1,4 +1,39 @@ import { Field, InputType, ObjectType } from '@nestjs/graphql'; +import { SMALL_TEXT_LENGTH } from '@src/common/constants'; +import { + IsOptional, + Validate, + ValidationArguments, + ValidatorConstraint, + ValidatorConstraintInterface, +} from 'class-validator'; +import GraphQLJSON from 'graphql-type-json'; + +const PROMPT_GRAPH_MAP_MAX_ENTRIES = 100; + +@ValidatorConstraint({ name: 'isPromptGraphMap', async: false }) +class IsPromptGraphMap implements ValidatorConstraintInterface { + validate(value: unknown): boolean { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + return false; + } + + const entries = Object.entries(value); + return ( + entries.length <= PROMPT_GRAPH_MAP_MAX_ENTRIES && + entries.every( + ([key, target]) => + key.length <= SMALL_TEXT_LENGTH && + typeof target === 'string' && + target.length <= SMALL_TEXT_LENGTH + ) + ); + } + + defaultMessage(args: ValidationArguments): string { + return `${args.property} must be an object of at most ${PROMPT_GRAPH_MAP_MAX_ENTRIES} string-to-string entries, with keys and values no longer than ${SMALL_TEXT_LENGTH} characters`; + } +} @InputType('PromptGraphEdgeInput') @ObjectType() @@ -8,4 +43,27 @@ export class PromptGraphEdge { @Field({ nullable: true }) to?: string; + + @Field(() => String, { + nullable: true, + description: + 'Conditional routing state field — validated by the engine at parse time.', + }) + on?: string; + + @Field(() => GraphQLJSON, { + nullable: true, + description: + 'Conditional routing-value to node-name map; keys match case-insensitively in the engine.', + }) + @IsOptional() + @Validate(IsPromptGraphMap) + map?: Record; + + @Field(() => String, { + nullable: true, + description: + 'Conditional routing fallback target — validated by the engine at parse time.', + }) + default?: string; } diff --git a/src/services/ai-server/prompt-graph/dto/prompt.graph.node.dto.ts b/src/services/ai-server/prompt-graph/dto/prompt.graph.node.dto.ts index b12be4024a..96c37596f1 100644 --- a/src/services/ai-server/prompt-graph/dto/prompt.graph.node.dto.ts +++ b/src/services/ai-server/prompt-graph/dto/prompt.graph.node.dto.ts @@ -1,4 +1,4 @@ -import { Field, InputType, ObjectType } from '@nestjs/graphql'; +import { Field, InputType, Int, ObjectType } from '@nestjs/graphql'; import { PromptGraphDataStruct } from './prompt.graph.data.struct.dto'; @InputType('PromptGraphNodeInput') @@ -18,4 +18,53 @@ export class PromptGraphNode { @Field(() => PromptGraphDataStruct, { nullable: true }) output?: PromptGraphDataStruct; + + @Field(() => String, { + nullable: true, + description: + 'Node type: llm (default) | retrieve | echo — validated by the engine at parse time.', + }) + type?: string; + + @Field(() => String, { + nullable: true, + description: + 'Echo node state key — validated by the engine contract at parse time.', + }) + source?: string; + + @Field(() => String, { + nullable: true, + description: + 'Retrieve collection template — validated by the engine contract at parse time.', + }) + collection_template?: string; + + @Field(() => String, { + nullable: true, + description: + 'Retrieve query template — validated by the engine contract at parse time.', + }) + query_template?: string; + + @Field(() => Int, { + nullable: true, + description: + 'Retrieve result count — validated by the engine contract at parse time.', + }) + n_results?: number; + + @Field(() => Int, { + nullable: true, + description: + 'Retrieve context character cap — validated by the engine contract at parse time.', + }) + max_context_chars?: number; + + @Field(() => String, { + nullable: true, + description: + 'Retrieve output state key — validated by the engine contract at parse time.', + }) + output_key?: string; } diff --git a/test/integration/ai-persona-prompt-graph/prompt-graph-attachment.spec.ts b/test/integration/ai-persona-prompt-graph/prompt-graph-attachment.spec.ts new file mode 100644 index 0000000000..d657c34846 --- /dev/null +++ b/test/integration/ai-persona-prompt-graph/prompt-graph-attachment.spec.ts @@ -0,0 +1,160 @@ +import { AiPersonaEngine } from '@common/enums/ai.persona.engine'; +import { EncryptionService } from '@hedger/nestjs-encryption'; +import { Test, TestingModule } from '@nestjs/testing'; +import { getRepositoryToken } from '@nestjs/typeorm'; +import { AiPersona } from '@services/ai-server/ai-persona/ai.persona.entity'; +import { AiPersonaService } from '@services/ai-server/ai-persona/ai.persona.service'; +import { AiPersonaEngineAdapter } from '@services/ai-server/ai-persona-engine-adapter/ai.persona.engine.adapter'; +import graphJson from '@services/ai-server/prompt-graph/config/prompt.graph.expert.json'; +import { MockWinstonProvider } from '@test/mocks/winston.provider.mock'; +import { defaultMockerFactory } from '@test/utils/default.mocker.factory'; +import { repositoryProviderMockFactory } from '@test/utils/repository.provider.mock.factory'; +import { type Mock, vi } from 'vitest'; + +describe('AI persona prompt-graph attachment matrix', () => { + let module: TestingModule; + let service: AiPersonaService; + let aiPersonaRepository: Record; + let aiPersonaEngineAdapter: Record; + + const storedGraph = { + nodes: [{ name: 'answer', system: false, prompt: 'Answer {question}' }], + edges: [{ from: 'START', to: 'answer' }], + }; + + beforeEach(async () => { + vi.restoreAllMocks(); + + module = await Test.createTestingModule({ + providers: [ + AiPersonaService, + repositoryProviderMockFactory(AiPersona), + MockWinstonProvider, + ], + }) + .useMocker(token => { + if (token === EncryptionService) { + return { + decrypt: vi.fn((value: string) => value), + encrypt: vi.fn((value: string) => value), + }; + } + return defaultMockerFactory(token); + }) + .compile(); + + service = module.get(AiPersonaService); + aiPersonaRepository = module.get(getRepositoryToken(AiPersona)); + aiPersonaEngineAdapter = module.get( + AiPersonaEngineAdapter + ) as unknown as Record; + aiPersonaEngineAdapter.invoke.mockResolvedValue(undefined); + }); + + afterEach(async () => { + await module.close(); + }); + + const attachmentCases: Array<{ + name: string; + engines: AiPersonaEngine[]; + graph?: typeof storedGraph; + override?: object; + expected: 'stored' | 'default' | 'absent'; + }> = [ + { + name: 'expert with stored graph', + engines: [AiPersonaEngine.EXPERT], + graph: storedGraph, + expected: 'stored', + }, + { + name: 'expert without stored graph', + engines: [AiPersonaEngine.EXPERT], + expected: 'default', + }, + { + name: 'expert with internal override', + engines: [AiPersonaEngine.EXPERT], + graph: storedGraph, + override: { nodes: [], edges: [] }, + expected: 'absent', + }, + { + name: 'generic-openai with stored graph', + engines: [AiPersonaEngine.GENERIC_OPENAI], + graph: storedGraph, + expected: 'stored', + }, + { + name: 'generic-openai without stored graph', + engines: [AiPersonaEngine.GENERIC_OPENAI], + expected: 'absent', + }, + { + name: 'generic-openai with internal override', + engines: [AiPersonaEngine.GENERIC_OPENAI], + graph: storedGraph, + override: { nodes: [], edges: [] }, + expected: 'absent', + }, + { + name: 'excluded engines with a stored graph', + engines: [ + AiPersonaEngine.GUIDANCE, + AiPersonaEngine.OPENAI_ASSISTANT, + AiPersonaEngine.LIBRA_FLOW, + AiPersonaEngine.COMMUNITY_MANAGER, + ], + graph: storedGraph, + expected: 'absent', + }, + ]; + + it.each(attachmentCases)('attaches the expected graph for $name', async ({ + engines, + graph, + override, + expected, + }) => { + for (const engine of engines) { + aiPersonaRepository.findOne.mockResolvedValue({ + id: 'persona-1', + engine, + prompt: ['test'], + externalConfig: {}, + promptGraph: graph, + }); + + await service.invoke( + { + aiPersonaID: 'persona-1', + message: 'Hello', + displayName: 'VC', + externalMetadata: {}, + resultHandler: { action: 'none' }, + promptGraph: override, + } as any, + [] + ); + + const lastCall = aiPersonaEngineAdapter.invoke.mock.calls.at(-1); + expect(lastCall).toBeDefined(); + const adapterInput = lastCall![0]; + if (expected === 'stored') { + expect(adapterInput.promptGraph).toBe(storedGraph); + } else if (expected === 'default') { + expect(adapterInput.promptGraph.nodes).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + name: 'check_input', + prompt: (graphJson.nodes[0].prompt as string[]).join('\n'), + }), + ]) + ); + } else { + expect('promptGraph' in adapterInput).toBe(false); + } + } + }); +});