feat(053): attach stored promptGraph for the generic-openai engine

docs/prompt-graphs.md

@@ -0,0 +1,124 @@
+# Generic-engine prompt graphs
+
+AI Personas can store a declarative prompt graph and send it to the engine at
+invocation. The graph is configuration: no deployment or feature flag is
+needed once it is stored on the persona.
+
+The ready-to-store workshop example is
+[`prompt-graphs/workshop-design.authoring.json`](./prompt-graphs/workshop-design.authoring.json).
+It is the server-authoring conversion of the Virtual Contributor's
+[`workshop-design.json`](https://github.com/alkem-io/virtual-contributor/blob/develop/docs/prompt-graphs/workshop-design.json).
+For engine semantics and validation errors, see the Virtual Contributor
+[prompt-graphs README](https://github.com/alkem-io/virtual-contributor/blob/develop/docs/prompt-graphs/README.md).
+
+## Engines that receive a stored graph
+
+| Engine | Stored graph attached at invocation? | Reason |
+| --- | --- | --- |
+| `expert` | Yes | The expert plugin executes prompt graphs. If no graph is stored, it receives its existing expert default graph. |
+| `generic-openai` | Yes | The generic plugin executes declarative prompt graphs. It receives no fallback graph when none is stored. |
+| `guidance` | No | Its plugin has no prompt-graph execution path. |
+| `openai-assistant` | No | Its plugin has no prompt-graph execution path. |
+| `libra-flow` | No | Its consumer does not execute prompt graphs. |
+| `community-manager` | No | It has no active prompt-graph queue consumer. |
+
+Only `expert` and `generic-openai` are an invocation allowlist. Storage is
+engine-independent, so a graph may be staged before an engine switch, but it
+will not be sent for the excluded engines.
+
+## Activate the workshop graph
+
+1. Ensure the target persona uses `generic-openai` and its Virtual Contributor
+   has a body of knowledge. The supplied graph contains `retrieve` nodes.
+2. Copy the complete JSON payload from
+   [`prompt-graphs/workshop-design.authoring.json`](./prompt-graphs/workshop-design.authoring.json)
+   into `promptGraph` in the following admin mutation variables.
+3. Run the mutation with an actor that has `UPDATE` on the AI Persona.
+
+```graphql
+mutation UpdatePersonaPromptGraph($aiPersonaData: UpdateAiPersonaInput!) {
+  aiServerUpdateAiPersona(aiPersonaData: $aiPersonaData) {
+    id
+    engine
+    promptGraph {
+      nodes {
+        name
+        type
+      }
+      edges {
+        from
+        to
+        on
+        map
+        default
+      }
+    }
+  }
+}
+```
+
+```json
+{
+  "aiPersonaData": {
+    "ID": "<ai-persona-id>",
+    "engine": "GENERIC_OPENAI",
+    "promptGraph": "<the complete JSON object from workshop-design.authoring.json>"
+  }
+}
+```
+
+The `promptGraph` value above is an object, not a JSON-encoded string. It is
+shown as a placeholder to keep this guide readable; use the fixture unchanged.
+
+Updates merge only the supplied top-level `promptGraph` keys, so omitted keys
+such as `start` and `end` are retained from a previously stored graph. For a
+clean replacement, first update the persona with `promptGraph: null`, then
+send the complete replacement graph in a subsequent update.
+
+### Retrieval precondition
+
+`retrieve` nodes require the Virtual Contributor invocation to carry a body
+of knowledge. Operators must verify that pairing at activation time; a
+misconfigured retrieve-bearing graph fails every invocation until it is
+corrected. The server deliberately does not validate the pairing when the
+persona is updated: `bodyOfKnowledgeID` is supplied per invocation, not stored
+on the persona. If it is absent, the engine fails at parse time and the member
+receives the standard error response at invocation. This is expected behavior,
+not a server-side validation failure.
+
+### External-provider egress check
+
+Before activating a retrieve-bearing graph, verify that the body of knowledge's
+data classification and the external model provider's processing basis permit
+the egress. Retrieval can send up to 95,000 characters of body-of-knowledge
+content to the external model provider in later prompts.
+
+## Authoring another engine payload
+
+The GraphQL input is deliberately a storage shape that matches the engine
+payload. Convert a Virtual Contributor payload as follows:
+
+1. Add `"system": false` to every node. `system` is required by the server
+   authoring API even though the generic engine does not read it.
+2. Keep typed-node fields verbatim: `type`, `source`, `collection_template`,
+   `query_template`, `n_results`, `max_context_chars`, and `output_key`.
+3. Keep conditional edge fields verbatim: `on`, `map`, and `default`.
+   `map` is a JSON object of routing-value to node-name; its keys are matched
+   case-insensitively by the engine.
+4. Convert JSON-Schema dictionary `state.properties` and node
+   `output.properties` to GraphQL's list form. For example,
+   `{"role":{"type":["string","null"]}}` becomes
+   `[{"name":"role","type":"string","optional":true}]`.
+5. Convert dictionary-form `required: ["name"]` to `optional: false` on the
+   named list entries, then drop `required`. Convert nested arrays of objects
+   recursively using the data point's `items` struct.
+6. Omit optional fields instead of passing `null`. Explicit nulls on retrieve
+   or conditional fields are stored but fail the engine parser with a named
+   error. The engine, not the server, is the validation authority for graph
+   semantics.
+
+The read surface returns stored graphs for any engine. A generic persona with
+no stored graph reads back `null`. Separately, the existing field resolver
+returns the expert default graph for `expert` and `libra-flow` personas without
+a stored graph; that pre-existing read-surface asymmetry is out of scope and
+does not change the invocation allowlist above.

docs/prompt-graphs/workshop-design.authoring.json

@@ -0,0 +1,263 @@
+{
+  "state": {
+    "type": "object",
+    "properties": [
+      {
+        "name": "role",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "duration",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "workshop_type",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "purpose",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "audience_size",
+        "type": "integer",
+        "optional": true
+      },
+      {
+        "name": "question",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "complete",
+        "type": "boolean",
+        "optional": true
+      },
+      {
+        "name": "action",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "current",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "knowledge_docs",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "result",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "conversation",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "current_question",
+        "type": "string",
+        "optional": true
+      },
+      {
+        "name": "bok_id",
+        "type": "string",
+        "optional": true
+      }
+    ]
+  },
+  "nodes": [
+    {
+      "name": "check_input",
+      "type": "llm",
+      "input_variables": ["conversation"],
+      "prompt": "You are a conversation analyser extracting key information points from a conversation between a human and an assistant. The conversation below is about a workshop creation and you are looking for the following data points: role, duration, workshop_type, purpose, audience_size.\n\nReply according to the output format instructions below, trying to fill in as many fields as possible.\n\nIf one or more data points are missing, generate a question asking for them. Ask for no more than 2 data points in a single question.\n\nIf the conversation is missing, assume all key points are missing.\n\nConversation:\n{conversation}\n\nOutput format instructions:\n{format_instructions}",
+      "output": {
+        "type": "object",
+        "properties": [
+          {
+            "name": "role",
+            "type": "string",
+            "description": "The role of the workshop creator/facilitator in the organisation.",
+            "optional": false
+          },
+          {
+            "name": "duration",
+            "type": "string",
+            "description": "The duration of the workshop.",
+            "optional": false
+          },
+          {
+            "name": "workshop_type",
+            "type": "string",
+            "description": "The type of the workshop - in-person, online, etc.",
+            "optional": false
+          },
+          {
+            "name": "purpose",
+            "type": "string",
+            "description": "The purpose/goal of the workshop.",
+            "optional": false
+          },
+          {
+            "name": "audience_size",
+            "type": "integer",
+            "description": "The size of the audience of the workshop.",
+            "optional": false
+          },
+          {
+            "name": "question",
+            "type": "string",
+            "description": "A question asking the workshop creator to provide a missing key variable.",
+            "optional": false
+          },
+          {
+            "name": "complete",
+            "type": "boolean",
+            "description": "True when all five data points are present.",
+            "optional": false
+          }
+        ]
+      },
+      "system": false
+    },
+    {
+      "name": "ask",
+      "type": "echo",
+      "source": "question",
+      "system": false
+    },
+    {
+      "name": "analyse_last_message",
+      "type": "llm",
+      "input_variables": ["conversation"],
+      "prompt": "You are a conversation analyser. You are provided with a conversation between a human and an assistant about a workshop design.\n\nBased on the last human message, determine if the user wants to refine an already generated workshop design ('refine') or wants a new one generated ('generate').\n\nConversation:\n{conversation}\n\nOutput format instructions:\n{format_instructions}",
+      "output": {
+        "type": "object",
+        "properties": [
+          {
+            "name": "action",
+            "type": "string",
+            "description": "Either 'refine' or 'generate'.",
+            "optional": false
+          }
+        ]
+      },
+      "system": false
+    },
+    {
+      "name": "extract",
+      "type": "llm",
+      "input_variables": ["conversation"],
+      "prompt": "You are a conversation analyser. Below you are provided with a conversation between a human and an assistant about a workshop design, formatted like:\n```\nhuman:\nmessage content\nassistant:\nmessage content\n...\n```\nOne of the messages contains a workshop design. Find it and return the contents of the message without any additional information or alterations.\n\nNever explain your reasoning or provide any additional information outside of the workshop design.\n\nConversation:\n{conversation}\n\nOutput format instructions:\n{format_instructions}",
+      "output": {
+        "type": "object",
+        "properties": [
+          {
+            "name": "current",
+            "type": "string",
+            "description": "The current iteration of the workshop design found in the conversation.",
+            "optional": false
+          }
+        ]
+      },
+      "system": false
+    },
+    {
+      "name": "retrieve_refine",
+      "type": "retrieve",
+      "collection_template": "{bok_id}-knowledge",
+      "query_template": "I need information about Liberating Structures suitable for a {workshop_type} for {audience_size} people held by {role} for {duration} in order to {purpose}.",
+      "n_results": 10,
+      "max_context_chars": 95000,
+      "output_key": "knowledge_docs",
+      "system": false
+    },
+    {
+      "name": "refine",
+      "type": "llm",
+      "input_variables": ["current", "current_question", "knowledge_docs"],
+      "prompt": "You are a renowned expert in Liberating Structures.\n\nYou are provided with a workshop design based on Liberating Structures and a human request to alter it.\n\nBased on the context provided below, apply the changes the user requested to the current workshop design.\n\nNever explain your reasoning or provide any additional information outside of the workshop design.\n\nThe design format is Markdown. Preserve the formatting, indentation and structure of the design. Add nice indentation to the design and do not remove any empty lines.\n\nWorkshop design:\n{current}\n\nHuman request:\n{current_question}\n\nContext:\n{knowledge_docs}",
+      "system": false
+    },
+    {
+      "name": "retrieve_generate",
+      "type": "retrieve",
+      "collection_template": "{bok_id}-knowledge",
+      "query_template": "I need information about Liberating Structures suitable for a {workshop_type} for {audience_size} people held by {role} for {duration} in order to {purpose}.",
+      "n_results": 10,
+      "max_context_chars": 95000,
+      "output_key": "knowledge_docs",
+      "system": false
+    },
+    {
+      "name": "generate",
+      "type": "llm",
+      "input_variables": [
+        "role",
+        "duration",
+        "workshop_type",
+        "purpose",
+        "audience_size",
+        "knowledge_docs"
+      ],
+      "prompt": "Here is information about Liberating Structures that are suitable for the scenario described below. Base your answer ONLY on that and ignore your previous knowledge about Liberating Structures.\n\nLiberating Structures information:\n\n{knowledge_docs}\n\nDesign a workshop for a {workshop_type} for {audience_size} people, held by a {role}, for {duration}, in order to {purpose}, using only the Liberating Structures information provided above.\n\nNever explain your reasoning or provide any additional information outside of the workshop design. The design format is Markdown.",
+      "system": false
+    }
+  ],
+  "edges": [
+    {
+      "from": "START",
+      "to": "check_input"
+    },
+    {
+      "from": "check_input",
+      "on": "complete",
+      "map": {
+        "true": "analyse_last_message",
+        "false": "ask"
+      }
+    },
+    {
+      "from": "ask",
+      "to": "END"
+    },
+    {
+      "from": "analyse_last_message",
+      "on": "action",
+      "map": {
+        "refine": "extract",
+        "generate": "retrieve_generate"
+      },
+      "default": "retrieve_generate"
+    },
+    {
+      "from": "extract",
+      "to": "retrieve_refine"
+    },
+    {
+      "from": "retrieve_refine",
+      "to": "refine"
+    },
+    {
+      "from": "refine",
+      "to": "END"
+    },
+    {
+      "from": "retrieve_generate",
+      "to": "generate"
+    },
+    {
+      "from": "generate",
+      "to": "END"
+    }
+  ]
+}