feat(server): adapt Hubble 2.0 & add graph/role management#3008
Conversation
- Add listProfile endpoint with default graph sorting and prefix filtering - Add setDefault/unsetDefault/getDefault endpoints for default graph management - Add manage(PUT) endpoint for graph nickname update - Add createByForm for form-urlencoded graph creation compatibility - Auto-fill HStore/PD defaults (backend/serializer/store) during graph creation
- Add setDefaultRole/checkDefaultRole/deleteDefaultRole in GraphSpaceAPI - Add checkDefaultRole endpoint in ManagerAPI - Add default role interfaces in AuthManager - Implement default role CRUD in StandardAuthManager and StandardAuthManagerV2 - Add stub proxy methods in HugeGraphAuthProxy
- Add new SchemaTemplateAPI with list/get/create/update/delete operations - Fix package path from api.profile to api.space - Use HugeGraphAuthProxy.username() instead of authManager.username()
There was a problem hiding this comment.
Pull request overview
This PR adds/extends HugeGraph Server REST endpoints and auth-layer capabilities needed by the Hubble 2.0 frontend, focusing on graph profile listing, default graph selection, default role management, and schema template CRUD within graphspaces.
Changes:
- Added graph profile listing + default-graph set/unset/query APIs, plus graph create compatibility tweaks.
- Implemented default-graph/default-role persistence methods in the
AuthManagerinterface and its implementations/proxies. - Introduced schema template CRUD API and corresponding
GraphManagerhelpers.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 16 comments.
Show a summary per file
| File | Description |
|---|---|
| hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/util/ConfigUtil.java | Adds config-to-string helper used by graph profile listing. |
| hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/auth/AuthManager.java | Extends auth interface for default graph/role operations. |
| hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/auth/StandardAuthManager.java | Implements new default graph/role methods (non-PD auth manager). |
| hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/auth/StandardAuthManagerV2.java | Implements new default graph/role methods (PD-mode auth manager). |
| hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeGraphAuthProxy.java | Proxies/delegates new AuthManager methods. |
| hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/core/GraphManager.java | Adds schema template management helpers. |
| hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/profile/GraphsAPI.java | Adds graph profile listing, default graph APIs, manage/update behavior, and create defaults. |
| hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/space/GraphSpaceAPI.java | Adds default role management endpoints and JSON tolerance. |
| hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/auth/ManagerAPI.java | Adds endpoint to query whether current user has a default role. |
| hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/space/SchemaTemplateAPI.java | New schema template CRUD endpoint implementation. |
Comments suppressed due to low confidence (1)
hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/profile/GraphsAPI.java:443
configsis only validated as non-null whenclone_graph_nameis empty. Ifclone_graph_nameis provided and the request body is omitted/empty,configscan be null andconvConfig(configs)will throw aNullPointerExceptionin the clone branch. Consider defaultingconfigsto an empty map (or makingconvConfig()null-safe) before using it for cloning.
// Check required parameters for creating graph
if (StringUtils.isEmpty(clone)) {
// Only check required parameters when creating new graph, not when cloning
E.checkArgument(configs != null, "Config parameters cannot be null");
// Auto-fill defaults for PD/HStore mode when not provided
configs.putIfAbsent("backend", "hstore");
configs.putIfAbsent("serializer", "binary");
configs.putIfAbsent("store", name);
// Map frontend 'schema' field to backend config key
Object schema = configs.remove("schema");
if (schema != null && !schema.toString().isEmpty()) {
configs.put("schema.init_template", schema.toString());
}
}
String creator = HugeGraphAuthProxy.username();
if (StringUtils.isNotEmpty(clone)) {
// Clone from existing graph
LOG.debug("Clone graph '{}' to '{}' in graph space '{}'", clone, name, graphSpace);
graph = manager.cloneGraph(graphSpace, clone, name, convConfig(configs));
} else {
// Create new graph
graph = manager.createGraph(graphSpace, name, creator,
convConfig(configs), true);
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…#3008) - fix: use @POST/@delete for setDefault/unsetDefault (REST semantics) - fix: add null/empty validation before role field access in GraphSpaceAPI to prevent NPE in setDefaultRole/checkDefaultRole/deleteDefaultRole - fix: change isPrefix to private static and guard nickname null in GraphSpaceAPI and GraphsAPI - fix: ConfigUtil.writeConfigToString always returns JSON regardless of whether config was loaded from file, fixing listProfile endpoint - fix: add @RolesAllowed annotations to SchemaTemplateAPI endpoints - fix: use ForbiddenException (403) instead of HugeException (400) for authorization failures in SchemaTemplateAPI and GraphSpaceAPI - fix: correct LOG placeholder count in SchemaTemplateAPI.delete - fix: use HugeException ('%s') format instead of SLF4J '{}' format - fix: replace com.alipay StringUtils with commons-lang3 in ManagerAPI - fix: add @consumes and checkUpdate() validation to SchemaTemplate.update - fix: add ensurePdModeEnabled guard to ManagerAPI.checkDefaultRole - fix: guard configs null access in GraphsAPI.create clone branch
新增 API 单机版(非 PD)兼容性对照目标:社区默认的 RocksDB 单机版通过 需要兼容单机版的端点(GraphsAPI)
PD 专属端点(无需兼容单机版)
总结:7 个 GraphsAPI 端点需要兼容单机版。其中 3 个需要代码改动(manage 需加 |
… mode ## Background Hubble 2.0 previously relied exclusively on PD mode (distributed HStore backend). This PR makes the server-side APIs fully compatible with the community default: single-node RocksDB without PD/HStore, so that Hubble remains functional out of the box for all deployment modes. ## Core bug fixes ### GraphsAPI - Fix `create()`: `backend=hstore` / `serializer=binary` defaults are now only injected when `manager.isPDEnabled()` is true, preventing graph creation failures on standalone RocksDB deployments. - Fix `manage()`: replace `exist.nickname(nickname)` (in-memory only) with `manager.updateGraphNickname()`, which persists the change to PD meta storage in distributed mode and gracefully falls back to in-memory update in standalone mode. - Fix `manage()`: relax `actionMap.size() == 2` validation to `containsKey(GRAPH_ACTION)`, so extra fields from the frontend no longer cause spurious 400 errors. - Guard `getDefaultGraph()`, `setDefault()`, `unsetDefault()`, and `getDefault()` with `isPDEnabled()` checks; return empty results in standalone mode instead of throwing NPE. - Fix `listProfile()`: guard `getDefaultGraph()` call with `isPDEnabled()`; add null-safe fallback for `gs.nickname()` in non-PD mode. ### GraphManager - `isExistedGraphNickname()`: add non-PD branch that scans in-memory graphs instead of accessing the uninitialized `metaManager`, preventing NPE in standalone mode. - New `updateGraphNickname()`: updates in-memory graph instance first, then persists nickname to `metaManager` only in PD mode. ### ConfigUtil - `writeConfigToString()`: always serializes config to JSON (previously could emit raw properties format), fixing `listProfile` deserialization. - New `isSensitiveKey()`: filters keys containing `password`, `secret`, `token`, `credential`, `private_key`, or `auth.key` from the serialized output to prevent credential leakage through the API. ### ManagerAPI - Add `ensurePdModeEnabled()` guard to all PD-specific endpoints (`createManager`, `deleteManager`, `list`, `checkRole`, `getRolesInGs`, `checkDefaultRole`). - Wrap `HugeDefaultRole.valueOf()` in try-catch to return HTTP 400 instead of HTTP 500 when an invalid role string is supplied. ### SchemaTemplateAPI - Fix incorrect `HugeException` import; replace with `ForbiddenException` for proper HTTP 403 semantics. - Add missing `@RolesAllowed` annotations and implement `checkUpdate()` validation. ### StandardAuthManager - Implement `setDefaultGraph` / `getDefaultGraph` / `unsetDefaultGraph` using marker-group pattern (HugeGroup + HugeBelong) for persistence. - Implement `createDefaultRole` / `createSpaceDefaultRole` / `isDefaultRole` / `deleteDefaultRole` with the same marker-group mechanism. - Add detailed design-note Javadoc explaining the workaround, its limitations, and the non-PD degradation path. ## Code quality improvements - Extract shared `isPrefix(Map, String)` helper and `DATE_FORMATTER` constant into the `API` base class, eliminating ~30 lines of duplicated code across `GraphsAPI` and `GraphSpaceAPI`. - Replace non-thread-safe `SimpleDateFormat` (constructed per-request) with a single static `DateTimeFormatter` (immutable, thread-safe). - Fix 12-hour clock format `hh` → 24-hour `HH` in `GraphSpaceAPI`.
imbajin
left a comment
There was a problem hiding this comment.
Follow-up review after the latest update: these are remaining current-PR issues that look small enough to fix directly, while already-covered findings are intentionally not duplicated.
- Restore SecurityManager and clean up prepared GremlinServer on startup failures - Resolve default-role owner type detection for group bindings - Validate graphspace and observer graph ownership in default-role APIs - Return 4xx for malformed graph manage requests instead of server errors - Invalidate user cache after default graph/default role mutations - Make V2 default-role deletion idempotent for repeated requests - Validate graph create/clone config values and reject null or non-scalar input - Avoid rolling back local nickname state after successful PD persistence - Remove misleading @consumes annotations from entity-less default-role endpoints - Add standalone tests for malformed graph API requests
- Add hstore-gated graph default lifecycle coverage for graphspace graphs - Add default role lifecycle coverage through the graphspace role API - Add schema template lifecycle coverage in PD/HStore mode - Clean trailing whitespace in GraphsAPI
|
Follow-up TODO for CI coverage: Add a real distributed-mode CI job that boots PD + Store and runs the hstore-backed graphspace API tests against that environment. The tests added in this PR compile and are gated to Suggested scope for a separate issue:
This is not necessarily a blocker for the current PR, but it is the right follow-up to prevent these Hubble-compatible APIs from only being validated at compile time. |
There was a problem hiding this comment.
Review summary
- Blocking: yes
- Summary: The PR introduces a cluster master safety regression and an authorization gap in default-role management.
- Evidence:
git diff origin/master...HEADmvn -pl hugegraph-server/hugegraph-api -am -DskipTests -Dmaven.javadoc.skip=true compilepassed
| page = PageInfo.pageInfo(servers); | ||
| } | ||
| } while (page != null); | ||
| } catch (Exception e) { |
There was a problem hiding this comment.
High: Existing live master detection is swallowed
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/ServerInfoManager.java:139
Evidence
- The uniqueness check at lines 131-133 throws when another alive master exists, but the new broad
catch (Exception e)at line 139 logs and continues tosaveServerInfo()at line 149.
Impact
- A node can register itself as master even after detecting an existing alive master, allowing multiple masters in the same cluster.
Requested fix
- Do not catch the invariant failure. Only handle the specific schema-mismatch/read error if needed, and rethrow when an existing alive master is found.
- rethrow duplicate master invariant failures - restrict default-role mutations to admin/space manager - add regression tests for master and role boundaries
- move master uniqueness regression into MasterServerInfoManagerTest - keep UnitTestSuite using normal imports and class references - avoid conflicting with master ServerInfoManagerTest coverage
4f69c52 to
682c2b5
Compare
|
Follow-up boundary notes after rechecking the current Hubble2 branch:
|
- clear auth proxy user-role cache after default-role mutations - keep graphspace profile behind the PD-mode guard - make hstore graphspace API tests fail fast when unavailable - add unit coverage for default-role cache invalidation
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: Current head still has a shared-storage server-info regression and a test coverage gap; visible hbase CI is failing. Evidence: final gate saw build-server (hbase, 11) and codecov/patch failures; static review of refs/remotes/pr/3008.
| * Re-save ServerInfo to recover automatically. | ||
| */ | ||
| LOG.warn("ServerInfo is missing: {}, re-saving it now", this.selfNodeId()); | ||
| serverInfo = this.saveServerInfo(this.selfNodeId(), this.selfNodeRole()); |
There was a problem hiding this comment.
heartbeat() now re-saves missing ServerInfo for master nodes too, and the latest hbase job is failing on that path. The log shows ServerInfo is missing: DEFAULT-hugegraph/server-1, re-saving it now, then GremlinApiTest.testClearAndInit hits Already existed master 'DEFAULT-hugegraph/server-1' in current cluster, followed by decreaseLoad() NPEs because self ServerInfo is still missing for DEFAULT-hugegraph/server1. Please make current-node master re-registration idempotent, or skip only the normalized current node in the uniqueness scan, and guard decreaseLoad() when self ServerInfo is unavailable.
|
|
||
| boolean result; | ||
| if (hasGraph) { | ||
| result = authManager.isDefaultRole(graphSpace, graph, user, |
There was a problem hiding this comment.
This endpoint checks raw default-role metadata without first validating that the path graphspace exists, and for OBSERVER it only checks that graph is non-empty. A typo or stale graph name can therefore return a metadata result instead of a clear 4xx. Please mirror GraphSpaceAPI: validate the graphspace before the auth lookup, and for observer roles verify that the graph exists in that graphspace before calling isDefaultRole().
| GraphSpaceApiTest.class, | ||
| GraphSpaceApiStandaloneTest.class, | ||
| ManagerApiStandaloneTest.class, | ||
| GraphsApiStandaloneTest.class, |
There was a problem hiding this comment.
Adding the backend-specific suites here does not make them run in CI as written. The new standalone and hstore tests gate on System.getProperty("backend"), but run-api-test.sh invokes Maven as mvn ... -P api-test,$BACKEND, and the surefire config does not pass the Maven backend profile property into the test JVM. That leaves backend null, so assumeStandaloneMode() skips the standalone suite and GraphSpaceApiTest skips as non-hstore. Please pass backend into surefire, for example with systemPropertyVariables, or call Maven with -Dbackend=$BACKEND.
- stop swallowing unexpected master scan failures - classify server-info startup failures as backend errors - cover skew, interrupt, and scan failure paths
- reject non-admin SPACE default-role checks - add unit coverage for admin and manager paths - keep standalone GraphSpaceAPI behavior unchanged
- include loaded DEFAULT local config graphs in PD graph lists - keep system graph hidden from graph list responses - add unit coverage for local config and SYS_GRAPH filtering - verify GraphSpaceAPITest and diff whitespace
- include loaded DEFAULT graph registry entries in PD graph listing - keep non-admin graph list filtering on READ roles - allow admin managers to list/profile loaded graphs - add regression coverage for loaded graph registry listing
- adopt soft-disabled ServerInfo behavior from master - keep GraphSpace API unit coverage in the merged suite - remove obsolete legacy master ServerInfo tests
- reject invalid memory monitor periods and roll back startup resources - validate default-role graphspace and observer graph targets - propagate backend profiles into API test JVMs - remove stale manager route coverage and add regressions
- support canonical default graph APIs in standalone mode - make marker group creation idempotent - remove state-changing GET compatibility routes - cover repeated set and canonical API semantics
- separate V2 schema cache names from V1 caches - prevent incompatible cache attachment casts - preserve cache sharing within each transaction generation
- derive V2 cache fixtures from production prefixes - verify V1 and V2 caches remain isolated - cover graph-specific V2 cache clearing
- keep backend-aware API tests on the selected profile - restore core and unit test JVM property boundaries - avoid enabling excluded HStore core scenarios
- link legacy HStore guards to issue 3090 - explain why backend remains API-test scoped - preserve the current test behavior
- use graphspace-scoped user and manager API routes - assert successful space manager creation in the fixture - expect the established 404 after schema template deletion
Purpose of the PR
Adapt the Server APIs needed by Hubble 2.0 so the community Server can support
Hubble's graph, GraphSpace, default graph, default role, schema template, and
manager flows.
Hubble2 will use only the canonical APIs listed below. Since Hubble2 has not
been released, legacy form-based APIs and state-changing GET aliases are not
compatibility requirements and should be removed before release.
Hubble2 reference: hugegraph/hugegraph-toolchain#4
Main Changes
GraphsAPI.java)GraphSpaceAPI.java)ManagerAPI.java)SchemaTemplateAPI.java)Integration Boundaries
POST /graphspaces/{graphspace}/graphs/{name}POST /graphspaces/{graphspace}/graphs/{name}/defaultDELETE /graphspaces/{graphspace}/graphs/{name}/defaultstate-changing GET aliases before Hubble2 is released.
PD-mode APIs. Standalone/non-PD Hubble should hide or degrade these flows
instead of requiring Server to emulate the full PD GraphSpace model.
DEFAULTGraphSpace and graphoperations that are meaningful in standalone mode.
data-only clear that preserves schema, and schema-and-data clear. Server must
implement and verify both semantics before Hubble wires both user actions.
Known Integration Follow-ups
GET /graphspaces/DEFAULT/graphsreturning an emptylist while direct
GET /graphspaces/DEFAULT/graphs/hugegraphsucceeds.maxcounts when Server-side Gremlinexecution does not provide the expected
galias.boundary is finalized.
Server follow-up status
memory_monitor.periodto be positive, while keepingmemory_monitor.threshold=1.0as the disable switch.construction or startup fails.
OBSERVERgraph existence in the managerdefault-role check.
nonexistent singular
auth/manager/defaulttest route.that preserves schema, and schema-and-data clear. Define canonical APIs,
confirmation/error semantics, and verify behavior across supported backends.
Hubble2 is released; retain only the canonical POST/DELETE operations.
Hubble migration status
POST/DELETEmethods to set and unset the default graph.204delete response.PUTcontract for backend-to-Server graphmetadata updates; the legacy Hubble GET
/updateroute is removed.facades; keep only canonical resource APIs.
/graphs/defaultand remove HubbleGET default-graph mutation aliases.
deletion scope, warn that it is irreversible, and require strong confirmation.
for now, but is explicitly deprecated because Server accepts only
update.mutation facade.
Hubble TODOs
Hubble actions to non-GET operations: data-only clear that preserves schema,
and schema-and-data clear. Keep both user-facing actions and add UI smoke
coverage proving the invoked mode and deletion scope.
default-role/account/role, loader peers, algorithms, and async tasks once the
Server API boundary and test environment are available.
Remaining design decisions
when a user and group can share the same name.
feature is exposed by Hubble.
Verifying these changes
boundaries.
rollback, manager resource validation, and backend-profile propagation.
canonical graph create path.
graph get, schema template list, and Gremlin schema/data read-write.
standalone and distributed backends.
mode and that its confirmation accurately describes the deletion scope.