RFC 091: Preservation identifiers across the LMS migration

[kenoir]

Preview

View the rendered RFC on this branch (renders the Mermaid diagrams and tables):

• RFC 091 README

---

What does this change?

Publishes the digitisation ingest discovery work as RFC 091, now titled Preservation identifiers across the LMS migration. The review discussion on this PR reversed the RFC's original proposal, and the current version records the agreed direction.

The RFC names the preservation identifier as a concept: the identifier Goobi, Archivematica and the storage service share for a digital object, which names the package, the METS and JP2 files, the storage location and the canonical IIIF Manifest URI. Today that is the Sierra b-number for digitised content and the CALM ref for born-digital. The decision recorded here is that preservation identifiers stay the canonical identifiers for digital objects: nothing is minted at ingest, IIIF Manifest URIs do not move to the catalogue Work id, and the Work id appears only as a redirect source. Three independent arguments from the review thread support this: the merge that binds a digital object to its public work is computed downstream from mutable metadata, so its outcome cannot be baked into immutable artefacts; works and digital objects are not reliably 1:1; and canvas ids, image URIs and annotation targets derive from METS filenames and can never be redirected, so moving the manifest URI alone would create permanent internal inconsistency in every document.

Ingest behaviour splits on the predecessor signal, found by a read-only reverse lookup on the Identifiers API (RFC 089):

• Cross-migration (a sierra-system-number sibling exists): reuse the old b-number as the preservation identifier. No transformer change, storage version history continues, and only the Folio-target merger rules are needed after migration.
• Post-migration (no sibling): assign a new preservation identifier at ingest. The Folio id rides in the METS record identifier so the transformer emits a folio-instance merge candidate; the pipeline mints canonical ids downstream exactly as today. The form of the new identifier is the main open question, with requirements (path-safe, lowercase, shape-distinct from Work ids, b-numbers and CALM refs, with a defined multi-volume suffix convention) and candidates; the recommendation is the Folio HRID, which also keeps the storage externalIdentifier equal to the METS record identifier.

The earlier design (a secured resolve-or-mint extension to RFC 089, a pre-minted canonical id as the storage handle, id_minter binding verification, and a hard RFC 081 gate) is withdrawn and recorded under alternatives considered. The Identifiers API stays read-only. The RFC also commits the Work-id redirect contract (extended to born-digital, where the Work id form does not currently resolve), covers born-digital alongside digitised content with worked examples for the four digitised/born-digital and Folio/Axiell permutations, and adds an open question on whether archival refs survive the CALM to Axiell move unchanged.

Files changed:

• rfcs/091-digitisation-ingest-identifiers/README.md: the RFC document (two Mermaid sequence diagrams for the current and proposed ingest flows).
• rfcs/README.md: RFC listing table regenerated by .scripts/create_table_summary.py.

How to test

• Read the rendered RFC and review the decision, proposal, worked examples, alternatives, and open questions.
• Confirm the RFC passes repo validation: .scripts/validate_rfc.py.
• Confirm the listing table is in sync: .scripts/create_table_summary.py --check-readme.
• Some relative links (../085…/, ../090…/) resolve only once those branches merge; they are correct against the merged layout.

How can we measure success?

No measurable runtime success criteria; this is a documentation RFC. Success is the team aligning on the recorded decision: preservation identifiers stay canonical, no minting at ingest, and the cross-migration / post-migration split, with the identifier-form question (open question 1) agreed with digital production, the pipeline team and Digirati.

Have we considered potential risks?

• Documentation only; no production code or infrastructure is changed by this PR.
• The principal design risk is the silent merge break that follows from mis-handling the package identifier; the proposal mitigates it by reusing the b-number cross-migration and by driving the post-migration merge from the METS record identifier exactly as today.
• Three open questions remain, of which the identifier form (including the multi-volume suffix convention) gates the post-migration path, and Axiell ref stability decides whether born-digital needs predecessor support.

Notes for reviewers

• This version reverses the PR's original proposal, following the review thread on this PR; the thread's arguments are recorded in the RFC under "The decision" and "Alternatives considered".
• Amending RFC 085 (RFC 085: Identifiers of and within IIIF resources after the migration #143) to invert its "Use the Work ID" position is a named deliverable of this RFC (next step 7), to be coordinated on that PR.
• RFC 089 stays read-only; a small follow-up amendment to its consumer framing will be proposed separately.

[tomcrane]

If I have understood correctly, the identifier defined here as C is the same concept as the identifier suggested by Paul in the on-site meeting we had a few weeks ago (described as Preservation identifier in this Slack conversation). That is, the identifier the Storage Service, Goobi and Archivematica use to name and save things, and recover old versions of things (packages in storage, workflows in Goobi). And that the RFC describes a way of ensuring that these systems always get given this identifier, which for old things may look like a b number or a Calm ref (see below), but for wholly new things will look like something else... examples will help.

I think this identifier needs a really clear name so we always know what we are talking about.

I know this is about b-numbers and FOLIO migration, but is it worth putting more examples into the RFC for the equivalent changes that have to happen for CALM Refs in both digitised and born digital content? For example in the sections

• METS works keep their own id
• Where each identifier lives after migration

And in the section

• Two kinds of digitisation

It's not just digitisation; a born digital item has the same issues, just with a different identifier form. It's really about preservation:

• Cross-migration preservation
• Post-migration preservation

Digitisation happens before preservation for physical objects, but digitisation is not a step for born digital archive material. Yet they face the same identity issues. Preservation is a precursor to all IIIF-ification, whether born digital or digitised, whether source identity is bibliographic or archival.

Unlike b numbers, hierarchical archival identifiers (AA/BB/37) still exist in the new world. So it's not quite the same, but there is one desired change that has some overlap and may have some implications for the proposal here. Our intention is that the canonical form for all IIIF representations is the Catalogue Work ID, not just those whose canonical form is currently a b number. At the moment the b number is the canonical form for all works that have b numbers, but some do not - in particular, born digital archival material from CALM, and old Miro images.

Consider:

https://iiif.wellcomecollection.org/presentation/SB/1/1/298

This redirects to https://iiif.wellcomecollection.org/presentation/b19995271, because, even though it's in CALM, it:

• has a Sierra b number, too
• It came through Goobi
• Is in storage under the digitised space - at digitised/b19995271

Its preservation identifier / externalIdentifier / C is b19995271 and therefore its current canonical identifier, as far as Goobi, Storage Service and IIIF is concerned, is b19995271. Goobi and the Storage Service will need to carry on treating this object as b19995271 forever, because it's what they currently know it as; it's reflected in METS IDs and storage location.

It has a work ID, which is dpv7wsth.

In the future, this IIIF Manifest's canonical URL will therefore be https://iiif.wellcomecollection.org/presentation/dpv7wsth

Now consider:

https://iiif.wellcomecollection.org/presentation/SABTS/A/2/10

It looks similar, but it doesn't redirect, because:

• it never had a Sierra b number
• It came through Archivematica as born digital
• It is in storage under the born-digital space, at born-digital/SABTS/A/2/10

Its preservation identifier / externalIdentifier / C is SABTS/A/2/10 and therefore its current canonical identifier, as far as Archivematica, Storage Service and IIIF is concerned, is SABTS/A/2/10. Archivematica and the Storage Service will need to carry on treating this object as SABTS/A/2/10 forever, because it's what they currently know it as; it's reflected in the storage location.

It has a work ID, which is h3jc4wga.

In the future, this IIIF Manifest's canonical URL will therefore be https://iiif.wellcomecollection.org/presentation/h3jc4wga

...we will start redirecting it if we receive the archival ref form ../SABTS/A/2/10, even though we don't now. (That's the difference between the two for IIIF delivery).

This is relevant to the PR because the notion of "preservation identifier" / C - the identifier that Goobi, Archivematica and the Storage Service see - is not just confined to b numbers. This isn't ignored in the RFC, e.g.,

"If it matches the Sierra system-number regex it emits a merge candidate sierra-system-number/<b-number>, otherwise it falls back to calm-ref-no for born-digital"

...but maybe it could be clearer and have a good name: there are CALM Ref preservation identifiers, too, that are not b numbers but continue to be the identifier the storage service understands the thing to be, forever.

I think that this is what's meant in the paragraph Canonical id continuity and in the proposal, but I find the terminology confusing (maybe not close enough to it) - different identifiers are canonical in different contexts: an old digitised object's canonical identifier is a b number as far as Goobi and Storage are concerned, and remain so forever. But the same item's canonical IIIF Manifest identifier uses the work ID.

If all Goobi or Archivematica ever rely on when processing is this preservation identifier / C, then they will continue to emit it to the SNS topic when they finish a workflow. And DDS will pick it up, and always see it as b19995271 or SABTS/A/2/10 or whatever.

Post-migration digitisation (no predecessor): mint a fresh catalogue-style id.

Can it be clearer what this catalogue-style id looks like, for new items after the migration has happened:

• Physical item catalogued in Folio, digitised through Goobi (e.g., printed book)
• Physical item catalogued in Calm/Axiell Collections, digitised through Goobi (e.g., some handwritten letters, an archival item)
• Born digital item catalogued in Calm/Axiell Collections, processed through Archivematica
• (Born digital item catalogued in Folio? That is the equivalent of the use case that triggered the DDS' own identity service, see below)

I assume this catalogue-style id is NOT the workId, but minted separately? And it's not the folio id or archival ref either.

I don't know what folio IDs look like but suppose there's an entirely new work - a printed book - being digitised with folio id fol123. It has a catalogue/workId abcd3456. We mint a C / preservation identifier with value wxyz9876, and so it gets preserved in storage at digitised/wxyz9876, and so Goobi makes a METS file called wxyz9876.xml, and all the page image JP2s are called wxyz9876_001.jp2, wxyz9876_002.jp2, etc., and Goobi fires an end-workflow message to SQS with the identifier wxyz9876 for iiif-builder (and others) to pick up.

In this scenario, the IIIF manifest that iiif-builder builds and saves to S3 should be stored as abcd3456.json not wxyz9876.json, because we want to save it in S3 as its public canonical form rather than always having to look up the "storage" form on every manifest request. If someone asks the Catalogue API for abcd3456, the digital location returned in the full API response should be ../presentation/abcd3456 and not .../presentation/wxyz9876 because the public-facing canonical IIIF URL is the semantic one not the preservation one; the work page URI on wc.org for this item has the same last path element as the IIIF Manifest, abcd3456.

On receipt of the workflow message, iiif-builder would pass "wxyz9876" to the identity service and get back a list that included blocks for fol123 and abcd3456. It knows how to select abcd3456 as the canonical identifier to publish the Manifest as.

A series of examples with identifiers for all of the digitised/born-digital, folio/axiell permutations would be really useful and may catch something lurking here.

Other random observations:

• Today, if I know the b number I know (for digitised things) where the preserved files are in storage and I know what the IIIF Manifest URL is, because they are all b numbers (assuming it's a Sierra book).
• Today, if I know the Catalogue workId, I can query the catalogue API and get back enough info to deduce the b number and hence where it lives in storage, and also the IIIF Manifest URL.
• In future, if I know the folio ID, I don't know where it's preserved or what its Manifest URI is. I query the catalogue API or identity service to learn these things.

iiif-builder prerequisite

IIdentityService (proposed in RFC 081 is implemented - it's internal to DDS. It's not fully implemented because the complete implementation depends on the external identity service.

I think it's worth having a comparison of what these two identity services do.

The main driver for the Identity Service in RFC 081 was:

[Collections would like to] ingest born-digital items through Archivematica under bnumbers from Sierra.

(This may be on hold due to the migration work).

Rather than have the DDS parse identifiers and deduce what they are (b numbers etc), where they are stored (digitised or born-digital space), what flavour of METS file they are expected to have (Goobi or Archivematica) etc., the DDS calls a service, passing the identifier (always the resolved currently canonical identifier, so currently never a Catalogue Work ID) and getting back a DdsIdentity object that carries all the info DDS needs to know.

The current implementation of this service still uses the same string parsing logic for now, but it also stores identities in a database, and validates that its conclusions about storage space and source are in fact correct. This was originally the groundwork for "bnumbers through Archivematica" - to remove the assumption that b-number means goobi, b-number means digitised storage space, etc., with the possibility that the information "this identifier's work is preserved in the born-digital storage space" is later obtained from something external to the DDS.

This implementation is not an identity resolution service as proposed in RFC 091. You can't give it a Catalogue workID and expect it to return a DdsIdentity object, because it expects the canonical form (b number or CALM Ref) and then tells you various things about it, rather than accepting any identifier form and telling you the canonical form and aliases.

It also does things that the proposed platform identity service isn't designed to do, such as understand that if you pass b12345678_002, it's a volume-level identifier for which the package-level identifier is b12345678. That is, it returns information about sub-package identifiers that the 091 Identity Service would not know.

However, the current PersistedIdentityService implementation could be extended to make calls to the proposed 091 service, so that inside the DDS, we can pass any identifier to our service and using a combination of existing logic and resolution via the 091 service.

You could say there should be one identity service to rule them all, the authoritative means of resolving an identity to something and learning about its aliases. But the distinction is that the 091 service is about the Catalogue: semantic records for works. Whereas the DDS service is about digital objects and their sub-structure. Inner structure of a digital object has no Catalogue description, therefore the 091 Identity service shouldn't be concerned with it. It's the DDS's job to be authoritative.

If it is useful for other Wellcome processes or workflows to know about these internal structure things, then the DDS should expose it as API.

[kenoir]

In this scenario, the IIIF manifest that iiif-builder builds and saves to S3 should be stored as abcd3456.json not wxyz9876.json, because we want to save it in S3 as its public canonical form rather than always having to look up the "storage" form on every manifest request. If someone asks the Catalogue API for abcd3456, the digital location returned in the full API response should be ../presentation/abcd3456 and not .../presentation/wxyz9876 because the public-facing canonical IIIF URL is the semantic one not the preservation one; the work page URI on wc.org for this item has the same last path element as the IIIF Manifest, abcd3456.

This is really useful feedback, but I want to focus in on this paragraph in particular because it will help clarify our position.

In you example above the following canonical ids exist:

• abcd3456 the LMS sourced work canonical ID, with source id like my_lms/123
• wxyz9876 the preservation system sourced work canonical ID, with source id like mets/123

In the catalogue pipeline matcher/merger we will have noticed that these refer to the same work from merge candidates, and merged them such that abcd3456 gets the digital locations from wxyz9876 and have generated a
redirection from wxyz9876 to abcd3456 such that requesting the former directs you to the latter.

In our proposal the digital locations can carry identifiers with whatever context we have at that point, it could be the METS source id mets/123, the corresponding canonical id, wxyz9876 or even the redirected target abcd3456 but I think the redirected target is the wrong choice as DDS will not have that ID available at point of ingest.

See https://github.com/wellcomecollection/docs/blob/rk/rfc-091-digitisation-ingest-identifiers/rfcs/091-digitisation-ingest-identifiers/README.md#mets-works-keep-their-own-id

The storage service will get the mets/123 mapped wxyz9876 canonical id as the externalIdentifier, with the original source identifier mets/123 inside the METS. So DDS will have that context only to decide manifest URLs.

A crucial point is that this re-direction from wxyz9876 to abcd3456 is only known at merge time based on merge candidates derived from inspecting work metadata, and that state is not fixed if we change the redirection target in the merger. The upshot of this is we cannot give preservation sourced works the canonical id of the LMS target as it breaks some of the assumptions of the pipeline.

If someone asks the Catalogue API for abcd3456, the digital location returned in the full API response should be ../presentation/abcd3456 and not .../presentation/wxyz9876 because the public-facing canonical IIIF URL is the semantic one not the preservation one

I think in this case in the IIIF APIs /presentation/wxyz9876 the preservation work canonical ID can serve as both the preservation and semantic URL.

Worth clarifying the relationship with predecessor ID which is different: We commit that it will always be true that mets/123 has the canonical ID wxyz9876, and in the case of a predecessor relationship being identified in the id minter that the successor will always get the same canonical_id as its predecessor, e.g. mets/b1234567 has wxyz9876, and also mets/folio-123 has wxyz9876. The predecessor relationship is intended to be used when migrating the source work records between systems that give them the same identifiers, works sourced from old & new systems cannot exist in the same pipeline, a fundamental difference from pipeline matching & merging.

[tomcrane]

A concrete example

Something that exists today:

https://wellcomecollection.org/works/km3uczhf is the work page
https://iiif.wellcomecollection.org/presentation/km3uczhf is the IIIF Manifest

atm the latter redirects to ../b28047345 but later it won't, we want to use the catalogue ID here.

Something created in 2027:

https://wellcomecollection.org/works/abcd3456 is the work page (if I understand correctly)

...but do we want the canonical IIIF Manifest to be
https://iiif.wellcomecollection.org/presentation/abcd3456 as well (like the above), or
https://iiif.wellcomecollection.org/presentation/wxyz9876 ?

You say:

...or even the redirected target abcd3456 but I think the redirected target is the wrong choice as DDS will not have that ID available at point of ingest.

Which implies that at initial Manifest construction time (after the Goobi workflow finishes) we'd have to build our Manifest as wxyz9876.json... if we looked up wxyz9876 at that point could we not learn that abcd3456 is its canonical ID?

That doesn't mean we can't have https://iiif.wellcomecollection.org/presentation/abcd3456 as the canonical IIIF Manifest URL, because we can resolve all these identities at the point of service, but we should decide whether that is in fact what we want. It seems neatest and user-friendliest to me that the work page and the manifest being shown on that work page have the same identifier element in them, but it is obviously not a technical necessity

[kenoir]

if we looked up wxyz9876 at that point could we not learn that abcd3456 is its canonical ID?

Yes, that's correct wxyz9876 is the canonical id of the mets sourced work, abcd3456 is the canonical id of the LMS sourced work that cannot be conflated with with the mets sourced work as the catalogue pipeline must continue to match and merge these downstream.

This could be hinting that we should not take this path and instead continue to represent METS sourced works by source identifiers in their current style, i.e. mets/123-lms-id-value. In that situation keeping the id value the same carries the benefit to digital production staff where the thing came from, rather than obscuring it, which the canonical ID would.

We could then simplify this RFC to include only the at point of ingestion tool that keeps records from the new LMS with sierra predecessor IDs on their previous system IDs.

[kenoir]

Alternatively we could keep minting the catalogue identify at point of ingestion, if we can be comfortable with the IIIF endpoints using the METS sourced work canonical identifier, to continue the example: wxyz9876, but on consideration I am leaning towards the non-opaque identifiers being useful for anyone handling these records between digitisation and preservation.

The identifiers API can still exchange these source identifiers for public catalogue identifiers, but cannot hold the result of redirection from merge, the catalogue-api is still the best place to find that.

[paul-butcher]

Yes! It looks like we should continue using source ids as storage identifiers.

I suppose there's an interesting question about who/what the identifiers are for and what they mean. I had pictured them as externally-facing so should be an external-style id.

However, that does not necessarily follow. Ids in the storage service or in METS or IIIF files or whatever are not intended to be shared manually like Wellcome Canonical Ids for Works and Concepts are. These ids could just as well be UUIDs as anything else.

Whatever id scheme we choose, it is not designed to carry meaning; but that doesn't mean it cannot be generated directly from a meaningful identifier (i.e. there's nothing wrong with saying that the id mets/123-lms-id-value is not meaningful in this context). That said, they definitely correspond to the proximal/source work, and not the result of any merge. Partly because that is where the relationship is most "real", but mainly because there is no reason why the resulting merged work would not be made up of multiple works that all have objects in the storage service.

I can imagine a scenario where matching/merging rules change meaning that where we did have a matching storage id and work id, they no longer match. It might cause a bit of a debugging headache where I try to find the storage object that matches the work id. If they never matched (because they are always in different schemes) then that confusion won't arise.

[tomcrane]

I think there is usability value in the web page about a work being https://wellcomecollection.org/works/abcd3456 and the digital object representation of that work (its IIIF Manifest JSON) being https://iiif.wellcomecollection.org/presentation/abcd3456, rather than https://iiif.wellcomecollection.org/presentation/wxyz9876 - if that is workable.

Obviously everything is still functional no matter what those URLs are as long as they eventually resolve to what they are supposed to. It depends on what value you attach to elegance and URL-hackability. These things have some value; the question is whether additional complexity elsewhere is worth that value.

Then the second value consideration is whether it is important, for digitised items, that the individual image URIs are clearly related to the work. This is true today, because the IIIF Manifest at https://iiif.wellcomecollection.org/presentation/b28855541 is full of links to images like https://iiif.wellcomecollection.org/image/b28855541_0001.jp2/full/654,1024/0/default.jpg.

We're going to weaken this in the current plan if the workID becomes the canonical manifest URL - the new canonical URI of this digital object https://iiif.wellcomecollection.org/presentation/k6y5ykqz will still be full of links to images like https://iiif.wellcomecollection.org/image/b28855541_0001.jp2/full/654,1024/0/default.jpg, because the mets source is the Sierra b number.

So for new post-migration digital items, this will either be equally weak if digitisation workflow uses a different mets-source ID: https://iiif.wellcomecollection.org/presentation/abcd3456 will be full of links to images like https://iiif.wellcomecollection.org/image/wxyz9876_0001.jp2/full/654,1024/0/default.jpg (because Goobi and storage service know this item as wxyz9876).

Or it will be stronger if digitisation workflow uses the work ID for new items: https://iiif.wellcomecollection.org/presentation/abcd3456 will be full of links to images like https://iiif.wellcomecollection.org/image/abcd3456_0001.jp2/full/654,1024/0/default.jpg (because Goobi and storage service know this item as abcd3456).

The only arrangement that makes all these URIs equally consistent is if the preservationId / C` continues to be used in the public canonical digital object identifier (IIIF Manifest URI): as today, not the workID. In this scenario, https://iiif.wellcomecollection.org/presentation/b28855541 stays canonical and is full of links to images like b28855541_0001.jp2, and the new item https://iiif.wellcomecollection.org/presentation/wxyz9876 is full of links to images like wxyz9876_0001.jp2, and neither old or new digital objects use the work ID (k6y5ykqz for old, abcd3456 for new) in their _canonical) digital object (IIIF Manifest) URI (but we'll redirect from those workIDs as we do now).

That now is the most consistent approach BUT is counter to where we started with the aim of all IIIF Manifest URIs being iiif.wellcomecollection.org/presentation/{workId} !!!

Maybe we decide which of these scenarios we want and work back?

there is no reason why the resulting merged work would not be made up of multiple works that all have objects in the storage service.

It was true in the past that a work could have multiple digital locations; that IIIF Manifest to Work was not always a 1:1 relationship. Is that still true? Our current informal redirection assumes that it's 1:1 and obviously this is true for the vast majority of cases. An obvious example is separate copies of the same work being digitised. These seem to get distinct work IDs though. Do those two have two different MARC records even though they are just different impressions/printings (not editions)? This example could be an outlier, it is a VERY early digitisation.

Ids in the storage service or in METS or IIIF files or whatever are not intended to be shared manually like Wellcome Canonical Ids for Works and Concepts are.

I don't think that statement is 100% true for IIIF Manifests. They are shared, bookmarked and cited in a way that the "internal" URIs are not. They do go out there in the wild. Not by the general web population, admittedly, but by a growing number of IIIF-aware users. That doesn't mean they have to have the same last path element to work, of course, but they are visible URLs.

[kenoir]

Taking up your suggestion that we decide which scenario we want and work back, I propose we choose the one you called most consistent. The preservation identifier stays the canonical IIIF Manifest URI for old and new content alike, and we don't mint catalogue-style ids upstream at all.

I think the deciding argument comes from RFC 085 itself. Canvas ids, image URIs and annotation targets all derive from METS filenames and can never be redirected, so flipping to the work id could only ever rename the outer manifest URI while everything inside the document keeps the preservation id. Keeping the preservation id canonical seems to be the only arrangement where a manifest and its contents agree, and it means existing b-number URIs, and the external annotations that target them, never move. I think it also means your question about whether manifest to work is still 1:1 stops mattering, because each digital object keeps its own URI regardless of how works merge.

The cost is the one you named, in that the work page and the manifest stop sharing a tail permanently. The work id form still redirects as it does today, and the catalogue API's digital locations carry the manifest URI, so I don't think anything breaks. We would be trading URL elegance for internal consistency and a much smaller system, which I think is the right trade, though it is a deliberate loss and worth recording as one.

If we agree, the "Use the Work ID" section of RFC 085 inverts, and I think the rest of that document becomes the rationale for the change rather than an obstacle to it. I'll rewrite this RFC around preservation identifiers, covering born digital as well as digitised content, with the digitised/born-digital and Folio/Axiell examples you asked for. The Identifiers API stays read-only, and the RFC 081 dependency drops out of the ingest path. What remains to design is the form of the preservation id for wholly new items, since it becomes a public, permanent, never-reused string assigned at ingest, and I'll bring concrete candidates to the rewrite.

[tomcrane]

So to be clear, it's this version from my previous comment:

"The only arrangement that makes all these URIs equally consistent is if the preservationId / C continues to be used in the public canonical digital object identifier (IIIF Manifest URI): as today, not the workID. In this scenario, https://iiif.wellcomecollection.org/presentation/b28855541 stays canonical and is full of links to images like b28855541_0001.jp2, and the new item https://iiif.wellcomecollection.org/presentation/wxyz9876 is full of links to images like wxyz9876_0001.jp2, and neither old or new digital objects use the work ID (k6y5ykqz for old, abcd3456 for new) in their _canonical) digital object (IIIF Manifest) URI (but we'll redirect from those workIDs as we do now)."

The DDS would continue to do redirect duty, so if you only have a workID you can construct a URL which will cheaply redirect to .../{preservation-id}

One to discuss tomorrow?

[kenoir]

Yes, exactly that scenario. One clarification so it doesn't get lost: in your quoted example wxyz9876 was the minted catalogue-style id from the original proposal, but under this version we don't mint anything upstream at all, so for wholly new items the preservation id is whatever string the ingest workflow assigns (the scheme being the open design question I'll bring candidates for), and the work id only ever appears as a redirect source, resolved by the DDS as today. I'll rewrite the RFC on that basis.

[tomcrane]

This is an interesting point. Ideally the P identifier is the same shape as all your other wellcome-minted identifiers - workIds, subjectIds etc. I'm not sure about the METS transformer, but the statement "lets the DDS classify identifiers cheaply, and preserves the debugging property that a storage id and a work id can never be confused" seems to me to be in the category of "very nice to have" rather than essential - because the new Identity Service RFC 091 (the "registry") should be able to tell the DDS what an identifier is.

Exploring this:

/presentation/wxyz9876_0002 (volume two of a new multi-volume item) must be split into package id + volume suffix by the DDS before anything can be resolved; the registry will never answer suffix (sub-work digital object) questions. wxyz9876_0002 is shape-identical to a CALM ref written with underscores (PPCRI_A_1), whose normalisation is underscores→slashes. The DDS dashboard uses this path-safe form in URIs to avoid lots of controller routing issues, but public IIIF API never does (it's always the tail). Lookup-based disambiguation (try the full string, then candidate stems) is workable but a syntactically recognisable id form makes it a non-problem. Either the form is distinguishable, or the multi-volume suffix convention is defined alongside it; it can't be left implicit, because Goobi will stamp it into METS filenames and JP2 names (wxyz9876_0002_0001.jp2) on day one, upstream of any service.

(needs more discussion - I will attempt to explore more in an update of RRC 085)

[tomcrane]

I would prefer that the DDS resolves the ID even if it has to make a few probes rather than relying on the form of the ID. Once it has an authoritative entry it doesn't need to ask the identity service/registry. At first sight it could see wxyz9876_0002 and request wxyz9876 to see if that is also an identifier, in which case it knows that wxyz9876_0002 must be a sub-part (volume) identifier rather than a complete identifier.

Anyway as I said I'll think some more.

[kenoir]

I agree on the DDS side and have rewritten the requirement to say so. Resolving by probe and persisting the answer costs one lookup per identifier. It also avoids baking a parsing rule into a place that would be expensive to change later. I think that is the better trade.

The part I would keep as a hard requirement sits in the pipeline rather than the DDS. The METS transformer picks between sierra-system-number, folio-instance and calm-ref-no by inspecting the record identifier during the transform, and no service call is available in that path. The form therefore has to be distinguishable from the b-number regex and from CALM refs. That is a narrower claim than the one I originally made. It says nothing about Work ids. Distinctness from Work ids only buys the debugging property, so I have demoted it to a preference in the text.

I think the question is moot in practice. in00012345 satisfies both anyway. Choosing the HRID means we never have to settle how much the Work-id distinction is worth.

I agree with your last point rather than having anything to add to it. The suffix convention cannot be left implicit. in00012345_0002 splits cleanly and does not collide with an underscore-written ref like PPCRI_A_1. It still needs writing down, because Goobi stamps it into METS and JP2 names on day one and nothing downstream can correct it afterwards. That is the half of open question 1 that remains live.

I am happy to wait on your RFC 085 update for the general principle. I do not think it blocks this RFC either way.

[tomcrane]

OK so the P-identifier is in00012345... will leave previous comment though as it's evidence for why to choose this form.

[kenoir]

Yes, in00012345. I have promoted it from one of three candidates to the RFC's stated proposal. The two alternatives were a dedicated prefix-plus-random scheme and UUIDs, and both are now a short note on why not.

Its structural advantage matters more than its legibility. It makes the storage externalIdentifier and the METS record identifier the same string again. That is the property they have today with the b-number, and it collapses open question 2 entirely.

Open question 1 is correspondingly narrower. The form is not really in doubt. What remains is sign-off from digital production and the pipeline team, who have not formally seen it, plus the multi-volume suffix convention.

[tomcrane]

This is great!

I agree with all of this. One tiny detail, for a born digital archive item you'd never end up with files that are named during the process like PPCRI_D_4_2_0001.jp2 because the born digital files retain their original filenames as far as possible, e.g., crick-notes-draft-3.doc rather than have new filenames minted by Archivematica. The eventual DLCS-served file would have a URI that retained the original file name at the tail (subject to any URL-safety mutations).

[kenoir]

Fixed, thank you. The born-digital worked example now carries a Files row showing retained original names. I took the example from the live manifest rather than inventing one. Canvas ids there look like /presentation/SABTS/A/2/10/canvases/SABTS_A_2_10---BTSARCH.LST.

The PPCRI_D_4_2_0001.jp2 you quoted is in the digitised, Axiell-catalogued example, where Goobi does mint those names, so it stays. What was missing was any statement that born-digital differs, which left the two examples looking like the same flow. There is now a paragraph after the table saying that Archivematica does not rename and that the DLCS URI keeps the original name at the tail, subject to URL-safety changes. There is therefore no <P>_0001.jp2 sequence.

That also means the multi-volume suffix convention in open question 1 applies only to Goobi digitisation. Born-digital never needs one.