diff --git a/src/main/xar-resources/data/configuration/configuration.xml b/src/main/xar-resources/data/configuration/configuration.xml index 4a896eda..23fe0d84 100644 --- a/src/main/xar-resources/data/configuration/configuration.xml +++ b/src/main/xar-resources/data/configuration/configuration.xml @@ -5,7 +5,7 @@ xmlns:xlink="http://www.w3.org/1999/xlink"> Application Server Configuration - 4Q25 + 3Q26 operations @@ -21,11 +21,11 @@ Main configuration file - The main configuration file for eXist-db is called - conf.xml, which is loaded the root directory of the + The main configuration file for eXist-db is called + conf.xml, which is loaded from the root directory of the distribution (as specified by the system property exist.home). - The configuration file conf.xml is divided into twelve + The configuration file conf.xml is divided into thirteen sections: @@ -279,7 +279,7 @@ size - This attributes sets the maximum allowed size of the journal file. + This attribute sets the maximum allowed size of the journal file. Once the journal reaches this limit, a checkpoint will be triggered and the journal will be cleaned. However, the database waits for running transactions to return before processing this checkpoint. In @@ -321,7 +321,7 @@ until the current file buffer (32kb) is really full. This can speed up eXist on some systems where a file sync is an expensive operation (mainly windows XP; not necessary on Linux). - However, group-comit="yes" will increase the + However, group-commit="yes" will increase the chance that an already committed operation is rolled back after a database crash. @@ -340,7 +340,7 @@ - consistency-check + consistency-check If set to yes, a consistency check will be run on the database if an error was detected during crash recovery. This option @@ -421,7 +421,7 @@ Tracking locks via the Lock Table imposes a small overhead per-Lock. Once users have finished testing their system to ensure correct operation, they may wish to disable this in production to ensure the absolute best performance. - Unless nessecary, it is recommened to leave this enabled. + Unless necessary, it is recommended to leave this enabled. @@ -429,7 +429,7 @@ Experimental: Causes path locks to be used for documents as well as collection locks. This has a performance and concurrency impact, but will ensure that you cannot have deadlocks between Collections and Documents. - Unless nessecary, it is recommened to leave this at its default value. + Unless necessary, it is recommended to leave this at its default value. @@ -573,8 +573,16 @@ be plugged into the indexing pipeline. The modules section lists and configures the indexes that will be available to the database: + Since eXist-db 7.0, all bundled index modules (Lucene full-text, + n-gram, range, and sort) are auto-discovered at startup via the Java + ServiceLoader mechanism. You no longer need an explicit + module entry in conf.xml to activate them. An explicit + entry takes precedence over SPI auto-discovery for the same id. + Add enabled="no" to an entry to suppress a bundled index without + removing the element from conf.xml. Third-party index modules still + require an explicit entry. The only common attributes for each module element are - class and id. The other attributes, as + class and id. The other attributes, as well as any nested elements, are specific to the index implementation. Detailed information is available in the article on Configuring Database Indexes. @@ -682,7 +690,7 @@ If the job is written in XQuery (not suitable for system jobs) this should be a path to the XQuery stored in the database, e.g. - /db/myCollection/myJob.xql. XQuery job's will be + /db/myCollection/myJob.xql. XQuery jobs will be launched under the guest account initially. The running XQuery may switch permissions through calls to xmldb:login(). @@ -728,6 +736,16 @@ indefinitely. + + enabled + + Set to no to keep the job entry in + conf.xml without activating it. The job will not be + registered with the scheduler at startup. Useful for preparing a + job definition or temporarily disabling a job during testing or + deployment transitions. Defaults to yes. + + Every job can take additional parameters, which are passed as name/value pairs. @@ -792,7 +810,7 @@ indent The serializer defaults to pretty-print the resulting XML source - code. Setingt this option to no disables + code. Setting this option to no disables pretty-printing. @@ -833,8 +851,8 @@ document XML Validation. The grammar-cache (added in eXist-db v7.0) configures the properties of the XML parser grammar cache. The cache contains - compiled versions of XSD and DTD grammar files that are generated by the XML parser when the XML parser is set in validating mode. - Re-use of these compiled versions can accellerate the validation process significantly. + compiled versions of XSD and DTD grammar files that are generated by the XML parser when the XML parser is set to validating mode. + Re-use of these compiled versions can accelerate the validation process significantly. The following items can be configured: @@ -849,7 +867,7 @@ Specifies the maximum number of entries the cache may contain. The following values can be used: - 0: grammars will be evicted immediately after being loaded into the cache. + 0: grammars will be evicted immediately after being loaded into the cache. -1: no maximum for the number of cached grammars. @@ -863,7 +881,7 @@ expire The time (in seconds) after which an unused grammar should expire and be removed - from the grammar pool. For value -1 grammars will not be removed over time. + from the grammar pool. For value -1, grammars will not be removed over time. @@ -997,7 +1015,7 @@ - enable-query-rewritingo + enable-query-rewriting the query engine can often achieve considerable performance improvements by rewriting an XQuery expression into a more efficient @@ -1012,7 +1030,7 @@ backwardCompatible - |Set to yes to enable XPath 1.0 backwards + Set to yes to enable XPath 1.0 backwards compatibility. The setting mainly effects automatic type conversions, which were less strict in XPath 1.0 than in later versions. @@ -1026,7 +1044,18 @@ This section lists the XQuery extension modules which will be known to the query engine. The modules in this list can be imported into a query without - specifying a location. For example: + specifying a location. + + Since eXist-db 7.0, all bundled XQuery extension modules are auto-discovered + at startup via the Java ServiceLoader mechanism — no + module entry is required to activate them. An explicit entry in + builtin-modules takes precedence over SPI for the same namespace URI, + allowing you to override the implementing class or supply module parameters. + Add enabled="no" to suppress a bundled module without removing it + from the configuration: + + Third-party modules that are not bundled with eXist-db still require an + explicit entry, for example: This establishes a static mapping between the module URI for the file module and the Java class which implements it. When using that module, it is sufficient @@ -1055,6 +1084,11 @@ configuration can still reference models by id (for example in a vector-field element’s model attribute). + The entire registry can be disabled without removing it from + conf.xml by setting enabled="no" on the + vector-models element. When disabled, the vector embedding module + falls back to resolving models by explicit path only. + Registering embedding models in conf.xml @@ -1114,11 +1148,9 @@ <tag>rpc-server</tag>/<tag>content-file</tag> element - This element configures the maximum amount of data in bytes held in the heap space before - the data is moved to the temporary disk space. While using memory improves the performance - and not triggering eventually existing virus scanners it comes with the cost of heap size usage. - The location, size and other parameters for this file can be set using the - content-file element. + This element configures the threshold at which data held in heap memory is spilled to + temporary disk space. Keeping data in memory improves performance and avoids triggering + antivirus scanners, but increases heap usage. content-file Attributes: @@ -1134,13 +1166,11 @@ <tag>rpc-server</tag>/<tag>content-file-pool</tag> element - This element configures the pool used to hold the content file entries in order to reuse the once - initialized instances to further improve the performance and to limit the amount of used heap size. - To calculate the maximum size of used memory the maximum in memory size of a single content file has - to be multiplied by the size of the pool. - The location, size and other parameters for this file can be set using the - content-file-pool element. - content-file Attributes: + This element configures the pool of reusable content-file instances. Reusing + instances improves performance and caps heap consumption. To estimate the maximum memory + footprint, multiply the max-in-memory-size value by the pool + size. + content-file-pool Attributes: size diff --git a/src/main/xar-resources/data/configuration/listings/listing-15.xml b/src/main/xar-resources/data/configuration/listings/listing-15.xml new file mode 100644 index 00000000..ca98d024 --- /dev/null +++ b/src/main/xar-resources/data/configuration/listings/listing-15.xml @@ -0,0 +1 @@ + diff --git a/src/main/xar-resources/data/configuration/listings/listing-5.xml b/src/main/xar-resources/data/configuration/listings/listing-5.xml index 621cb521..eb164d0e 100644 --- a/src/main/xar-resources/data/configuration/listings/listing-5.xml +++ b/src/main/xar-resources/data/configuration/listings/listing-5.xml @@ -1,4 +1,7 @@ - - + + + + + diff --git a/src/main/xar-resources/data/devguide_indexes/devguide_indexes.xml b/src/main/xar-resources/data/devguide_indexes/devguide_indexes.xml index 59b26f2b..2046c95a 100644 --- a/src/main/xar-resources/data/devguide_indexes/devguide_indexes.xml +++ b/src/main/xar-resources/data/devguide_indexes/devguide_indexes.xml @@ -5,7 +5,7 @@ xmlns:xlink="http://www.w3.org/1999/xlink"> Developer's Guide to Modularized Indexes - 2Q19 + 3Q26 java-development indexes @@ -359,6 +359,54 @@ + + Registering an Index via ServiceLoader (SPI) + + Since eXist-db 7.0, index modules can be auto-discovered at startup through the + Java ServiceLoader mechanism without requiring an explicit + module entry in conf.xml. This is the recommended + registration mechanism for bundled and reusable third-party indexes. + + To register an index via SPI: + + + + Implement the org.exist.indexing.IndexFactory + interface. It has a single method: + public interface IndexFactory { + AbstractIndex create(BrokerPool pool, Path dataDir, Element config) + throws DatabaseConfigurationException; + String getId(); +} + getId() must return the same stable identifier as + AbstractIndex.getIndexId() in the index it creates. + create() constructs and configures the index. + + + Register the factory in the JAR's service descriptor. Create the file + META-INF/services/org.exist.indexing.IndexFactory + containing the fully-qualified class name of your factory, one per line: + com.example.myindex.MyIndexFactory + + + At startup, IndexManager scans all JARs on the + classpath for IndexFactory providers and calls + create() for each one whose + id has not been mentioned in conf.xml + (either as an active entry or as enabled="no"). The scan + runs once inside the Configuration constructor — + there is no per-query overhead. + + + + An explicit module entry in conf.xml for the same + id always takes precedence over SPI discovery. To suppress a + SPI-registered index without removing its JAR, add a + module entry with enabled="no". + + + + org.exist.indexing.IndexWorker diff --git a/src/main/xar-resources/data/extensions/extensions.xml b/src/main/xar-resources/data/extensions/extensions.xml index 4a9f0321..efd89c8c 100644 --- a/src/main/xar-resources/data/extensions/extensions.xml +++ b/src/main/xar-resources/data/extensions/extensions.xml @@ -5,7 +5,7 @@ xmlns:xlink="http://www.w3.org/1999/xlink"> Extension Modules - 2Q21 + 3Q26 java-development @@ -36,15 +36,20 @@ may then be added to $EXIST_HOME/extensions/indexes/pom.xml. They will be compiled automatically by the standard build targets or as indicated above. - eXist-db must also be told which modules to load at startup, this is done in - conf.xml and the Class name and Namespace for each module is - listed below. + Since eXist-db 7.0, bundled extension modules are auto-discovered at startup via + the Java ServiceLoader mechanism — no conf.xml + entry is needed to activate them. Third-party modules (not bundled with eXist-db) + still require an explicit module entry in the + builtin-modules section of conf.xml. An explicit entry + always takes precedence over SPI auto-discovery for the same namespace URI; set + enabled="no" on an entry to suppress a bundled module. The class name + and namespace for each module are listed below. eXist-db will require a restart to load any new modules added. - Once a Module is configured and loaded eXist-db will display the module and its + Once a module is configured and loaded, eXist-db will display the module and its function definitions as part of the function - library page or through util:decribe-function(). + library page or through util:describe-function(). @@ -80,12 +85,12 @@ - The Cache module can be configured for with a bounded size, or time, or both. eXist-db cannot + The Cache module can be configured with a bounded size, or time, or both. eXist-db cannot know how much memory the data you will put in the cache will take, so it is up to you to manage your own memory needs here. A named cache can either be explicitly created by calling the cache:create XQuery - function, or it can be implicitly created lazing on the first operation performed on the cache. + function, or it can be implicitly created lazily on the first operation performed on the cache. Configuration of the Cache module is specified within the module definition of eXist-db's conf.xml file: @@ -116,7 +121,7 @@ - The SQL Module can be configured with additional which allow Connection Pooling of the SQL connections to be utilised. + The SQL Module can be configured with additional options that enable connection pooling for SQL connections. For this purpose the HikariCP connection pool implementation is used. Multiple connection pools can be configured and used from XQuery via the sql:get-connection-from-pool function. diff --git a/src/main/xar-resources/data/incompatibilities/incompatibilities.xml b/src/main/xar-resources/data/incompatibilities/incompatibilities.xml index 9ba186bd..46ebdd80 100644 --- a/src/main/xar-resources/data/incompatibilities/incompatibilities.xml +++ b/src/main/xar-resources/data/incompatibilities/incompatibilities.xml @@ -3,7 +3,7 @@
Known Issues when upgrading - 1Q20 + 3Q26 operations @@ -12,6 +12,36 @@ This article lists known incompatibilities when upgrading from an older version of eXist-db. + + + Upgrading to 7.0.0 + + + + conf.xml template trimmed — SPI auto-discovery for bundled + modules and indexes. Built-in XQuery modules and index modules are now + auto-discovered at startup via the Java ServiceLoader mechanism. + The default conf.xml template no longer lists them explicitly. + Installations upgrading with a hand-maintained conf.xml are + unaffected — explicit module entries are still honoured and take + precedence over SPI discovery for the same namespace URI or index + id. To suppress a bundled module or index without removing its + JAR, add enabled="no" to its entry. + + + New enabled attribute on + conf.xml elements. The trigger, + job, module, feature, parameter, and + property elements in conf.xml now accept an + enabled="yes|no" attribute (default yes). Setting + enabled="no" disables the entry at startup without removing it from + the file. Existing configuration files without the attribute continue to work + unchanged. + + + + + Upgrading to 5.0.0 diff --git a/src/main/xar-resources/data/indexing/indexing.xml b/src/main/xar-resources/data/indexing/indexing.xml index 8a5e8e2d..642960ec 100644 --- a/src/main/xar-resources/data/indexing/indexing.xml +++ b/src/main/xar-resources/data/indexing/indexing.xml @@ -3,7 +3,7 @@ schematypens="http://purl.oclc.org/dsdl/schematron"?>
Configuring Database Indexes - 2Q21 + 3Q26 application-development @@ -284,13 +284,20 @@ Enabling Index Modules - To activate an index plug-in, it needs to be added to the modules section - within the global configuration file conf.xml: + Since eXist-db 7.0, bundled index modules (Lucene, ngram, range, sort) are + auto-discovered at startup via the Java ServiceLoader mechanism. No + module entry in conf.xml is required to activate them. + Third-party index modules (not bundled with eXist-db) still require an explicit + module entry in the modules section of + conf.xml: - Every module element needs at least an id and - class attribute. The class attribute contains the name of the plug-in + An explicit module entry always takes precedence over SPI + auto-discovery for the same id. To suppress a bundled index without + removing its JAR from the classpath, add enabled="no" to the entry. + Every module element needs at least an id and + class attribute. The class attribute contains the name of the plug-in class, which has to be an implementation of org.exist.indexing.Index. All other attributes or nested configuration elements below the module diff --git a/src/main/xar-resources/data/scheduler/listings/listing-4.xml b/src/main/xar-resources/data/scheduler/listings/listing-4.xml new file mode 100644 index 00000000..c2d522a6 --- /dev/null +++ b/src/main/xar-resources/data/scheduler/listings/listing-4.xml @@ -0,0 +1,4 @@ + + + + diff --git a/src/main/xar-resources/data/scheduler/scheduler.xml b/src/main/xar-resources/data/scheduler/scheduler.xml index 56eb0da6..2b6dda65 100644 --- a/src/main/xar-resources/data/scheduler/scheduler.xml +++ b/src/main/xar-resources/data/scheduler/scheduler.xml @@ -5,7 +5,7 @@ xmlns:xlink="http://www.w3.org/1999/xlink"> Scheduler Module - 2Q19 + 3Q26 operations application-development @@ -28,11 +28,16 @@ Jobs may be statically scheduled by configuring them in the scheduler element of eXist-db's conf.xml configuration file. When eXist-db starts-up this configuration is read and - the jobs will be scheduled with the scheduler. The configuration file - contains (commented out) example jobs. An example of a scheduler + the jobs will be scheduled with the scheduler. An example of a scheduler entry: + + Add enabled="no" to a job element to keep the + entry in conf.xml without activating it at startup. This is + useful for preparing a job configuration in advance or disabling a job + temporarily without losing its parameters: + @@ -119,8 +124,8 @@ An XQuery job is a standard XQuery Main Module which is stored in the database. You configure the scheduling of the job by providing the database path to the XQuery, for example: /db/my-collection/my-job.xq. - XQuery job's are launched under the guest account. If - you wish to perform tasks as another user , either switch permissions by + XQuery jobs are launched under the guest account. If + you wish to perform tasks as another user, either switch permissions by calling xmldb:login() from within your job, or set the SetUid/SetGid bits on the XQuery file's permissions (see Security for more @@ -145,13 +150,13 @@ Job Schedule - Job's may be scheduled using one of two mechanisms, a simple mechanism for periodic + Jobs may be scheduled using one of two mechanisms: a simple mechanism for periodic execution, or a more complex mechanism which uses the Cron syntax to offer greater flexibility. Periodic Scheduling - For period scheduling enables you can specify for the job to run every + Periodic scheduling lets you specify that the job runs every n milliseconds. There are additional options to specify a delay before the first execution of the job, and to only repeat the execution of the job schedule a fixed number of times. diff --git a/src/main/xar-resources/data/xquery/xquery.xml b/src/main/xar-resources/data/xquery/xquery.xml index 0328f67b..1fa814e3 100644 --- a/src/main/xar-resources/data/xquery/xquery.xml +++ b/src/main/xar-resources/data/xquery/xquery.xml @@ -5,7 +5,7 @@ xmlns:xlink="http://www.w3.org/1999/xlink"> XQuery in eXist-db - 1Q26 + 3Q26 xquery @@ -270,9 +270,13 @@ Preloaded Modules Preloaded modules do not need to be explicitly imported or declared in the prolog - of queries. The builtin-modules element in conf.xml - lists the namespaces and the corresponding Java class that implements all modules to - be preloaded: + of queries. Since eXist-db 7.0, bundled modules are auto-discovered at startup via + the Java ServiceLoader mechanism — no builtin-modules + entry in conf.xml is required to activate them. + An explicit module entry in builtin-modules always takes + precedence over SPI auto-discovery for the same namespace URI. Third-party modules + still require an explicit entry. To suppress a bundled module without removing its + JAR, set enabled="no" on its module entry: