Fix label leakage in BNCI2025_001, the montages gh-700 never shipped, and the NEMAR prefetch guard (#1161)

- BNCI2025_001: type the 7 kinematic channels (target position, hand velocity) as misc. read_raw_eeglab typed them eeg, so every paradigm fed the labels to the classifier as features. EOG named explicitly so the declared channel_types is true by construction.
- Rodrigues2017, Cattan2019_PHMD and Cattan2019_VR: the montage gh-700 announced and never implemented. Fc5/Fc6 were the only two of their 16 names standard_1020 cannot resolve.
- Cattan2019_PHMD interval [0, 1] -> [0, 60]; its own metadata records 60 s blocks.
- NEMAR prefetch settles presence per subject, offline, instead of trusting any non-empty store.
- moabb.analysis no longer imports plotting eagerly, so importing a dataset stops restyling the caller's global matplotlib rcParams.
- The monthly download job pointed at a file that does not exist and collected nothing.

Author: bruAristimunha

.github/workflows/download-test.yml

@@ -38,5 +38,4 @@ jobs:
       - name: Run download tests
         run: |
           echo "Running tests"
-          cd moabb/tests
-          pytest -vv -s --tb=long --durations=0 --maxfail=5 --log-cli-level=INFO --dl-data download.py
+          pytest -vv -s --tb=long --durations=0 --maxfail=5 --log-cli-level=INFO --dl-data moabb/tests/test_download.py

docs/source/whats_new.rst

@@ -18,8 +18,8 @@ What's new
 
 .. _current:
 
-Version 1.7  (Source - GitHub)
--------------------------------
+Version 1.6.1  (Source - GitHub)
+---------------------------------
 
 Enhancements
 ~~~~~~~~~~~~
@@ -35,7 +35,13 @@ Requirements
 
 Bugs
 ~~~~
-- None yet.
+- Point the monthly download job at ``moabb/tests/test_download.py``; it ran ``download.py``, a file that does not exist, so it collected nothing and has been failing every month while no ``@pytest.mark.download`` test executed anywhere in CI (by `Bruno Aristimunha`_).
+- Stop importing moabb from restyling the caller's matplotlib. ``moabb.analysis.plotting`` applies a seaborn theme to the global ``rcParams`` when it is imported, and ``moabb/analysis/__init__.py`` imported it eagerly -- so ``import moabb.datasets`` alone changed ``font.family``, ``axes.grid`` and ``axes.spines.*`` for every figure the caller drew afterwards, values matplotlib reads at axes creation and a caller therefore cannot undo. The path was indirect: :mod:`moabb.datasets.bids_interface` imports ``moabb.analysis.results`` for ``get_digest``, which runs that ``__init__``. The plotting imports are now deferred to the functions that use them and to a module-level ``__getattr__`` for the three re-exported plotting helpers, so every public name still resolves and an explicit ``import moabb.analysis.plotting`` still applies the theme, leaving MOABB's own figures unchanged (by `Bruno Aristimunha`_).
+- Type the seven non-EEG channels of :class:`moabb.datasets.BNCI2025_001` as ``misc``. ``read_raw_eeglab`` types anything it does not recognise as ``eeg``, so ``x``/``y``/``vx``/``vy``/``validity``/``targetPosX``/``targetPoxY`` -- the hand kinematics and the target position of the reaching task -- were picked as EEG by every paradigm and fed to classifiers as features, leaking the labels they encode. The declared ``n_channels`` (67) also disagreed with the dataset's own 60-electrode montage; it is now 71 with ``channel_types={"eeg": 60, "eog": 4, "misc": 7}``. Accuracies on this dataset will fall, which is the point (by `Bruno Aristimunha`_).
+- Give :class:`moabb.datasets.Rodrigues2017` the montage :gh:`700` announced but never shipped, the same omission as :class:`moabb.datasets.Cattan2019_PHMD` below: both share the 16-electrode setup, both spelled ``Fc5``/``Fc6`` -- the only two names ``standard_1020`` cannot resolve -- and neither loader called ``set_montage``. Its ``METADATA`` also declared ``standard_1010``, which is not a montage MNE can build (by `Bruno Aristimunha`_).
+- Give :class:`moabb.datasets.Cattan2019_PHMD` the montage :gh:`700` announced but never shipped: that PR fixed only the unit scaling, leaving the loader with no ``set_montage`` call at all. Its channel list also spelled the two frontal-central electrodes ``Fc5``/``Fc6``, the only two of its sixteen names that ``standard_1020`` cannot resolve; they are now ``FC5``/``FC6`` (by `Bruno Aristimunha`_).
+- Correct :class:`moabb.datasets.Cattan2019_PHMD` ``interval`` from ``[0, 1]`` to ``[0, 60]``. Each marker starts a one-minute relaxation block -- as the dataset's own ``block_duration_s=60.0`` records -- but ``SetRawAnnotations`` derives annotation durations from ``interval``, so every block was annotated as lasting one second. ``interval[0]`` is unchanged, so onsets do not move (by `Bruno Aristimunha`_).
+- Prefetch the NEMAR sourcedata store per subject rather than per dataset. The guard returned as soon as the store held anything, and the deposit's provenance manifest lands inside it, so a store holding only a manifest counted as complete. Presence is now settled per subject from the cached manifest, at no network cost: :func:`nemar.download` walks index, version, metadata and manifest before it consults ``trust_existing``, so a call with nothing to do is still four round-trips. Note that deposits whose manifest predates the ``subject`` field -- which is every deposit today -- are fetched as a whole tree, so for those the older whole-store rule was already right and is kept; the per-subject check matters once manifests record subjects, and today for a store left partial by an interrupted fetch (by `Bruno Aristimunha`_).
 
 Code health
 ~~~~~~~~~~~

moabb/__init__.py

@@ -1,5 +1,5 @@
 # flake8: noqa
-__version__ = "1.7.0dev0"
+__version__ = "1.6.1dev0"
 
 from .benchmark import benchmark
 from .utils import (

moabb/analysis/__init__.py

@@ -1,11 +1,11 @@
+import importlib
 import logging
 import os
 import platform
 from datetime import datetime
 
 from mne.utils import _open_lock
 
-from moabb.analysis import plotting as plt
 from moabb.analysis.chance_level import (  # noqa: F401
     adjusted_chance_level,
     chance_by_chance,
@@ -14,11 +14,6 @@
     compute_dataset_statistics,
     find_significant_differences,
 )
-from moabb.analysis.plotting import (  # noqa: F401
-    codecarbon_plot,
-    distribution_plot,
-    emissions_summary,
-)
 from moabb.analysis.results import Results  # noqa: F401
 from moabb.analysis.style import MOABB_PALETTE, apply_moabb_style  # noqa: F401
 from moabb.analysis.timeline import (  # noqa: F401
@@ -73,6 +68,8 @@ def analyze(results, out_path, name="analysis", plot=False):
     else:
         analysis_path = os.path.join(out_path, name)
 
+    from moabb.analysis import plotting as plt
+
     unique_ids = [plt._simplify_names(x) for x in results.pipeline.unique()]
     simplify = True
     if len(unique_ids) != len(set(unique_ids)):
@@ -97,3 +94,27 @@ def analyze(results, out_path, name="analysis", plot=False):
         fig.savefig(os.path.join(analysis_path, "scores.pdf"))
         fig = plt.summary_plot(P, T, simplify=simplify)
         fig.savefig(os.path.join(analysis_path, "ordering.pdf"))
+
+
+# ``plotting`` applies a seaborn theme to the global matplotlib rcParams at
+# import time. Re-exporting its names lazily keeps that off the path of anyone
+# who reaches this package for something else -- ``moabb.datasets`` imports
+# ``moabb.analysis.results`` for ``get_digest``, so an eager import here
+# restyled the figures of every caller who merely imported a dataset.
+_PLOTTING_EXPORTS = ("codecarbon_plot", "distribution_plot", "emissions_summary")
+
+
+def __getattr__(name):
+    # ``plotting`` itself is included: the eager import used to bind it as an
+    # attribute of this package, so ``moabb.analysis.plotting`` resolved without
+    # the caller importing the submodule. Dropping that would be a break.
+    if name == "plotting" or name in _PLOTTING_EXPORTS:
+        # import_module, not ``from . import plotting``: the latter falls back to
+        # getattr on this package, which re-enters __getattr__ forever.
+        plotting = importlib.import_module("moabb.analysis.plotting")
+        return plotting if name == "plotting" else getattr(plotting, name)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__():
+    return sorted(set(globals()) | set(_PLOTTING_EXPORTS) | {"plotting"})

moabb/datasets/alphawaves.py

@@ -91,16 +91,16 @@ class Rodrigues2017(BaseDataset):
             sampling_rate=512.0,
             n_channels=16,
             channel_types={"eeg": 16},
-            montage="standard_1010",
+            montage="standard_1020",
             hardware="g.tec g.USBamp",
             sensor_type="wet electrodes",
             reference="right earlobe",
             software="OpenViBE",
             filters="no digital filter",
             sensors=[
                 "Cz",
-                "Fc5",
-                "Fc6",
+                "FC5",
+                "FC6",
                 "Fp1",
                 "Fp2",
                 "Fz",
@@ -206,9 +206,9 @@ def _get_single_subject_data(self, subject):
         chnames = [
             "Fp1",
             "Fp2",
-            "Fc5",
+            "FC5",
             "Fz",
-            "Fc6",
+            "FC6",
             "T7",
             "Cz",
             "T8",
@@ -229,6 +229,7 @@ def _get_single_subject_data(self, subject):
             ch_names=chnames, sfreq=512, ch_types=chtypes, verbose=False
         )
         raw = mne.io.RawArray(data=X, info=info, verbose=False)
+        raw.set_montage("standard_1020")
 
         return {"0": {"0": raw}}
 

moabb/datasets/base.py

@@ -34,6 +34,7 @@
     active_sourcedata_store,
     nemar_dl,
     nemar_sourcedata_dl,
+    nemar_sourcedata_is_local,
     nemar_store,
 )
 from moabb.datasets.preprocessing import FixedPipeline, SetRawAnnotations
@@ -1106,17 +1107,19 @@ def _prefetch_nemar_sourcedata(self, subjects, verbose=None):
         ``"upstream"`` skips NEMAR entirely, ``"nemar"`` treats a failure as
         fatal rather than silently reaching the host the caller opted out
         of, and ``"auto"`` warns per subject and leaves that subject to the
-        dataset's own downloader. A non-empty store is trusted as-is and
-        costs no network at all -- refresh or extend it with
-        :meth:`download` (``force_update=True`` to refetch).
+        dataset's own downloader. Subjects already in the store cost no
+        network at all: their presence is settled from the cached manifest,
+        without contacting NEMAR. Refresh one with :meth:`download`
+        (``force_update=True``).
         """
         provider = get_download_provider()
         if self.nemar_id is None or provider == "upstream":
             return
-        store = self._sourcedata_store()
-        if store is not None and store.is_dir() and any(store.iterdir()):
-            return
+        # Resolved once: nemar_store() re-reads MNE's config file each call.
+        store_root = nemar_store(self.code, self.nemar_id)
         for subject in subjects:
+            if nemar_sourcedata_is_local(store_root, self._nemar_subject_ids(subject)):
+                continue
             try:
                 self.sourcedata_path(subject=subject, verbose=verbose)
             except NemarDownloadError as exc:
@@ -1130,6 +1133,22 @@ def _prefetch_nemar_sourcedata(self, subjects, verbose=None):
                     stacklevel=2,
                 )
 
+    def _nemar_subject_ids(self, subject):
+        """The identifiers a deposit may file this subject under.
+
+        Provenance manifests observed in the wild record raw MOABB ids, but a
+        dataset's ``nemar_subject_template`` documents how its deposit labels
+        subjects -- match both, exactly as :meth:`sourcedata_path` does.
+        """
+        label = self._nemar_subject(subject)
+        # Return the scalar untouched unless the deposit really files this
+        # subject under a second label: a None label would stringify to "None"
+        # and collide with manifest entries that carry no subject at all, and
+        # wrapping a lone id in a list would change what downstream callers see.
+        if label is None or str(label) == str(subject):
+            return subject
+        return [subject, label]
+
     def _sourcedata_store(self):
         """Local NEMAR sourcedata store to serve this dataset's loads from.
 
@@ -1184,9 +1203,7 @@ def sourcedata_path(self, subject=None, path=None, force_update=False, verbose=N
         # a dataset's nemar_subject_template documents how its deposit labels
         # subjects -- match both rather than betting on one convention.
         if subject is not None:
-            label = self._nemar_subject(subject)
-            if label is not None and str(label) != str(subject):
-                subject = [subject, label]
+            subject = self._nemar_subject_ids(subject)
         return nemar_sourcedata_dl(
             self.nemar_id,
             self.code,

moabb/datasets/bnci/bnci_2025.py

@@ -45,6 +45,25 @@
 # Base URL for the BNCI 2025-001 dataset (hosted at TU Graz)
 BNCI_2025_001_URL = "https://lampx.tugraz.at/~bci/database/001-2025/"
 
+# Channels the EEGLAB ``.set`` files carry alongside the EEG montage: hand
+# kinematics from the reaching task, the target position, and a validity flag.
+# ``read_raw_eeglab`` types anything it cannot recognise as ``eeg``, which would
+# feed the target position and hand velocity -- i.e. the labels -- to any
+# paradigm picking ``eeg``.
+_EOG_CHANNELS_001_2025 = ("EOGL1", "EOGL2", "EOGL3", "EOGR1")
+
+_NON_EEG_CHANNELS_001_2025 = (
+    "x",
+    "y",
+    "vx",
+    "vy",
+    "validity",
+    "targetPosX",
+    "targetPoxY",  # sic: the published files misspell targetPosY
+    "targetPosY",  # ...accept the corrected spelling too
+)
+
+
 # Event code mapping for 001-2025 dataset
 # Format: XYZ where X=speed (1=slow, 2=fast), Y=distance (1=near, 2=far), Z=direction (1-4)
 # Direction codes: 1=up, 2=down, 3=left, 4=right
@@ -156,7 +175,18 @@ def _load_data_001_2025(
         return [str(set_file)]
 
     # Load the EEGLAB file
-    raw = mne.io.read_raw_eeglab(str(set_file), preload=True, verbose=verbose)
+    raw = mne.io.read_raw_eeglab(
+        str(set_file), eog=_EOG_CHANNELS_001_2025, preload=True, verbose=verbose
+    )
+
+    # Type the non-EEG channels before the montage, so they are neither placed
+    # on the scalp nor picked as EEG.
+    # set_channel_types raises on a name that is not in info, so filter first.
+    raw.set_channel_types(
+        {ch: "misc" for ch in _NON_EEG_CHANNELS_001_2025 if ch in raw.ch_names},
+        on_unit_change="ignore",
+        verbose=False,
+    )
 
     # Remap annotation descriptions from numeric codes to descriptive names
     # The data contains codes like "111", "112", etc. which we map to
@@ -266,9 +296,9 @@ class BNCI2025_001(BNCIBaseDataset):
     METADATA = DatasetMetadata(
         acquisition=AcquisitionMetadata(
             sampling_rate=500.0,
-            n_channels=67,
-            channel_types={"eeg": 67, "eog": 4},
-            montage="af7 af3 afz af4 af8 f7 f5 f3 f1 fz f2 f4 f6 f8 ft7 fc5 fc3 fc1 fcz fc2 fc4 fc6 ft8 t7 c5 c3 c1 cz c2 c4 c6 t8 tp7 cp5 cp3 cp1 cpz cp2 cp4 cp6 tp8 p7 p5 p3 p1 pz p2 p4 p6 p8 ppo1h ppo2h po7 po3 poz po4 po8 o1 oz o2",
+            n_channels=71,
+            channel_types={"eeg": 60, "eog": 4, "misc": 7},
+            montage="standard_1005",
             sensor_type="EEG",
             hardware="BrainAmp",
             reference="common average",
@@ -1047,7 +1077,7 @@ class BNCI2025_002(BNCIBaseDataset):
             sampling_rate=200.0,
             n_channels=60,
             channel_types={"eeg": 60, "eog": 4},
-            montage="af7 af3 afz af4 af8 f7 f5 f3 f1 fz f2 f4 f6 f8 ft7 fc5 fc3 fc1 fcz fc2 fc4 fc6 ft8 t7 c5 c3 c1 cz c2 c4 c6 t8 tp7 cp5 cp3 cp1 cpz cp2 cp4 cp6 tp8 p7 p5 p3 p1 pz p2 p4 p6 p8 ppo1h ppo2h po7 po3 poz po4 po8 o1 oz o2",
+            montage="standard_1005",
             sensor_type="EEG",
             hardware="actiCAP, Brain Products GmbH",
             reference="right mastoid",

moabb/datasets/braininvaders.py

@@ -182,9 +182,9 @@ def _bi_get_subject_data(ds, subject):  # noqa: C901
             chnames = [
                 "Fp1",
                 "Fp2",
-                "Fc5",
+                "FC5",
                 "Fz",
-                "Fc6",
+                "FC6",
                 "T7",
                 "Cz",
                 "T8",
@@ -209,10 +209,13 @@ def _bi_get_subject_data(ds, subject):  # noqa: C901
         info = mne.create_info(
             ch_names=chnames, sfreq=sfreq, ch_types=chtypes, verbose=False
         )
+        # Set it here rather than on each Raw: the Cattan2019-VR branch below
+        # builds many Raws from this Info, and used to leave them with no
+        # channel positions at all.
+        info.set_montage(make_standard_montage("standard_1020"), on_missing="ignore")
 
         if not ds.code == "Cattan2019-VR":
             raw = mne.io.RawArray(data=X, info=info, verbose=False)
-            raw.set_montage(make_standard_montage("standard_1020"))
 
             if ds.code == "BrainInvaders2012":
                 # get rid of the Fz channel (it is the ground)
@@ -941,7 +944,7 @@ class BI2014a(BaseDataset):
             sampling_rate=512.0,
             n_channels=16,
             channel_types={"eeg": 16},
-            montage="standard_1010",
+            montage="standard_1020",
             hardware="g.USBamp (g.tec, Schiedlberg, Austria)",
             sensor_type="dry electrodes",
             reference="right earlobe",
@@ -1143,7 +1146,7 @@ class BI2014b(BaseDataset):
             sampling_rate=512.0,
             n_channels=32,
             channel_types={"eeg": 32},
-            montage="standard_1010",
+            montage="standard_1020",
             hardware="g.USBamp (g.tec, Schiedlberg, Austria)",
             sensor_type="wet electrodes",
             reference="right earlobe",
@@ -1356,7 +1359,7 @@ class BI2015a(BaseDataset):
             sampling_rate=512.0,
             n_channels=32,
             channel_types={"eeg": 32},
-            montage="10-10",
+            montage="standard_1020",
             hardware="g.USBamp (g.tec, Schiedlberg, Austria)",
             sensor_type="wet electrodes",
             reference="right earlobe",
@@ -1557,7 +1560,7 @@ class BI2015b(BaseDataset):
             sampling_rate=512.0,
             n_channels=32,
             channel_types={"eeg": 32},
-            montage="10-10",
+            montage="standard_1020",
             hardware="g.USBamp (g.tec, Schiedlberg, Austria)",
             sensor_type="wet Silver/Silver Chloride electrodes",
             reference="right earlobe",
@@ -1769,7 +1772,7 @@ class Cattan2019_VR(BaseDataset):
             sampling_rate=512.0,
             n_channels=16,
             channel_types={"eeg": 16},
-            montage="10-10",
+            montage="standard_1020",
             hardware="g.USBamp (g.tec, Schiedlberg, Austria)",
             sensor_type="wet electrodes",
             reference="right earlobe",
@@ -1778,9 +1781,9 @@ class Cattan2019_VR(BaseDataset):
             sensors=[
                 "Fp1",
                 "Fp2",
-                "Fc5",
+                "FC5",
                 "Fz",
-                "Fc6",
+                "FC6",
                 "T7",
                 "Cz",
                 "T8",