Skip to content

Include part fields in the KiCad category listing (one request per category instead of one per part) - #1489

Open
arlenarlenarlen wants to merge 1 commit into
Part-DB:masterfrom
arlenarlenarlen:kicad-inline-fields-in-category-listing
Open

Include part fields in the KiCad category listing (one request per category instead of one per part)#1489
arlenarlenarlen wants to merge 1 commit into
Part-DB:masterfrom
arlenarlenarlen:kicad-inline-fields-in-category-listing

Conversation

@arlenarlenarlen

@arlenarlenarlen arlenarlenarlen commented Aug 13, 2026

Copy link
Copy Markdown

What

getCategoryParts() returns only id/name/description, so KiCad's symbol chooser has to fetch every part individually when it enumerates the library. This inlines the record getKiCADPart() already builds, which lets KiCad skip those requests.

Why

SCH_IO_HTTP_LIB::EnumerateSymbolLib() back-fills each part from parts/{id}.json unless the category listing already carried a fields object — HTTP_LIB_CONNECTION::SelectAll() uses its presence to set detailsLoaded (KiCad ≥ 10.0.5, common/http_lib/http_lib_connection.cpp). Because the listing omits it, the skip never triggers.

On the 351-part / 14-category instance I measured, that's 366 requests per cold library open instead of 15. Most of each request is connection setup and round trip rather than server time — for that instance, ~105 ms TCP+TLS and ~50 ms RTT against ~60 ms of server work — so the request count dominates and a cold open takes tens of seconds.

Measured on the same database, same disk, cold cache both times:

before after
HTTP requests per cold open 366 15
per-part back-fill requests 351 0
server time, cold cache 2.39 s 0.21 s
server time, warm cache 1.96 s 0.09 s
total payload 331 KiB 302 KiB
slowest single listing 10 ms 45 ms (174 parts, 149 KiB)

The total payload gets smaller: 14 batched responses carry less overhead than 351 individual ones. The largest category costs 45 ms cold to assemble, and the result is cached with the same Part/Category/Footprint tags as before, so it is rebuilt exactly as often as it was.

How

By wiring up the $minimal argument getCategoryParts() already accepts and that KiCadApiController already exposes as ?minimal=1. Today that flag has no effect on the response — the cache closure captures only $category, so $minimal changes the cache key and nothing else. After this change:

  • ?minimal=1 → the light id/name/description shape (what the endpoint returns for everyone today),
  • default → the same record the per-part endpoint returns, which is a superset of it.

So it also gives users the control asked for in #1464, and keeps a cheap listing available for clients that only need names.

Tests

Three cases added to tests/Services/EDA/KiCadHelperTest.php, using the existing EDADataFixtures (category 1 has a KiCad symbol, so its parts are EDA-visible): the default listing carries fields plus symbolIdStr, ?minimal=1 still returns exactly id/name/description, and both listings describe the same parts in the same order.

Checks run locally

On PHP 8.4 + SQLite (a cell of the CI matrix), in a container with the extensions CI installs:

bin/phpunit (full suite) 1955 tests, 4852 assertions, 0 failures, 1 skipped
bin/phpunit --filter KiCadHelperTest 27 tests (24 existing + the 3 added), 69 assertions
composer phpstan (level 5) no errors
bin/console lint:yaml config --parse-tags 66 files OK
bin/console lint:twig templates --env=prod 176 files OK
bin/console doctrine:schema:validate --skip-sync mapping OK

Not run: the MySQL and PostgreSQL matrix cells, PHP 8.2/8.3/8.5, and Codecov upload. The change builds a PHP array and issues no new queries, so it should be database-agnostic, but I didn't verify that empirically.

Two notes on the tooling while I was in there, both pre-existing and unrelated to this PR:

  • vendor/bin/ecs can't run: ecs.php uses the deprecated return function (ECSConfig $ecsConfig) format, and the installed Easy Coding Standard passes an ECSConfig where the closure's signature demands a ContainerConfigurator, so it fails while building its container before analysing anything. Happy to send a separate PR converting ecs.php to the fluent API if that's wanted. I matched the surrounding style in KiCadHelper.php by hand instead.
  • CONTRIBUTING.md says phpstan runs at --level=2; the actual composer phpstan script and phpstan.dist.neon both say level 5. I used level 5.

Refs #1464 — this covers the listing side of that request (a light vs. full listing); it does not add per-parameter selection.

KiCad's symbol chooser makes one HTTP request per part when it enumerates an
HTTP library. SCH_IO_HTTP_LIB::EnumerateSymbolLib() back-fills every part from
parts/{id}.json unless the category listing already carried a "fields" object,
which HTTP_LIB_CONNECTION::SelectAll() uses to set detailsLoaded (KiCad >=
10.0.5, common/http_lib/http_lib_connection.cpp).

getCategoryParts() returns only id/name/description, so that skip never
triggers. On a 351-part instance that is 366 requests per cold library open
instead of 15. Since most of each request is connection setup and round trip
rather than server time, it dominates: measured on that instance, 2.4s of
server time and tens of seconds of wall clock, against 0.2s and a few seconds
with the fields inlined. The response also gets slightly SMALLER in total
(302 KiB vs 331 KiB), because 14 batched responses carry less overhead than
351 individual ones.

This wires up the $minimal argument that getCategoryParts() already accepts
and that KiCadApiController already exposes as ?minimal=1. It currently has no
effect on the response: the cache closure captures only $category, so the flag
changes the cache key and nothing else. Now ?minimal=1 keeps the light
id/name/description shape, and the default returns the same record the
per-part endpoint returns, which is a superset of it.

Refs Part-DB#1464

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant