[bugfix] Parse multipart/form-data for all HTTP methods, not only POST - #6581
Conversation
HttpRequestWrapper only recognized a request as multipart/form-data when the method was POST, so request:is-multipart-content() returned false and the form data was never parsed for PUT (or PATCH) uploads. Separately, XQueryURLRewrite built its request wrapper with parseMultipart=false, so on the controller and RESTXQ path the uploaded file parts were never exposed for any method — only Jetty's getParameterMap() form fields were, and those omit file parts (and, for non-POST methods, omit them entirely). Together these meant a multipart PUT upload was unusable: the file part was silently dropped and the request stream consumed, so it could not be recovered downstream. Recognize a multipart body by its content type regardless of the HTTP method, and let XQueryURLRewrite parse the multipart content, so controllers and RESTXQ resource functions can read uploaded files via the request: module for every method. Closes eXist-db#6580 Relates to eXist-db#6578 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
line-o
left a comment
There was a problem hiding this comment.
I built this branch and tested it live against both the direct REST API path (RESTServer) and the controller/RESTXQ path (XQueryURLRewrite). The PUT/PATCH fix itself works as described, but it has a side effect: GET requests are now treated as multipart too.
Before this PR, a GET request carrying a multipart/form-data body already leaked non-file form fields (a pre-existing, unrelated quirk of the servlet container's parameter map), but request:is-multipart-content() reported false and uploaded files were silently dropped. After this PR, GET is handled identically to POST/PUT: is-multipart-content() returns true and uploaded files become fully visible via request:get-uploaded-file-name(), on both paths I tested.
GET is specified as a safe method with no defined semantics for a request body. RFC 9110 §9.3.1 ("GET") is explicit about this: a client SHOULD NOT generate content in a GET request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported — since intermediaries along the request chain are often unaware of such private agreements. (The predecessor spec, RFC 7231 §4.3.1, put it more bluntly: "A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.")
Requesting changes: rather than dropping the method check entirely, could isMultipartContent use an explicit allow-list of methods that legitimately carry a body (POST, PUT, PATCH), or otherwise explicitly exclude GET (and HEAD/DELETE)? That keeps the PUT/PATCH fix intact without also making GET a vector for parsed, attacker-suppliable multipart content (including file uploads) on every RESTXQ/controller function bound to it — which is a meaningful change of trust boundary for any handler that wasn't written expecting a body on GET.
See inline comment on the specific line.
| // https://github.com/eXist-db/exist/issues/6578 | ||
| @Nullable final String contentType = servletRequest.getContentType(); | ||
| isMultipartContent = "POST".equalsIgnoreCase(servletRequest.getMethod()) && contentType != null && contentType.toLowerCase(Locale.ENGLISH).startsWith("multipart/"); | ||
| isMultipartContent = contentType != null && contentType.toLowerCase(Locale.ENGLISH).startsWith("multipart/"); |
There was a problem hiding this comment.
This drops the POST-only gate entirely instead of replacing it with an explicit allow-list. Confirmed live: a GET request with a multipart/form-data body now gets is-multipart-content()=true and its uploaded files fully parsed and exposed via request:get-uploaded-file-name(), on both the RESTServer and XQueryURLRewrite paths — previously GET never exposed uploaded files at all.
Per RFC 9110 §9.3.1, a GET request body has no defined semantics and a client SHOULD NOT send one absent prior agreement with the origin server. Suggest scoping this to an explicit set of methods that carry a body (POST, PUT, PATCH) rather than every method, e.g.:
private static final Set<String> MULTIPART_METHODS = Set.of("POST", "PUT", "PATCH");
...
isMultipartContent = MULTIPART_METHODS.contains(servletRequest.getMethod().toUpperCase(Locale.ENGLISH))
&& contentType != null && contentType.toLowerCase(Locale.ENGLISH).startsWith("multipart/");There was a problem hiding this comment.
@line-o Great, thanks. Work on these changes is in progress.
There was a problem hiding this comment.
[This response was prompted by Joe, drafted by Claude Code, and reviewed by Joe.]
Thanks @line-o — good catch, and agreed. Treating GET as multipart is a real change of trust boundary, and dropping the method gate entirely was too broad. Fixed in d836b11 with your suggested approach: an explicit allow-list of body-carrying methods.
private static final Set<String> MULTIPART_METHODS = Set.of("POST", "PUT", "PATCH");
...
isMultipartContent = MULTIPART_METHODS.contains(servletRequest.getMethod().toUpperCase(Locale.ENGLISH))
&& contentType != null && contentType.toLowerCase(Locale.ENGLISH).startsWith("multipart/");So GET/HEAD/DELETE are excluded while the PUT/PATCH fix stays intact. Added a test alongside the existing one: a GET carrying a multipart/form-data body now reports is-multipart-content()=false and exposes no uploaded file, while POST/PUT/PATCH all parse it.
You're right that the pre-existing leak of non-file form fields into the container's parameter map on GET is separate — this change doesn't alter it, and it seemed out of scope here; happy to file it as its own issue if you think it's worth chasing.
There was a problem hiding this comment.
hmm, why do we call this MULTIPART_METHODS rather then making it clear that these methods can carry a body.
How about METHOD_CAN_CARY_A_BODY and I would also propose to match against lower-case strings.
Do we want to make this an enum, @reinhapa?
There was a problem hiding this comment.
[This response was prompted by Joe, drafted by Claude Code, and reviewed by Joe.]
Good points, @line-o.
Name — agreed, it should say why these methods qualify rather than what we use them for. I'd suggest METHODS_WITH_REQUEST_BODY (clear, and avoids the singular/plural mismatch). Happy to use whatever wording you prefer.
Case — the current check is already case-insensitive (getMethod().toUpperCase(...) against an uppercase set). I'd lean towards keeping uppercase: HTTP method names are canonically uppercase, and the existing method checks in this codebase compare against uppercase literals (e.g. PathFilter does "PUT".equalsIgnoreCase(...)), so an uppercase set reads most obviously as "these HTTP methods". But it's a one-line flip either way — glad to match lower-case if you'd rather.
Enum — over to @reinhapa, since it's really your call. For context: there's no existing HTTP-method enum in the codebase today; method dispatch is done with string comparisons throughout, so this would be net-new infrastructure for a three-element set. I don't feel strongly and am happy to add one if you'd prefer it.
I'll hold the rename until we've settled on the name and the enum question, so it lands in one clean change rather than renaming twice.
There was a problem hiding this comment.
- Name:
METHODS_WITH_REQUEST_BODY👍🏼 - Case: Agreed, uppercase it is
- Enum: let's do this in a followup (if at all)
Address review feedback: recognize a multipart/form-data body only for POST, PUT and PATCH rather than for every method. Dropping the method gate entirely also let GET (and HEAD/DELETE) parse and expose an uploaded multipart body; but GET has no defined semantics for a request body (RFC 9110 §9.3.1), and treating it as multipart would expose attacker-suppliable file uploads to handlers not written to expect a body. Adds a test asserting GET is not parsed as multipart while POST/PUT/PATCH are. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The set gates multipart parsing on the request method, but its members are simply the methods for which a request body is defined (POST, PUT, PATCH). Naming it for that broader property, rather than for multipart specifically, reads more clearly at the use site and matches how the allow-list is reasoned about in review. No behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
[This response was prompted by Joe, drafted by Claude Code, and reviewed by Joe.] Done — renamed to |
[This PR was prompted by Joe, drafted by Claude Code, and reviewed by Joe.]
Summary
Multipart/form-data uploads only worked for
POST. A multipartPUT(orPATCH) was effectively unusable:request:is-multipart-content()returnedfalse, the uploaded file part was silently dropped, and the request stream was consumed so nothing downstream could recover it. This fixes multipart parsing so it works for every HTTP method that carries a multipart body, and so uploaded files are exposed to controllers and RESTXQ resource functions (via therequest:module) regardless of method.Closes #6580. Substantially addresses #6578 (see Scope below).
Root cause — two coupled defects
HttpRequestWrappergated multipart detection onPOST.isMultipartContentwas only evertruewhengetMethod()wasPOST, so forPUT/PATCHthe request was never treated as multipart:request:is-multipart-content()returnedfalseandgetParts()was never called.XQueryURLRewritebuilt its request wrapper withparseMultipart=false. On the controller and RESTXQ path — the path aPUTcan actually reach, since RESTPUTstores rather than executes — the wrapper fell back to the servlet container'sgetParameterMap(). That exposes non-file form fields but omits file parts entirely, so an uploaded file was never visible there for any method, and reading the parameters consumed the body so it could not be re-parsed by hand either.The two are coupled: fixing only (1) would have flipped
request:is-multipart-content()totruewithout making the file available on that path — which would move consumers onto a code path that returns an empty file. Both are fixed together.What changed
HttpRequestWrapper— recognize a multipart body by itsContent-Type(multipart/…) regardless of the HTTP method, instead of requiringPOST.XQueryURLRewrite— construct the request wrapper withparseMultipart=true, so multipart content (including file parts) is parsed viagetParts()and exposed to controllers and RESTXQ resource functions through therequest:module.Reproduction / test
MultipartMethodControllerTeststores acontroller.xqlthat reports, via therequest:module, what it sees of an identical multipart body sent asPOSTand asPUT— the same path a third-party router (e.g. the roaster library, eeditiones/roaster#107) takes. Before the fix,PUTreportedis-multipart-content()=falseand exposed neither the file field nor the uploaded file; the controller path never exposed the uploaded file for either method. After the fix, both methods see the form field, the uploaded file, andis-multipart-content()=true.Behavior before vs. after, from that test (controller path):
is-multipart-content()pathfileScope
POSTmethods / on the controller and RESTXQ path) is fixed: files are now parsed and available through therequest:module for every method, so the "parse the raw body yourself" workaround is no longer needed. One aspect it deliberately does not add is a way to read an individual part's own headers (e.g. a file part'sContent-Type) through therequest:module — that would be a small additive API and is better discussed separately. Marked "Relates to" rather than "Closes" [BUG] auto-parsing of multipart-formdata is incomplete #6578 for that reason; happy to close it if reviewers consider the remaining item out of scope.Test plan
MultipartMethodControllerTest(new) — multipartPOSTandPUTthrough acontroller.xql, asserting the field, the file, andis-multipart-content().org.exist.http.urlrewrite.*andorg.exist.xquery.functions.request.*— full packages green (incl.ControllerTest,GetParameterTest,PatchTest,GetData*Test).RESTServiceTestgreen.