Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ import pekko.util.ByteString
* INTERNAL API: Use [[pekko.stream.scaladsl.JsonFraming]] instead.
*
* **Mutable** framing implementation that given any number of [[pekko.util.ByteString]] chunks, can emit JSON objects contained within them.
* Typically JSON objects are separated by new-lines or commas, however a top-level JSON Array can also be understood and chunked up
* into valid JSON objects by this framing implementation.
* Typically JSON objects are separated by newlines or commas. A top-level JSON array is supported as a container
* of objects, but top-level non-object values are not supported.
*
* Leading whitespace between elements will be trimmed.
*/
Expand Down Expand Up @@ -90,7 +90,8 @@ import pekko.util.ByteString

/**
* Attempt to locate next complete JSON object in buffered ByteString and returns `Some(it)` if found.
* May throw a [[pekko.stream.scaladsl.Framing.FramingException]] if the contained JSON is invalid or max object size is exceeded.
* May throw a [[pekko.stream.scaladsl.Framing.FramingException]] if content outside an object is not an array
* delimiter, comma, or whitespace, or if the max object size is exceeded.
*/
def poll(): Option[ByteString] =
try {
Expand All @@ -112,7 +113,7 @@ import pekko.util.ByteString
throw new FramingException(s"Invalid JSON encountered at position [$pos] of [${ByteString(buffer).utf8String}]")
}

/** @return true if an entire valid JSON object was found, false otherwise */
/** @return true if an entire JSON object was found, false otherwise */
private def seekObject(): Boolean = {
completedObject = false
val bufSize = buffer.length
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,16 @@ import org.apache.pekko
import pekko.NotUsed
import pekko.util.ByteString

/** Provides JSON framing operators that can separate valid JSON objects from incoming [[pekko.util.ByteString]] objects. */
/** Provides JSON framing operators that can separate JSON objects from incoming [[pekko.util.ByteString]] objects. */
object JsonFraming {

/**
* Returns a Flow that implements a "brace counting" based framing operator for emitting valid JSON chunks.
* Returns a Flow that implements a "brace counting" based framing operator for emitting JSON objects.
*
* This operator only frames JSON objects; it is not a general-purpose JSON parser and does not validate the
* object contents. A top-level JSON array is supported only when all its elements are objects. Other top-level JSON
* values, such as strings, numbers, booleans, or `null`, are not supported. JSON Lines or NDJSON input is supported
* when every record is an object. A leading UTF-8 byte-order mark (BOM) must be removed before framing.
*
* Typical examples of data that one may want to frame using this operator include:
*
Expand All @@ -36,10 +41,13 @@ object JsonFraming {
* {"id": 1}, {"id": 2}, [...], {"id": 999}
* }}}
*
* The framing works independently of formatting, i.e. it will still emit valid JSON elements even if two
* The framing works independently of formatting, i.e. it will still emit JSON objects even if two
* elements are separated by multiple newlines or other whitespace characters. And of course is insensitive
* (and does not impact the emitting frame) to the JSON object's internal formatting.
*
* For streaming nested JSON structures, see
* <a href="https://pekko.apache.org/docs/pekko-connectors/current/data-transformations/json.html">Apache Pekko Connectors JSON support</a>.
*
* @param maximumObjectLength The maximum length of allowed frames while decoding. If the maximum length is exceeded
* this Flow will fail the stream.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,21 @@ import pekko.stream.scaladsl.Framing.FramingException
import pekko.stream.stage.{ GraphStageLogic, InHandler, OutHandler }
import pekko.util.ByteString

/** Provides JSON framing operators that can separate valid JSON objects from incoming [[pekko.util.ByteString]] objects. */
/** Provides JSON framing operators that can separate JSON objects from incoming [[pekko.util.ByteString]] objects. */
object JsonFraming {

/** Thrown if upstream completes with a partial object in the buffer. */
class PartialObjectException(msg: String = "JSON stream completed with partial content in the buffer!")
extends FramingException(msg)

/**
* Returns a Flow that implements a "brace counting" based framing operator for emitting valid JSON chunks.
* It scans the incoming data stream for valid JSON objects and returns chunks of ByteStrings containing only those valid chunks.
* Returns a Flow that implements a "brace counting" based framing operator for emitting JSON objects.
* It scans the incoming data stream for JSON objects and returns chunks of ByteStrings containing those objects.
*
* This operator only frames JSON objects; it is not a general-purpose JSON parser and does not validate the
* object contents. A top-level JSON array is supported only when all its elements are objects. Other top-level JSON
* values, such as strings, numbers, booleans, or `null`, are not supported. JSON Lines or NDJSON input is supported
* when every record is an object. A leading UTF-8 byte-order mark (BOM) must be removed before framing.
*
* Typical examples of data that one may want to frame using this operator include:
*
Expand All @@ -48,10 +53,13 @@ object JsonFraming {
* {"id": 1}, {"id": 2}, [...], {"id": 999}
* }}}
*
* The framing works independently of formatting, i.e. it will still emit valid JSON elements even if two
* The framing works independently of formatting, i.e. it will still emit JSON objects even if two
* elements are separated by multiple newlines or other whitespace characters. And of course is insensitive
* (and does not impact the emitting frame) to the JSON object's internal formatting.
*
* For streaming nested JSON structures, see
* <a href="https://pekko.apache.org/docs/pekko-connectors/current/data-transformations/json.html">Apache Pekko Connectors JSON support</a>.
*
* If the stream completes while mid-object, the stage will fail with a [[PartialObjectException]].
*
* @param maximumObjectLength The maximum length of allowed frames while decoding. If the maximum length is exceeded
Expand Down