Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 76 additions & 9 deletions dependencies/stdout-log-generator/README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,86 @@
# Stdout Log Generator

Small tool for generating logs to stdout, used in the e2e tests of the telemetry-manager.
A small tool that continuously writes log lines to stdout. Used in the e2e tests of telemetry-manager to produce configurable log traffic inside Kubernetes workloads.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A small tool that continuously writes log lines to stdout. Used in the e2e tests of telemetry-manager to produce configurable log traffic inside Kubernetes workloads.
`stdout-log-generator` is a tool that continuously writes log lines to stdout. Telemetry Manager e2e tests use it to generate configurable log traffic inside Kubernetes workloads.


Available command line args:
The tool exposes a Prometheus metrics endpoint on port `2112` (`/metrics`) with two metrics:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The tool exposes a Prometheus metrics endpoint on port `2112` (`/metrics`) with two metrics:
The tool exposes a Prometheus metrics endpoint on port `2112` (`/metrics`) with the following metrics:

never mention the exact number when introducing a list, easier for maintenancce


- format
- bytes
- rate
- fields
- text
- `logs_generated_total` — total log lines written since start
- `logs_generated_rate` — actual throughput in logs/second since start

## Build locally
## Usage

```sh
stdout-log-generator [flags]
```

## Flags

| Flag | Short | Default | Description |
|------|-------|---------|-------------|
| `--format` | | `json` | Output format: `json` or `plaintext` |
| `--bytes` | `-b` | `2048` | Size of each log line in bytes |
| `--rate` | `-r` | `1` | Target log lines per second. `0` means unlimited |
| `--fields` | `-f` | | Additional key=value fields to include in every JSON log record (can be repeated or comma-separated). Ignored when `--format plaintext` is used |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

less ambiguity

Suggested change
| `--fields` | `-f` | | Additional key=value fields to include in every JSON log record (can be repeated or comma-separated). Ignored when `--format plaintext` is used |
| `--fields` | `-f` | | Additional key=value fields to include in every JSON log record. You can specify it multiple times or provide a comma-separated list. Ignored when `--format plaintext` is used |

| `--text` | `-t` | | Fixed text to write for every plaintext log line. Ignored if `--bytes` is set. Only relevant when `--format plaintext` is used |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `--text` | `-t` | | Fixed text to write for every plaintext log line. Ignored if `--bytes` is set. Only relevant when `--format plaintext` is used |
| `--text` | `-t` | | Fixed text to write for every plaintext log line. Applies only if you use `--format plaintext`. Ignored if you set `--bytes`. |


## Output formats

### JSON (default)

Each line is a JSON object with a `padding` field filled with random characters to reach the requested byte size, plus any custom fields added via `--fields`:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Each line is a JSON object with a `padding` field filled with random characters to reach the requested byte size, plus any custom fields added via `--fields`:
Each line is a JSON object with a `padding` field filled with random characters to reach the requested byte size, plus any custom fields added with `--fields`:


```json
{"padding":"aBcDeFgH...","environment":"prod","app":"myservice"}
```

The `padding` field is sized so that the total serialized JSON line matches `--bytes` exactly (including the trailing newline added by the JSON encoder).

If the combined size of the custom fields already exceeds `--bytes`, the tool exits with an error.

### Plaintext

Each line is either the fixed string given by `--text`, or a random string of `--bytes` length when `--text` is not set.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Each line is either the fixed string given by `--text`, or a random string of `--bytes` length when `--text` is not set.
The content of each line depends on whether you set the `--bytes` flag:
- With `--bytes`, each line is a random string of the specified length.
- Without `--bytes` but with `--text`, each line is the fixed string you provide.


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if neither flag (--bytes or --text) is set? Random string with default byte size of 2048.

## Rate limiting

The tool uses a token-bucket rate limiter (`golang.org/x/time/rate`). On Linux, the minimum achievable sleep granularity is approximately 2 ms. To compensate, the burst size is set to `2 × (rate / 1000)` so that the generator can emit multiple logs back-to-back before sleeping, keeping the long-run average close to the requested rate even at high throughputs.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Splitting the sentence for clarity

Suggested change
The tool uses a token-bucket rate limiter (`golang.org/x/time/rate`). On Linux, the minimum achievable sleep granularity is approximately 2 ms. To compensate, the burst size is set to `2 × (rate / 1000)` so that the generator can emit multiple logs back-to-back before sleeping, keeping the long-run average close to the requested rate even at high throughputs.
The tool uses a token-bucket rate limiter (`golang.org/x/time/rate`). On Linux, the minimum achievable sleep granularity is approximately 2 ms. To compensate for this, the burst size is set to `2 × (rate / 1000)`. This setting lets the generator emit multiple logs back-to-back before sleeping, keeping the long-run average close to the requested rate even at high throughput.


Setting `--rate 0` disables throttling entirely (write as fast as possible).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

purpose first, then instruction

Suggested change
Setting `--rate 0` disables throttling entirely (write as fast as possible).
To disable throttling and write logs as fast as possible, set `--rate 0`.


## Examples

Generate 100 JSON logs per second, each 1 KiB in size:

To build the image locally run:
```sh
stdout-log-generator --rate 100 --bytes 1024
```

Generate JSON logs with custom fields at the default rate:

```sh
stdout-log-generator --fields app=myservice,env=prod
```

Generate plaintext logs with a fixed message at 50 logs/second:

```sh
stdout-log-generator --format plaintext --text "hello world" --rate 50
```

Generate random plaintext logs of 512 bytes each, unlimited rate:

```sh
stdout-log-generator --format plaintext --bytes 512 --rate 0
```

## Build locally

```sh
docker build -t stdout-log-generator:local .
```

Run the local image:

```sh
docker run --rm stdout-log-generator:local --rate 10 --fields app=test
```
Loading