Describe the bug
A clear and concise description of what the bug is.
Screenshots or console output
If applicable, add screenshots to help explain your problem. When doing so., please ensure you have the first command that triggered the trace and/or the command that started up your build included, otherwise it is difficult to diagnose.
Branch / Deployment used
Let us know if it's the master branch, or the stable branch indicated in the readme, as well as if you're running it locally, in the cloud, via the docker compose (cpu or gpu), or direct docker run commands. Please include the exact commands used to run in the latter cases.
Operating System
Include the platform, version numbers of your docker, etc. Whether its GPU (Nvidia or other) or CPU, Mac, Linux, Windows, etc.
Additional context
Add any other context about the problem here.
During high concurrency, I use the fastapi interface with workers=8 and run python -m uvicorn api.src.main:app --host 0.0.0.0 --port 7081 --workers 8. Frequent interruptions during processing can cause worker crashes. When I interrupt a request mid-process, there's no output showing "Yielding final chunk Split completed in." Is this a leak causing the crash?
Describe the bug
A clear and concise description of what the bug is.
Screenshots or console output
If applicable, add screenshots to help explain your problem. When doing so., please ensure you have the first command that triggered the trace and/or the command that started up your build included, otherwise it is difficult to diagnose.
Branch / Deployment used
Let us know if it's the master branch, or the stable branch indicated in the readme, as well as if you're running it locally, in the cloud, via the docker compose (cpu or gpu), or direct docker run commands. Please include the exact commands used to run in the latter cases.
Operating System
Include the platform, version numbers of your docker, etc. Whether its GPU (Nvidia or other) or CPU, Mac, Linux, Windows, etc.
Additional context
Add any other context about the problem here.
During high concurrency, I use the fastapi interface with workers=8 and run python -m uvicorn api.src.main:app --host 0.0.0.0 --port 7081 --workers 8. Frequent interruptions during processing can cause worker crashes. When I interrupt a request mid-process, there's no output showing "Yielding final chunk Split completed in." Is this a leak causing the crash?