JamForge M1 delivery#18
Conversation
|
T&C Assistant Lite bot All Contributors have signed the T&Cs. |
|
I have read and hereby sign the JAM Prize Terms and Conditions. |
semuelle
left a comment
semuelle
left a comment
There was a problem hiding this comment.
Hey @philoniare, please see the the reports and traces here. Please let me know once I should have another look.
@semuelle Thank you for your time, we'll resolve those traces asap. |
@semuelle We've pushed a new image, could you please pull the changes and try running again? Thank you |
|
Hey @philoniare, please see updated reports and traces here. |
@semuelle thank you, the traces have been very helpful, we've managed to fix an underlying discrepancy in the PVM memory mapping. We've pushed an updated image, should be able to get a lot further now. |
|
Hey @philoniare, thanks for the update. I'm currently failing to generate any new traces, mainly because the fuzzing stops with "broken pipe", "early eof" or some variation of IO error. I don't see any meaningful logs from the client, even in the |
@semuelle Yep, I've added a verbose flag that you could use, you can just update the I wish we could reproduce the issue, but it's passing all our trace files. Please let me know if there's anything else I could do, thanks. |
|
Hi @midegdugarova, could you please run our solution as well? |
|
Hi @philoniare, apologies for the lack of communication, there have been a lot of internal changes. I just started fuzzing your target this morning, will keep you updated, thanks! |
No worries, @CrackTheCode016. Great, thank you very much, looking forward to the results. |
|
@philoniare Apologies, I did not see the previous comments about running it verbose. I will do another run now with it. You may find a summary below, as well as the traces / some logs here. |
|
@CrackTheCode016 This is actually great, exactly what we needed. We've addressed one bug in our host call implementation and found a memory leak that grew unbounded during subsequent runs eventually running out of memory, becoming unresponsive. Both issues have been addressed in our new docker image. Could you please rerun the tests with the latest image at your convenience? Thanks |
|
Hey @philoniare, great to hear, I pushed up some more traces with the verbose logs. I will try with the new Docker image now and let you know (most likely in the morning) |
|
@CrackTheCode016 Thanks, you may run the image without verbose now as we've addressed the unbounded memory growth issue. |
|
@philoniare It did better than before; but did not reach 1 million, you may find the summary below (and traces in the branch): |
|
@CrackTheCode016 Thank you for the run, glad to see that there's no state diff errors. We're just running into some long running edge cases that we were not able to test before. We've added some performance improvements that hopefully address these. Could you please run it again whenever you can? If it's compliant with the rules of review process, it might be helpful to everyone to have one 100,000 step test case that they can run sequentially. |
|
@philoniare thank you, so currently, it is 100k x 10 runs running in parallel, so I think that accomplishes what you suggested (unless I am not understanding it correctly) |
|
@CrackTheCode016 All good, I just meant making a single sequential 100k step trace public so that implementers can test their solution to make sure they're meeting the 30 second timeout performance requirement. Looking forward to the next run results. |
|
@philoniare, the latest results are in the branch. You may find the summary below, it got farther but ran into a couple of issues.. I will also investigate that our environment is not blame: |
|
@CrackTheCode016 Environment looks great, and I think having a timeout is totally reasonable — it's a good way to catch issues with long sequential block imports. On our end, we found and fixed a bug where an operation ordering issue was causing the node to hit an invalid state and crash. We've also made a bunch of performance improvements to reduce GC pressure, which should help with the timeout issues. Could you run it again when you get a chance? Hoping this one's the charm! |
|
Hey @CrackTheCode016 — just wanted to bump this in case it got buried. No rush, but whenever you get a chance to re-run it, we'd love to see the results. Thanks! |
|
Hi @philoniare, apologies for the delay, a lot going on internally ;) Just run the fuzzer again with the new target. I also increased the memory in the Docker instance to ensure that it isn't a bottleneck. Thank you for your patience! |
|
@philoniare new results in the branch - unfortunately not passing the 1 million yet: |
|
Hi @CrackTheCode016, thank you for your patience and continued support. Apologies for the multiple run requests — we've been tracking down some tricky issues. In this update, we've fixed the state diff mismatches (a subtle shallow-copy bug during accumulation) and hardened our error handling to prevent crashes from broken pipes and unexpected exceptions. We're confident this should resolve the remaining failures. |
|
Hi @CrackTheCode016, with our latest changes, we're now passing the additional traces. Since we can't run the fuzzer locally, we've built our own stress testing harness and made some improvements. Could you please run it again whenever you can? |
|
We have passed all of the conformance fuzzer tests for M1: |
3 participants
Milestone Delivery Checklist