What happened when I tried a model that doesn't generate text at all
I spend a lot of time trying different AI models, especially mode...
For further actions, you may consider blocking this person and/or reporting abuse
Hi @ayush7614
I was trying out laya on my local today and encountering this eror:
/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/site-packages/laya/router.py:260: RuntimeWarning: laya: this checkpoint ships invalid temperatures or values outside [0.5, 5]; using choice:11+=0.10058280825614929 -> 0.5. Treat confidence from the affected entries as uncalibrated.Yesterday it worked fine, only today I am facing this. I am using python 3.14 and I have tried removing and reinstalling laya. Also created multiple virual environments to ensure I didn't mess anything there that caused that. Still same error in the fresh venvs as well.
Can you please help or guide me here.
Thank You
Hi Siam, thanks for reporting this. I’ve actually seen the same warning while testing Laya locally. It’s related to the checkpoint’s temperature values: choice:11+ is currently shipped below the supported [0.5, 5] range, so Laya clamps it to 0.5 and warns that confidence for those affected entries is uncalibrated.
Since you’re using Python 3.14, I’d also recommend trying the same test with Python 3.13 in a fresh virtual environment, especially since the current package metadata lists 3.10–3.13 as its supported classifiers.
But importantly, the warning itself is not necessarily a failure — it’s a warning about confidence calibration. Let me know whether the model still produces predictions for you after this warning, and we can narrow down whether there’s a separate Python 3.14 issue.
Hi Ayush,
I have tried with a fresh virtual env with Python 3.10 & Python 3.12 still I had issues. As I was on mac this one seemed to have worked without any issues - github.com/mizorewww/laya-mlx
Still I am unsure was that a mac problem or a python version issue 🤔
Hi Siam, glad you got it working with laya-mlx!
Based on what you tested, it looks more like the issue may be related to the Mac/MLX vs. the standard Laya runtime, rather than Python itself, since you saw the same behavior on Python 3.10 and 3.12.
laya-mlx uses a different backend specifically optimized for Apple Silicon, so that could explain why it behaves differently. I wouldn't say for certain yet that it's a Mac issue, though.
Thanks for sharing this — it's a useful finding. I’ll also keep an eye on this when testing Laya again.
a small model that only returns a decision and a probability is basically a router. the real question is what happens when laya is confident but wrong, does the app still show that confidence to the user or does something downstream catch it before it acts on a bad route
That’s a great point. In my testing, Laya returns a probability/confidence score, but that shouldn’t be treated as proof that the decision is correct. If Laya is confident and wrong, the downstream application would need its own guardrails — for example, confidence thresholds, human review, or a fallback path before taking action. I haven’t tested that failure-handling layer yet, but I think it’s an important next step to explore.
How stable are the returned probabilities when an input contains multiple intents or a category absent from the configured choices?
Great question. I haven't tested that specific edge case yet. In my experiments so far, Laya has been quite sensitive to the way the decision options are defined, especially with larger choice sets. The model scores each configured option and returns a probability distribution, so an input that contains multiple intents or an intent outside the configured categories is something I'd want to test rather than assume. The project also notes that confidence needs to be validated/calibrated on your own data.
This is actually a really interesting test case. I’ll add it to my experiments and share the results.
I am curious about time takes to generate the output. And how fast is compared with Jev.
Yes, that was one of the interesting parts of testing Laya. The current Laya benchmark reports 32.8 ms p50 latency for a single question on a Tesla T4, compared with 236–276 ms p50 reported for Jev. These aren't a controlled head-to-head benchmark—the Jev numbers are third-party measurements with different prompts/sample sizes—but they give a useful indication of the latency difference.
congrats