RAG 与知识库·github.com
[Bug]: Title: GAMOptSettings.random_state is unused — GAM exploration is non-deterministic
目标用户 · developer
阅读原文 ↗[Bug]: Title: GAMOptSettings.random_state is unused — GAM exploration is non-deterministic: ### Bug Description # GAMOptSettings.random_state is unused — GAM exploration order is non-deterministic ## Summary `GAMOptSettings` exposes `random_state` (default `64`), but `GAMOptimizer` never uses it. Initial hyperp
个性化解读(发生了什么 / 为什么重要 / 影响 / 建议)将在接入 LLM 后按你的角色生成。
痛点信号
- [bug]: title: gamoptsettings.random_state is unused — gam exploration is non-deterministic.
- ### bug description # gamoptsettings.random_state is unused — gam exploration order is non-deterministic ## summary `gamoptsettings` exposes `random_state` (default `64`), but `gamoptimizer` never uses it.
- ## related improvements (separate / follow-up) | item | why | |------|-----| | benchmark mode `temperature=0` | reduce llm score variance during eval | | `evaluate_fixed_patterns(patterns: list[dict])` or document `known_observations` for replay | prompt-only a/b without gam confounding | | persist exploration trace (evaluated params + scores per step) | debug cross-run differences | ## references - hpo api: https://ibm.github.io/ai4rag/latest/api-reference/core/hpo/ - comparative analysis: rhoai/autorag william benchmark notebooks using `gamoptsettings(max_evals=10, n_random_nodes=4)` ## environment - ai4rag: `move-autorag-components-code-to-ai4rag` / `fix-prompts` branches - optimizer: `gamoptimizer` via `ai4ragexperiment.search()` - metric: `faithfulness` (unitxt) ### steps to reproduce run gam based experiment multiple times.
#developer#rag#issues