混合检索的 RRF:把两份排名合成可解释的结果
保留每一路的名次和来源,手算并复现候选怎样合并,以及精确错误码为什么没有一直排第一。
- 按名次贡献求和,不混加原始分数
- 缺席为零,未知 ID 明确拒绝
- k、候选窗口、最终条数分别控制
- 保留来源、逐路名次与贡献
- 融合不保证精确错误码优先
1. 错误码命中了,为什么融合后反而排到第三?
设想一个帮助中心检索问题:“导出出现 E42,重试后还是失败,应该先看哪篇说明?”关键词检索很容易把含 E42 的文章放到前面。如果另一份候选排名更偏向超时和重试,把两路结果合并后,第一名会怎样变化?本文用六篇原创的短文档,把这个问题缩小为一次能手算、能运行、能解释的排名融合实验。
第一路实际创建 SQLite FTS5 索引并执行查询;第二路是明确写在源码中的人工排名,用来观察同一文档出现在不同位置时发生的变化。我们没有运行向量模型,也没有用人工列表冒充语义检索。你会得到完整的文档夹具、RRF 实现、14 项测试以及一份逐项可核对的输出。
实验最终把 timeout 放到第一,retry 第二,含错误码的 code 第三。这个结果说明合并规则确实生效,也直接展示了它的边界:RRF 不保证精确错误码命中始终优先。至于这个顺序是否更能解决用户问题,需要另外准备真实查询与相关性判断,本篇没有这样的质量结论。
文章只处理“两份有序候选怎样成为一份结果”。它不生成回答,不搭完整 RAG,不部署搜索服务。已有 FTS5 检索或向量格式检查之后,可以把这里作为独立的融合层练习;阅读本文不需要先完成其他教程。
2. 把原始分数留在各自的检索器里
两种检索器可能使用不同的评分方向、取值范围和分布。把 BM25 的原始分数直接加到另一路相似度上,会让结果依赖这些数值尺度。RRF 选择只读取每路的名次:谁排第一、第二、第三,随后用同一个倒数函数换算贡献。原始分数仍可保留作排查记录,但不进入本文的求和。
这里尤其要注意 SQLite FTS5 的方向。SQLite 官方 bm25() 文档说明,较小的数值对应更好的匹配,因此 SQL 使用 ORDER BY bm25(ft), ft.id 升序排列。这个约定属于 SQLite FTS5 的实现,不能把“越小越好”套到所有名字叫 BM25 的接口。
本次实际测得 code 的 bm25 为 -2.4378963801346473e-06,retry 为 -1.1042471042471042e-06,timeout 为 -1.032490974729242e-06。这些数字只说明本夹具上的 FTS5 排名,不能按数值大小解释为命中概率。融合器收到的第一路输入只有 code、retry、timeout 三个 ID。
3. 先把公式读成一笔笔贡献
Cormack、Clarke 与 Büttcher 的 SIGIR 2009 论文给出了按名次倒数求和的 RRF 方法,并在其研究中固定 k = 60。本文采用等权形式:
RRF(d) = sum(1 / (k + rank_r(d)))
对每一个包含文档 d 的候选列表 r 求和
rank 从 1 开始;不在某路候选中的文档,该路贡献为 0。
例如,一篇文档在第一路排第 3、第二路排第 1,当 k = 60 时得分就是 1/63 + 1/61。只在第一路排第 1 的另一篇文档得到 1/61。缺席不是“排到最后”,所以不用为缺席文档捏造一个名次。两路都没取回的文档不会凭空出现在融合结果里。
原论文讨论完整排名;本篇处理有限候选列表,按列表成员关系累加贡献。这个使用方式可以对照 Elasticsearch 的 RRF 参考理解。后面的 Python 函数是独立教学实现,其参数验证、去重、并列排序和输出格式均由本篇定义,不是 Elasticsearch API 的复刻。
RRF 分数是排序用的合计值,不是概率、答案置信度或来源真实性证明。两路、k = 60 时,单篇文档两路都第一才会达到 2/61;所有文档的分数也没有归一化为总和 1。看到 0.032,不应解释成“3.2% 的正确率”。
4. 六篇文档和一个明确的检索范围
文档正文用简短英文构造,是为了让 FTS5 的结果容易复现,不承担中文分词评测。所有记录都在同一个固定快照中,每篇有唯一 doc_id、用于回查的 source,以及 product、status 两个范围字段。source 是夹具中的逻辑路径,不是对磁盘文件存在性的承诺。完整夹具包含在后面的源码 DOCS 常量中。
- code:当前桌面版的 E42 与 checksum mismatch 说明
- retry:当前桌面版的导出重试说明
- timeout:当前桌面版的导出超时说明
- rate:当前桌面版的限流等待说明
- old:归档的桌面版说明
- mobile:当前移动版说明
本次范围固定为 product = desktop 且 status = current。FTS5 查询在 SQL 中加入这两个条件,再按 bm25 与 ID 排序。查询表达式固定为应用预先写好的 "E42" OR "export",不是把用户整段问题转换为 FTS 语法,也没有演示任意自然语言输入的解析。
人工路原始顺序是 old、mobile、timeout、retry、rate。过滤掉不属于本次范围的前两项后,得到 timeout、retry、rate。这份顺序是为了观察融合而设计的输入;它不是一个运行过的 embedding 检索器,也没有相似度数值。
这里的 scope 是演示用的数据筛选。它不验证用户身份,没有租户隔离或权限撤销,不能充当鉴权机制。若未来接入实际服务,检索范围如何执行以及哪些结果允许返回,需要在实际系统中另行验证,不能从这个字典过滤函数推出安全结论。
5. 统一身份,再处理同路重复与跨路重合
两路都必须用同一套文档身份。timeout 在两路里出现时,应当累加到同一篇记录上,而不是输出两条内容相同的结果。本文只用文档级 ID;如果某一路返回的是 chunk,另一路返回整篇文档,就先需要定义清楚粒度和映射。多个 chunk 如何折叠为文档是另一条规则,不应靠复用一个 ID 含糊处理。
输入列表中重复同一个 ID 的情况,也需要有可检查的约定。本文保留同一路的第一次出现,删除后续重复,然后重新连续编号,最后截取 window。比如 code、code、retry 会变成 code、retry,retry 的名次为 2。规范的上游候选通常应当已经唯一;这里做归一化是为了让演示对重复输入有确定行为,不宣称所有 RRF 接口都这样处理。
“没有进入某路候选”和“输入了未知 ID”是两回事。已知的 code 可以只在关键词路出现,其余路贡献为 0;但 missing 如果根本不在 DOCS 中,程序就抛出 ValueError。否则结果可能有分数,却没有可靠的正文和 source 可以回查。scoped_unique 会检查传入的整份列表,所以即使未知 ID 位于窗口之后,也会被拒绝。
最终输出每个 doc_id 只出现一次,从 DOCS 取回 source、正文和范围字段,并附上各路 rank、contribution、精确分数 exact_score。保留这些字段,就能区分“这篇文档在哪一路靠前”与“最终为什么排到这个位置”。
6. 亲手算一次:两路支持让 timeout 超过 code
把 window 设为 3,每路最多保留三个符合范围的候选。第一路是 code、retry、timeout,第二路是 timeout、retry、rate。代入 k = 60,会得到下面四笔账。
| 最终顺序 | 文档 | 精确分数 | 小数展示 |
|---|---|---|---|
| 1 | timeout | 124/3843 | 0.032266458495966696 |
| 2 | retry | 1/31 | 0.03225806451612903 |
| 3 | code | 1/61 | 0.01639344262295082 |
| 4 | rate | 1/63 | 0.015873015873015872 |
timeout 与 retry 的分差很小。若先把两者四舍五入成 0.0323 再排序,就会人为制造同分。代码内部用 Fraction 精确累加和比较,只在输出展示字段中转为 float。最终真正同分时,按 doc_id 升序稳定排序;这也是本文自己选择的规则,不是 RRF 数学公式自带的第二排序键。
code 只收到一路支持,即使那一路是第一,也落到两篇双路命中文档之后。如果业务明确要求“输入精确错误码时,对应说明必须排第一”,单靠这里的 RRF 不足以满足要求。应先明确这一需求怎样与其他查询意图共存,再独立设计并验证对应策略;本文不把任何置顶或加权策略当成默认生产建议。
7. k、候选窗口与最终条数是三个开关
k 改变名次之间的相对贡献。k 较大时,第一与后续名次的相对差距缩小,但不同排名仍然不同。它也可能改变胜出者:测试里另外构造了 a 只在一路第 1、b 在两路第 4 的情况。k = 1 时,a 得 1/2,b 得 2/5,a 领先;k = 60 时,a 得 1/61,b 得 2/64,b 领先。其他填充候选各只出现一路,因此不会遮住这个对照。
window 控制每一路进入融合的候选深度。它不会把分数重新缩放,而是直接改变谁还能贡献。本例 window = 1 时,两路只剩 code 和 timeout,各得 1/61。按照 ID 升序,结果显示 code、timeout;window = 3 时则以 timeout 开头。窗口改变了候选集合,也改变了结果。
size 是融合排序完成之后展示的条数。它与 window 分开:window = 3、size = 1 会保留同样的候选参与求和,最后只返回 timeout。把 window 改成 1 再只显示一条,返回的却是 code。不能把这两个参数当成同一个 top-k。
教学函数要求 k、window、size 都是正整数,显式拒绝布尔值;因此这里不能用 size = 0 表示空页。这是 Python 接口的选择。函数没有分页,没有服务端候选窗口上限,也没有实现 Elasticsearch 对相关参数的全部校验。k = 60 只是用于复现原理的起点,不是对所有数据最优的宣称。
8. 先过滤还是先截断,会得到不同的问题
人工路的前两项恰好不在目标范围中。这让顺序差异一眼可见:先取原始第一项 old,再过滤 current,结果为空;先过滤整份人工列表,再取第一项,结果是 timeout。程序选择后者,并在去重、过滤后重新编号。
在这个内存夹具里,完整人工列表已经可用,所以可以先过滤再截断。真实检索服务如果只取回全局前若干项,然后在应用层过滤,应用并不知道未取回的合格候选是什么,融合阶段无法把它们补齐。应核对实际检索器的过滤位置和返回语义;这里的实验只能展示列表操作的差异,不能替代服务端验证。
FTS5 这一支虽然在 SQL 中限制返回范围,bm25 的统计仍来自本例整张 FTS 索引。范围外文档不出现在结果里,不代表它们完全不影响词项统计。不要把本次结果解释为“仅用当前桌面文档重新建索引”的分数。这个区别不影响按实际返回名次融合,却影响你怎样说明原始分数的来源。
9. 在空目录运行完整实验
需要 Python 3 和带 FTS5 的 SQLite,代码只使用标准库,不下载模型、不安装包、不启网络服务,也不读取用户文档。本次运行环境是 Python 3.12.14、SQLite 3.53.1,FTS5 建表和查询实际成功。其他 Python 构建是否启用 FTS5,以运行结果为准;若出现 no such module: fts5,说明当前构建缺少该扩展,尚未到融合算法这一层。
将下面完整代码保存为 rrf_demo.py,在该文件所在的空目录运行。源码已经包含全部六篇文档、人工列表与 14 项测试,无须额外准备数据文件。标准输出是 JSON,测试记录写到标准错误,方便分别保存。
python3 rrf_demo.py > actual.json 2> tests.log
成功时进程退出码为 0,tests.log 的结尾是 Ran 14 tests 与 OK;actual.json 中 tests_run 为 14、tests_ok 为 true。输出里的 full.hits 顺序应是 timeout、retry、code、rate;truncated.hits 顺序应是 code、timeout。第二路字段刻意命名为 manual_fixture,提醒读者它的输入性质。
"""Offline RRF tutorial: real FTS5 plus manually authored ranks, no embedding model."""
import json
import platform
import sqlite3
from fractions import Fraction
import unittest
DOCS = {
'code': dict(doc_id='code', source='help/code.txt', product='desktop', status='current', text='E42 E42 export fails checksum mismatch'),
'retry': dict(doc_id='retry', source='help/retry.txt', product='desktop', status='current', text='export retry after temporary interruption'),
'timeout': dict(doc_id='timeout', source='help/timeout.txt', product='desktop', status='current', text='export timeout when connection is slow'),
'rate': dict(doc_id='rate', source='help/rate.txt', product='desktop', status='current', text='wait before another request after throttling'),
'old': dict(doc_id='old', source='archive/code.txt', product='desktop', status='archived', text='E42 E42 E42 export export export old instructions'),
'mobile': dict(doc_id='mobile', source='mobile/code.txt', product='mobile', status='current', text='E42 E42 E42 export export export phone instructions'),
}
SCOPE = dict(product='desktop', status='current')
# Entirely hand-authored fixture; no query was embedded and no semantic retrieval ran.
MANUAL = ['old', 'mobile', 'timeout', 'retry', 'rate']
def scoped_unique(ids, docs, scope):
seen, out = set(), []
for doc_id in ids:
if doc_id not in docs:
raise ValueError('unknown document: ' + doc_id)
if doc_id in seen:
continue
seen.add(doc_id)
if all(docs[doc_id].get(k) == v for k, v in scope.items()):
out.append(doc_id)
return out
def fuse(rankings, docs, scope, k=60, window=10, size=10):
if any(type(x) is not int or x < 1 for x in (k, window, size)):
raise ValueError('k/window/size must be positive integers')
totals, traces = {}, {}
candidates = {}
for name, ids in rankings.items():
candidates[name] = scoped_unique(ids, docs, scope)[:window]
for rank, doc_id in enumerate(candidates[name], 1):
term = Fraction(1, k + rank)
totals[doc_id] = totals.get(doc_id, Fraction()) + term
traces.setdefault(doc_id, {})[name] = dict(rank=rank, contribution=float(term))
ordered = sorted(totals, key=lambda doc_id: (-totals[doc_id], doc_id))[:size]
return dict(scope=dict(scope), k=k, window=window, candidates=candidates, hits=[
dict(docs[d], rrf_score=float(totals[d]), exact_score=str(totals[d]), ranks=traces[d])
for d in ordered])
def lexical():
db = sqlite3.connect(':memory:')
db.executescript('CREATE TABLE docs(id TEXT PRIMARY KEY, product TEXT, status TEXT); CREATE VIRTUAL TABLE ft USING fts5(id UNINDEXED, text);')
for d, row in DOCS.items():
db.execute('INSERT INTO docs VALUES(?,?,?)', (d,row['product'],row['status']))
db.execute('INSERT INTO ft VALUES(?,?)', (d,row['text']))
sql = '''SELECT ft.id,bm25(ft) FROM ft JOIN docs d ON ft.id=d.id
WHERE ft MATCH ? AND d.product=? AND d.status=? ORDER BY bm25(ft),ft.id'''
# Static application-controlled query; no natural-language parsing claim.
hits = db.execute(sql, ('"E42" OR "export"', SCOPE['product'], SCOPE['status'])).fetchall()
db.close()
return hits
def ids(result): return [r['doc_id'] for r in result['hits']]
class Tests(unittest.TestCase):
def test_fts_real(self): self.assertEqual([d for d,s in lexical()], ['code','retry','timeout'])
def test_missing_branch_zero(self):
r=fuse({'lex':['code'], 'manual':['retry']},DOCS,SCOPE)
self.assertEqual([x['exact_score'] for x in r['hits']], ['1/61','1/61'])
def test_rank_starts_one(self):
self.assertEqual(fuse({'a':['code']},DOCS,SCOPE)['hits'][0]['exact_score'],'1/61')
def test_same_id_fuses(self):
r=fuse({'a':['code'], 'b':['code']},DOCS,SCOPE)
self.assertEqual(len(r['hits']),1); self.assertEqual(r['hits'][0]['exact_score'],'2/61')
def test_intra_list_duplicate_no_bonus(self):
a=fuse({'a':['code','code','retry']},DOCS,SCOPE)
b=fuse({'a':['code','retry']},DOCS,SCOPE)
self.assertEqual(a,b)
def test_unknown_doc_fails(self):
with self.assertRaises(ValueError): fuse({'a':['missing']},DOCS,SCOPE)
def test_filter_before_window(self):
r=fuse({'a':MANUAL},DOCS,SCOPE,window=1)
self.assertEqual(ids(r),['timeout'])
self.assertEqual(scoped_unique(MANUAL[:1],DOCS,SCOPE),[]) # Wrong sequence starves candidates.
def test_both_branches_same_scope(self):
r=fuse({'a':['mobile','code'], 'b':MANUAL},DOCS,SCOPE)
self.assertFalse({'old','mobile'} & set(ids(r)))
self.assertEqual(r['scope'],SCOPE)
def test_stable_ties(self):
a=fuse({'a':['retry','code'],'b':['code','retry']},DOCS,SCOPE)
b=fuse({'b':['code','retry'],'a':['retry','code']},DOCS,SCOPE)
self.assertEqual(ids(a),['code','retry']); self.assertEqual(ids(a),ids(b))
def test_empty(self): self.assertEqual(ids(fuse({'a':[],'b':[]},DOCS,SCOPE)),[])
def test_truncation_changes_winner(self):
inp={'lex':[d for d,s in lexical()],'manual':MANUAL}
self.assertEqual(ids(fuse(inp,DOCS,SCOPE,window=1))[0],'code')
self.assertEqual(ids(fuse(inp,DOCS,SCOPE,window=3))[0],'timeout')
def test_k_can_change_order(self):
ds={x:dict(doc_id=x) for x in ['a','b','c','d','e','f','g']}
inp={'one':['a','c','d','b'],'two':['e','f','g','b']}
self.assertEqual(ids(fuse(inp,ds,{},k=1))[0],'a')
self.assertEqual(ids(fuse(inp,ds,{},k=60))[0],'b')
def test_preserve_source(self):
hit=fuse({'a':['code']},DOCS,SCOPE)['hits'][0]
self.assertEqual(hit['source'],'help/code.txt'); self.assertEqual(hit['doc_id'],'code')
def test_invalid_params(self):
for k in (True,0,-1,1.5):
with self.assertRaises(ValueError): fuse({},DOCS,SCOPE,k=k)
if __name__ == '__main__':
suite=unittest.defaultTestLoader.loadTestsFromTestCase(Tests)
res=unittest.TextTestRunner(verbosity=2).run(suite)
inp={'lexical':[d for d,s in lexical()], 'manual_fixture':MANUAL}
report=dict(python=platform.python_version(),sqlite=sqlite3.sqlite_version,
input_kind='real FTS5 plus manually authored synthetic candidate ranks; no embedding model',
lexical_scores=lexical(),full=fuse(inp,DOCS,SCOPE,window=3),truncated=fuse(inp,DOCS,SCOPE,window=1),
tests_run=res.testsRun, tests_ok=res.wasSuccessful())
print(json.dumps(report,indent=2,ensure_ascii=False))
raise SystemExit(not res.wasSuccessful())
10. 核对输出,而不只看测试绿灯
交付目录中的 expected.json 保存本次运行结果,包括实际版本、FTS5 原始分数、两种候选窗口和完整来源字段。版本与原始浮点数属于这一环境的观测值。复现时首先看候选 ID 顺序、精确分数和每路贡献;如果换了文档正文、索引或查询表达式,关键词顺序可能改变,原先的预期结果也要重新审查。
将下面完整 JSON 保存为 expected.json。即使只有这份 HTML,也可以复制本文的主程序、预期输出和检查器到同一个空目录,按前述命令运行。
{
"python": "3.12.14",
"sqlite": "3.53.1",
"input_kind": "real FTS5 plus manually authored synthetic candidate ranks; no embedding model",
"lexical_scores": [
[
"code",
-2.4378963801346473e-06
],
[
"retry",
-1.1042471042471042e-06
],
[
"timeout",
-1.032490974729242e-06
]
],
"full": {
"scope": {
"product": "desktop",
"status": "current"
},
"k": 60,
"window": 3,
"candidates": {
"lexical": [
"code",
"retry",
"timeout"
],
"manual_fixture": [
"timeout",
"retry",
"rate"
]
},
"hits": [
{
"doc_id": "timeout",
"source": "help/timeout.txt",
"product": "desktop",
"status": "current",
"text": "export timeout when connection is slow",
"rrf_score": 0.032266458495966696,
"exact_score": "124/3843",
"ranks": {
"lexical": {
"rank": 3,
"contribution": 0.015873015873015872
},
"manual_fixture": {
"rank": 1,
"contribution": 0.01639344262295082
}
}
},
{
"doc_id": "retry",
"source": "help/retry.txt",
"product": "desktop",
"status": "current",
"text": "export retry after temporary interruption",
"rrf_score": 0.03225806451612903,
"exact_score": "1/31",
"ranks": {
"lexical": {
"rank": 2,
"contribution": 0.016129032258064516
},
"manual_fixture": {
"rank": 2,
"contribution": 0.016129032258064516
}
}
},
{
"doc_id": "code",
"source": "help/code.txt",
"product": "desktop",
"status": "current",
"text": "E42 E42 export fails checksum mismatch",
"rrf_score": 0.01639344262295082,
"exact_score": "1/61",
"ranks": {
"lexical": {
"rank": 1,
"contribution": 0.01639344262295082
}
}
},
{
"doc_id": "rate",
"source": "help/rate.txt",
"product": "desktop",
"status": "current",
"text": "wait before another request after throttling",
"rrf_score": 0.015873015873015872,
"exact_score": "1/63",
"ranks": {
"manual_fixture": {
"rank": 3,
"contribution": 0.015873015873015872
}
}
}
]
},
"truncated": {
"scope": {
"product": "desktop",
"status": "current"
},
"k": 60,
"window": 1,
"candidates": {
"lexical": [
"code"
],
"manual_fixture": [
"timeout"
]
},
"hits": [
{
"doc_id": "code",
"source": "help/code.txt",
"product": "desktop",
"status": "current",
"text": "E42 E42 export fails checksum mismatch",
"rrf_score": 0.01639344262295082,
"exact_score": "1/61",
"ranks": {
"lexical": {
"rank": 1,
"contribution": 0.01639344262295082
}
}
},
{
"doc_id": "timeout",
"source": "help/timeout.txt",
"product": "desktop",
"status": "current",
"text": "export timeout when connection is slow",
"rrf_score": 0.01639344262295082,
"exact_score": "1/61",
"ranks": {
"manual_fixture": {
"rank": 1,
"contribution": 0.01639344262295082
}
}
}
]
},
"tests_run": 14,
"tests_ok": true
}
下面这个独立检查器会比较候选、命中项和测试状态,版本仅打印供检查,原始 BM25 浮点值保留在 actual.json 中。把它保存为 verify_output.py,与 expected.json 放在同一目录。运行主程序后执行 python3 verify_output.py;它不会在差异出现时自动改写预期文件。
import json
from pathlib import Path
actual = json.loads(Path('actual.json').read_text())
expected = json.loads(Path('expected.json').read_text())
for key in ('input_kind', 'full', 'truncated', 'tests_run', 'tests_ok'):
if actual[key] != expected[key]:
raise SystemExit('Mismatch: ' + key)
if [d for d, score in actual['lexical_scores']] != ['code', 'retry', 'timeout']:
raise SystemExit('Mismatch: lexical order')
print('Expected candidates, hits, traces and tests match')
print('Python', actual['python'], 'SQLite', actual['sqlite'])
14 项单元测试分别检查实际 FTS5 顺序、缺席贡献、名次起点、跨路合并、同路去重、未知 ID、先过滤后截断、两路共同范围、同分稳定、空列表、窗口改变胜者、k 改变顺序、来源保留和非法 k。空输入返回空 hits,不应被自动补成一篇“看起来相关”的文章。参数实现同时校验 window 和 size,但原始 14 项测试的非法参数用例只直接枚举 k,不能把它描述成所有参数组合的穷举测试。
通过这些断言,可以确认这个小函数按约定处理了这些输入。它们没有测出真实检索的 recall、precision 或用户满意度,也没有测回答是否正确。第二路仍是人工数据,所以“双路结果更好”不是测试通过后的合理结论。
11. 把这一层的责任留清楚
完成这个实验后,你手里的是一个可解释的排名合并器:输入文档级候选顺序,明确处理范围、重复与未知身份;逐路记录贡献;按精确合计和稳定并列规则返回结果。它适合帮助读者理解候选为什么上升或下降,也便于之后替换上游输入时查出哪条规则发生了变化。
真正接入向量检索时,先需要验证模型与索引确实返回了候选,再核对文档粒度、版本、过滤位置与身份映射。之后还要用有相关性判断的查询集比较单路和融合结果。本篇没有执行 embedding、真实召回质量评估、答案生成、ANN 性能测试、生产 API 隔离或浏览器像素验收,不将这些工作隐含在“混合检索”这个标题里。
准备把人工排名换成真实向量检索时,可先参考EmbeddingGemma 2 验证指南中的向量格式检查与中文查询样本。格式检查通过只说明输出满足约定;还要实际生成候选,并在同一套带相关性标注的查询上比较单路与融合结果。该指南尚未运行模型,不能作为第二路召回效果已获验证的依据。
如果你只记住一个现象,就记住 code 的位置:关键词第一,融合第三。两路支持可以超过单路首位,窗口缩短又可以改变胜者。理解这些变化,并留下每一步的来源和贡献,比只得到一个新分数更有助于解释搜索结果。
12. 参考资料
- Cormack, Clarke, Büttcher:Reciprocal Rank Fusion,SIGIR 2009。原始公式与 k 的研究背景;论文自己的效果结论不移用于本夹具。
- Elasticsearch 官方 RRF 参考。有限候选列表、名次与窗口的说明;本文未安装或调用 Elasticsearch。
- SQLite FTS5:bm25() 函数。核对本例原始分数的排序方向。
13. 补充实验(2026-10-13):候选没变,第一名为什么变了?
RRF 不相加两路的原始分数,却仍然依赖上游把每一路排对。如果同一个逻辑检索路拆成两个分区,两个分区各自有序,按返回先后直接拼接,得到的整路未必有序。把数组位置当成名次,网络先后就会间接改变融合结果。
本次补充给出一个很小的反例:只交换两个批次的到达顺序,沿用前文同一个 fuse(),第一名就从 a 变成 target。按事先约定的可比分数整理第一路后,全部 12 种输入排列给出相同的完整结果。这里验证的是输入构造规则,target 只是文档 ID,并不表示真实问题的正确答案。
前文 FTS5 已用 ORDER BY bm25(ft), ft.id 排序,另一路本来就是人工写好的有序列表;它们没有这个多批次拼接问题。原实现、原输出和 14 项测试均保留,下面是单独追加的实验。
这个问题也有一份有日期的开发者讨论:WeKnora #3796 于 2026-09-27 提出;9 月 28 日回复已说明当时 main 不再有该机制,报告者将观察限定到 v0.8.2;10 月 7 日按已修复关闭。本文借它提出检查点,不将它写成当前产品故障。讨论中的按列表取名次,与下面明确选择的“单路候选统一排序”也不是同一种合并语义。
14. 先限定实验:只整理同一路、同一尺度的候选
以下分数全部是原创人工数值,不是余弦相似度、FTS5 输出或相关性标注。第一路拆为两个不重叠分区;第二路固定为 target、a、b、c,没有原始分数。
| 逻辑路 / 分区 | doc_id | 人工分数 | 分区内初始名次 |
|---|---|---|---|
| 第一路 / A | a | 0.81 | 1 |
| 第一路 / A | b | 0.79 | 2 |
| 第一路 / A | c | 0.77 | 3 |
| 第一路 / B | target | 0.96 | 1 |
我们显式假设:第一路来自同一查询、同一快照、同一可比较的度量,且越大越好;同分时按 doc_id 升序。这是本夹具的前提,不是看到 score 字段、相同模型名或相同检索类型就能推断出的保证。真实系统还得核对查询向量、度量、归一化、模型版本及分数尺度。代码里的 score_contract 字符串只是声明和拒绝检查,不能认证这些条件。
这里只在第一路内部排序,不把 BM25 与向量原始分数混排,也不宣称跨分片 BM25 天然可比。若产品本来就希望每个分区各算一份排名,应保留独立列表并定义贡献规则;那是另一种合理设计,不能用本例直接判错。
15. 把变动写成名次与分数,而不是“结果似乎更好”
参数固定为 k = 60、每路 window = 4,先查看全部四条结果;size = 1 只用来观察第一名。A 先到时,直接拼接得到 a、b、c、target;B 先到时,得到 target、a、b、c。两次没有增删候选,也没有改分数。
| 第一路构造 | target 在第一路的名次 | 融合第一名 |
|---|---|---|
| 直接拼接 A → B | 4 | a |
| 直接拼接 B → A | 1 | target |
| 按可比分数整理(任一到达顺序) | 1 | target |
下面列出全部分数,不只列胜者。B → A 的直接拼接恰好已是整理后的顺序,完整结果与表的后半部分相同。
| 构造 | 最终名次 | ID | 第一路名次 | 第二路名次 | 两路贡献 | 精确合计 |
|---|---|---|---|---|---|---|
| A → B 直接拼接 | 1 | a | 1 | 2 | 1/61 + 1/62 | 123/3782 |
| A → B 直接拼接 | 2 | target | 4 | 1 | 1/64 + 1/61 | 125/3904 |
| A → B 直接拼接 | 3 | b | 2 | 3 | 1/62 + 1/63 | 125/3906 |
| A → B 直接拼接 | 4 | c | 3 | 4 | 1/63 + 1/64 | 127/4032 |
| 先整理第一路 | 1 | target | 1 | 1 | 1/61 + 1/61 | 2/61 |
| 先整理第一路 | 2 | a | 2 | 2 | 1/62 + 1/62 | 1/31 |
| 先整理第一路 | 3 | b | 3 | 3 | 1/63 + 1/63 | 2/63 |
| 先整理第一路 | 4 | c | 4 | 4 | 1/64 + 1/64 | 1/32 |
a 在错误构造中是 1/61 + 1/62,target 是 1/64 + 1/61,所以 a 领先。整理之后,target 两路都第一,得到 2/61。第二路始终未变;变化来自第一路的名次,不是原始分数直接进入了 RRF 求和。
为了不只覆盖“哪个批次先到”,再枚举 A 的三个元素的 6 种内部顺序,与 2 种批次顺序组合,共 12 种。后五种内部顺序是故意打乱的输入,不再声称每个批次都已局部有序。整理后的全部四条 ID、名次和精确分数均与固定预期值一致。
| 编号 | A 内部顺序 | 批次顺序 | 直接拼接第一名 | 整理后第一名 |
|---|---|---|---|---|
| 1 | a → b → c | A → B | a | target |
| 2 | a → b → c | B → A | target | target |
| 3 | a → c → b | A → B | a | target |
| 4 | a → c → b | B → A | target | target |
| 5 | b → a → c | A → B | b | target |
| 6 | b → a → c | B → A | target | target |
| 7 | b → c → a | A → B | b | target |
| 8 | b → c → a | B → A | target | target |
| 9 | c → a → b | A → B | a | target |
| 10 | c → a → b | B → A | target | target |
| 11 | c → b → a | A → B | c | target |
| 12 | c → b → a | B → A | target | target |
16. 排序应发生在名次生成和候选截断之前
新增的 merge_comparable() 检查声明、拒绝重复分区和重复 ID,把已给候选放进新列表,再按 (score.copy_negate(), doc_id) 排序。copy_negate() 直接取反且不受当前 Decimal 精度舍入影响;用普通负号可能把两个很接近的分数压成假同分。它不改调用者的数据,也不改前文的 fuse()。拒绝重叠 ID 是为了不顺手引入另一套“重叠分区应该取哪个分数”的规则。
return [hit.doc_id for hit in sorted(
rows, key=lambda h: (h.score.copy_negate(), h.doc_id)
)]
这里只有一个已排序批次时,整理前后相同;等分候选用 ID 固定次序;不同契约、NaN、无穷值及非 Decimal 分数会被拒绝。空批次允许通过。这些是本例定义的边界,不是某个搜索服务的兼容接口。
如果先把 A → B 的拼接结果截成前三项,target 已从第一路消失。后面再排序,最多得到 a、b、c,不能补回第一路本应有的那笔贡献。第二路仍有 target,所以它最终还有 1/61,不能说它从所有结果中消失。
| 操作 | 参与融合的第一路 | size = 1 的结果 |
|---|---|---|
| 先整理,再取 window = 1 | target | target,2/61 |
| 直接拼接 A → B,再取 window = 1 | a | a,1/61;与 target 同分,按 ID 决胜 |
“整理全部已给候选”也不等于“找齐全库最优候选”。缺失分区、上游召回遗漏、提前截断和真实并发调度均不在这个函数的检查范围内。融合后再加一个排序或重排器,也无法从这份候选列表恢复没有收到的证据。
17. 在空目录复现:13 项测试、12 种排列、7 个故障变体
本次在 CPython 3.12.14、仅标准库环境运行。前文未修改的 14 项测试通过;补充的 13 项具名测试通过,其中一个测试枚举 12 种排列。fixed_oracle.py 把两组完整 ID、名次、精确分数写成字面量,测试不会调用待测 fuse() 来生成自己的预期答案。
将下方四个新增 Python 文件和前文 rrf_demo.py 放在同一目录,运行:
python3 -B rrf_demo.py
python3 -B probe.py
python3 -B test_probe.py
python3 -B fixed_oracle.py
python3 -B check_mutants.py
check_mutants.py 在各自的临时目录实际修改一处源代码,然后启动独立 Python 进程跑补充测试。七次非零退出分别覆盖:用到达顺序代替排序、排序方向颠倒、删去同分 ID 规则、跳过契约检查、排序前截断、将 Decimal 排序键舍入、改错 RRF 常量。它只检验这些故障能否被发现,不表示穷尽全部错误。
| 源代码故障变体 | 测试退出码 | 结果 |
|---|---|---|
| arrival_as_rank | 1 | 已被测试检出 |
| wrong_score_direction | 1 | 已被测试检出 |
| unstable_equal_scores | 1 | 已被测试检出 |
| skip_score_contract | 1 | 已被测试检出 |
| cut_before_ordering | 1 | 已被测试检出 |
| rounded_decimal_key | 1 | 已被测试检出 |
| changed_rrf_constant | 1 | 已被测试检出 |
下方保留完整源码与输出。输出中的 input_batches 是全量输入,cases 是两种初始到达顺序,permutation_results 给出 12 种排列各自的完整输入及输出。连续独立运行输出逐字节相同;公开 ZIP 在新临时目录解包后也重新通过原测试、补充测试、固定预期值和七个变体检查。
probe.py
"""One offline RRF input-order experiment; no search engine or model is run.
rrf_demo.py is the unchanged code copied from the existing published article.
All scores and rankings here are authored fixtures, not measured similarities.
"""
from dataclasses import dataclass
from decimal import Decimal
from itertools import permutations
import json
import platform
from rrf_demo import fuse
@dataclass(frozen=True)
class Hit:
doc_id: str
score: Decimal
@dataclass(frozen=True)
class Batch:
partition: str
score_contract: str
hits: tuple[Hit, ...]
# This label declares a toy assumption. Matching labels cannot certify real
# score comparability, embedding identity, query identity, or index versions.
CONTRACT = "authored-score-v1/same-query/same-snapshot/higher-is-better"
A = Batch("partition_a", CONTRACT, (
Hit("a", Decimal("0.81")),
Hit("b", Decimal("0.79")),
Hit("c", Decimal("0.77")),
))
B = Batch("partition_b", CONTRACT, (Hit("target", Decimal("0.96")),))
DOCS = {d: {"doc_id": d, "source": "fixture/" + d + ".txt"}
for d in ("a", "b", "c", "target")}
SECOND_ROUTE = ("target", "a", "b", "c")
def arrival_ids(batches):
"""Intentionally flawed baseline: transport position becomes search rank."""
return [hit.doc_id for batch in batches for hit in batch.hits]
def merge_comparable(batches, expected_contract=CONTRACT):
"""Order only the supplied candidates under one explicit comparable scale.
No claim of complete corpus recall. Duplicate IDs are rejected rather than
silently deciding between overlapping partitions or inconsistent scores.
This is an independent toy merge convention, not a product implementation.
"""
rows = []
seen_partitions, seen_ids = set(), set()
for batch in batches:
if batch.score_contract != expected_contract:
raise ValueError("score_contract_mismatch")
if not batch.partition or batch.partition in seen_partitions:
raise ValueError("duplicate_or_empty_partition")
seen_partitions.add(batch.partition)
for hit in batch.hits:
if not hit.doc_id or hit.doc_id in seen_ids:
raise ValueError("duplicate_or_empty_document")
if type(hit.score) is not Decimal or not hit.score.is_finite():
raise ValueError("finite_decimal_score_required")
seen_ids.add(hit.doc_id)
rows.append(hit)
# New list: do not mutate a caller's batch, row, score, or existing order.
return [hit.doc_id for hit in sorted(rows, key=lambda h: (h.score.copy_negate(), h.doc_id))]
def fused(first_route, *, window=4, size=4):
"""Run the EXISTING article's equal-weight RRF without changing its formula."""
result = fuse({"partitioned_fixture": first_route,
"second_fixture": list(SECOND_ROUTE)},
DOCS, {}, k=60, window=window, size=size)
return {"candidates": result["candidates"], "hits": [
{"doc_id": row["doc_id"], "exact_score": row["exact_score"],
"ranks": {route: trace["rank"] for route, trace in row["ranks"].items()}}
for row in result["hits"]]}
def report():
cases = []
for batches in ((A, B), (B, A)):
raw, ordered = arrival_ids(batches), merge_comparable(batches)
cases.append({
"arrival": [b.partition for b in batches],
"naive_route": raw,
"naive_fusion": fused(raw),
"merged_route": ordered,
"merged_fusion": fused(ordered),
"naive_top1": fused(raw, size=1)["hits"][0]["doc_id"],
"merged_top1": fused(ordered, size=1)["hits"][0]["doc_id"],
})
permutations_checked = 0
permutation_results = []
for a_hits in permutations(A.hits):
a = Batch(A.partition, CONTRACT, a_hits)
for batches in ((a, B), (B, a)):
if merge_comparable(batches) != list(SECOND_ROUTE):
raise AssertionError("merged route changed across permutations")
if fused(merge_comparable(batches)) != fused(list(SECOND_ROUTE)):
raise AssertionError("fusion changed across permutations")
permutations_checked += 1
permutation_results.append({
"arrival": [b.partition for b in batches],
"a_internal_order": [h.doc_id for h in a_hits],
"naive_route": arrival_ids(batches),
"naive_fusion": fused(arrival_ids(batches)),
"merged_route": merge_comparable(batches),
"merged_fusion": fused(merge_comparable(batches)),
})
# Information already discarded cannot be recreated by subsequent sorting.
early_cut = (Batch("already_truncated", CONTRACT, A.hits),)
recovered = merge_comparable(early_cut)
full = merge_comparable((A, B))
return {
"python": platform.python_version(),
"input_kind": "original authored scores and ranked lists; no retrieval/model run",
"policy": "one logical route; globally order its supplied comparable candidates",
"input_batches": [{"partition": b.partition, "score_contract": b.score_contract,
"hits": [{"doc_id": h.doc_id, "score": str(h.score)} for h in b.hits]}
for b in (A, B)],
"second_route": list(SECOND_ROUTE),
"parameters": {"k": 60, "window": 4, "size": 4, "top1_size": 1},
"cases": cases,
"permutation_results": permutation_results,
"permutations_checked": permutations_checked,
"single_sorted_batch": {"arrival": arrival_ids((A,)),
"merged": merge_comparable((A,))},
"cut_before_merge": {"kept_ids": recovered,
"target_in_first_route": "target" in recovered,
"target_still_in_second_route": "target" in SECOND_ROUTE,
"fusion_after_lost_first_route_contribution": fused(recovered),
"warning": "second route may still retrieve target; missing first-route evidence is not restored"},
"merge_then_route_window_1": fused(full, window=1, size=1),
"naive_then_route_window_1": fused(arrival_ids((A, B)), window=1, size=1),
"limits": [
"Same-contract label is only a declared fixture assumption, not proof of comparable real scores",
"Never sort unrelated BM25 and vector scores together",
"Separate-list RRF is another valid design with different semantics; not implemented here",
"No async scheduler, WeKnora deployment, Elasticsearch instance, embeddings, ANN or reranker tested",
"Only supplied candidates are ordered; missing candidates are not recovered",
],
}
if __name__ == "__main__":
print(json.dumps(report(), ensure_ascii=False, indent=2))
test_probe.py
"""Deterministic positive/negative controls for one RRF input-order boundary."""
from dataclasses import replace
from decimal import Decimal, localcontext
from fractions import Fraction
from itertools import permutations
import unittest
from fixed_oracle import BAD, GOOD
from probe import (A, B, CONTRACT, SECOND_ROUTE, Batch, Hit, arrival_ids,
fused, merge_comparable, report)
class ArrivalOrderTests(unittest.TestCase):
def test_local_sorted_batches_are_not_global_sorted(self):
self.assertEqual(arrival_ids((A, B)), ["a", "b", "c", "target"])
self.assertEqual(merge_comparable((A, B)), ["target", "a", "b", "c"])
def test_only_arrival_order_changes_naive_top1(self):
self.assertEqual(fused(arrival_ids((A, B))), BAD)
self.assertEqual(fused(arrival_ids((A, B)), size=1)["hits"][0]["doc_id"], "a")
self.assertEqual(fused(arrival_ids((B, A))), GOOD)
self.assertEqual(fused(arrival_ids((B, A)), size=1)["hits"][0]["doc_id"], "target")
def test_all_12_fixture_permutations_preserve_correct_fusion(self):
expected = GOOD
count = 0
for a_hits in permutations(A.hits):
for batches in ((replace(A, hits=a_hits), B), (B, replace(A, hits=a_hits))):
self.assertEqual(fused(merge_comparable(batches)), expected)
count += 1
self.assertEqual(count, 12)
def test_exact_contributions_explain_flip(self):
bad = {r["doc_id"]: r for r in fused(arrival_ids((A, B)))["hits"]}
good = {r["doc_id"]: r for r in fused(merge_comparable((A, B)))["hits"]}
self.assertEqual(Fraction(bad["a"]["exact_score"]), Fraction(1,61)+Fraction(1,62))
self.assertEqual(Fraction(bad["target"]["exact_score"]), Fraction(1,64)+Fraction(1,61))
self.assertEqual(good["target"]["exact_score"], "2/61")
self.assertEqual(good["target"]["ranks"]["partitioned_fixture"], 1)
def test_single_sorted_batch_is_noop(self):
self.assertEqual(merge_comparable((A,)), arrival_ids((A,)))
self.assertEqual(merge_comparable((B,)), arrival_ids((B,)))
def test_exact_decimal_order_and_stable_ties(self):
x = Batch("x", CONTRACT, (Hit("z", Decimal("0.9")),))
y = Batch("y", CONTRACT, (Hit("a", Decimal("0.9")),))
self.assertEqual(merge_comparable((x,y)), ["a","z"])
self.assertEqual(merge_comparable((y,x)), ["a","z"])
# Unary minus would round these keys at the default Decimal precision.
lower = Batch("lower", CONTRACT, (Hit("a", Decimal("0.90000000000000000000000000001")),))
higher = Batch("higher", CONTRACT, (Hit("z", Decimal("0.90000000000000000000000000002")),))
with localcontext() as context:
context.prec = 2
self.assertEqual(merge_comparable((lower,higher)), ["z","a"])
self.assertEqual(merge_comparable((higher,lower)), ["z","a"])
def test_other_score_contract_is_rejected(self):
for label in ("bm25/lower-is-better", "different-query", "different-model", "different-snapshot"):
with self.subTest(label=label), self.assertRaisesRegex(ValueError, "score_contract_mismatch"):
merge_comparable((A, replace(B, score_contract=label)))
def test_nonfinite_or_nondecimal_scores_are_rejected(self):
for score in (Decimal("NaN"), Decimal("Infinity"), Decimal("-Infinity"), 0.9, True):
with self.subTest(score=score), self.assertRaisesRegex(ValueError, "finite_decimal"):
merge_comparable((replace(B, hits=(Hit("target", score),)),))
def test_duplicate_partition_or_document_is_rejected(self):
with self.assertRaisesRegex(ValueError, "duplicate_or_empty_partition"):
merge_comparable((A,A))
with self.assertRaisesRegex(ValueError, "duplicate_or_empty_document"):
merge_comparable((A,replace(B, hits=(Hit("a",Decimal("0.99")),))))
def test_empty_batch_and_no_mutation(self):
before = repr((A,B))
empty = Batch("empty", CONTRACT, ())
self.assertEqual(merge_comparable((empty,)), [])
self.assertEqual(merge_comparable((A,empty,B)), list(SECOND_ROUTE))
self.assertEqual(repr((A,B)), before)
def test_early_cut_cannot_restore_missing_route_evidence(self):
cut = (Batch("cut", CONTRACT, A.hits),)
merged = merge_comparable(cut)
self.assertNotIn("target", merged)
hit = next(r for r in fused(merged)["hits"] if r["doc_id"] == "target")
self.assertEqual(hit["exact_score"], "1/61")
self.assertEqual(hit["ranks"], {"second_fixture":1})
def test_route_window_and_display_size_remain_distinct(self):
bad = fused(arrival_ids((A,B)),window=1,size=1)
good = fused(merge_comparable((A,B)),window=1,size=1)
self.assertEqual(bad["hits"][0]["doc_id"], "a")
self.assertEqual(bad["hits"][0]["exact_score"], "1/61")
self.assertEqual(good["hits"][0]["doc_id"], "target")
self.assertEqual(good["hits"][0]["exact_score"], "2/61")
def test_report_is_deterministic(self):
self.assertEqual(report(), report())
self.assertEqual(report()["permutations_checked"], 12)
if __name__ == "__main__":
unittest.main(verbosity=2)
fixed_oracle.py
"""Literal independently written expectations, not computed by the target code."""
# Each row is (doc_id, exact RRF score, first-route rank, second-route rank).
BAD_ROWS = (
("a", "123/3782", 1, 2),
("target", "125/3904", 4, 1),
("b", "125/3906", 2, 3),
("c", "127/4032", 3, 4),
)
GOOD_ROWS = (
("target", "2/61", 1, 1),
("a", "1/31", 2, 2),
("b", "2/63", 3, 3),
("c", "1/32", 4, 4),
)
EXPECTED_ROUTE = ["target", "a", "b", "c"]
def expected(rows, first_route):
"""Formatting only: every rank and exact score above is a literal."""
return {"candidates": {
"partitioned_fixture": list(first_route),
"second_fixture": ["target", "a", "b", "c"],
}, "hits": [{"doc_id": d, "exact_score": s, "ranks": {
"partitioned_fixture": r1, "second_fixture": r2,
}} for d, s, r1, r2 in rows]}
BAD = expected(BAD_ROWS, ["a", "b", "c", "target"])
GOOD = expected(GOOD_ROWS, EXPECTED_ROUTE)
def require_equal(actual, expected_value):
if actual != expected_value:
raise AssertionError(f"oracle mismatch: {actual!r} != {expected_value!r}")
def verify():
# Import only after the fixed expectations have been defined.
from dataclasses import replace
from itertools import permutations
from probe import A, B, arrival_ids, fused, merge_comparable
require_equal(fused(arrival_ids((A, B))), BAD)
require_equal(fused(arrival_ids((B, A))), GOOD)
count = 0
for order in permutations(A.hits):
a = replace(A, hits=order)
for batches in ((a, B), (B, a)):
ordered = merge_comparable(batches)
require_equal(ordered, EXPECTED_ROUTE)
require_equal(fused(ordered), GOOD)
count += 1
require_equal(count, 12)
return {"literal_oracle_passed": True, "permutations_checked": count,
"bad": BAD, "good": GOOD}
if __name__ == "__main__":
import json
print(json.dumps(verify(), ensure_ascii=False, indent=2))
check_mutants.py
"""Run seven real source mutations in isolated directories; no external packages."""
import hashlib
import json
from pathlib import Path
import re
import subprocess
import sys
import tempfile
ROOT = Path(__file__).resolve().parent
MUTANTS = (
("arrival_as_rank", "sorted(rows, key=lambda h: (h.score.copy_negate(), h.doc_id))", "rows",
"test_all_12_fixture_permutations_preserve_correct_fusion"),
("wrong_score_direction", "(h.score.copy_negate(), h.doc_id)", "(h.score, h.doc_id)",
"test_local_sorted_batches_are_not_global_sorted"),
("unstable_equal_scores", "(h.score.copy_negate(), h.doc_id)", "(h.score.copy_negate(),)",
"test_exact_decimal_order_and_stable_ties"),
("skip_score_contract", "if batch.score_contract != expected_contract:", "if False:",
"test_other_score_contract_is_rejected"),
("cut_before_ordering", "sorted(rows, key=lambda h: (h.score.copy_negate(), h.doc_id))",
"sorted(rows[:3], key=lambda h: (h.score.copy_negate(), h.doc_id))",
"test_all_12_fixture_permutations_preserve_correct_fusion"),
("rounded_decimal_key", "(h.score.copy_negate(), h.doc_id)", "(-h.score, h.doc_id)",
"test_exact_decimal_order_and_stable_ties"),
("changed_rrf_constant", "DOCS, {}, k=60, window=window, size=size)",
"DOCS, {}, k=61, window=window, size=size)",
"test_exact_contributions_explain_flip"),
)
def run():
original = (ROOT / "probe.py").read_text()
results = []
for name, old, new, required_failure in MUTANTS:
if original.count(old) != 1:
raise RuntimeError("mutation target must occur exactly once: " + name)
changed = original.replace(old, new, 1)
with tempfile.TemporaryDirectory(prefix="rrf-mutant-") as tmp:
directory = Path(tmp)
for filename in ("rrf_demo.py", "test_probe.py", "fixed_oracle.py"):
(directory / filename).write_bytes((ROOT / filename).read_bytes())
(directory / "probe.py").write_text(changed)
proc = subprocess.run([sys.executable, "-B"] + (["-O"] if sys.flags.optimize else []) + ["test_probe.py"], cwd=directory,
text=True, capture_output=True, timeout=30)
failures = sorted(set(re.findall(r"^(?:FAIL|ERROR): (test_[A-Za-z0-9_]+)",
proc.stderr, re.MULTILINE)))
detected = proc.returncode != 0 and required_failure in failures
results.append({"mutant": name, "old_source": old, "new_source": new,
"mutated_probe_sha256": hashlib.sha256(changed.encode()).hexdigest(),
"test_exit_code": proc.returncode,
"failed_named_tests": failures, "detected": detected})
if not detected:
raise AssertionError(json.dumps(results[-1]))
return {"source_mutants_executed": len(results), "all_detected": True,
"baseline_probe_sha256": hashlib.sha256(original.encode()).hexdigest(),
"mutants": results}
if __name__ == "__main__":
print(json.dumps(run(), ensure_ascii=False, indent=2))
完整输入与运行输出 results.json
{
"python": "3.12.14",
"input_kind": "original authored scores and ranked lists; no retrieval/model run",
"policy": "one logical route; globally order its supplied comparable candidates",
"input_batches": [
{
"partition": "partition_a",
"score_contract": "authored-score-v1/same-query/same-snapshot/higher-is-better",
"hits": [
{
"doc_id": "a",
"score": "0.81"
},
{
"doc_id": "b",
"score": "0.79"
},
{
"doc_id": "c",
"score": "0.77"
}
]
},
{
"partition": "partition_b",
"score_contract": "authored-score-v1/same-query/same-snapshot/higher-is-better",
"hits": [
{
"doc_id": "target",
"score": "0.96"
}
]
}
],
"second_route": [
"target",
"a",
"b",
"c"
],
"parameters": {
"k": 60,
"window": 4,
"size": 4,
"top1_size": 1
},
"cases": [
{
"arrival": [
"partition_a",
"partition_b"
],
"naive_route": [
"a",
"b",
"c",
"target"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"a",
"b",
"c",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "a",
"exact_score": "123/3782",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 2
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "b",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
},
"naive_top1": "a",
"merged_top1": "target"
},
{
"arrival": [
"partition_b",
"partition_a"
],
"naive_route": [
"target",
"a",
"b",
"c"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
},
"naive_top1": "target",
"merged_top1": "target"
}
],
"permutation_results": [
{
"arrival": [
"partition_a",
"partition_b"
],
"a_internal_order": [
"a",
"b",
"c"
],
"naive_route": [
"a",
"b",
"c",
"target"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"a",
"b",
"c",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "a",
"exact_score": "123/3782",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 2
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "b",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_b",
"partition_a"
],
"a_internal_order": [
"a",
"b",
"c"
],
"naive_route": [
"target",
"a",
"b",
"c"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_a",
"partition_b"
],
"a_internal_order": [
"a",
"c",
"b"
],
"naive_route": [
"a",
"c",
"b",
"target"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"a",
"c",
"b",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "a",
"exact_score": "123/3782",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 2
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "c",
"exact_score": "63/1984",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 4
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_b",
"partition_a"
],
"a_internal_order": [
"a",
"c",
"b"
],
"naive_route": [
"target",
"a",
"c",
"b"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"c",
"b"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_a",
"partition_b"
],
"a_internal_order": [
"b",
"a",
"c"
],
"naive_route": [
"b",
"a",
"c",
"target"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"b",
"a",
"c",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "b",
"exact_score": "124/3843",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 3
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "c",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_b",
"partition_a"
],
"a_internal_order": [
"b",
"a",
"c"
],
"naive_route": [
"target",
"b",
"a",
"c"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"b",
"a",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_a",
"partition_b"
],
"a_internal_order": [
"b",
"c",
"a"
],
"naive_route": [
"b",
"c",
"a",
"target"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"b",
"c",
"a",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "b",
"exact_score": "124/3843",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 3
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 2
}
},
{
"doc_id": "c",
"exact_score": "63/1984",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_b",
"partition_a"
],
"a_internal_order": [
"b",
"c",
"a"
],
"naive_route": [
"target",
"b",
"c",
"a"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"b",
"c",
"a"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "b",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 3
}
},
{
"doc_id": "a",
"exact_score": "63/1984",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 2
}
},
{
"doc_id": "c",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 4
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_a",
"partition_b"
],
"a_internal_order": [
"c",
"a",
"b"
],
"naive_route": [
"c",
"a",
"b",
"target"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"c",
"a",
"b",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "c",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 4
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_b",
"partition_a"
],
"a_internal_order": [
"c",
"a",
"b"
],
"naive_route": [
"target",
"c",
"a",
"b"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"c",
"a",
"b"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 2
}
},
{
"doc_id": "c",
"exact_score": "63/1984",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 4
}
},
{
"doc_id": "b",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 3
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_a",
"partition_b"
],
"a_internal_order": [
"c",
"b",
"a"
],
"naive_route": [
"c",
"b",
"a",
"target"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"c",
"b",
"a",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "c",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 4
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 3
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
},
{
"arrival": [
"partition_b",
"partition_a"
],
"a_internal_order": [
"c",
"b",
"a"
],
"naive_route": [
"target",
"c",
"b",
"a"
],
"naive_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"c",
"b",
"a"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "63/1984",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 2
}
},
{
"doc_id": "c",
"exact_score": "63/1984",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 4
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
}
]
},
"merged_route": [
"target",
"a",
"b",
"c"
],
"merged_fusion": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
}
],
"permutations_checked": 12,
"single_sorted_batch": {
"arrival": [
"a",
"b",
"c"
],
"merged": [
"a",
"b",
"c"
]
},
"cut_before_merge": {
"kept_ids": [
"a",
"b",
"c"
],
"target_in_first_route": false,
"target_still_in_second_route": true,
"fusion_after_lost_first_route_contribution": {
"candidates": {
"partitioned_fixture": [
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "a",
"exact_score": "123/3782",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 4
}
},
{
"doc_id": "target",
"exact_score": "1/61",
"ranks": {
"second_fixture": 1
}
}
]
},
"warning": "second route may still retrieve target; missing first-route evidence is not restored"
},
"merge_then_route_window_1": {
"candidates": {
"partitioned_fixture": [
"target"
],
"second_fixture": [
"target"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
}
]
},
"naive_then_route_window_1": {
"candidates": {
"partitioned_fixture": [
"a"
],
"second_fixture": [
"target"
]
},
"hits": [
{
"doc_id": "a",
"exact_score": "1/61",
"ranks": {
"partitioned_fixture": 1
}
}
]
},
"limits": [
"Same-contract label is only a declared fixture assumption, not proof of comparable real scores",
"Never sort unrelated BM25 and vector scores together",
"Separate-list RRF is another valid design with different semantics; not implemented here",
"No async scheduler, WeKnora deployment, Elasticsearch instance, embeddings, ANN or reranker tested",
"Only supplied candidates are ordered; missing candidates are not recovered"
]
}
固定预期值校验输出
{
"literal_oracle_passed": true,
"permutations_checked": 12,
"bad": {
"candidates": {
"partitioned_fixture": [
"a",
"b",
"c",
"target"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "a",
"exact_score": "123/3782",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 2
}
},
{
"doc_id": "target",
"exact_score": "125/3904",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 1
}
},
{
"doc_id": "b",
"exact_score": "125/3906",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "127/4032",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 4
}
}
]
},
"good": {
"candidates": {
"partitioned_fixture": [
"target",
"a",
"b",
"c"
],
"second_fixture": [
"target",
"a",
"b",
"c"
]
},
"hits": [
{
"doc_id": "target",
"exact_score": "2/61",
"ranks": {
"partitioned_fixture": 1,
"second_fixture": 1
}
},
{
"doc_id": "a",
"exact_score": "1/31",
"ranks": {
"partitioned_fixture": 2,
"second_fixture": 2
}
},
{
"doc_id": "b",
"exact_score": "2/63",
"ranks": {
"partitioned_fixture": 3,
"second_fixture": 3
}
},
{
"doc_id": "c",
"exact_score": "1/32",
"ranks": {
"partitioned_fixture": 4,
"second_fixture": 4
}
}
]
}
}
七次真实源代码变体结果
{
"source_mutants_executed": 7,
"all_detected": true,
"baseline_probe_sha256": "23477c3e48d340eb27eb3fc456465f4c89eb1b8a900d52fb85f5efbc5e65a1e0",
"mutants": [
{
"mutant": "arrival_as_rank",
"old_source": "sorted(rows, key=lambda h: (h.score.copy_negate(), h.doc_id))",
"new_source": "rows",
"mutated_probe_sha256": "9f4aa9e951af000675b3d4304635652c4e5a703be8623a17db3d70f120821470",
"test_exit_code": 1,
"failed_named_tests": [
"test_all_12_fixture_permutations_preserve_correct_fusion",
"test_empty_batch_and_no_mutation",
"test_exact_contributions_explain_flip",
"test_exact_decimal_order_and_stable_ties",
"test_local_sorted_batches_are_not_global_sorted",
"test_report_is_deterministic",
"test_route_window_and_display_size_remain_distinct"
],
"detected": true
},
{
"mutant": "wrong_score_direction",
"old_source": "(h.score.copy_negate(), h.doc_id)",
"new_source": "(h.score, h.doc_id)",
"mutated_probe_sha256": "39c027932d350d61468e982df4de99f440f3f5ab1761e67508506ff245145800",
"test_exit_code": 1,
"failed_named_tests": [
"test_all_12_fixture_permutations_preserve_correct_fusion",
"test_empty_batch_and_no_mutation",
"test_exact_contributions_explain_flip",
"test_exact_decimal_order_and_stable_ties",
"test_local_sorted_batches_are_not_global_sorted",
"test_report_is_deterministic",
"test_route_window_and_display_size_remain_distinct",
"test_single_sorted_batch_is_noop"
],
"detected": true
},
{
"mutant": "unstable_equal_scores",
"old_source": "(h.score.copy_negate(), h.doc_id)",
"new_source": "(h.score.copy_negate(),)",
"mutated_probe_sha256": "1ffeabb6f585882518501e4b63872b4c4faac0136b89caa81b6212e726caae2f",
"test_exit_code": 1,
"failed_named_tests": [
"test_exact_decimal_order_and_stable_ties"
],
"detected": true
},
{
"mutant": "skip_score_contract",
"old_source": "if batch.score_contract != expected_contract:",
"new_source": "if False:",
"mutated_probe_sha256": "94ac914d1f70b98771fae67700d6396f906a3daf92eb9abfc5ea5daa53a06070",
"test_exit_code": 1,
"failed_named_tests": [
"test_other_score_contract_is_rejected"
],
"detected": true
},
{
"mutant": "cut_before_ordering",
"old_source": "sorted(rows, key=lambda h: (h.score.copy_negate(), h.doc_id))",
"new_source": "sorted(rows[:3], key=lambda h: (h.score.copy_negate(), h.doc_id))",
"mutated_probe_sha256": "8c132a7513135726b334543ca4e44c4a6861b644c44f25a39e0d55a0f068da6b",
"test_exit_code": 1,
"failed_named_tests": [
"test_all_12_fixture_permutations_preserve_correct_fusion",
"test_empty_batch_and_no_mutation",
"test_exact_contributions_explain_flip",
"test_local_sorted_batches_are_not_global_sorted",
"test_report_is_deterministic",
"test_route_window_and_display_size_remain_distinct"
],
"detected": true
},
{
"mutant": "rounded_decimal_key",
"old_source": "(h.score.copy_negate(), h.doc_id)",
"new_source": "(-h.score, h.doc_id)",
"mutated_probe_sha256": "7d50975a39aa0cd194a25dbae9ee6da224ec7b667eff319918188d046c2ec49d",
"test_exit_code": 1,
"failed_named_tests": [
"test_exact_decimal_order_and_stable_ties"
],
"detected": true
},
{
"mutant": "changed_rrf_constant",
"old_source": "DOCS, {}, k=60, window=window, size=size)",
"new_source": "DOCS, {}, k=61, window=window, size=size)",
"mutated_probe_sha256": "d81a9baba863c956cfa9ef5fe87e85563d3c382cd2b5a3f07a2399e51d756686",
"test_exit_code": 1,
"failed_named_tests": [
"test_all_12_fixture_permutations_preserve_correct_fusion",
"test_early_cut_cannot_restore_missing_route_evidence",
"test_exact_contributions_explain_flip",
"test_only_arrival_order_changes_naive_top1",
"test_route_window_and_display_size_remain_distinct"
],
"detected": true
}
]
}
这次新增实验没有运行 embedding、实际检索服务、ANN、Go、WeKnora、Elasticsearch、重排器或 Agent,也没有检索质量、性能与生产修复结论。前文的 SQLite FTS5 基线另行重跑,不把它算作这次人工分区夹具的真实检索结果。新增图经离线原生 SVG 渲染检查;页面未做浏览器视觉验证。
Elasticsearch 官方 RRF 参考(2026-10-13 核对)把 rank 定义为文档在相应查询结果中的名次,示例先取得各子检索器的全局 top 结果再融合。它支持这里“输入名次应有明确定义”的区分,不为本教学合并器、真实相关性或通用分区策略背书。
记住这一条就够了:RRF 可以不把原始分数相加,但它收到的每一路必须真的是按既定规则排好的名次;请求返回先后不能代替这个规则。
下载完整源码、文章 HTML、原创 SVG 与输出 ZIP。若阅读器不支持 data 下载,可从上面的源码展开项复制新增文件,并沿用前文的 rrf_demo.py。
第二路运行了向量模型吗?
没有。第二路是明确的人工候选排名,用于验证合并;第一路实际运行 SQLite FTS5。
RRF 会保证精确错误码排第一吗?
不会。本例 code 在关键词路第一,融合后第三。
这个分数能当作回答置信度吗?
不能。RRF 分数是名次贡献之和,不是概率或答案质量判断。
先过滤和先截断为什么不一样?
先截断可能只留下范围外候选,过滤后为空;未取回的合格文档无法由融合器补回。
这是 Elasticsearch 的替代 API 吗?
不是。参数校验、同路去重、重编号和同分规则是本文教学约定,未验证服务接口兼容性。