📝 18 新 Issues
✅ 8 关闭
✨ 6 新 PRs
🎉 9 合并
🔥 核心要点
- 🎉 PR #3238 合并 Spark lake table 谓词下推功能
- 🎉 PR #2972 合并 Flink TableSource Watermark 下推功能
- 🐛 PR #3208 修复空 datalake 表批查询报错问题
- ✨ 用户 fhan688 新增 10+ 个 Hudi 集成相关 Issue (FIP-24)
💬 Issue/PR 动态
Issue 讨论
Issue #3254 FIP-24: Support tiering Fluss data to Hudi
- @fhan688: 列出了 Hudi 集成任务的子 Issue 清单
- @MaheshS08: 询问是否可以参与 Hudi 集成协作
- @fhan688: 解释代码已实现，仅是拆分提交
Issue #3229 [server] Add metrics of log startOffset
- @luoyuxia: 建议无远程日志时指标返回-1并更新文档
- @zuston: 认同建议并采纳修改
Issue #3227 [spark] Fix empty projection causing Invalid metadata length for COUNT(*)/COUNT(1)
- @beryllw: 建议增加过滤器和分区表测试用例
Issue #3206 [docs] AskAi fails to load on Fluss website
- @wuchong: 暂时回滚 AI 组件，将申请正式授权
PR Review
PR #3238 [spark] lake table predicate pushdown
- @luoyuxia: 建议在日志中打印已接受的谓词
- @Yohahaha: 建议调整日志级别
PR #3260 [spark] Add scan.max.records.per.partition config
- @fresh-borzoni: 指出 earliest 模式下可能存在 Bug
PR #2920 [lake/tiering] add table dropped handling and nullable write results
- @luoyuxia: 讨论取消状态的语义及 abort 清理逻辑
PR #3030 [server] Implement JBOD Phase 1 Local Multi-Directory Support
- @hanliu0830: 修复重复线程池及锁冲突等问题
📝 新建 Issue/PR
Issues
- #3284 Flink Union Read Test of Hudi @fhan688
- #3283 Introduce Hudi Compaction @fhan688
- #3282 Introduce Hudi LakeCommitter @fhan688
- #3280 Introduce Hudi LakeWriter to support tiering table @fhan688
- #3279 Introduce Tag field in TieringSplit @fhan688
- #3278 Hudi source sortedReader and PredicateConverter @fhan688
- #3277 Introduce Hudi source reader @fhan688
- #3276 Introduce Hudi source split planner @fhan688
- #3275 Introduce Hudi LakeCatalog to create table @fhan688
- #3274 Introduce HudiBucketingFunction for bucket strategy @fhan688
- #3273 Trim Flink multi-version CI tests to reduce pipeline time @luoyuxia
- #3270 [flink] COUNT(column) aggregate pushdown produces incorrect results for nullable columns @beryllw
- #3269 Optional channel selection for source split in FlinkSourceEnumerator @zuston
- #3267 [ci] Optimize the log prompt of NoticeFileChecker @zhuyufeng0809
- #3265 [flink] Support hint-based watermark configuration for Flink connector @swuferhong
- #3262 [client] Refactor RemoteLogDownloader to use chunked streaming instead of downloading whole segment file @swuferhong
- #3261 Tiering hang due to the BucketOffsetsRetrieverImpl.listOffsets @zuston
- #3259 [tiering] Slow manifest compaction during lake commit blocks other tables' tiering progress @beryllw
Pull Requests
- #3281 [catalog] Preserve exception cause when rethrowing catalog exceptions @XuQianJin-Stars
- #3272 [flink] add split assignment mechanism w/o partitionId @zuston
- #3271 [flink] Fix COUNT(column) aggregate pushdown to reject nullable columns @beryllw
- #3266 [flink] Support hint-based watermark configuration for Flink connector @swuferhong
- #3263 [client] Refactor RemoteLogDownloader to use chunked file append instead of downloading whole log file @swuferhong
- #3260 [spark] Add scan.max.records.per.partition config to split log table input partitions @Yohahaha
✅ 关闭 Issue/PR
已关闭 Issues
- #3267 [ci] Optimize the log prompt of NoticeFileChecker @zhuyufeng0809
- #3261 Tiering hang due to the BucketOffsetsRetrieverImpl.listOffsets @zuston
- #3239 [spark] lake table predicate pushdown @fresh-borzoni
- #3207 [flink] Batch query on empty datalake-enabled table should return 0 rows instead of failing with UnsupportedOperationException @binary-signal
- #2971 [flink] FlinkTableSource should implement SupportsWatermarkPushDown @LiebingYu
- #2624 Unstable Test: KvTabletSnapshotTargetTest.testAddToSnapshotToStoreFail @loserwang1024
- #2497 Support report the number of lake tables to metric @luoyuxia
- #2418 Upgrade Fluss cluster cause tiering service stuck @luoyuxia
已合并 PRs
- #3285 Update team @fresh-borzoni
- #3268 [ci] Optimize the log prompt of NoticeFileChecker @zhuyufeng0809
- #3264 [kv] Fix flaky KvTabletSnapshotTargetTest.testAddToSnapshotToStoreFail @Kaixuan-Duan
- #3238 [spark] lake table predicate pushdown @fresh-borzoni
- #3229 [server] Add metrics of log startOffset @zuston
- #3216 Revert "[fluss/ai agent] enable ai doc agent on website (#3152)" @MehulBatra
- #3208 [flink] Fix batch query on empty datalake-enabled table to return 0 rows instead of failing @matrixsparse
- #3043 [client] release pending fetch semaphore on discarding @zuston
- #2972 [flink] Implement SupportsWatermarkPushDown for FlinkTableSource @LiebingYu