Skip to content

fix: fix EOF error when decoding columns with empty string or zero po…#155

Open
betterlmy wants to merge 1 commit intoapache:mainfrom
betterlmy:fix-column-decoder-null-handling
Open

fix: fix EOF error when decoding columns with empty string or zero po…#155
betterlmy wants to merge 1 commit intoapache:mainfrom
betterlmy:fix-column-decoder-null-handling

Conversation

@betterlmy
Copy link

Bug 复现与修复报告

一、问题描述

在使用 IoTDB Go Client 查询包含空字符串("")的 TEXT 列时,客户端报 EOF 错误,导致查询失败。同一条 SQL 在 IoTDB 命令行中执行正常。


二、复现步骤

环境: IoTDB 2.0.5

第一步:写入一条 TEXT 列值为空字符串的数据

INSERT INTO root.dev.property.`09354fae7b31c824d5888bcd184af887`.Ew7dSmO5m0PdZt7dU9sytGEyPgUlHH
  (timestamp, mouldNo_value)
VALUES (now(), '');

第二步:用 Go Client 查询该列

sql := "SELECT mouldNo_value FROM root.dev.property.09354fae7b31c824d5888bcd184af887.Ew7dSmO5m0PdZt7dU9sytGEyPgUlHH ORDER BY time DESC LIMIT 2"
dataset, err := session.ExecuteQueryStatement(sql, nil)
// err == nil

for {
    hasNext, err := dataset.Next()
    // err = EOF  <-- 此处报错
}

复现条件:

  • 查询列为单列 TEXT 类型
  • 结果集中最后一行的该列值为空字符串 ""(非 null)

报错信息:

EOF

三、根本原因分析

问题出在 client/column_decoder.goBinaryArrayColumnDecoder.ReadColumn

IoTDB 服务端对 TEXT 类型的每个值序列化格式为:

+---------------+-------+
| value length  | value |
+---------------+-------+
| int32         | bytes |
+---------------+-------+

当值为空字符串时,服务端发送 length = 0,后跟 0 字节内容。

旧代码:

var length int32
binary.Read(reader, binary.BigEndian, &length)

value := make([]byte, length)  // length=0,创建空 slice
_, err = reader.Read(value)    // 对空 slice 调用 Read,reader 恰好处于 EOF 边界时返回 io.EOF

Go 标准库 bytes.Reader.Read 在 reader 已无剩余字节时,即使传入空 slice(读取 0 字节),也会返回 io.EOF。当空字符串恰好是 TsBlock 字节流中最后一个值时,reader.Read([]byte{}) 触发 EOF,导致解码失败。

同时发现的第二个 bug: 当服务端返回 positionCount = 0 的列时,所有类型的 ColumnDecoder 都会调用 deserializeNullIndicators,后者对空 reader 执行 ReadByte(),同样返回 io.EOF


四、修复方案

Bug 1(空字符串 EOF):BinaryArrayColumnDecoder 中,当 length == 0 时直接创建空 Binary,跳过 reader.Read

if length == 0 {
    values[i] = NewBinary([]byte{})
} else {
    value := make([]byte, length)
    _, err = reader.Read(value)
    if err != nil {
        return nil, err
    }
    values[i] = NewBinary(value)
}

Bug 2(positionCount=0 EOF): 在四种 ColumnDecoder 的 ReadColumn 入口处,对 positionCount == 0 提前返回空 Column:

if positionCount == 0 {
    return NewBinaryColumn(0, 0, nil, []*Binary{})
}

五、验证结果

修复前:

迭代结果集失败(第 1 行): EOF
--- FAIL: TestQueryControlSignal

修复后:

--- 第 1 行 --- mouldNo_value (TEXT) = ""
--- 第 2 行 --- mouldNo_value (TEXT) = ""
共返回 2 行
--- PASS: TestQueryControlSignal

六、新增 Feature:SessionDataSet.GetCurrentRowTime()

SessionDataSet 新增 GetCurrentRowTime() 方法,返回当前行的原始时间戳(int64,毫秒级 Unix 时间戳)。

背景: 原有的 GetTimestamp(columnName) 方法返回 time.Time,需要指定列名,且经过时区转换。对于需要直接操作原始时间戳数值的场景(如比较、存储、序列化),需要再做一次转换,不够便捷。

用法:

for {
    hasNext, _ := dataset.Next()
    if !hasNext {
        break
    }
    ts := dataset.GetCurrentRowTime() // 直接获取行时间戳,单位毫秒
    fmt.Println(ts)
}

Copilot AI review requested due to automatic review settings March 9, 2026 14:06
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes two EOF-related bugs in the IoTDB Go Client's column decoder and adds defensive nil-checks to session reconnect code paths. It also adds a convenience method GetCurrentRowTime() to SessionDataSet.

Changes:

  • Fix EOF error when decoding TEXT columns with empty string values (length == 0) by skipping reader.Read for zero-length values in BinaryArrayColumnDecoder
  • Fix EOF error when all column decoders receive positionCount == 0 by adding early return guards before calling deserializeNullIndicators
  • Add nil-checks (err == nil && resp != nil) to reconnect retry paths in ExecuteQueryStatement, ExecuteAggregationQuery, ExecuteAggregationQueryWithLegalNodes, and ExecuteFastLastDataQueryForOnePrefixPath

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
client/column_decoder.go Adds positionCount == 0 early returns for all four decoders and handles length == 0 in BinaryArrayColumnDecoder
client/column_decoder_test.go New test file covering empty string, null+empty string, null interleaving, and zero position count scenarios
client/session.go Adds err == nil && resp != nil nil guards to four reconnect retry paths
client/sessiondataset.go Adds GetCurrentRowTime() convenience wrapper method

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@betterlmy
Copy link
Author

@HTHou 帮忙看一下这个pr💗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants