fix: :bug:invalid TypeDatetime 0000-00-00 00:00:00.000 by tiansin · Pull Request #285 · XiaoMi/Gaea

tiansin · 2025-07-27T15:11:16Z

Session write response error, connId: 10007, err: invalid TypeDatetime 0000-00-00 00:00:00.000

gongna-au · 2025-07-28T02:40:40Z

gongna-au · 2025-07-28T02:55:28Z

@tiansin hi~ What is the specific type of the field that has this problem? MySQL Timestamp should not store fractional seconds. I'm trying to reproduce this step

tiansin · 2025-07-28T08:15:00Z

@tiansin hi~ What is the specific type of the field that has this problem? MySQL Timestamp should not store fractional seconds. I'm trying to reproduce this step

package main

import (
	"database/sql"
	"fmt"
	"log"
	"time"

	"gorm.io/driver/mysql"
	"gorm.io/gorm"
)

type User struct {
	ID        uint `gorm:"primaryKey"`
	Name      string
	CreatedAt time.Time
}

func main() {
	dsnRoot := "admin:123456@tcp(127.0.0.1:3306)/?parseTime=true"
	sqlDB, err := sql.Open("mysql", dsnRoot)
	if err != nil {
		log.Fatal(err)
	}
	defer sqlDB.Close()

	dbName := "gorm_test"
	_, err = sqlDB.Exec("CREATE DATABASE IF NOT EXISTS " + dbName)
	if err != nil {
		log.Fatal("create database error:", err)
	}

	dsn := fmt.Sprintf("admin:123456@tcp(127.0.0.1:3306)/%s?parseTime=true", dbName)
	db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
	if err != nil {
		log.Fatal("connect gorm error:", err)
	}

	if err := db.AutoMigrate(&User{}); err != nil {
		log.Fatal("auto migrate error:", err)
	}

	db.Create(&User{Name: "Alice"})

	tableName := "users"
	var result struct {
		Table       string
		CreateTable string
	}
	db.Raw("SHOW CREATE TABLE " + tableName).Scan(&result)
	fmt.Println(result.CreateTable)
}

感谢回复, 最后我发现这个问题原来是由gorm造成的.如果不设置'DisableDatetimePrecision'参数就会自动创建带毫秒精度的时间.
另外无效时间的判断 v == "0000-00-00 00:00:00" 代码可以优化下吗,因为这里一但失败数据库就会报错重连.
https://github.com/go-gorm/mysql/blob/d744156745998478ffa3c5395c3336ae6f0b0c0c/README.md?plain=1#L29

gongna-au · 2025-08-01T08:54:58Z

感谢回复, 最后我发现这个问题原来是由gorm造成的.如果不设置'DisableDatetimePrecision'参数就会自动创建带毫秒精度的时间. 另外无效时间的判断 v == "0000-00-00 00:00:00" 代码可以优化下吗,因为这里一但失败数据库就会报错重连. https://github.com/go-gorm/mysql/blob/d744156745998478ffa3c5395c3336ae6f0b0c0c/README.md?plain=1#L29

@tiansin hi~ 我正尝试复现以获取到invalid TypeDatetime Error 但是似乎没有任何错误（例如连接断开）

package main

import (
	"database/sql"
	"fmt"
	"log"
	"time"

	"gorm.io/driver/mysql"
	"gorm.io/gorm"
)

type User struct {
	ID        uint `gorm:"primaryKey"`
	Name      string
	CreatedAt time.Time
}

func main() {
	dsnRoot := "superroot:superroot@tcp(127.0.0.1:13306)/?parseTime=true"
	sqlDB, err := sql.Open("mysql", dsnRoot)
	if err != nil {
		log.Fatal(err)
	}
	defer sqlDB.Close()

	dbName := "gorm_test"
	_, err = sqlDB.Exec("CREATE DATABASE IF NOT EXISTS " + dbName)
	if err != nil {
		log.Fatal("create database error:", err)
	}
	time.Sleep(5 * time.Second)
	var datetimePrecision = 3
	db, err := gorm.Open(mysql.New(mysql.Config{
		DSN:                       "superroot:superroot@tcp(127.0.0.1:13306)/gorm_test?charset=utf8&parseTime=True&loc=Local", // data source name, refer https://github.com/go-sql-driver/mysql#dsn-data-source-name
		DefaultStringSize:         256,                                                                                        // add default size for string fields, by default, will use db type `longtext` for fields without size, not a primary key, no index defined and don't have default values
		DisableDatetimePrecision:  false,                                                                                      // disable datetime precision support, which not supported before MySQL 5.6
		DefaultDatetimePrecision:  &datetimePrecision,                                                                         // default datetime precision
		DontSupportRenameIndex:    true,                                                                                       // drop & create index when rename index, rename index not supported before MySQL 5.7, MariaDB
		DontSupportRenameColumn:   true,                                                                                       // use change when rename column, rename rename not supported before MySQL 8, MariaDB
		SkipInitializeWithVersion: false,                                                                                      // smart configure based on used version
	}), &gorm.Config{})

	if err != nil {
		fmt.Println("open database error:", err)
	}
	if err := db.AutoMigrate(&User{}); err != nil {
		log.Fatal("auto migrate error:", err)
	}

	db.Create(&User{Name: "Alice"})

	tableName := "users"
	var result struct {
		Table       string
		CreateTable string
	}
	db.Raw("SHOW CREATE TABLE " + tableName).Scan(&result)
	fmt.Println("结果是", result.CreateTable)
	time.Sleep(30 * time.Second)
}

执行SQL成功到程序退出之前的日志如下⬇️

[2025-08-01 16:52:01.713] [INFO] [900000001] OK - 1.5ms - ns=test_demo, superroot@127.0.0.1:54534->127.0.0.1:3349/gorm_test, connect_id=10004, mysql_connect_id=165, prepare=false, transaction=false|SHOW CREATE TABLE users
[2025-08-01 16:52:31.714] [INFO] [900000001] Quit - conn_id=10003, ns=test_demo, superroot@127.0.0.1:54530/
[2025-08-01 16:52:31.714] [INFO] [900000001] Session Close - conn_id=10003, ns=test_demo, superroot@127.0.0.1:54530/, capability: 696965
[2025-08-01 16:52:31.714] [INFO] [900000001] Session Close - conn_id=10004, ns=test_demo, superroot@127.0.0.1:54534/gorm_test, capability: 696973

出现Session Close的原因是因为程序退出，除此之外没看到执行SQL出错。

tiansin · 2025-08-02T09:59:37Z

dsnRoot := "admin:123456@tcp(127.0.0.1:3306)/?parseTime=true"

@gongna-au 抱歉,之前的测试代码只有为什么会产生时间精度的问题,并没有SQL出错的原因

package main

import (
	"database/sql"
	"fmt"
	"log"
	"time"

	"gorm.io/driver/mysql"
	"gorm.io/gorm"
)

type User struct {
	ID         uint `gorm:"primaryKey"`
	Name       string
	TestNullAt time.Time
}

func main() {
	// dsnRoot := "superroot:superroot@tcp(127.0.0.1:13306)/?parseTime=true"
	dsnRoot := "admin:123456@tcp(ubuntu:13306)/?parseTime=true"
	sqlDB, err := sql.Open("mysql", dsnRoot)
	if err != nil {
		log.Fatal(err)
	}
	defer sqlDB.Close()

	dbName := "gorm_test"
	_, err = sqlDB.Exec("CREATE DATABASE IF NOT EXISTS " + dbName)
	if err != nil {
		log.Fatal("create database error:", err)
	}
	// time.Sleep(5 * time.Second)
	datetimePrecision := 3
	db, err := gorm.Open(mysql.New(mysql.Config{
		// DSN:                       "superroot:superroot@tcp(127.0.0.1:13306)/gorm_test?charset=utf8&parseTime=True&loc=Local", // data source name, refer https://github.com/go-sql-driver/mysql#dsn-data-source-name
		DSN:                       "admin:123456@tcp(ubuntu:13306)/gorm_test?charset=utf8&parseTime=True&loc=Local", // data source name, refer https://github.com/go-sql-driver/mysql#dsn-data-source-name
		DefaultStringSize:         256,                                                                              // add default size for string fields, by default, will use db type `longtext` for fields without size, not a primary key, no index defined and don't have default values
		DisableDatetimePrecision:  false,                                                                            // disable datetime precision support, which not supported before MySQL 5.6
		DefaultDatetimePrecision:  &datetimePrecision,                                                               // default datetime precision
		DontSupportRenameIndex:    true,                                                                             // drop & create index when rename index, rename index not supported before MySQL 5.7, MariaDB
		DontSupportRenameColumn:   true,                                                                             // use change when rename column, rename rename not supported before MySQL 8, MariaDB
		SkipInitializeWithVersion: false,                                                                            // smart configure based on used version
	}), &gorm.Config{})
	if err != nil {
		fmt.Println("open database error:", err)
	}

	// 执行下边这句就会报错
	// Session write response error, connId: 10018, err: invalid TypeDatetime 0000-00-00 00:00:00.000
	if err := db.AutoMigrate(&User{}); err != nil {
		log.Fatal("auto migrate error:", err)
	}

	db.Create(&User{Name: "Alice"})

	tableName := "users"
	var result struct {
		Table       string
		CreateTable string
	}
	db.Raw("SHOW CREATE TABLE " + tableName).Scan(&result)
	fmt.Println("结果是", result.CreateTable)
	time.Sleep(30 * time.Second)
}

namespace配置如下:

{
    "name": "test_namespace_1",
    "online": true,
    "read_only": false,
    "allowed_dbs": {
        "information_schema": true,
        "gorm_test": true
    },
    "slow_sql_time": "1000",
    "black_sql": [
        ""
    ],
    "allowed_ip": null,
    "slices": [
        {
            "name": "slice-0",
            "user_name": "admin",
            "password": "123456",
            "master": "192.168.1.58:3306",
            "slaves": ["192.168.1.58:3306"],
            "statistic_slaves": null,
            "capacity": 12,
            "max_capacity": 24,
            "idle_timeout": 60,
            "init_connect": ""
        }
    ],
    "shard_rules": null,
    "users": [
        {
            "user_name": "admin",
            "password": "123456",
            "namespace": "test_namespace_1",
            "rw_flag": 2,
            "rw_split": 1,
            "other_property": 0
        }
    ],
    "default_slice": "slice-0",
    "global_sequences": null,
    "max_sql_execute_time": 0,
    "max_sql_result_size": 1000000
}

上述代码第一次没有报错, 但再次运行时会就会抛出错误.

gongna-au · 2025-12-17T12:56:11Z

@tiansin hi，Thank you very much for giving us advance warning about this problem. I couldn't reproduce the issue before, but we recently encountered the same problem in our production environment, and this time I successfully reproduced it. I think the following fix would be a more robust solution.

case TypeDatetime, TypeTimestamp:
			encoded, err := encodeDateTime(v)
			if err != nil {
				mysqlTypeStr := "TypeDatetime"
				if fieldType == TypeTimestamp {
					mysqlTypeStr = "TypeTimestamp"
				}
				return nil, fmt.Errorf("invalid %s %s: %v", mysqlTypeStr, v, err)
			}
			t = encoded

// encodeDateTime 将时间字符串编码为 MySQL 协议的二进制格式
func encodeDateTime(v string) ([]byte, error) {
	var t []byte

	// 1. 处理所有形式的零值 (0000-00-00...)
	if strings.HasPrefix(v, "0000-00-00") {
		// 根据 MySQL 协议，长度为 0 的日期字节表示零值
		return append(t, 0), nil
	}

	// 2. 匹配解析格式
	var layout string
	switch {
	case len(v) == 10: // 2024-01-01
		layout = "2006-01-02"
	case len(v) == 19: // 2024-01-01 12:00:00
		layout = "2006-01-02 15:04:05"
	case len(v) > 19: // 2024-01-01 12:00:00.000...
		layout = "2006-01-02 15:04:05.999999999"
	default:
		return nil, fmt.Errorf("invalid time string length: %s", v)
	}

	ts, err := time.Parse(layout, v)
	if err != nil {
		return nil, fmt.Errorf("parse time error: %v (value: %s)", err, v)
	}

	// 3. 构建二进制报文
	// 长度 11 表示包含: year(2), month(1), day(1), hour(1), minute(1), second(1), microsecond(4)
	t = append(t, 11)
	t = AppendUint16(t, uint16(ts.Year()))
	t = append(t, byte(int(ts.Month())), byte(ts.Day()), byte(ts.Hour()), byte(ts.Minute()), byte(ts.Second()))

	// 计算微秒 (1 毫秒 = 1000 微秒)
	microseconds := uint32(ts.Nanosecond() / 1000)
	t = AppendUint32(t, microseconds)
	return t, nil
}

tiansin · 2025-12-18T09:15:36Z

@gongna-au 感谢回复,我也认为你的处理方案比较完美,我把这个pr关闭掉.

fix: 🐛

5d8e7fb

Session write response error, connId: 10007, err: invalid TypeDatetime 0000-00-00 00:00:00.000

update encoding.go

36fdec6

tiansin closed this Dec 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: 🐛invalid TypeDatetime 0000-00-00 00:00:00.000#285

fix: 🐛invalid TypeDatetime 0000-00-00 00:00:00.000#285
tiansin wants to merge 2 commits intoXiaoMi:mainfrom
tiansin:main

tiansin commented Jul 27, 2025

Uh oh!

gongna-au commented Jul 28, 2025 •

edited

Loading

Uh oh!

gongna-au commented Jul 28, 2025

Uh oh!

tiansin commented Jul 28, 2025

Uh oh!

gongna-au commented Aug 1, 2025 •

edited

Loading

Uh oh!

tiansin commented Aug 2, 2025

Uh oh!

gongna-au commented Dec 17, 2025 •

edited

Loading

Uh oh!

tiansin commented Dec 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

tiansin commented Jul 27, 2025

Uh oh!

gongna-au commented Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how it works?

Check List

Uh oh!

gongna-au commented Jul 28, 2025

Uh oh!

tiansin commented Jul 28, 2025

Uh oh!

gongna-au commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tiansin commented Aug 2, 2025

Uh oh!

gongna-au commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tiansin commented Dec 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gongna-au commented Jul 28, 2025 •

edited

Loading

gongna-au commented Aug 1, 2025 •

edited

Loading

gongna-au commented Dec 17, 2025 •

edited

Loading