Skip to content

MySQL: fix handling of TIMESTAMP when using TEXT COLUMN#36612

Open
martykulma wants to merge 5 commits into
MaterializeInc:mainfrom
martykulma:mz-bugfix
Open

MySQL: fix handling of TIMESTAMP when using TEXT COLUMN#36612
martykulma wants to merge 5 commits into
MaterializeInc:mainfrom
martykulma:mz-bugfix

Conversation

@martykulma
Copy link
Copy Markdown
Contributor

@martykulma martykulma commented May 18, 2026

When using TEXT COLUMN for a timestamp column, Materialize did not have proper handling for timestamps read from binlog that were using either the legacy (pre-5.6) representation MYSQL_TYPE_TIMESTAMP or the post 5.6 MYSQL_TYPE_TIMESTAMP2.

This resulted in user not being able to ingest invalid timestamps (like MySQL's special "zero" value timestamps "0000-00-00 00:00:00")

Fixes SS-151: MySQL: decode fails for TIMESTAMP column when using TEXT COLUMNS

@martykulma martykulma marked this pull request as ready for review May 18, 2026 20:33
@martykulma martykulma requested a review from a team as a code owner May 18, 2026 20:33
@martykulma martykulma requested a review from a team May 18, 2026 20:33
@def-
Copy link
Copy Markdown
Contributor

def- commented May 19, 2026

Copy link
Copy Markdown
Contributor

@def- def- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running bin/mzcompose --find mysql-cdc down && bin/mzcompose --find mysql-cdc run cdc text-columns-timestamp.td shows me scary temporary retraction errors:

> SELECT id, created_at FROM products ORDER BY id;
executing query failed

Caused by:
    0: db error
    1: ERROR: Invalid data in source, saw retractions (1) for row that does not exist: [Int32(5), String("1970-01-01 00:00:00")]
rows didn't match; sleeping to see if dataflow catches up 🕑 50ms
rows match; continuing at ts 1779156671.0067012, took 0.949501151s

If we change the handling of the binlog decoding we also have to make the same change in the snapshot code!

@martykulma
Copy link
Copy Markdown
Contributor Author

Running bin/mzcompose --find mysql-cdc down && bin/mzcompose --find mysql-cdc run cdc text-columns-timestamp.td shows me scary temporary retraction errors:

Definitely a scary error! I ran this over a dozen times yesterday (against 9.5, 8.4, 8.0), and again for each version this morning, but still no failures.

I definitely missed some thing in looking at this yesterday. I just came across https://dev.mysql.com/doc/refman/8.0/en/datetime.html -- and this has got me worried because the behavior or DATETIME and TIMESTAMP are different, but in MZ we don't distinguish -- which is a problem!

@def-
Copy link
Copy Markdown
Contributor

def- commented May 19, 2026

I think they are caused by this:

If we change the handling of the binlog decoding we also have to make the same change in the snapshot code!

@martykulma
Copy link
Copy Markdown
Contributor Author

I think they are caused by this:

If we change the handling of the binlog decoding we also have to make the same change in the snapshot code!

Seems like the most likely case (the snapshot case is decoding as Value::Date, where binlog will decode as Value::Bytes). I'm not sure why I could not repro the error locally (seems like it should happen fairly consistently). PR is updated with additional settings.

@martykulma martykulma requested a review from def- May 19, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants