Skip to content

Modernize cdc#12720

Open
dveeden wants to merge 3 commits into
pingcap:masterfrom
dveeden:modern_cdc
Open

Modernize cdc#12720
dveeden wants to merge 3 commits into
pingcap:masterfrom
dveeden:modern_cdc

Conversation

@dveeden

@dveeden dveeden commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: ref #12691

What is changed and how it works?

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

None

@ti-chi-bot ti-chi-bot Bot added the release-note-none Denotes a PR that doesn't merit a release note. label Jun 16, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign 5kbpers, yudongusa for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jun 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors and modernizes the codebase by replacing interface{} with any, adopting for range loops for index-only iterations, utilizing Go 1.21+ slices and built-in min/max functions, and simplifying goroutine spawning in tests with wg.Go. The review feedback recommends using fmt.Fprintf with strings.Builder in cdc/model/owner.go to avoid allocations and prevent nil pointer dereferences, simplifying the allowed check in cdc/api/middleware/middleware.go with slices.Contains, and adding a defensive check in placeHolder within cdc/sink/dmlsink/txn/mysql/dml.go to prevent a runtime panic when n <= 0.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread cdc/model/owner.go
Comment on lines 263 to 265
for id, sinfo := range p {
s += fmt.Sprintf("%s: %+v,", id, *sinfo)
s.WriteString(fmt.Sprintf("%s: %+v,", id, *sinfo))
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using fmt.Fprintf directly with the strings.Builder avoids the intermediate string allocation caused by fmt.Sprintf. Additionally, we should defensively check if sinfo is nil before dereferencing it to prevent potential runtime panics.

Suggested change
for id, sinfo := range p {
s += fmt.Sprintf("%s: %+v,", id, *sinfo)
s.WriteString(fmt.Sprintf("%s: %+v,", id, *sinfo))
}
for id, sinfo := range p {
if sinfo == nil {
fmt.Fprintf(&s, "%s: nil,", id)
} else {
fmt.Fprintf(&s, "%s: %+v,", id, *sinfo)
}
}

Comment thread cdc/api/middleware/middleware.go Outdated
Comment on lines 156 to 160
allowed := false
serverCfg := config.GetGlobalServerConfig()
for _, user := range serverCfg.Security.ClientAllowedUser {
if user == username {
allowed = true
break
}
if slices.Contains(serverCfg.Security.ClientAllowedUser, username) {
allowed = true
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The allowed boolean variable can be directly initialized with the result of slices.Contains, making the code cleaner and more concise.

Suggested change
allowed := false
serverCfg := config.GetGlobalServerConfig()
for _, user := range serverCfg.Security.ClientAllowedUser {
if user == username {
allowed = true
break
}
if slices.Contains(serverCfg.Security.ClientAllowedUser, username) {
allowed = true
}
serverCfg := config.GetGlobalServerConfig()
allowed := slices.Contains(serverCfg.Security.ClientAllowedUser, username)

Comment on lines 198 to 204
func placeHolder(n int) string {
var builder strings.Builder
builder.Grow((n-1)*2 + 1)
for i := 0; i < n; i++ {
for i := range n {
if i > 0 {
builder.WriteString(",")
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If n is 0 (e.g., when there are no columns to update or insert), (n-1)*2 + 1 evaluates to -1. Passing a negative value to builder.Grow will cause a runtime panic. Adding a defensive check for n <= 0 prevents this panic.

func placeHolder(n int) string {
	if n <= 0 {
		return ""
	}
	var builder strings.Builder
	builder.Grow((n-1)*2 + 1)
	for i := range n {
		if i > 0 {
			builder.WriteString(",")
		}

@dveeden

dveeden commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

/cc @kennytm @GMHDBJD

@ti-chi-bot ti-chi-bot Bot requested review from GMHDBJD and kennytm June 16, 2026 07:37
@dveeden

dveeden commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

/test pull-cdc-integration-kafka-test

- owner.go: write ProcessorsInfos.String() via fmt.Fprintf and guard
  against a nil *TaskStatus to avoid a panic on dereference
- middleware.go: initialize allowed directly from slices.Contains
- dml.go: guard placeHolder against n <= 0 so builder.Grow never gets
  a negative value

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dveeden

dveeden commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

/retest

2 similar comments
@dveeden

dveeden commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@kennytm

kennytm commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

/retest

@kennytm

kennytm commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

The error seems legit

 === FAIL: cdc/sink/dmlsink/txn/mysql TestHolderString (0.00s)
    mysql_test.go:767: 
        	Error Trace:	/home/prow/go/src/github.com/pingcap/tiflow/cdc/sink/dmlsink/txn/mysql/mysql_test.go:767
        	Error:      	func (assert.PanicTestFunc)(0xb86c120) should panic
        	            		Panic value:	<nil>
        	Test:       	TestHolderString
        	Messages:   	strings.Builder.Grow: negative count

The modernization added a defensive `if n <= 0 { return "" }` guard to
placeHolder, changing its long-standing contract (panic on n <= 0). The
pre-existing TestHolderString still asserts that placeHolder(0) and
placeHolder(-1) panic, so unit tests failed. The only production caller
passes len(columnNames), which is guaranteed >= 1, so the guard added no
value while breaking the test. Revert to the original fail-fast behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dveeden

dveeden commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

/test pull-verify

@ti-chi-bot

ti-chi-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

@dveeden: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test pull-build
/test pull-cdc-integration-kafka-test
/test pull-cdc-integration-mysql-test
/test pull-cdc-integration-pulsar-test
/test pull-cdc-integration-storage-test
/test pull-check
/test pull-dm-compatibility-test
/test pull-dm-integration-test
/test pull-dm-integration-test-next-gen
/test pull-error-log-review
/test pull-syncdiff-integration-test
/test pull-unit-test-cdc
/test pull-verify
/test wip-pull-unit-test-dm
/test wip-pull-unit-test-engine

Use /test all to run the following jobs that were automatically triggered:

pingcap/tiflow/ghpr_verify
pingcap/tiflow/pull_cdc_integration_kafka_test
pingcap/tiflow/pull_cdc_integration_pulsar_test
pingcap/tiflow/pull_cdc_integration_storage_test
pingcap/tiflow/pull_cdc_integration_test
pingcap/tiflow/pull_dm_compatibility_test
pingcap/tiflow/pull_syncdiff_integration_test
pull-build
pull-check
pull-error-log-review
pull-unit-test-cdc
Details

In response to this:

/test pull-verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@dveeden

dveeden commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants