Skip to content

[test/flink] Cover full data types in UnionRead tests and polish some tests#1167

Merged
leonardBang merged 6 commits into
apache:mainfrom
naivedogger:fulltype-unionread-test
Jun 25, 2025
Merged

[test/flink] Cover full data types in UnionRead tests and polish some tests#1167
leonardBang merged 6 commits into
apache:mainfrom
naivedogger:fulltype-unionread-test

Conversation

@naivedogger

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #1127

Brief change log

Tests

API and Format

Documentation

@naivedogger naivedogger marked this pull request as ready for review June 24, 2025 02:36

@leonardBang leonardBang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @naivedogger for the contribution, I left two comments

Comment on lines +346 to +350
"[+I[false, 1, 2, 3, 4, 5.1, 6.0, string, 0.09, 10, 2023-10-25T12:01:13.182005Z, 2023-10-25T12:01:13.183006, [1, 2, 3, 4], 2025], +I[false, 1, 2, 3, 4, 5.1, 6.0, string, 0.09, 10, 2023-10-25T12:01:13.182005Z, 2023-10-25T12:01:13.183006, [1, 2, 3, 4], 2026], +I[true, 10, 20, 30, 40, 50.1, 60.0, another_string, 0.90, 100, 2023-10-25T12:01:13.200005Z, 2023-10-25T12:01:13.201006, [1, 2, 3, 4], 2025], +I[true, 10, 20, 30, 40, 50.1, 60.0, another_string, 0.90, 100, 2023-10-25T12:01:13.200005Z, 2023-10-25T12:01:13.201006, [1, 2, 3, 4], 2026]]");
} else {
assertThat(paimonSnapshotRows.toString().replace("+U", "+I"))
.isEqualTo(
"[+I[false, 1, 2, 3, 4, 5.1, 6.0, string, 0.09, 10, 2023-10-25T12:01:13.182005Z, 2023-10-25T12:01:13.183006, [1, 2, 3, 4], null], +I[true, 10, 20, 30, 40, 50.1, 60.0, another_string, 0.90, 100, 2023-10-25T12:01:13.200005Z, 2023-10-25T12:01:13.201006, [1, 2, 3, 4], null]]");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can split the long string to multiple lines to make sure it readable

Comment on lines +417 to +347
"[+I[false, 1, 2, 3, 4, 5.1, 6.0, string, 0.09, 10, 2023-10-25T12:01:13.182005Z, 2023-10-25T12:01:13.183006, [1, 2, 3, 4], 2025], +I[false, 1, 2, 3, 4, 5.1, 6.0, string, 0.09, 10, 2023-10-25T12:01:13.182005Z, 2023-10-25T12:01:13.183006, [1, 2, 3, 4], 2026], +I[true, 100, 200, 30, 400, 500.1, 600.0, another_string_2, 9.00, 1000, 2023-10-25T12:01:13.400007Z, 2023-10-25T12:01:13.501008, [5, 6, 7, 8], 2025], +I[true, 100, 200, 30, 400, 500.1, 600.0, another_string_2, 9.00, 1000, 2023-10-25T12:01:13.400007Z, 2023-10-25T12:01:13.501008, [5, 6, 7, 8], 2026]]");
} else {
assertThat(result.toString().replace("+U", "+I"))
.isEqualTo(
"[+I[false, 1, 2, 3, 4, 5.1, 6.0, string, 0.09, 10, 2023-10-25T12:01:13.182005Z, 2023-10-25T12:01:13.183006, [1, 2, 3, 4], null], +I[true, 100, 200, 30, 400, 500.1, 600.0, another_string_2, 9.00, 1000, 2023-10-25T12:01:13.400007Z, 2023-10-25T12:01:13.501008, [5, 6, 7, 8], null]]");
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above.


@ParameterizedTest
@ValueSource(booleans = {false, true})
void testUnionReadFullType(Boolean isPartitioned) throws Exception {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All tests in this class are pretty heavy as they need mini-cluster resources, we can reduce testUnionReadFullType and testPrimaryKeyTable to one for example testPrimaryKeyTableWithFullTypes to save the test cost, and testUnionReadWithTimeStamp can be improve the testUnionReadWithTimeStampAndFullTypes. In this way, we improve our test coverage but not waste unnecessary test cost.

@leonardBang leonardBang force-pushed the fulltype-unionread-test branch from 329a396 to 32abfe6 Compare June 25, 2025 03:49

@leonardBang leonardBang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @naivedogger for the update, LGTM, I append a commit to improve existing tests

@leonardBang leonardBang force-pushed the fulltype-unionread-test branch from bdab959 to 004b6c3 Compare June 25, 2025 07:45
@leonardBang leonardBang merged commit ced8cba into apache:main Jun 25, 2025
4 checks passed
@leonardBang leonardBang changed the title Fulltype unionread test [test/flink] Cover full data types in UnionRead tests and polish some tests Jun 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[test] Add full data type test for flink connector for user case Union Read/ PK table/ Log table and similar cases.

2 participants