[fix](test) Remove wrapper_field dependency from unit tests#60530
Merged
yiguolei merged 10 commits intoapache:masterfrom Feb 7, 2026
Merged
[fix](test) Remove wrapper_field dependency from unit tests#60530yiguolei merged 10 commits intoapache:masterfrom
yiguolei merged 10 commits intoapache:masterfrom
Conversation
Update FromStringTest, TextSerde, and CsvSerde unit tests to remove dependency on the deleted wrapper_field.h header. Replace WrapperField usage with hardcoded min/max values for IP addresses and direct string comparisons for other types. Test logic and coverage are preserved. - Remove wrapper_field.h includes - Replace WrapperField::create_by_type() and related calls with explicit values - Update test data structures to remove wrapper field expectations - Simplify nullable string tests to use test string directly - Update comments to reflect serde-only testing approach Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Author
|
run buildall |
yiguolei
reviewed
Feb 5, 2026
TPC-H: Total hot run time: 31324 ms |
ClickBench: Total hot run time: 28.21 s |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 31351 ms |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 31008 ms |
ClickBench: Total hot run time: 28.68 s |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 30150 ms |
ClickBench: Total hot run time: 28.56 s |
yiguolei
reviewed
Feb 6, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run buildall |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 30216 ms |
ClickBench: Total hot run time: 28.26 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run buildall |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 30857 ms |
ClickBench: Total hot run time: 28.33 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
yiguolei
approved these changes
Feb 7, 2026
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
zclllyybb
approved these changes
Feb 7, 2026
Gabriel39
added a commit
to Gabriel39/incubator-doris
that referenced
this pull request
Feb 9, 2026
…0530) Update FromStringTest, TextSerde, and CsvSerde unit tests to remove dependency on the deleted wrapper_field.h header. Replace WrapperField usage with hardcoded min/max values for IP addresses and direct string comparisons for other types. Test logic and coverage are preserved. - Remove wrapper_field.h includes - Replace WrapperField::create_by_type() and related calls with explicit values - Update test data structures to remove wrapper field expectations - Simplify nullable string tests to use test string directly - Update comments to reflect serde-only testing approach Issue Number: close #xxx Related PR: #xxx Problem Summary: None - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
yiguolei
pushed a commit
that referenced
this pull request
Feb 10, 2026
…60593) Update FromStringTest, TextSerde, and CsvSerde unit tests to remove dependency on the deleted wrapper_field.h header. Replace WrapperField usage with hardcoded min/max values for IP addresses and direct string comparisons for other types. Test logic and coverage are preserved. - Remove wrapper_field.h includes - Replace WrapperField::create_by_type() and related calls with explicit values - Update test data structures to remove wrapper field expectations - Simplify nullable string tests to use test string directly - Update comments to reflect serde-only testing approach Issue Number: close #xxx Related PR: #xxx Problem Summary: None - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
ybtsdst
pushed a commit
to ybtsdst/doris
that referenced
this pull request
Feb 27, 2026
…0530) (apache#60593) Update FromStringTest, TextSerde, and CsvSerde unit tests to remove dependency on the deleted wrapper_field.h header. Replace WrapperField usage with hardcoded min/max values for IP addresses and direct string comparisons for other types. Test logic and coverage are preserved. - Remove wrapper_field.h includes - Replace WrapperField::create_by_type() and related calls with explicit values - Update test data structures to remove wrapper field expectations - Simplify nullable string tests to use test string directly - Update comments to reflect serde-only testing approach Issue Number: close #xxx Related PR: #xxx Problem Summary: None - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
16 tasks
yiguolei
pushed a commit
that referenced
this pull request
Apr 16, 2026
### What problem does this PR solve? Fix memory corruption in PredicateColumnType::insert_duplicate_fields for string types Fixed a bug in the pointer arithmetic when inserting duplicate string values. The original code used dst += i * str.size() which incorrectly multiplied by the loop index, causing: - Iteration 0: no advance (correct by accident) - Iteration 1: advance by 1×size (correct) - Iteration 2: advance by 2×size (wrong - overwrites beyond allocated memory) This resulted in memory corruption and incorrect string data when inserting more than 2 duplicates. The fix changes to dst += str.size() to correctly advance the pointer by one string length per iteration. Added unit tests covering: - String type with multiple duplicate insertions - Integer type duplicates - LargeInt (Int128) type duplicates Introduced by #60530 ==13605==ERROR: AddressSanitizer: use-after-poison on address 0x7d639d26bc30 at pc 0x55e8b71df91e bp 0x7b4e8fc1e510 sp 0x7b4e8fc1dcd0 WRITE of size 6 at 0x7d639d26bc30 thread T620 (ls_normal [work) #0 0x55e8b71df91d in __asan_memcpy (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2205791d) #1 0x55e8cce8b1ab in doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:327:17 #2 0x55e8b7db4e83 in doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 #3 0x55e8ccf8ba2c in doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 #4 0x55e8ccf3c705 in doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 #5 0x55e8cd4f9313 in doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 #6 0x55e8cd4f9313 in doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 #7 0x55e8cd50621b in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 #8 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 #9 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 #10 0x55e8cd37cca1 in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 #11 0x55e8cccc0b05 in doris::Status doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 #12 0x55e8ccc98668 in doris::BetaRowsetReader::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 #13 0x55e8ca62ee8f in doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h #14 0x55e8ca611036 in doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 #15 0x55e8ca601a08 in doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 #16 0x55e8ca604ea2 in doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 #17 0x55e8ca608b20 in doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 #18 0x55e8ca5dc1ae in doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 #19 0x55e8ca5d7b9d in doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 #20 0x55e8d02df6d6 in doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 #21 0x55e8d02cd697 in doris::Scanner::open(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 #22 0x55e8d02b576a in doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 #23 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 #24 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 #25 0x55e8d02bf603 in bool std::__invoke_impl<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 #26 0x55e8d02bf603 in std::enable_if<is_invocable_r_v<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 #27 0x55e8d02bf603 in std::_Function_handler<bool (), doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 #28 0x55e8d02bed85 in std::function<bool ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #29 0x55e8d02bed85 in doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 #30 0x55e8d02aa844 in doris::PrioritizedSplitRunner::process() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 #31 0x55e8d0272674 in doris::TimeSharingTaskExecutor::_dispatch_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 #32 0x55e8d3fa8ad1 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #33 0x55e8d3fa8ad1 in doris::Thread::supervise_thread(void*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 #34 0x55e8b71ddd26 in asan_thread_start(void*) (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22055d26) #35 0x7f539b835ac2 in start_thread nptl/pthread_create.c:442:8 #36 0x7f539b8c78cf misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 0x7d639d26bc30 is located 816 bytes inside of 4096-byte region [0x7d639d26b900,0x7d639d26c900) allocated by thread T620 (ls_normal [work) here: #0 0x55e8b71e1b34 in malloc (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22059b34) #1 0x55e8b78d2140 in doris::DefaultMemoryAllocator::malloc(unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.h:90:55 #2 0x55e8b78d2140 in doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>::alloc(unsigned long, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.cpp:337:23 #3 0x55e8b7ccdfaf in doris::Arena::Chunk::Chunk(unsigned long, doris::Arena::Chunk*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:59:63 #4 0x55e8b7ccdad9 in doris::Arena::_init_head_if_needed() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:131:24 #5 0x55e8b7ccd758 in doris::Arena::alloc(unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:148:9 #6 0x55e8cce8b118 in doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:325:32 #7 0x55e8b7db4e83 in doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 #8 0x55e8ccf8ba2c in doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 #9 0x55e8ccf3c705 in doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 #10 0x55e8cd4f9313 in doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 #11 0x55e8cd4f9313 in doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 #12 0x55e8cd50621b in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 #13 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 #14 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 #15 0x55e8cd37cca1 in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 #16 0x55e8cccc0b05 in doris::Status doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 #17 0x55e8ccc98668 in doris::BetaRowsetReader::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 #18 0x55e8ca62ee8f in doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h #19 0x55e8ca611036 in doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 #20 0x55e8ca601a08 in doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 #21 0x55e8ca604ea2 in doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 #22 0x55e8ca608b20 in doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 #23 0x55e8ca5dc1ae in doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 #24 0x55e8ca5d7b9d in doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 #25 0x55e8d02df6d6 in doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 #26 0x55e8d02cd697 in doris::Scanner::open(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 #27 0x55e8d02b576a in doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 #28 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 #29 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 #30 0x55e8d02bf603 in bool std::__invoke_impl<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 #31 0x55e8d02bf603 in std::enable_if<is_invocable_r_v<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 #32 0x55e8d02bf603 in std::_Function_handler<bool (), doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 #33 0x55e8d02bed85 in std::function<bool ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #34 0x55e8d02bed85 in doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 #35 0x55e8d02aa844 in doris::PrioritizedSplitRunner::process() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 #36 0x55e8d0272674 in doris::TimeSharingTaskExecutor::_dispatch_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 #37 0x55e8d3fa8ad1 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #38 0x55e8d3fa8ad1 in doris::Thread::supervise_thread(void*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 Thread T620 (ls_normal [work) created by T0 here: #0 0x55e8b71c5e91 in pthread_create (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2203de91) #1 0x55e8d3fa7b8a in doris::Thread::start_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::function<void ()> const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:412:15 #2 0x55e8d02849fd in doris::Status doris::Thread::create<void (doris::TimeSharingTaskExecutor::*)(), doris::TimeSharingTaskExecutor*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, void (doris::TimeSharingTaskExecutor::* const&)(), doris::TimeSharingTaskExecutor* const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.h:47:16 #3 0x55e8d026e6b3 in doris::TimeSharingTaskExecutor::_create_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:671:12 #4 0x55e8d026b628 in doris::TimeSharingTaskExecutor::_try_create_thread(int, std::lock_guard<std::mutex>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:360:25 #5 0x55e8d026967a in doris::TimeSharingTaskExecutor::init() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:221:27 #6 0x55e8d025a85a in doris::TaskExecutorSimplifiedScanScheduler::start(int, int, int, int) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.h:288:9 #7 0x55e8d368d649 in doris::WorkloadGroup::upsert_thread_pool_no_lock(doris::WorkloadGroupInfo*, std::shared_ptr<doris::CgroupCpuCtl>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:571:38 #8 0x55e8d368f14d in doris::WorkloadGroup::upsert_task_scheduler(doris::WorkloadGroupInfo*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:667:12 #9 0x55e8d36bd223 in doris::WorkloadGroupMgr::create_internal_wg() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group_manager.cpp:900:5 #10 0x55e8d32bbd3d in doris::ExecEnv::_create_internal_workload_group() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:475:5 #11 0x55e8d32b27fc in doris::ExecEnv::_init(std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:424:5 #12 0x55e8d32ad7ec in doris::ExecEnv::init(doris::ExecEnv*, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:199:17 #13 0x55e8b722cc8b in main /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/service/doris_main.cpp:531:14 #14 0x7f539b7cad8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
github-actions bot
added a commit
that referenced
this pull request
Apr 16, 2026
### What problem does this PR solve? Fix memory corruption in PredicateColumnType::insert_duplicate_fields for string types Fixed a bug in the pointer arithmetic when inserting duplicate string values. The original code used dst += i * str.size() which incorrectly multiplied by the loop index, causing: - Iteration 0: no advance (correct by accident) - Iteration 1: advance by 1×size (correct) - Iteration 2: advance by 2×size (wrong - overwrites beyond allocated memory) This resulted in memory corruption and incorrect string data when inserting more than 2 duplicates. The fix changes to dst += str.size() to correctly advance the pointer by one string length per iteration. Added unit tests covering: - String type with multiple duplicate insertions - Integer type duplicates - LargeInt (Int128) type duplicates Introduced by #60530 ==13605==ERROR: AddressSanitizer: use-after-poison on address 0x7d639d26bc30 at pc 0x55e8b71df91e bp 0x7b4e8fc1e510 sp 0x7b4e8fc1dcd0 WRITE of size 6 at 0x7d639d26bc30 thread T620 (ls_normal [work) #0 0x55e8b71df91d in __asan_memcpy (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2205791d) #1 0x55e8cce8b1ab in doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:327:17 #2 0x55e8b7db4e83 in doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 #3 0x55e8ccf8ba2c in doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 #4 0x55e8ccf3c705 in doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 #5 0x55e8cd4f9313 in doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 #6 0x55e8cd4f9313 in doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 #7 0x55e8cd50621b in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 #8 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 #9 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 #10 0x55e8cd37cca1 in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 #11 0x55e8cccc0b05 in doris::Status doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 #12 0x55e8ccc98668 in doris::BetaRowsetReader::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 #13 0x55e8ca62ee8f in doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h #14 0x55e8ca611036 in doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 #15 0x55e8ca601a08 in doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 #16 0x55e8ca604ea2 in doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 #17 0x55e8ca608b20 in doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 #18 0x55e8ca5dc1ae in doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 #19 0x55e8ca5d7b9d in doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 #20 0x55e8d02df6d6 in doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 #21 0x55e8d02cd697 in doris::Scanner::open(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 #22 0x55e8d02b576a in doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 #23 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 #24 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 #25 0x55e8d02bf603 in bool std::__invoke_impl<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 #26 0x55e8d02bf603 in std::enable_if<is_invocable_r_v<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 #27 0x55e8d02bf603 in std::_Function_handler<bool (), doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 #28 0x55e8d02bed85 in std::function<bool ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #29 0x55e8d02bed85 in doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 #30 0x55e8d02aa844 in doris::PrioritizedSplitRunner::process() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 #31 0x55e8d0272674 in doris::TimeSharingTaskExecutor::_dispatch_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 #32 0x55e8d3fa8ad1 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #33 0x55e8d3fa8ad1 in doris::Thread::supervise_thread(void*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 #34 0x55e8b71ddd26 in asan_thread_start(void*) (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22055d26) #35 0x7f539b835ac2 in start_thread nptl/pthread_create.c:442:8 #36 0x7f539b8c78cf misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 0x7d639d26bc30 is located 816 bytes inside of 4096-byte region [0x7d639d26b900,0x7d639d26c900) allocated by thread T620 (ls_normal [work) here: #0 0x55e8b71e1b34 in malloc (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22059b34) #1 0x55e8b78d2140 in doris::DefaultMemoryAllocator::malloc(unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.h:90:55 #2 0x55e8b78d2140 in doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>::alloc(unsigned long, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.cpp:337:23 #3 0x55e8b7ccdfaf in doris::Arena::Chunk::Chunk(unsigned long, doris::Arena::Chunk*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:59:63 #4 0x55e8b7ccdad9 in doris::Arena::_init_head_if_needed() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:131:24 #5 0x55e8b7ccd758 in doris::Arena::alloc(unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:148:9 #6 0x55e8cce8b118 in doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:325:32 #7 0x55e8b7db4e83 in doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 #8 0x55e8ccf8ba2c in doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 #9 0x55e8ccf3c705 in doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 #10 0x55e8cd4f9313 in doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 #11 0x55e8cd4f9313 in doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 #12 0x55e8cd50621b in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 #13 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 #14 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 #15 0x55e8cd37cca1 in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 #16 0x55e8cccc0b05 in doris::Status doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 #17 0x55e8ccc98668 in doris::BetaRowsetReader::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 #18 0x55e8ca62ee8f in doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h #19 0x55e8ca611036 in doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 #20 0x55e8ca601a08 in doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 #21 0x55e8ca604ea2 in doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 #22 0x55e8ca608b20 in doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 #23 0x55e8ca5dc1ae in doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 #24 0x55e8ca5d7b9d in doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 #25 0x55e8d02df6d6 in doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 #26 0x55e8d02cd697 in doris::Scanner::open(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 #27 0x55e8d02b576a in doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 #28 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 #29 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 #30 0x55e8d02bf603 in bool std::__invoke_impl<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 #31 0x55e8d02bf603 in std::enable_if<is_invocable_r_v<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 #32 0x55e8d02bf603 in std::_Function_handler<bool (), doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 #33 0x55e8d02bed85 in std::function<bool ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #34 0x55e8d02bed85 in doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 #35 0x55e8d02aa844 in doris::PrioritizedSplitRunner::process() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 #36 0x55e8d0272674 in doris::TimeSharingTaskExecutor::_dispatch_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 #37 0x55e8d3fa8ad1 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 #38 0x55e8d3fa8ad1 in doris::Thread::supervise_thread(void*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 Thread T620 (ls_normal [work) created by T0 here: #0 0x55e8b71c5e91 in pthread_create (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2203de91) #1 0x55e8d3fa7b8a in doris::Thread::start_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::function<void ()> const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:412:15 #2 0x55e8d02849fd in doris::Status doris::Thread::create<void (doris::TimeSharingTaskExecutor::*)(), doris::TimeSharingTaskExecutor*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, void (doris::TimeSharingTaskExecutor::* const&)(), doris::TimeSharingTaskExecutor* const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.h:47:16 #3 0x55e8d026e6b3 in doris::TimeSharingTaskExecutor::_create_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:671:12 #4 0x55e8d026b628 in doris::TimeSharingTaskExecutor::_try_create_thread(int, std::lock_guard<std::mutex>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:360:25 #5 0x55e8d026967a in doris::TimeSharingTaskExecutor::init() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:221:27 #6 0x55e8d025a85a in doris::TaskExecutorSimplifiedScanScheduler::start(int, int, int, int) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.h:288:9 #7 0x55e8d368d649 in doris::WorkloadGroup::upsert_thread_pool_no_lock(doris::WorkloadGroupInfo*, std::shared_ptr<doris::CgroupCpuCtl>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:571:38 #8 0x55e8d368f14d in doris::WorkloadGroup::upsert_task_scheduler(doris::WorkloadGroupInfo*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:667:12 #9 0x55e8d36bd223 in doris::WorkloadGroupMgr::create_internal_wg() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group_manager.cpp:900:5 #10 0x55e8d32bbd3d in doris::ExecEnv::_create_internal_workload_group() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:475:5 #11 0x55e8d32b27fc in doris::ExecEnv::_init(std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:424:5 #12 0x55e8d32ad7ec in doris::ExecEnv::init(doris::ExecEnv*, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:199:17 #13 0x55e8b722cc8b in main /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/service/doris_main.cpp:531:14 #14 0x7f539b7cad8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Gabriel39
added a commit
to Gabriel39/incubator-doris
that referenced
this pull request
Apr 17, 2026
Fix memory corruption in PredicateColumnType::insert_duplicate_fields for string types Fixed a bug in the pointer arithmetic when inserting duplicate string values. The original code used dst += i * str.size() which incorrectly multiplied by the loop index, causing: - Iteration 0: no advance (correct by accident) - Iteration 1: advance by 1×size (correct) - Iteration 2: advance by 2×size (wrong - overwrites beyond allocated memory) This resulted in memory corruption and incorrect string data when inserting more than 2 duplicates. The fix changes to dst += str.size() to correctly advance the pointer by one string length per iteration. Added unit tests covering: - String type with multiple duplicate insertions - Integer type duplicates - LargeInt (Int128) type duplicates Introduced by apache#60530 ==13605==ERROR: AddressSanitizer: use-after-poison on address 0x7d639d26bc30 at pc 0x55e8b71df91e bp 0x7b4e8fc1e510 sp 0x7b4e8fc1dcd0 WRITE of size 6 at 0x7d639d26bc30 thread T620 (ls_normal [work) (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2205791d) doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:327:17 doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22055d26) apache#35 0x7f539b835ac2 in start_thread nptl/pthread_create.c:442:8 0x7d639d26bc30 is located 816 bytes inside of 4096-byte region [0x7d639d26b900,0x7d639d26c900) allocated by thread T620 (ls_normal [work) here: (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22059b34) long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.h:90:55 doris::DefaultMemoryAllocator, true>::alloc(unsigned long, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.cpp:337:23 doris::Arena::Chunk*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:59:63 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:131:24 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:148:9 doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:325:32 doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 Thread T620 (ls_normal [work) created by T0 here: (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2203de91) doris::Thread::start_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::function<void ()> const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:412:15 (doris::TimeSharingTaskExecutor::*)(), doris::TimeSharingTaskExecutor*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, void (doris::TimeSharingTaskExecutor::* const&)(), doris::TimeSharingTaskExecutor* const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.h:47:16 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:671:12 doris::TimeSharingTaskExecutor::_try_create_thread(int, std::lock_guard<std::mutex>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:360:25 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:221:27 doris::TaskExecutorSimplifiedScanScheduler::start(int, int, int, int) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.h:288:9 doris::WorkloadGroup::upsert_thread_pool_no_lock(doris::WorkloadGroupInfo*, std::shared_ptr<doris::CgroupCpuCtl>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:571:38 doris::WorkloadGroup::upsert_task_scheduler(doris::WorkloadGroupInfo*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:667:12 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group_manager.cpp:900:5 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:475:5 doris::ExecEnv::_init(std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:424:5 std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:199:17 /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/service/doris_main.cpp:531:14 csu/../sysdeps/nptl/libc_start_call_main.h:58:16 - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
29 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update FromStringTest, TextSerde, and CsvSerde unit tests to remove dependency on the deleted wrapper_field.h header. Replace WrapperField usage with hardcoded min/max values for IP addresses and direct string comparisons for other types. Test logic and coverage are preserved.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)