-
Notifications
You must be signed in to change notification settings - Fork 3.6k
branch-3.1: [feature](maxcompute)support maxcompte catalog read project-schema-table. #57012 #57286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
...ble. (#57012) ### What problem does this PR solve? Problem Summary: In the previous MaxCompute architecture, and its mapping in Doris, the hierarchy was: `project / database -> table / table`. When creating a catalog, users needed to specify the property `mc.default.project`, which indicated the default project to access. In this structure, executing `SHOW DATABASES` would list other projects. After MaxCompute introduced the concept of schemas, the hierarchy changed to: `project / catalog -> schema / database -> table / table`. Here, the project is at a higher level, and `SHOW DATABASES` should now list all schemas under the current project. As a result, users need to create a separate catalog for each project, specifying a different `mc.default.project` property. doc: https://help.aliyun.com/zh/maxcompute/user-guide/schema-related-operations?spm=a2c4g.11186623.help-menu-search-27797.d_0#9d8326491bagj To maintain compatibility with the old version, a catalog property `mc.project.schema.table` is introduced: - When the property is true, the new architecture is used. - When the property is false, the old architecture is used.
@github-actions
github-actions
bot
requested a review
from morrySnow
as a code owner
October 24, 2025 02:48
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:
- What problem was fixed (it's best to include specific error reporting information). How it was fixed.
- Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
- What features were added. Why was this function added?
- Which code was refactored and why was this part of the code refactored?
- Which functions were optimized and what is the difference before and after the optimization?
run buildall
doris-robot
commented
Oct 24, 2025
TPC-H: Total hot run time: 32417 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 67561b71a108f1d7a5ac7d895f108292297470ec, data reload: false
------ Round 1 ----------------------------------
q1 17610 5424 5484 5424
q2 2018 395 283 283
q3 12069 1211 734 734
q4 10312 866 451 451
q5 9168 2357 2105 2105
q6 184 162 132 132
q7 881 741 604 604
q8 9325 1454 1141 1141
q9 5260 4961 4912 4912
q10 6770 2285 1806 1806
q11 475 275 255 255
q12 334 354 213 213
q13 17790 3585 3010 3010
q14 220 231 224 224
q15 549 470 456 456
q16 424 420 368 368
q17 581 851 356 356
q18 6886 6430 6324 6324
q19 1220 957 517 517
q20 315 329 201 201
q21 2760 2176 1908 1908
q22 1032 1029 993 993
Total cold run time: 106183 ms
Total hot run time: 32417 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5552 5522 5557 5522
q2 237 330 234 234
q3 2245 2628 2423 2423
q4 1412 1792 1368 1368
q5 4558 5091 5056 5056
q6 171 172 135 135
q7 2137 2045 1851 1851
q8 2677 2886 2739 2739
q9 7459 7442 7329 7329
q10 3116 3314 2806 2806
q11 596 550 508 508
q12 685 790 645 645
q13 3490 3911 3334 3334
q14 293 291 283 283
q15 527 469 492 469
q16 442 477 440 440
q17 1225 1723 1263 1263
q18 7657 7513 7409 7409
q19 782 979 1131 979
q20 2039 2094 1908 1908
q21 5399 4792 4551 4551
q22 1133 1075 1037 1037
Total cold run time: 53832 ms
Total hot run time: 52289 ms
doris-robot
commented
Oct 24, 2025
TPC-DS: Total hot run time: 193004 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 67561b71a108f1d7a5ac7d895f108292297470ec, data reload: false
query1 964 397 396 396
query2 6177 1918 1860 1860
query3 8689 207 196 196
query4 33585 24070 23951 23951
query5 4305 599 471 471
query6 310 202 172 172
query7 4217 481 316 316
query8 303 241 228 228
query9 9499 2577 2578 2577
query10 490 320 258 258
query11 18155 15498 15215 15215
query12 154 112 116 112
query13 1584 534 409 409
query14 9158 6638 7423 6638
query15 235 192 175 175
query16 8120 673 506 506
query17 1541 792 613 613
query18 2154 416 323 323
query19 225 212 168 168
query20 140 121 125 121
query21 229 137 128 128
query22 4580 4524 4368 4368
query23 35842 34513 34281 34281
query24 7254 2688 2709 2688
query25 527 502 436 436
query26 794 286 175 175
query27 2149 473 363 363
query28 4824 2194 2157 2157
query29 712 634 473 473
query30 243 198 158 158
query31 1014 902 846 846
query32 87 65 59 59
query33 550 411 306 306
query34 753 852 521 521
query35 796 842 725 725
query36 1040 1078 960 960
query37 109 92 74 74
query38 4074 3996 4027 3996
query39 1536 1497 1495 1495
query40 215 121 106 106
query41 49 49 50 49
query42 120 101 108 101
query43 508 509 491 491
query44 1327 844 827 827
query45 203 178 179 178
query46 873 1043 675 675
query47 2011 2015 1932 1932
query48 409 419 335 335
query49 752 495 402 402
query50 662 721 423 423
query51 7233 7417 7200 7200
query52 104 105 93 93
query53 237 262 192 192
query54 544 536 482 482
query55 79 85 83 83
query56 295 305 252 252
query57 1276 1256 1222 1222
query58 252 218 216 216
query59 3122 3154 3070 3070
query60 301 286 280 280
query61 129 110 154 110
query62 820 765 704 704
query63 242 199 192 192
query64 3703 1010 644 644
query65 3427 3310 3335 3310
query66 1066 409 339 339
query67 16572 15828 15711 15711
query68 8393 804 527 527
query69 500 308 263 263
query70 1194 1129 1123 1123
query71 424 299 268 268
query72 5139 3842 3821 3821
query73 651 743 348 348
query74 10318 9357 9308 9308
query75 4369 3235 2658 2658
query76 3779 1185 756 756
query77 766 356 266 266
query78 10362 10493 9636 9636
query79 5129 869 585 585
query80 639 517 453 453
query81 487 253 218 218
query82 227 119 88 88
query83 169 155 141 141
query84 290 101 78 78
query85 770 375 303 303
query86 341 294 299 294
query87 4291 4350 4240 4240
query88 4329 2406 2400 2400
query89 431 330 288 288
query90 2128 196 191 191
query91 136 136 109 109
query92 68 55 53 53
query93 3615 874 528 528
query94 657 410 305 305
query95 353 280 267 267
query96 486 603 282 282
query97 3175 3260 3148 3148
query98 224 204 204 204
query99 1446 1393 1322 1322
Total cold run time: 298935 ms
Total hot run time: 193004 ms
doris-robot
commented
Oct 24, 2025
ClickBench: Total hot run time: 29.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 67561b71a108f1d7a5ac7d895f108292297470ec, data reload: false
query1 0.03 0.03 0.03
query2 0.07 0.04 0.03
query3 0.22 0.07 0.06
query4 1.63 0.11 0.11
query5 0.51 0.52 0.52
query6 1.13 0.74 0.72
query7 0.02 0.01 0.02
query8 0.04 0.03 0.03
query9 0.57 0.49 0.50
query10 0.56 0.54 0.55
query11 0.15 0.11 0.10
query12 0.14 0.11 0.11
query13 0.63 0.60 0.60
query14 0.79 0.80 0.82
query15 0.85 0.84 0.83
query16 0.38 0.38 0.38
query17 1.07 1.05 1.05
query18 0.24 0.22 0.22
query19 1.88 1.84 1.91
query20 0.02 0.01 0.01
query21 15.37 0.91 0.59
query22 0.74 0.90 0.56
query23 15.10 1.43 0.58
query24 2.93 1.40 1.50
query25 0.15 0.11 0.07
query26 0.25 0.15 0.14
query27 0.05 0.06 0.06
query28 13.94 1.05 0.43
query29 12.56 3.89 3.29
query30 0.26 0.09 0.06
query31 2.83 0.62 0.38
query32 3.24 0.54 0.46
query33 3.04 3.02 3.04
query34 16.55 5.19 4.55
query35 4.59 4.56 4.61
query36 0.66 0.50 0.51
query37 0.08 0.06 0.05
query38 0.05 0.04 0.03
query39 0.03 0.02 0.02
query40 0.17 0.14 0.13
query41 0.08 0.03 0.03
query42 0.03 0.02 0.02
query43 0.04 0.02 0.03
Total cold run time: 103.67 s
Total hot run time: 29.43 s
FE UT Coverage Report
Increment line coverage 0.00% (0/100) 🎉
Increment coverage report
Complete coverage report
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-picked from #57012