How to improve the performance of tpcds with spark3.3.3? #408
Unanswered
MrFireChow
asked this question in
Q&A
Replies: 1 comment
|
If you wish to assess the acceleration effects of Blaze, here are some details that are easily overlooked:
|
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hello!I build the environment of spark3.3.3 and blaze2.0.8, then i do some tests based on 100G tpcds data,however,I did not receive any benefits compared to not using blaze,this is my spark-sql command:
spark-sql --master spark://xxxx:xxxx --conf spark.sql.extensions=org.apache.spark.sql.blaze.BlazeSparkSessionExtension --conf spark.shuffle.manager=org.apache.spark.sql.execution.blaze.shuffle.BlazeShuffleManager --conf spark.blaze.enable.smjInequalityJoin=true
I want to know if I need to add any other parameters. Looking forward to your reply!
All reactions