[AURON #1937][CI] Enable TPCDS test q14b

[cxzl25]

Which issue does this PR close?

Closes #1937

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

How was this patch tested?

[Copilot]

Pull request overview

This PR addresses a thread-safety issue in the deserializeExpression method by adding synchronization around object deserialization when a TaskContext is present. The fix re-enables the previously flaky q14b TPC-DS query test.

Changes:

• Added synchronization on taskMetrics() during expression deserialization to fix thread-safety issues with Spark's TaskMetrics#externalAccums
• Re-enabled q14b in the TPC-DS test suite after addressing the flaky failure

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeConverters.scala	Added TaskContext import and synchronized block around readObject calls in deserializeExpression to fix thread-safety issue
.github/workflows/tpcds-reusable.yml	Re-enabled q14b test and removed the comment about the flaky failure

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[yew1eb]

Nice fix @cxzl25! Could you share the root cause and troubleshooting clues for this q14b issue?

[cxzl25]

Could you share the root cause and troubleshooting clues for this q14b issue

Judging from the call stack, the accumUpdates of TaskResult contains null values.

ERROR TaskResultGetter: Exception while getting task result
java.lang.NullPointerException
	at org.apache.spark.scheduler.TaskResultGetter$$anon$3.$anonfun$run$3(TaskResultGetter.scala:109)

The only place where TaskMetrics#externalAccums might be updated is TaskMetrics#registerAccumulator, which in turn is invoked via taskContext.registerAccumulator(this) called by the deserialization of AccumulatorV2#readObject. From the perspective of the code, it is highly unlikely for a null value to be written here.

Since externalAccums is an ArrayBuffer and not thread-safe, I tried adding the synchronized keyword to TaskMetrics#registerAccumulator, and the NPE issue was resolved. I then added some logging to record which threads were accessing this method concurrently, and ultimately identified that Auron's deserialization of expressions may be the root cause of this problem.