[SPARK-59433][SQL] Render TIMESTAMP_LTZ explicitly when spark.sql.timestampType=TIMESTAMP_NTZ - #58734

[P1] Do not use client-local config to render LTZ. SqlApiConf is explicitly hardcoded in sql-api-only/Spark Connect, so this branch is never taken by a Scala Connect client even when the remote session has spark.sql.timestampType=TIMESTAMP_NTZ. For example, DataFrameReader sends a concrete StructType as schema.toDDL; the server reparses that DDL under the remote NTZ session, so a requested TimestampType is sent as bare TIMESTAMP and silently becomes TimestampNTZType. The concrete-schema overloads of from_json, from_csv, and from_xml have the same issue. Please use an unambiguous schema transport/rendering for concrete types and add Scala Connect coverage under an NTZ remote session.

This changes auto-generated column names too, not only DDL. Cast.sql and Literal.sql use dataType.sql, so under spark.sql.timestampType=TIMESTAMP_NTZ, SELECT CAST(c AS TIMESTAMP_LTZ) is now named CAST(c AS TIMESTAMP_LTZ) instead of CAST(c AS TIMESTAMP).

This can break existing persistent views after upgrade. A view created in an NTZ session, e.g. CREATE VIEW v AS SELECT CAST(ntz_col AS TIMESTAMP_LTZ) FROM t, stores the old query output column name and captures spark.sql.timestampType=TIMESTAMP_NTZ. On read, the view query is re-analyzed under the captured conf, produces the new name, and GetViewColumnByNameAndOrdinal fails with INCOMPATIBLE_VIEW_SCHEMA_CHANGE.

CTAS / DataFrame column names and toSQLType error messages change in the same way. Could you check this case and mention it in the user-facing change section?

SHOW CREATE TABLE ... AS SERDE also goes through StructField.toDDL (showHiveTableHeader in tables.scala). It is documented as generating Hive DDL, but under NTZ it will now emit TIMESTAMP_LTZ for a timestamp column of a Hive SerDe table, which Hive cannot parse. Is this intended? Either way, it would be good to cover the AS SERDE path in a test.