SNOW-2912540: add IS_V5_DRIVER constant; use it for SecretDetector/TelemetryData/TelemetryClient/ReauthenticationRequest imports and pandas/pyarrow/numpy resolution - #4313

@snowflake-security-bot snowflake-security-bot Bot left a comment

@snowflake-security-bot snowflake-security-bot Bot left a comment

@snowflake-security-bot snowflake-security-bot Bot left a comment

@sfc-gh-yuwang sfc-gh-yuwang left a comment

...ctor imports
connector_version is already imported in utils.py; this one-liner exposes a
boolean flag so callers can gate imports or behavior that differs between the
legacy connector (v3/v4) and the Universal Driver (v5+).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

...; drop connector.options imports
snowflake.connector.options is a backward-compat shim in v5 (UD) that
will eventually be removed. Define MissingOptionalDependency,
MissingPandas, MissingPyarrow, ModuleLikeObject, pandas, pyarrow,
installed_pandas, and installed_pyarrow directly in _internal/utils.py
and redirect all thirteen source-file imports there.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The v5 (Universal Driver) public snowflake.connector.telemetry shim does not
always expose TelemetryData.TRUE/.FALSE (present on _internal.telemetry in some
UD builds), which made every telemetry-sending Snowpark test raise
AttributeError: type object 'TelemetryData' has no attribute 'FALSE'. Gate the
TelemetryData import on IS_V5_DRIVER: on v5 prefer _internal.telemetry and fall
back to the public shim; on v4 keep the legacy public import.
Co-authored-by: Cursor <cursoragent@cursor.com>

...edefinition; re-source from connector
04c82d2 assumed connector.options had a gap and redefined MissingOptionalDependency,
MissingPandas, MissingPyarrow, ModuleLikeObject, pyarrow, and installed_pyarrow locally.
connector.options already provides all of these except installed_pyarrow (verified
against the actually-installed v4.7.2 connector, which only exposes pyarrow itself and
couples its availability to pandas's import tuple, not a standalone name). Re-import the
four names that do exist there, gated on IS_V5_DRIVER so this keeps working once UD's
connector._common.extras lands, and derive installed_pyarrow locally via isinstance
instead of maintaining an independent, unprecedented MissingPyarrow resolution.
pandas/installed_pandas (produced by the pre-existing _pandas_importer(), unrelated to
04c82d2) are intentionally left untouched -- unifying those is a separate follow-up.

...rent shape
UD PR #1151 was updated since cb9f590 landed: MissingPandas is deleted
outright from _common/extras.py (BehaviorDifferences.yaml #66), not kept as a
deprecated re-export. Its suggested replacement, MissingOptionalDependency("pandas"),
only works on v5 -- the real v4 connector's MissingOptionalDependency defines no
__init__ override, so it only supports the no-arg-subclass pattern.
Import MissingPandas only where it's real (v4). Add _missing_pandas() to build
the sentinel with whichever construction style the active driver generation
supports, centralizing the branch in _internal/utils.py rather than spreading
IS_V5_DRIVER awareness to callers. Fix _pandas_importer() and mock/_options.py,
both of which referenced MissingPandas directly and would otherwise NameError/
ImportError under IS_V5_DRIVER=True.
installed_pyarrow also moves to a direct v5 import: #1151 fixed it to check
pyarrow independently instead of mirroring installed_pandas, so it's now
correct to import there instead of re-deriving locally. v4 still doesn't
export it at all, so the local isinstance derivation stays there.

...n v5
The previous try/except hedged between two locations, neither of which
actually has TelemetryData.TRUE/.FALSE on the current UD main: the public
snowflake.connector.telemetry shim's TelemetryData has no TRUE/FALSE at all,
and _internal.telemetry currently has no TelemetryData class either. The
fallback branch was silently reachable and silently wrong.
UD PR #1106 (open, not draft) adds TelemetryData/TelemetryField to
_internal/telemetry.py specifically to match Snowpark's exact usage
(PCTelemetryData(message=..., timestamp=...), .TRUE/.FALSE) -- confirmed by
reading its actual diff. Import from there unconditionally on v5, no
try/except: both branches now import from one definite, verified location.

...as from connector
_pandas_importer() predates this whole effort and duplicated resolution the
connector already does correctly on both driver generations -- including the
"relative imports without dots" DataFrame workaround, now folded into UD's
own _common.extras.pandas (confirmed on the not-yet-merged UD PR #1151/#1152;
v4's options.py already had it). Add pandas/installed_pandas to the existing
IS_V5_DRIVER-gated import block and delete the local resolution entirely.
Verified the workaround isn't needed on Snowpark's side by running the exact
invocation style its comment called out (pytest with tests/unit/ as cwd) --
no failure, consistent with both driver generations now handling it
internally.

....extras
mock/_options.py's MissingNumpy/numpy try-except was functionally identical
to _common/extras.py's own numpy resolution (confirmed: pure duplicate, no
fix to merge, per UD PR #1152's investigation). Import numpy from
_common.extras on v5; v4 keeps its own MissingNumpy class since v4's
options.py has no numpy handling to delegate to.
Does not touch the pandas try/except in this file -- Local Testing
deliberately never resolves pyarrow, unlike every other pandas-resolution
path in this codebase (commit #1628).

Removing the function left only one blank line before class TempObjectType; black requires two before a top-level class definition.

...block
- F401: pandas is imported purely for other modules to re-import from
 here, so it's never referenced elsewhere in this file. Split into
 its own import with a noqa, rather than noqa-ing a name inside a
 multi-line parenthesized import (which flake8 attributes to the
 opening line, not the name's own line).
- E402: the IS_V5_DRIVER conditional-import block and _missing_pandas()
 ended up sitting between the top-of-file imports and two later ones
 (Row, VERSION). Moved those two imports up to stay contiguous.

...ield imports on IS_V5_DRIVER
network.py doesn't exist in UD at all -- legacy's errors.py/network.py split is
consolidated into errors.py. UD PR #1224 (open, stacked on #1133) adds
ReauthenticationRequest(ProgrammingError) to errors.py and removes network.py
outright, naming Snowpark's import site explicitly as the target. Gate the
import in server_connection.py and its unit test mock, same pattern as every
other IS_V5_DRIVER import in this stack.
TelemetryClient/TelemetryField were imported unconditionally from the
top-level snowflake.connector.telemetry module, which is a stub on v5 (the
real implementation lives in _common.telemetry per UD PR #1106's current
branch). Also fixes this same file's PCTelemetryData import, added in an
earlier commit against _internal.telemetry -- that class moved to _common
too on the same #1106 branch since that commit landed, so it was already
stale for the identical reason.
Adds test coverage for _missing_pandas() (added earlier in this stack to
replace direct MissingPandas() construction), which had zero coverage after
test__pandas_importer() was deleted alongside _pandas_importer() itself.

... import
from snowflake.connector.options import installed_pandas is unconditional in
tests/integ/test_function.py, test_cte.py, scala/test_datatype_suite.py, and
scala/test_update_delete_merge_suite.py -- ModuleNotFoundError once UD deletes
options.py outright (confirmed: f61156c7a, ancestor of SNOW-2912540-extras-to-common's
current tip, already relied on elsewhere in this stack).
Swap to snowflake.snowpark._internal.utils, which already re-exports
installed_pandas correctly gated on IS_V5_DRIVER internally (from this PR's
earlier _pandas_importer()-removal commit) -- no IS_V5_DRIVER awareness needed
in these test files themselves.

...s.py
_internal/utils.py's IS_V5_DRIVER-gated block (MissingOptionalDependency,
ModuleLikeObject, pandas, pyarrow, installed_pandas, installed_pyarrow,
_missing_pandas()) was a self-contained concern mirroring the connector's own
options.py/_common.extras 1:1, buried in an already-large kitchen-sink file.
Moved to a dedicated module, mirroring mock/_options.py's existing role as
the scoped equivalent for the local-testing side.
options.py computes its own IS_V5_DRIVER independently rather than importing
it from utils.py, since utils.py itself needs names back from options.py
(MissingOptionalDependency/ModuleLikeObject/installed_pandas, used by its
modin-optional-dependency code) -- importing in both directions would be
circular. One-line duplication, avoids import-order fragility entirely.
Updated all 17 consumers (found types.py's multi-line import via a
regex-based sweep after a naive single-line grep missed it) to import these
names from _internal.options instead. mock/_options.py and
event_table_telemetry.py keep their other _internal.utils imports
(IS_V5_DRIVER, parse_table_name) unchanged.

...ield
Reuse the canonical IS_V5_DRIVER from _internal.options instead of
redefining it in utils, and gate the leftover Modin connector.telemetry
import on the same flag as Snowpark telemetry.
Co-authored-by: Filip Pawłowski <sfc-gh-fpawlowski@users.noreply.github.com>

flake8 reports unused IS_V5_DRIVER on the `from` line, so a noqa on the
name itself never applied and Check linting failed from this PR upstack.
Co-authored-by: Filip Pawłowski <sfc-gh-fpawlowski@users.noreply.github.com>

Moving noqa to the from-import line was not enough for Check linting on
this PR. Bind the options module constant to a local name instead of an
unused import.
Co-authored-by: Filip Pawłowski <sfc-gh-fpawlowski@users.noreply.github.com>

...secret-detector-import
Co-authored-by: Filip Pawłowski <sfc-gh-fpawlowski@users.noreply.github.com>

...secret-detector-import
Co-authored-by: Filip Pawłowski <sfc-gh-fpawlowski@users.noreply.github.com>

...secret-detector-import
Co-authored-by: Filip Pawłowski <sfc-gh-fpawlowski@users.noreply.github.com>

...secret-detector-import