Work Item / Issue Reference

AB#39896

GitHub Issue: #306

Summary

Fixes #306 by introducing a connection string parser for mssql-python.

PR Title Guide

FEAT: Improve connection string parsing, honor the allow list, and improve error messaging.

Connection String Allowlist Feature - Review Guide

Overview

This PR introduces a comprehensive connection string validation and allowlist system that validates all ODBC connection parameters before passing them to the driver, improving usability and providing better error messages.

What This Changes

Before: Connection strings were passed directly to the ODBC driver with minimal or no validation. Unknown parameters were silently ignored by ODBC, malformed strings caused cryptic ODBC errors.

After: All connection strings are parsed, validated against an allowlist of ODBC Driver 18 parameters, and reconstructed with proper escaping. Clear error messages are provided for any issues.

Data Flow

User Input (connection string + kwargs)
 ↓
┌─────────────────────────────────────────────┐
│ Connection.__init__() │
│ _construct_connection_string() │
└─────────────────────────────────────────────┘
 ↓
┌─────────────────────────────────────────────┐
│ Step 1: Parse Connection String │
│ _ConnectionStringParser.parse() │
│ - Tokenizes key=value pairs │
│ - Handles braced values: {val} │
│ - Processes escape sequences: }}-> } │
│ - Detects syntax errors │
│ Output: Dict[str, str] │
└─────────────────────────────────────────────┘
 ↓
┌─────────────────────────────────────────────┐
│ Step 2: Validate Against Allowlist │
│ ConnectionStringAllowList.normalize_keys() │
│ - Checks unknown parameters │
│ - Normalizes synonyms (host→Server) │
│ - Blocks reserved params (Driver, APP) │
│ - Warns about rejected params │
│ Output: Dict[str, str] (filtered) │
└─────────────────────────────────────────────┘
 ↓
┌─────────────────────────────────────────────┐
│ Step 3: Process kwargs │
│ - Normalize each kwarg key │
│ - Block reserved parameters │
│ - Merge into filtered params │
│ - kwargs override connection string │
└─────────────────────────────────────────────┘
 ↓
┌─────────────────────────────────────────────┐
│ Step 4: Build Connection String │
│ _ConnectionStringBuilder.build() │
│ - Add Driver (hardcoded) │
│ - Add APP=MSSQL-Python │
│ - Escape special characters │
│ - Format: key1=value1;key2=value2; │
│ Output: str (final connection string) │
└─────────────────────────────────────────────┘
 ↓
ODBC Driver (validated, safe connection string)

File Structure & Review Order

Recommended Review Order

Review in this sequence to understand the architecture:

1. Core Components (understand the building blocks)

1. mssql_python/connection_string_parser.py Start here

   • Purpose: Parses ODBC connection strings per MS-ODBCSTR spec
   • Key Classes:
     • ConnectionStringParseError - Custom exception
     • _ConnectionStringParser - Main parser class
   • Key Methods:
     • parse(connection_str) - Main entry point
     • _parse_value(str, pos) - Handles braced values & escaping
   • What to Look For:
     • Proper handling of escape sequences ({{, }})
     • Error collection and batch reporting
     • Edge cases: empty strings, whitespace, special chars

2. mssql_python/connection_string_allowlist.py Critical

   • Purpose: Validates parameters against ODBC Driver 18 keywords
   • Key Classes:
     • ConnectionStringAllowList - Singleton allowlist manager
   • Key Methods:
     • normalize_key(key) - Maps synonyms to canonical names
     • filter_params(params) - Validates and filters parameters
   • What to Look For:
     • Completeness of allowlist (compare with ODBC docs)
     • Synonym mappings (host→Server, user→UID, etc.)
     • Reserved parameter handling (Driver, APP)

3. mssql_python/connection_string_builder.py

   • Purpose: Safely constructs connection strings with escaping
   • Key Classes:
     • _ConnectionStringBuilder - Builder with escape logic
   • Key Methods:
     • add_param(key, value) - Adds a parameter
     • _needs_braces(value) - Determines if braces needed
     • _escape_value(value) - Escapes special characters
     • build() - Constructs final string
   • What to Look For:
     • Correct escaping logic (;, {, }, =)
     • Proper brace placement
     • Semicolon formatting

2. Integration (see how it fits together)

4. mssql_python/connection.py Integration point

   • Modified Method: _construct_connection_string()
   • What Changed:
     • Lines 241-303: New implementation replacing old concat logic
     • Now uses parser-> filter-> builder pipeline
     • Handles kwargs with allowlist validation
     • Raises ValueError for reserved parameters
   • What to Look For:
     • Error handling for parse failures
     • kwargs override behavior
     • Reserved parameter rejection
     • Logging of warnings

5. mssql_python/__init__.py

   • What Changed:
     • Added export: ConnectionStringParseError
   • What to Look For:
     • Proper exception exposure for users

3. Tests (validate functionality)

6. tests/test_010_connection_string_parser.py Parser tests

   • Coverage: ~40 test cases
   • Test Categories:
     • Valid parsing scenarios
     • Braced value handling
     • Escape sequence processing
     • Error detection and reporting
     • Edge cases (empty, whitespace, unicode)
   • What to Look For:
     • Test coverage of MS-ODBCSTR spec
     • Error message clarity
     • Edge case handling

7. tests/test_011_connection_string_allowlist.py Allowlist tests

   • Coverage: ~25 test cases
   • Test Categories:
     • Key normalization (synonyms)
     • Parameter filtering
     • Reserved parameter blocking
     • Unknown parameter detection
   • What to Look For:
     • All ODBC parameters tested
     • Synonym mappings validated
     • Security (Driver/APP blocking)

8. tests/test_012_connection_string_integration.py Integration tests

   • Coverage: ~20 test cases
   • Test Categories:
     • End-to-end parsing-> filtering-> building
     • Real connection string scenarios
     • Error propagation from connect() API
     • kwargs override behavior
   • What to Look For:
     • Real-world usage patterns
     • Error messages match user expectations
     • Backward compatibility where possible

9. tests/test_003_connection.py (Updated)

   • What Changed:
     • Updated assertions in test_construct_connection_string()
     • Updated assertions in test_connection_string_with_attrs_before()
     • Updated assertions in test_connection_string_with_odbc_param()
   • What to Look For:
     • Assertions match new builder output format
     • No semicolons in middle of assertions (builder handles formatting)

10. tests/test_006_exceptions.py (Updated)

    • What Changed:
      • test_connection_error() now expects ConnectionStringParseError
      • Updated error message assertions
    • What to Look For:
      • Proper exception type changes
      • Error messages are helpful

4. Documentation (understand design decisions)

11. docs/connection_string_allow_list_design.md Read this

    • Content:
      • Design rationale and motivation
      • Architecture decisions
      • Security considerations
      • Future enhancements
    • What to Look For:
      • Justification for approach
      • Trade-offs discussed
      • Security implications understood

12. docs/parser_state_machine.md

    • Content:
      • Detailed state machine for parser
      • Character-by-character processing flow
      • Error handling states
      • Escape sequence examples
    • What to Look For:
      • Parser logic is well-documented
      • Edge cases covered in examples

Areas to Focus On

1. Security

• Reserved Parameters: Verify Driver and APP cannot be set by users
• Allowlist Completeness: Check all ODBC Driver 18 params are included
• Escape Handling: Ensure no injection via special characters

2. Error Handling

• Error Messages: Are they clear and actionable?
• Error Collection: Multiple errors reported together?
• Backward Compatibility: Do meaningful errors replace cryptic ODBC errors?

3. Performance

• Parsing Overhead: There is no string split being used during parsing. Hence the cost of multiple allocation of strings is avoided.
• No Regex in Hot Path: Parser uses character-by-character processing. Regex have been known to cause problems and its advisable to stay away from them.
• Allowlist Lookup: O(1) dict lookups, minimal overhead

4. Correctness

• Synonym Handling: All common aliases map correctly
• Case Insensitivity: Keys normalized consistently

Testing Strategy

Test Coverage Map

Behavior Changes

Should be aware of these behavioral changes:

1. Unknown Parameters Now Raise Errors

Before:

connect("Server=localhost;FakeParam=value") # Silently ignored

After:

connect("Server=localhost;FakeParam=value")
# Raises: ConnectionStringParseError: Unknown keyword 'FakeParam' is not recognized

2. Malformed Strings Caught Early

Before:

connect("ServerLocalhost") # ODBC error later

After:

connect("ServerLocalhost")
# Raises: ConnectionStringParseError: Incomplete specification: keyword 'ServerLocalhost' has no value

3. Reserved Parameters Blocked

Before:

connect("Server=localhost;Driver={SQL Server}") # Maybe ignored

After:

connect("Server=localhost;Driver={SQL Server}")
# Raises: ValueError: Connection parameter 'Driver' is reserved and controlled by the driver

Key Concepts to Understand

ODBC Connection String Format

key1=value1;key2=value2;key3={val;ue}

Braced Values

Used when value contains semicolons or special characters:

PWD={my;pass;word} -> Password is "my;pass;word"

Escape Sequences

• {{-> { (escaped left brace)
• }}-> } (escaped right brace)

Example:

PWD={a}}b{{c} -> Password is "a}b{c"