refactor(auth): replace pyOpenSSL with standard ssl and cryptography

[nbayati]

Replace pyOpenSSL with standard library ssl for mTLS transport and update key decryption to use cryptography library.

This change also enhances security for handling private keys by:

• Using Linux memfd_create for RAM-backed in-memory files to avoid writing secrets to physical storage.
• Encrypting plaintext keys on-the-fly before writing to fallback temporary files on disk.
• Securely wiping temporary files with null bytes before deletion.

Fixes #16920
Closes #16921
fixes #17412

[gemini-code-assist]

Code Review

This pull request refactors the library to remove the pyOpenSSL dependency, migrating certificate and key management to the cryptography library and Python's standard ssl module. It introduces a secure three-tier fallback strategy for handling mTLS credentials via the new secure_cert_key_paths utility, which utilizes RAM-backed virtual files on Linux to minimize disk exposure. Review feedback suggests simplifying the manual context manager orchestration within this utility and using os.fdopen instead of os.write for more robust and idiomatic file descriptor operations.

[parthea]

@nbayati , Please could you address the failing presubmits?

[nbayati]

/gemini review

[gemini-code-assist]

Code Review

This pull request removes the dependency on pyOpenSSL across the library, replacing it with Python's standard ssl library and the cryptography package. It also introduces a secure three-tier fallback strategy for handling client certificates and private keys. The review feedback highlights a potential NameError in urllib3.py due to an unimported module reference, suggests simplifying context manager exit logic in _mtls_helper.py, recommends verifying write permissions for /dev/shm to prevent failures in restricted environments, and advises defensively encoding leaf certificate callback values to bytes.

[daniel-sanche]

nit: I wonder if making a helper or loop for these could be cleaner?

[nbayati]

Done! Added a loop.

[nbayati]

I tested the refactored paths manually by using the External Account X.509 provider to exchange certificates for access tokens and retrieve GCS buckets across different transports and credential initiation paths. The default transport (requests), urllib3, and aiohttp (async) are all verified and working as expected after the refactoring. I've also verified the encrypted private key loading/decryption flow using the new cryptography-based helper.

A copy of the test script can be found at: paste/5209556048281600
Results: screenshot/7tvfZhJ6qoYEGbY

[parthea]

/gemini review

[gemini-code-assist]

Code Review

This pull request removes the dependency on pyOpenSSL and cffi across the codebase, transitioning to the standard library ssl module and the cryptography library for handling mTLS certificates and keys. It introduces a new secure context manager, secure_cert_key_paths, in _mtls_helper.py to securely manage certificate and key paths using Linux in-memory virtual files (memfd_create) or encrypted temporary files as a fallback. The setup configuration, tests, and environment files are updated accordingly. The feedback recommends removing a redundant import os statement inside the _secure_wipe_and_remove function since os is already imported at the module level.

[lsirac]

1. Critical Packaging Bug in setup.py

• File: packages/google-auth/setup.py (PR-level)
• Technical Note:
  The list cryptography_base_require is placed directly inside the DEPENDENCIES tuple without unpacking on line 27. This results in a nested structure: ("pyasn1-modules>=0.2.1", ["cryptography >= 38.0.3"]). When setuptools parses install_requires during packaging or installation, it iterates over the requirements and expects each item to be a flat string specifier. Encountering a nested list triggers a TypeError (e.g., TypeError: expected string or bytes-like object), completely blocking building or installing the package from source. Unpack the list inside DEPENDENCIES:

  DEPENDENCIES = (
      "pyasn1-modules>=0.2.1",
      *cryptography_base_require,
  )

---

2. Standard SSLContext Pointer Alignment and Safety

• File: packages/google-auth/google/auth/transport/_custom_tls_signer.py (Line 270)
• Technical Note:
  The pointer calculation in _cast_ssl_ctx_to_void_p_stdlib relies on standard CPython release alignment id(context) + ctypes.sizeof(ctypes.c_void_p) * 2 to obtain the raw pointer to the underlying OpenSSL structure SSL_CTX. This offset assumes a 2-pointer PyObject_HEAD header structure. This alignment is highly fragile and shifts under debug builds with Py_TRACE_REFS enabled (increasing header size to 4 pointers), free-threaded CPython (Python 3.13+, PEP 703), or alternative runtimes (PyPy). Reading from the wrong offset results in arbitrary memory pointer retrieval and uncatchable segmentation faults. Reject unsupported environments or enforce safe alternative C-API casting.

---

3. Missing Type Validation on Context Parameter

• File: packages/google-auth/google/auth/transport/_custom_tls_signer.py (Line 270)
• Technical Note:
  In attach_to_ssl_context, there is no type validation on ctx. If a caller passes None or an invalid object type, _cast_ssl_ctx_to_void_p_stdlib calculates a memory address offset beyond the bounds of the object structure and reads raw heap garbage as an SSL_CTX pointer. When passed to the Go offload library ConfigureSslContext, this bogus address is dereferenced, causing a hard segmentation fault that crashes the entire Python host process instead of raising a clean Python TypeError. Add an explicit assertion before pointer casting:

  if not isinstance(ctx, ssl.SSLContext):
      raise TypeError("ctx must be an instance of ssl.SSLContext")