Better explanation of using custom/self-signed certificates

[Kilo59]

Feature Request

I'm trying to understand how to properly use Custom Certificates (or Self Signed Certificates).
The documentation and related error messages are confusing.

The documentation here says that if I want to use a custom/self-signed certificate I need to use the neo4j+ssc scheme.
https://neo4j.com/docs/api/python-driver/5.28/api.html#uri

However when I do that I get an error.

neo4j.exceptions.ConfigurationError: The config settings "encrypted", "trust", "trusted_certificates", and "ssl_context" can only be used with the URI schemes ['bolt', 'neo4j']. Use the other URI schemes ['bolt+ssc', 'bolt+s', 'neo4j+ssc', 'neo4j+s'] for setting encryption settings.

I've tried using both SSL ssl_context and trusted_certificates, they both throw the same error.

MRE(ish) below 👇

import pathlib
import ssl

import neo4j
from neo4j import GraphDatabase

from my_app import CUSTOM_CERT_PATH # pathlib.Path object

def get_config() -> dict:
   return {"user_name": "username", "password": "pw", "uri": "neo4j+ssc://12345.databases.neo4j.io"}

def get_neo4j_driver_ssl_context(config: dict[str, str]) -> neo4j.Driver:
    return GraphDatabase.driver(
        config["uri"],
        auth=(config["username"], config["password"]),
        ssl_context=ssl.create_default_context(cafile=CUSTOM_CERT_PATH),
    )

def get_neo4j_driver_custom_ca(config: dict[str, str]) -> neo4j.Driver:
    return GraphDatabase.driver(
        config["uri"],
        auth=(config["username"], config["password"]),
        trusted_certificates=neo4j.TrustCustomCAs(str(CUSTOM_CERT_PATH))
    )

if __name__ == "__main__":
  config = get_config()
  ssl_ctx_driver = get_neo4j_driver_ssl_context(config)
  ssl_ctx_driver.verify_connectivity()

  custom_ca_driver = get_neo4j_driver_custom_ca(config)
  custom_ca_driver.verify_connectivity()

I should note I've used this exact cert file with multiple other (non Aura/Neo4J) services without issue.

The DB in question is on/in Aura if the uri didn't make that clear.

I can of course connect when I drop the the +ssc scheme (example: neo4j+ssc://12345.databases.neo4j.io) but it goes without saying I don't want to to use the driver without encryption.

Version Info

neo4j v5.28.2
Python 3.13.5

Pitch

Please update or expand your documentation around using custom certificates along with examples, or fix the underlying issue that prevents using self signed certs in the manor that is described.

[Kilo59]

Here's the more detailed stacktrace.

Notice 👉 parsed = ParseResult(scheme='neo4j+ssc', netloc='12345.databases.neo4j.io', path='', params='', query='', fragment=''), client_certificate = None
seems to indicate that it parsed the uri scheme correctly. Which is what makes the error all the more confusing.

It seems like this is just a bug.

cls = <class 'neo4j._sync.driver.GraphDatabase'>, uri = 'neo4j+ssc://12345.databases.neo4j.io', auth = <neo4j._sync.auth_management.StaticAuthManager object at 0x10aad42f0>
config = {'auth': <neo4j._sync.auth_management.StaticAuthManager object at 0x10aad42f0>, 'trusted_certificates': <neo4j._conf.TrustCustomCAs object at 0x10aa13b60>}
driver_type = 'DRIVER_NEO4J', security_type = 'SECURITY_TYPE_SELF_SIGNED_CERTIFICATE'
parsed = ParseResult(scheme='neo4j+ssc', netloc='12345.databases.neo4j.io', path='', params='', query='', fragment=''), client_certificate = None

    @classmethod
    def driver(
        cls,
        uri: str,
        *,
        auth: _TAuth | AuthManager = None,
        **config,
    ) -> Driver:
        """
        Create a driver.
    
        :param uri: the connection URI for the driver,
            see :ref:`uri-ref` for available URIs.
        :param auth: the authentication details,
            see :ref:`auth-ref` for available authentication details.
        :param config: driver configuration key-word arguments,
            see :ref:`driver-configuration-ref` for available
            key-word arguments.
        """
        driver_type, security_type, parsed = parse_neo4j_uri(uri)
    
        if not isinstance(auth, AuthManager):
            auth = AuthManagers.static(auth)
        config["auth"] = auth
    
        client_certificate = config.get("client_certificate")
        if isinstance(client_certificate, ClientCertificate):
            # using internal class until public factory is GA:
            # AsyncClientCertificateProviders.static
            config["client_certificate"] = (
                _StaticClientCertificateProvider(client_certificate)
            )
    
        # TODO: 6.0 - remove "trust" config option
        if "trust" in config and config["trust"] not in {
            TRUST_ALL_CERTIFICATES,
            TRUST_SYSTEM_CA_SIGNED_CERTIFICATES,
        }:
            raise ConfigurationError(
                "The config setting `trust` values are {!r}".format(
                    [
                        TRUST_ALL_CERTIFICATES,
                        TRUST_SYSTEM_CA_SIGNED_CERTIFICATES,
                    ]
                )
            )
    
        if "trusted_certificates" in config and not isinstance(
            config["trusted_certificates"], TrustStore
        ):
            raise ConfigurationError(
                'The config setting "trusted_certificates" must be of '
                "type neo4j.TrustAll, neo4j.TrustCustomCAs, or"
                "neo4j.TrustSystemCAs but was {}".format(
                    type(config["trusted_certificates"])
                )
            )
    
        if security_type in {
            SECURITY_TYPE_SELF_SIGNED_CERTIFICATE,
            SECURITY_TYPE_SECURE,
        } and (
            "encrypted" in config
            or "trust" in config
            or "trusted_certificates" in config
            or "ssl_context" in config
        ):
            # TODO: 6.0 - remove "trust" from error message
>           raise ConfigurationError(
                'The config settings "encrypted", "trust", '
                '"trusted_certificates", and "ssl_context" can only be '
                "used with the URI schemes {!r}. Use the other URI "
                "schemes {!r} for setting encryption settings.".format(
                    [
                        URI_SCHEME_BOLT,
                        URI_SCHEME_NEO4J,
                    ],
                    [
                        URI_SCHEME_BOLT_SELF_SIGNED_CERTIFICATE,
                        URI_SCHEME_BOLT_SECURE,
                        URI_SCHEME_NEO4J_SELF_SIGNED_CERTIFICATE,
                        URI_SCHEME_NEO4J_SECURE,
                    ],
                )
            )
E           neo4j.exceptions.ConfigurationError: The config settings "encrypted", "trust", "trusted_certificates", and "ssl_context" can only be used with the URI schemes ['bolt', 'neo4j']. Use the other URI schemes ['bolt+ssc', 'bolt+s', 'neo4j+ssc', 'neo4j+s'] for setting encryption settings.

.venv/lib/python3.13/site-packages/neo4j/_sync/driver.py:251: ConfigurationError

[robsdedude]

Thanks for reaching out and sorry for the long wait I was on vacation.

This is not a bug, but arguably a confusing design. Let me investigate what our other official drivers do before I consider changing the code. Should I decide to keep the current implementation, I will certainly update the docs and potentially the error message.

Now to explain the design:

The +s[sc] suffixes are to be thought of as shorthand notation for configuring the security settings of the driver

• +s: turn on encryption (TLS) & verify the certificate + host-name etc.
• +ssc: turn on encryption, trust any certificate (vulnerable to man-in-the-middle attacks)
• no suffix: by default no encryption, unless explicit configuration states otherwise.

I can totally see how that latter is not obvious from the docs nor really intuitive.

So in your case, if you use neo4j://... and pass encrypted=True, you still get TLS. If you enable debug logging, you should be able to see some log entries along the lines of SECURE or such that confirm this.

Note that changing the design requires deciding what should happen in unclear situations and what situations are to be considered unclear

• suffix +s & encrypted=False => error? if not, what has precedence?
• no suffix & encrypted=True => error? if not, what has precedence? (note also that an error would mean a breaking change without a smooth migration path)
• ...

[Kilo59]

Clarification, in my initial post I used self-signed and custom certificates interchangeably.
I don't care at all about working with self-signed certificates, I just need to be able to set the sdk to use a custom certificate (still signed by a valid certificate authority).

[robsdedude]

I don't care at all about working with self-signed certificates, I just need to be able to set the sdk to use a custom certificate

Same idea: use no suffix to be able to take full control over the TLS settings. Then set

• trusted_certificates=TrustCustomCAs(...)
  https://neo4j.com/docs/api/python-driver/5.28/api.html#neo4j.TrustCustomCAs
  https://neo4j.com/docs/api/python-driver/5.28/api.html#trusted-certificates-ref
• encrypted=True
  As the last link states is necessary for the first option to take effect.
  https://neo4j.com/docs/api/python-driver/5.28/api.html#encrypted-ref

[Kilo59]

@robsdedude
Thanks I'm testing this now. Seems to be working locally, now I'm just in the process of testing it in our dev environment.

As far as the debug logger goes, is there a specific logger I can grab for the debug logs rather than seeing all of the debug logs for the neo4j package?
Something like "neo4j.secure" ???

[robsdedude]

is there a specific logger I can grab

https://neo4j.com/docs/api/python-driver/5.28/api.html#logging

From the top of my head, I'd say this will be logged on the neo4j.io logger which also logs all bolt messages exchanged between server and driver. Therefore, it's rather verbose. So a clear yesn't on that front 🙃

Just to be explicit: I recommend to stay clear of programmatically consuming the driver logs. They're not an API, but solely meant for debugging the driver. Even using them for observability is a bit of stretch even though I appreciate that there's a desire for the latter in a production system.

[Kilo59]

@robsdedude

I'm being told by a support engineer that Aura only supports neo4j+s:// or neo4j+ssc://.
So it seems like it's currently impossible to specify a custom cert (when using the python driver) with Aura?

---

As an aside I still don't think I understand why there is a separate trusted_certificates parameter for driver configuration, most other python libraries just use ssl_context for this.

https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification
https://www.python-httpx.org/advanced/ssl/
https://yanyongyu.github.io/githubkit/usage/configuration/#ssl_verify
https://pymongo.readthedocs.io/en/stable/examples/tls.html#tls-ssl-and-pymongo
https://python-gitlab.readthedocs.io/en/stable/api-usage-advanced.html#ssl-certificate-verification

[robsdedude]

I'm being told by a support engineer that Aura only supports neo4j+s:// or neo4j+ssc://.
So it seems like it's currently impossible to specify a custom cert (when using the python driver) with Aura?

I think that could be framed a little better. It's true, that Aura does not accept unencrypted traffic. Again, the +s and +ssc suffixes are just short-hand notation. The following examples show 100% equivalent driver configs (for driver 5.28 - some things might be implementation details that could change) (i.e., the server has no chance of knowing the difference):

`+s` equivalent config options

short-hand

with neo4j.GraphDatabase.driver("neo4j+s://...", auth=...) as driver:
    ...

with encrypted config

with neo4j.GraphDatabase.driver("neo4j://...", auth=..., encrypted=True) as driver:
    ...

with custom ssl context

# The exact SSL context values chosen by `+s` are to be considered an implementation detail
SSL_CTX = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
SSL_CTX.minimum_version = ssl.TLSVersion.TLSv1_2
SSL_CTX.check_hostname = True
SSL_CTX.verify_mode = ssl.CERT_REQUIRED
SSL_CTX.load_default_certs()

with neo4j.GraphDatabase.driver("neo4j://...", auth=..., ssl_context=SSL_CTX) as driver:
    ...

`+ssc` equivalent config options

short-hand

with neo4j.GraphDatabase.driver("neo4j+ssc://...", auth=...) as driver:
    ...

with encrypted config

with neo4j.GraphDatabase.driver(
    "neo4j://...",
    auth=...,
    encrypted=True,
    trusted_certificates=neo4j.TrustAll()
) as driver:
    ...

with custom ssl context

# The exact SSL context values chosen by `+ssc` are to be considered an implementation detail
SSL_CTX = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
SSL_CTX.minimum_version = ssl.TLSVersion.TLSv1_2
SSL_CTX.check_hostname = False
SSL_CTX.verify_mode = ssl.CERT_NONE

with neo4j.GraphDatabase.driver("neo4j://...", auth=..., ssl_context=SSL_CTX) as driver:
    ...

As an aside I still don't think I understand why there is a separate trusted_certificates parameter for driver configuration, most other python libraries just use ssl_context for this.

I guess it's a mixture of historically grown like this and because the driver tries to offer a more beginner-friendly API. You don't have to mess with configuring your own SSL context if you want to do a common task as configuring the driver to only accept a custom/specific certificate. Note how many of these option in the API docs state that they're incompatible with some other options. Here's an overview of the different (mutually exclusive) approaches:

• use short-hand +s or +ssc
• use encrypted=True
  • optionally also trusted_certificates
  • optionally also client_certificate
• use ssl_context and gain full control (and responsibility) over the SSL configuration
• nothing of the above and get an unencrypted connection (if the server supports it - Aura doesn't)