[BUG] promptflow_vectordb.tool.common_index_lookup.search' is not found in the current environment

[OlegDStassenko]

Describe the bug
I encounter an error when trying to run my flow in VS Code:

pf.flow.test failed with PackageToolNotFoundError: Package tool 'promptflow_vectordb.tool.common_index_lookup.search' is not found in the current environment. All available package tools are: [].

Running Information
Promptflow Package Versions :
prompt_toolkit 3.0.48
promptflow 1.17.2
promptflow-azure 1.17.2
promptflow-core 1.17.2
promptflow-devkit 1.17.2
promptflow-tools 1.6.0
promptflow-tracing 1.17.2
promptflow_vectordb 0.2.16

• Operating System: 20.04.1-Ubuntu
• Python Version : 3.10.15

Additional context
I have tried the following solutions, but the issue persists:

1. Setting the environment variable PF_DISABLE_TRACING
   Tried both false and true → Same bug

2. Manually setting BUILD_INFO
   Attempted to add BUILD_INFO but the error remains.

if "BUILD_INFO" not in os.environ:
    build_number = os.environ.get("BUILD_BUILDNUMBER", "")
    build_info = {"build_number": f"ci-{build_number}" if build_number else "local-pytest"}
    os.environ["BUILD_INFO"] = json.dumps(build_info)

[vsantele]

I have the same issue after upgrading promptflow-tools from 1.4.0 to 1.6.0.

Here is the error I have. It's related to protobuf and grpc.

[2025-03-13 11:22:39 +0100][promptflow._core.tools_manager][WARNING] - Failed to load tools from package promptflow_vectordb: cannot import name 'runtime_version' from 'google.protobuf' ([redacted]\Lib\site-packages\google\protobuf\__init__.py), traceback: Traceback (most recent call last):
  File "[redacted]\Lib\site-packages\promptflow\_core\tools_manager.py", line 99, in collect_package_tools
    importlib.import_module(m)  # Import the module to make sure it is valid
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[redacted]\Lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "[redacted]\Lib\site-packages\promptflow_vectordb\tool\common_index_lookup.py", line 18, in <module>
    from .common_index_lookup_extensions import build_search_func
  File "[redacted]\Lib\site-packages\promptflow_vectordb\tool\common_index_lookup_extensions\__init__.py", line 1, in <module>
    from .utils import build_search_func  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[redacted]\Lib\site-packages\promptflow_vectordb\tool\common_index_lookup_extensions\utils.py", line 9, in <module>
    from ..common_index_lookup_utils.constants import QueryTypes
  File "[redacted]\Lib\site-packages\promptflow_vectordb\tool\common_index_lookup_utils\__init__.py", line 28, in <module>
    from .weaviate import (  # noqa: F401
  File "[redacted]\Lib\site-packages\promptflow_vectordb\tool\common_index_lookup_utils\weaviate.py", line 4, in <module>
    import weaviate
  File "[redacted]\Lib\site-packages\weaviate\__init__.py", line 15, in <module>
    from .client import Client, WeaviateAsyncClient, WeaviateClient
  File "[redacted]\Lib\site-packages\weaviate\client.py", line 11, in <module>
    from weaviate.backup.backup import _BackupAsync
  File "[redacted]\Lib\site-packages\weaviate\backup\__init__.py", line 7, in <module>
    from weaviate.backup.backup import BackupStorage
  File "[redacted]\Lib\site-packages\weaviate\backup\backup.py", line 11, in <module>
    from weaviate.connect import ConnectionV4
  File "[redacted]\Lib\site-packages\weaviate\connect\__init__.py", line 7, in <module>
    from .v4 import ConnectionV4
  File "[redacted]\Lib\site-packages\weaviate\connect\v4.py", line 16, in <module>
    from grpc_health.v1 import health_pb2  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[redacted]\Lib\site-packages\grpc_health\v1\health_pb2.py", line 9, in <module>
    from google.protobuf import runtime_version as _runtime_version
ImportError: cannot import name 'runtime_version' from 'google.protobuf' ([redacted]\Lib\site-packages\google\protobuf\__init__.py)

Running Information
Promptflow Package Versions :
promptflow 1.17.1
promptflow-azure 1.17.1
promptflow-core 1.17.1
promptflow-devkit 1.17.1
promptflow-tools 1.6.0
promptflow-tracing 1.17.1
promptflow_vectordb 0.2.16
protobuf==4.25.6
grpcio==1.70.0
grpcio-tools==1.69.0
grpcio-health-checking==1.69.0
weviate-client==4.10.2

I tried to reinstall all the packages in a new env. It's working with these versions:
promptflow 1.17.2
promptflow-azure 1.17.2
promptflow-core 1.17.2
promptflow-devkit 1.17.2
promptflow-tools 1.6.0
promptflow-tracing 1.17.2
promptflow_vectordb 0.2.16
protobuf==5.29.3
grpcio==1.71.0
grpcio-tools==1.71.0
grpcio-health-checking==1.71.0
weviate-client==4.11.1

I don't know why pip replace my protobuf==5.29.3 package by protobuf==4.25.0 when I upgraded promptflow-tools to 1.6.0.

[OlegDStassenko]

Hi! Thank you for your responsiveness.
I tried reinstalling protobuf==5.29.3 but got the same result :(

Here is my package list.

Package Version

---

adal 1.2.7
aiohappyeyeballs 2.4.4
aiohttp 3.9.5
aiosignal 1.3.1
aniso8601 10.0.0
annotated-types 0.7.0
anyio 4.6.2.post1
argcomplete 3.6.0
asttokens 3.0.0
async-timeout 4.0.3
attrs 24.2.0
Authlib 1.3.1
azure-ai-ml 1.22.4
azure-common 1.1.28
azure-core 1.32.0
azure-cosmos 4.9.0
azure-graphrbac 0.61.2
azure-identity 1.19.0
azure-mgmt-authorization 4.0.0
azure-mgmt-containerregistry 10.3.0
azure-mgmt-core 1.5.0
azure-mgmt-keyvault 10.3.1
azure-mgmt-network 28.1.0
azure-mgmt-resource 23.3.0
azure-mgmt-storage 22.0.0
azure-monitor-opentelemetry-exporter 1.0.0b35
azure-search-documents 11.5.2
azure-storage-blob 12.19.0
azure-storage-file-datalake 12.14.0
azure-storage-file-share 12.20.0
azureml-core 1.59.0.post1
azureml-dataprep 5.1.6
azureml-dataprep-native 41.0.0
azureml-dataprep-rslex 2.22.5
azureml-fsspec 1.3.1
azureml-inference-server-http 1.4.0
azureml-mlflow 1.57.0
azureml-rag 0.2.38
backports.tarfile 1.2.0
backports.tempfile 1.0
backports.weakref 1.0.post1
bcrypt 4.3.0
beautifulsoup4 4.13.3
blinker 1.9.0
bs4 0.0.2
cachetools 5.5.0
cattrs 24.1.2
certifi 2024.8.30
cffi 1.17.1
chardet 5.2.0
charset-normalizer 3.4.0
click 8.1.7
cloudevents 1.11.0
cloudpickle 2.2.1
colorama 0.4.6
comm 0.2.2
contextlib2 21.6.0
cryptography 44.0.0
databricks-sdk 0.38.0
dataclasses-json 0.6.7
debugpy 1.8.9
decorator 5.1.1
defusedxml 0.7.1
Deprecated 1.2.15
deprecation 2.1.0
distro 1.9.0
dnspython 2.7.0
docker 7.1.0
docopt 0.6.2
docstring_parser 0.16
elastic-transport 8.17.0
elasticsearch 8.17.2
et_xmlfile 2.0.0
ete3 3.1.3
exceptiongroup 1.2.2
executing 2.1.0
faiss-cpu 1.7.4
fastapi 0.115.11
filelock 3.17.0
filetype 1.2.0
fixedint 0.1.6
Flask 3.0.3
Flask-Cors 5.0.0
flask-restx 1.3.0
frozenlist 1.5.0
fsspec 2023.10.0
future 1.0.0
gitdb 4.0.11
GitPython 3.1.41
google-api-core 2.23.0
google-auth 2.36.0
google_search_results 2.4.1
googleapis-common-protos 1.66.0
greenlet 3.1.1
grpcio 1.71.0
grpcio-health-checking 1.71.0
grpcio-tools 1.71.0
gunicorn 23.0.0
h11 0.14.0
h2 4.2.0
hpack 4.1.0
httpcore 1.0.7
httpx 0.28.0
httpx-sse 0.4.0
huggingface-hub 0.29.3
humanfriendly 10.0
hyperframe 6.1.0
idna 3.10
importlib_metadata 8.5.0
importlib_resources 6.5.2
inference-schema 1.8
iniconfig 2.0.0
ipykernel 6.29.5
ipython 8.30.0
isodate 0.7.2
itsdangerous 2.2.0
jaraco.classes 3.4.0
jaraco.context 6.0.1
jaraco.functools 4.1.0
jedi 0.19.2
jeepney 0.8.0
Jinja2 3.1.4
jiter 0.8.0
jmespath 1.0.1
joblib 1.4.2
jsonpatch 1.33
jsonpickle 4.0.0
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-path 0.3.3
jsonschema-specifications 2023.12.1
jupyter_client 8.6.3
jupyter_core 5.7.2
keyring 24.3.1
keyrings.alt 5.0.2
knack 0.12.0
langchain 0.3.20
langchain-community 0.3.19
langchain-core 0.3.43
langchain-openai 0.2.11
langchain-pinecone 0.2.0
langchain-qdrant 0.2.0
langchain-text-splitters 0.3.6
langdetect 1.0.9
langsmith 0.1.147
lazy-object-proxy 1.10.0
Markdown 3.7
MarkupSafe 3.0.2
marshmallow 3.23.1
matplotlib-inline 0.1.7
mlflow-skinny 2.18.0
mmh3 5.1.0
more-itertools 10.5.0
msal 1.31.1
msal-extensions 1.2.0
msrest 0.7.1
msrestazure 0.6.4.post1
multidict 6.1.0
mypy-extensions 1.0.0
ndg-httpsclient 0.5.1
nest-asyncio 1.6.0
numpy 1.26.4
oauthlib 3.2.2
openai 1.56.2
openapi-core 0.19.4
openapi-schema-validator 0.6.2
openapi-spec-validator 0.7.1
opencensus 0.11.4
opencensus-context 0.1.3
opencensus-ext-azure 1.1.13
opencensus-ext-logging 0.1.1
openpyxl 3.1.5
opentelemetry-api 1.30.0
opentelemetry-exporter-otlp-proto-common 1.30.0
opentelemetry-exporter-otlp-proto-http 1.30.0
opentelemetry-proto 1.30.0
opentelemetry-sdk 1.30.0
opentelemetry-semantic-conventions 0.51b0
orjson 3.10.12
packaging 24.2
pandas 2.2.3
paramiko 3.5.1
parse 1.20.2
parso 0.8.4
pathable 0.4.3
pathspec 0.12.1
pexpect 4.9.0
pgvector 0.2.5
pillow 11.0.0
pinecone-client 5.0.1
pinecone-plugin-inference 1.1.0
pinecone-plugin-interface 0.0.7
pip 25.0.1
pkginfo 1.12.1.2
platformdirs 4.3.6
pluggy 1.5.0
portalocker 2.10.1
prance 23.6.21.0
prompt_toolkit 3.0.48
promptflow 1.17.2
promptflow-azure 1.17.2
promptflow-core 1.17.2
promptflow-devkit 1.17.2
promptflow-tools 1.6.0
promptflow-tracing 1.17.2
promptflow_vectordb 0.2.16
propcache 0.2.1
proto-plus 1.25.0
protobuf 5.29.3
psutil 6.1.0
psycopg2-binary 2.9.10
ptyprocess 0.7.0
pure_eval 0.2.3
pyarrow 19.0.1
pyasn1 0.6.1
pyasn1_modules 0.4.1
pybars4 0.9.13
pycparser 2.22
pydantic 2.9.2
pydantic_core 2.23.4
pydantic-settings 2.6.1
pydash 7.0.7
Pygments 2.18.0
PyJWT 2.10.1
PyMeta3 0.5.1
pymongo 4.11.2
pymongo-schema 0.4.1
PyNaCl 1.5.0
pyOpenSSL 24.3.0
PySocks 1.7.1
pytest 8.3.4
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
pytz 2024.2
PyYAML 6.0.2
pyzmq 26.2.0
qdrant-client 1.13.3
rank-bm25 0.2.2
referencing 0.35.1
regex 2024.11.6
requests 2.32.3
requests-cache 1.1.1
requests-oauthlib 2.0.0
requests-toolbelt 1.0.0
rfc3339-validator 0.1.4
rpds-py 0.22.3
rsa 4.9
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.12
safetensors 0.5.3
scikit-learn 1.6.1
scipy 1.15.2
SecretStorage 3.3.3
semantic-kernel 1.16.0
setuptools 75.6.0
six 1.17.0
smmap 5.0.1
sniffio 1.3.1
soupsieve 2.6
SQLAlchemy 2.0.36
sqlparse 0.5.2
stack-data 0.6.3
starlette 0.46.1
strictyaml 1.7.3
tabulate 0.9.0
tenacity 9.0.0
threadpoolctl 3.5.0
tiktoken 0.8.0
tokenizers 0.21.0
tomli 2.2.1
tornado 6.4.2
tqdm 4.67.1
traitlets 5.14.3
transformers 4.49.0
typing_extensions 4.12.2
typing-inspect 0.9.0
tzdata 2025.1
url-normalize 1.4.3
urllib3 2.2.3
validators 0.34.0
waitress 3.0.2
wcwidth 0.2.13
weaviate-client 4.11.1
Werkzeug 3.1.3
wheel 0.44.0
wrapt 1.16.0
XlsxWriter 3.2.2
xlwt 1.3.0
yarl 1.18.3
zipp 3.21.0

How did you obtain the error logs? "[2025-03-13 11:22:39 +0100][promptflow._core.tools_manager][WARNING]" by running it or another way?

[vsantele]

I simply ran pf flow test --flow . --interactive --ui, sent a message and saw this log in my terminal.

I will try with your packages list.

[vsantele]

Update:
I tried with your packages list. The only thing I had to change was the psutil package. I remove the pin version and pip resolved it to 5.9.8.

The conflict is caused by:
    The user requested psutil==6.1.0
    azure-monitor-opentelemetry-exporter 1.0.0b35 depends on psutil<7 and >=5.9
    azureml-inference-server-http 1.4.0 depends on psutil<6.0.0; platform_system == "Windows"

But I'm on Windows, not Ubuntu like you.

My flow is working with a faiss index stored in an Azure ML workspace and using the same tool promptflow_vectordb.tool.common_index_lookup.search.

You don't have other errors before the PackageToolNotFoundError one?

[OlegDStassenko]

Thank you for your fast feedback !
I think I just stumbled upon a clue (by the way I set psutil==5.9.8)

[2025-03-13 13:54:45 +0000][promptflow._core.tools_manager][WARNING] - Failed to load tools from package promptflow_vectordb: [Errno 21] Is a directory: '\n(\n    (\n        (\n            float, \n            ((boolean) integer) biginteger\n        ) number,\n        (\n            oid, \n            dbref\n        ) string,\n        date,\n        timestamp,\n        unknown\n    ) general_scalar,\n    OBJECT\n) mixed_scalar_object\n;', traceback: Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.10/site-packages/promptflow/_core/tools_manager.py", line 99, in collect_package_tools
    importlib.import_module(m)  # Import the module to make sure it is valid
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/vscode/.local/lib/python3.10/site-packages/promptflow_vectordb/tool/common_index_lookup.py", line 18, in <module>
    from .common_index_lookup_extensions import build_search_func
  File "/home/vscode/.local/lib/python3.10/site-packages/promptflow_vectordb/tool/common_index_lookup_extensions/__init__.py", line 1, in <module>
    from .utils import build_search_func  # noqa: F401
  File "/home/vscode/.local/lib/python3.10/site-packages/promptflow_vectordb/tool/common_index_lookup_extensions/utils.py", line 9, in <module>
    from ..common_index_lookup_utils.constants import QueryTypes
  File "/home/vscode/.local/lib/python3.10/site-packages/promptflow_vectordb/tool/common_index_lookup_utils/__init__.py", line 12, in <module>
    from .mongodb import (  # noqa: F401
  File "/home/vscode/.local/lib/python3.10/site-packages/promptflow_vectordb/tool/common_index_lookup_utils/mongodb.py", line 5, in <module>
    from pymongo_schema.extract import extract_pymongo_client_schema
  File "/home/vscode/.local/lib/python3.10/site-packages/pymongo_schema/extract.py", line 48, in <module>
    from pymongo_schema.mongo_sql_types import get_type_string, common_parent_type
  File "/home/vscode/.local/lib/python3.10/site-packages/pymongo_schema/mongo_sql_types.py", line 93, in <module>
    TYPES_STRING_TREE = Tree(NEWICK_TYPES_STRING_TREE, format=8)
  File "/home/vscode/.local/lib/python3.10/site-packages/ete3/coretype/tree.py", line 212, in __init__
    read_newick(newick, root_node = self, format=format,
  File "/home/vscode/.local/lib/python3.10/site-packages/ete3/parser/newick.py", line 252, in read_newick
    with open(newick) as INPUT:
IsADirectoryError: [Errno 21] Is a directory: '\n(\n    (\n        (\n            float, \n            ((boolean) integer) biginteger\n        ) number,\n        (\n            oid, \n            dbref\n        ) string,\n        date,\n        timestamp,\n        unknown\n    ) general_scalar,\n    OBJECT\n) mixed_scalar_object\n;'

When I try to understand the error message, it seems to indicate an issue with the file path.
For more context, I work in Azure AI Foundry. My project runs on an Ubuntu virtual machine generated by the platform. VS Code opens on Windows and connects to this VM for development.

[OlegDStassenko]

Update:

This is very strange; it seems there is an issue in the pymongo-schema package version 0.4.1, specifically in the mongo_sql_types.py file.

I navigated to the directory:

cd /home/vscode/.local/lib/python3.10/site-packages/pymongo_schema/

And I edited the file using:

nano /home/vscode/.local/lib/python3.10/site-packages/pymongo_schema/mongo_sql_types.py

I replaced this:

NEWICK_TYPES_STRING_TREE = """
(
    (
        (
            float, 
            ((boolean) integer) biginteger
        ) number,
        (
            oid, 
            dbref
        ) string,
        date,
        timestamp,
        unknown
    ) general_scalar,
    OBJECT
) mixed_scalar_object
;"""

With this:

NEWICK_TYPES_STRING_TREE = "((float, (boolean, integer, biginteger) number), (oid, dbref) string, date, timestamp, unknown) general_scalar"

I ran the script, and it seems to be working, but I’m not sure if this is a viable solution. Curious.