Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 83 additions & 5 deletions libs/partners/ollama/langchain_ollama/llms.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,93 @@


class OllamaLLM(BaseLLM):
"""OllamaLLM large language models.

Example:
"""Ollama large language models.

Setup:
Install ``langchain-ollama`` and install/run the Ollama server locally:

.. code-block:: bash

pip install -U langchain-ollama
# Visit https://ollama.com/download to download and install Ollama
# (Linux users): start the server with ``ollama serve``

Download a model to use:

.. code-block:: bash

ollama pull llama3.1

Key init args — generation params:
model: str
Name of the Ollama model to use (e.g. ``'llama4'``).
temperature: Optional[float]
Sampling temperature. Higher values make output more creative.
num_predict: Optional[int]
Maximum number of tokens to predict.
top_k: Optional[int]
Limits the next token selection to the K most probable tokens.
top_p: Optional[float]
Nucleus sampling parameter. Higher values lead to more diverse text.
mirostat: Optional[int]
Enable Mirostat sampling for controlling perplexity.
seed: Optional[int]
Random number seed for generation reproducibility.

Key init args — client params:
base_url: Optional[str]
Base URL where Ollama server is hosted.
keep_alive: Optional[Union[int, str]]
How long the model stays loaded into memory.
format: Literal["", "json"]
Specify the format of the output.

See full list of supported init args and their descriptions in the params section.

Instantiate:
.. code-block:: python

from langchain_ollama import OllamaLLM

model = OllamaLLM(model="llama3")
print(model.invoke("Come up with 10 names for a song about parrots"))
llm = OllamaLLM(
model="llama3.1",
temperature=0.7,
num_predict=256,
# base_url="http://localhost:11434",
# other params...
)

Invoke:
.. code-block:: python

input_text = "The meaning of life is "
response = llm.invoke(input_text)
print(response)

.. code-block:: none

"a philosophical question that has been contemplated by humans for
centuries..."

Stream:
.. code-block:: python

for chunk in llm.stream(input_text):
print(chunk, end="")

.. code-block:: none

a philosophical question that has been contemplated by humans for
centuries...

Async:
.. code-block:: python

response = await llm.ainvoke(input_text)

# stream:
# async for chunk in llm.astream(input_text):
# print(chunk, end="")

"""

Expand Down
137 changes: 125 additions & 12 deletions libs/partners/openai/langchain_openai/llms/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,120 @@ def _stream_response_to_generation_chunk(


class BaseOpenAI(BaseLLM):
"""Base OpenAI large language model class."""
"""Base OpenAI large language model class.

Setup:
Install ``langchain-openai`` and set environment variable ``OPENAI_API_KEY``.

.. code-block:: bash

pip install -U langchain-openai
export OPENAI_API_KEY="your-api-key"

Key init args — completion params:
model_name: str
Name of OpenAI model to use.
temperature: float
Sampling temperature.
max_tokens: int
Max number of tokens to generate.
top_p: float
Total probability mass of tokens to consider at each step.
frequency_penalty: float
Penalizes repeated tokens according to frequency.
presence_penalty: float
Penalizes repeated tokens.
n: int
How many completions to generate for each prompt.
best_of: int
Generates best_of completions server-side and returns the "best".
logit_bias: Optional[dict[str, float]]
Adjust the probability of specific tokens being generated.
seed: Optional[int]
Seed for generation.
logprobs: Optional[int]
Include the log probabilities on the logprobs most likely output tokens.
streaming: bool
Whether to stream the results or not.

Key init args — client params:
openai_api_key: Optional[SecretStr]
OpenAI API key. If not passed in will be read from env var
``OPENAI_API_KEY``.
openai_api_base: Optional[str]
Base URL path for API requests, leave blank if not using a proxy or
service emulator.
openai_organization: Optional[str]
OpenAI organization ID. If not passed in will be read from env
var ``OPENAI_ORG_ID``.
request_timeout: Union[float, tuple[float, float], Any, None]
Timeout for requests to OpenAI completion API.
max_retries: int
Maximum number of retries to make when generating.
batch_size: int
Batch size to use when passing multiple documents to generate.

See full list of supported init args and their descriptions in the params section.

Instantiate:
.. code-block:: python

from langchain_openai.llms.base import BaseOpenAI

llm = BaseOpenAI(
model_name="gpt-3.5-turbo-instruct",
temperature=0.7,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
# openai_api_key="...",
# openai_api_base="...",
# openai_organization="...",
# other params...
)

Invoke:
.. code-block:: python

input_text = "The meaning of life is "
response = llm.invoke(input_text)
print(response)

.. code-block:: none

"a philosophical question that has been debated by thinkers and
scholars for centuries."

Stream:
.. code-block:: python

for chunk in llm.stream(input_text):
print(chunk, end="")

.. code-block:: none

a philosophical question that has been debated by thinkers and
scholars for centuries.

Async:
.. code-block:: python

response = await llm.ainvoke(input_text)

# stream:
# async for chunk in llm.astream(input_text):
# print(chunk, end="")

# batch:
# await llm.abatch([input_text])

.. code-block:: none

"a philosophical question that has been debated by thinkers and
scholars for centuries."

"""

client: Any = Field(default=None, exclude=True) #: :meta private:
async_client: Any = Field(default=None, exclude=True) #: :meta private:
Expand Down Expand Up @@ -80,7 +193,7 @@ class BaseOpenAI(BaseLLM):
openai_api_base: Optional[str] = Field(
alias="base_url", default_factory=from_env("OPENAI_API_BASE", default=None)
)
"""Base URL path for API requests, leave blank if not using a proxy or service
"""Base URL path for API requests, leave blank if not using a proxy or service
emulator."""
openai_organization: Optional[str] = Field(
alias="organization",
Expand Down Expand Up @@ -116,26 +229,26 @@ class BaseOpenAI(BaseLLM):
disallowed_special: Union[Literal["all"], Collection[str]] = "all"
"""Set of special tokens that are not allowed。"""
tiktoken_model_name: Optional[str] = None
"""The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
"""The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here."""
default_headers: Union[Mapping[str, str], None] = None
default_query: Union[Mapping[str, object], None] = None
# Configure a custom httpx client. See the
# [httpx documentation](https://www.python-httpx.org/api/#client) for more details.
http_client: Union[Any, None] = None
"""Optional ``httpx.Client``. Only used for sync invocations. Must specify
"""Optional ``httpx.Client``. Only used for sync invocations. Must specify
``http_async_client`` as well if you'd like a custom client for async
invocations.
"""
http_async_client: Union[Any, None] = None
"""Optional ``httpx.AsyncClient``. Only used for async invocations. Must specify
"""Optional ``httpx.AsyncClient``. Only used for async invocations. Must specify
``http_client`` as well if you'd like a custom client for sync invocations."""
extra_body: Optional[Mapping[str, Any]] = None
"""Optional additional JSON properties to include in the request parameters when
Expand Down