Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://python.langchain.com/docs/integrations/vectorstores/google_bigquery_vector_search below:

Google BigQuery Vector Search | 🦜️🔗 LangChain

Google Cloud BigQuery Vector Search lets you use GoogleSQL to do semantic search, using vector indexes for fast approximate results, or using brute force for exact results.

This tutorial illustrates how to work with an end-to-end data and embedding management system in LangChain, and provides a scalable semantic search in BigQuery using theBigQueryVectorStore class. This class is part of a set of 2 classes capable of providing a unified data storage and flexible vector search in Google Cloud:

• BigQuery Vector Search: with BigQueryVectorStore class, which is ideal for rapid prototyping with no infrastructure setup and batch retrieval.
• Feature Store Online Store: with VertexFSVectorStore class, enables low-latency retrieval with manual or scheduled data sync. Perfect for production-ready user-facing GenAI applications.

%pip install --upgrade --quiet  langchain langchain-google-vertexai "langchain-google-community[featurestore]"

To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which restarts the current kernel.

import IPython

app = IPython.Application.instance()
app.kernel.do_shutdown(True)

If you don't know your project ID, try the following:

• Run gcloud config list.
• Run gcloud projects list.
• See the support page: Locate the project ID.

PROJECT_ID = ""  


! gcloud config set project {PROJECT_ID}

You can also change the REGION variable used by BigQuery. Learn more about BigQuery regions.

They will be your BigQuery Vector Store.

DATASET = "my_langchain_dataset"  
TABLE = "doc_and_vectors"  

• If you are using Colab to run this notebook, uncomment the cell below and continue.
• If you are using Vertex AI Workbench, check out the setup instructions here.

You may need to enable Vertex AI API in your project by running gcloud services enable aiplatform.googleapis.com --project {PROJECT_ID} (replace {PROJECT_ID} with the name of your project).

You can use any LangChain embeddings model.

from langchain_google_vertexai import VertexAIEmbeddings

embedding = VertexAIEmbeddings(
    model_name="textembedding-gecko@latest", project=PROJECT_ID
)

BigQuery Dataset and Table will be automatically created if they do not exist. See class definition here for all optional paremeters.

from langchain_google_community import BigQueryVectorStore

store = BigQueryVectorStore(
    project_id=PROJECT_ID,
    dataset_name=DATASET,
    table_name=TABLE,
    location=REGION,
    embedding=embedding,
)

all_texts = ["Apples and oranges", "Cars and airplanes", "Pineapple", "Train", "Banana"]
metadatas = [{"len": len(t)} for t in all_texts]

store.add_texts(all_texts, metadatas=metadatas)

query = "I'd like a fruit."
docs = store.similarity_search(query)
print(docs)

query_vector = embedding.embed_query(query)
docs = store.similarity_search_by_vector(query_vector, k=2)
print(docs)

The vectorstore supports two methods for applying filters to metadata fields when performing document searches:

• Dictionary-based Filters
  • You can pass a dictionary (dict) where the keys represent metadata fields and the values specify the filter condition. This method applies an equality filter between the key and the corresponding value. When multiple key-value pairs are provided, they are combined using a logical AND operation.
• SQL-based Filters
  • Alternatively, you can provide a string representing an SQL WHERE clause to define more complex filtering conditions. This allows for greater flexibility, supporting SQL expressions such as comparison operators and logical operators. Learn more about BigQuery operators.

docs = store.similarity_search_by_vector(query_vector, filter={"len": 6})
print(docs)

docs = store.similarity_search_by_vector(query_vector, filter="len = 6 AND len > 17")
print(docs)

BigQueryVectorStore offers a batch_search method for scalable Vector similarity search.

results = store.batch_search(
    embeddings=None,  
    queries=["search_query", "search_query"],  
)

You can also bring your own embeddings with the add_texts_with_embeddings method. This is particularly useful for multimodal data which might require custom preprocessing before the embedding generation.

items = ["some text"]
embs = embedding.embed(items)

ids = store.add_texts_with_embeddings(
    texts=["some text"], embs=embs, metadatas=[{"len": 1}]
)

You can simply use the method .to_vertex_fs_vector_store() to get a VertexFSVectorStore object, which offers low latency for online use cases. All mandatory parameters will be automatically transferred from the existing BigQueryVectorStore class. See the class definition for all the other parameters you can use.

Moving back to BigQueryVectorStore is equivalently easy with the .to_bq_vector_store() method.

store.to_vertex_fs_vector_store()  

• Vector store conceptual guide
• Vector store how-to guides

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4