Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://developers.google.com/bigquery/docs/multimodal-data-dataframes-tutorial below:

Analyze multimodal data in Python with BigQuery DataFrames

Stay organized with collections Save and categorize content based on your preferences.

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This tutorial shows you how to analyze multimodal data in a Python notebook by using BigQuery DataFrames classes and methods.

This tutorial uses the product catalog from the public Cymbal pet store dataset.

To upload a notebook already populated with the tasks covered in this tutorial, see BigFrames Multimodal DataFrame.

• Create multimodal DataFrames.
• Combine structured and unstructured data in a DataFrame.
• Transform images.
• Generate text and embeddings based on image data.
• Chunk PDFs for further analysis.

In this document, you use the following billable components of Google Cloud:

• BigQuery: you incur costs for the data that you process in BigQuery.
• BigQuery Python UDFs: you incur costs for using BigQuery DataFrames image transformation and chunk PDF methods.
• Cloud Storage: you incur costs for the objects stored in Cloud Storage.
• Vertex AI: you incur costs for calls to Vertex AI models.

To generate a cost estimate based on your projected usage, use the pricing calculator.

New Google Cloud users might be eligible for a

.

For more information about, see the following pricing pages:

• BigQuery pricing
• BigQuery Python UDF pricing
• Cloud Storage pricing
• Vertex AI pricing

1. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

2. Verify that billing is enabled for your Google Cloud project.

3. Enable the BigQuery, BigQuery Connection, Cloud Storage, and Vertex AI APIs.

   Enable the APIs

To get the permissions that you need to complete this tutorial, ask your administrator to grant you the following IAM roles:

• Create a connection: BigQuery Connection Admin (roles/bigquery.connectionAdmin)
• Grant permissions to the connection's service account: Project IAM Admin (roles/resourcemanager.projectIamAdmin)
• Create a Cloud Storage bucket: Storage Admin (roles/storage.admin)
• Run BigQuery jobs: BigQuery User (roles/bigquery.user)
• Create and call Python UDFs: BigQuery Data Editor (roles/bigquery.dataEditor)
• Create URLs that let you read and modify Cloud Storage objects: BigQuery ObjectRef Admin (roles/bigquery.objectRefAdmin)
• Use notebooks:
  • BigQuery Read Session User (roles/bigquery.readSessionUser)
  • Notebook Runtime User (roles/aiplatform.notebookRuntimeUser)
  • Notebook Runtime User (roles/aiplatform.notebookRuntimeUser)
  • Code Creator (roles/dataform.codeCreator)

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

In this section, you create the Cloud Storage bucket, connection, and notebook used in this tutorial.

Create a Cloud Storage bucket for storing transformed objects:

1. In the Google Cloud console, go to the Buckets page.

   Go to Buckets

2. Click add_box Create.

3. On the Create a bucket page, in the Get started section, enter a globally unique name that meets the bucket name requirements.

4. Click Create.

Create a Cloud resource connection and get the connection's service account. BigQuery uses the connection to access objects in Cloud Storage.

1. Go to the BigQuery page.

   Go to BigQuery

2. In the Explorer pane, click add Add data.

   The Add data dialog opens.

3. In the Filter By pane, in the Data Source Type section, select Business Applications.

   Alternatively, in the Search for data sources field, you can enter Vertex AI.

4. In the Featured data sources section, click Vertex AI.

5. Click the Vertex AI Models: BigQuery Federation solution card.

6. In the Connection type list, select Vertex AI remote models, remote functions and BigLake (Cloud Resource).

7. In the Connection ID field, type bigframes-default-connection.

8. Click Create connection.

9. Click Go to connection.

10. In the Connection info pane, copy the service account ID for use in a later step.

Grant the connection's service account the roles that it needs to access Cloud Storage and Vertex AI. You must grant these roles in the same project you created or selected in the Before you begin section.

To grant the role, follow these steps:

1. Go to the IAM & Admin page.

   Go to IAM & Admin

2. Click person_add Grant access.

3. In the New principals field, enter the service account ID that you copied earlier.

4. In the Select a role field, choose Cloud Storage, and then select Storage Object User.

5. Click Add another role.

6. In the Select a role field, select Vertex AI, and then select Vertex AI User.

7. Click Save.

Create a notebook where you can run Python code:

1. Go to the BigQuery page.

   Go to BigQuery

2. In the tab bar of the editor pane, click the arrow_drop_down drop-down arrow next to add_box SQL query, and then click Notebook.

3. In the Start with a template pane, click Close.

4. Click Connect > Connect to a runtime.

5. If you have an existing runtime, accept the default settings and click Connect. If you don't have an existing runtime, select Create new Runtime, and then click Connect.

   It might take several minutes for the runtime to get set up.

Create a multimodal DataFrame that integrates structured and unstructured data by using the from_glob_path method of the Session class:

1. In the notebook, create a code cell and copy the following code into it:

2. Click play_circle_filled Run.

   The final call to df_image returns the images that have been added to the DataFrame. Alternatively, you could call the .display method.

Combine text and image data in the multimodal DataFrame:

1. In the notebook, create a code cell and copy the following code into it:

2. Click Run play_circle_filled.

   The code returns the DataFrame data.

3. In the notebook, create a code cell and copy the following code into it:

4. Click Run play_circle_filled.

   The code returns images from the DataFrame where the author column value is alice.

Transform image data by using the following methods of the Series.BlobAccessor class:

• image_blur
• image_normalize
• image_resize

The transformed images are written to Cloud Storage.

Transform images:

1. In the notebook, create a code cell and copy the following code into it:
2. Update all references to {dst_bucket} to refer to the bucket that you created, in the format gs://mybucket.

3. Click Run play_circle_filled.

   The code returns the original images as well as all of their transformations.

Generate text from multimodal data by using the predict method of the GeminiTextGenerator class:

1. In the notebook, create a code cell and copy the following code into it:

2. Click Run play_circle_filled.

   The code returns the first two images in df_image, along with text generated in response to the question what item is it? for both images.

3. In the notebook, create a code cell and copy the following code into it:

4. Click Run play_circle_filled.

   The code returns the first two images in df_image, with text generated in response to the question what item is it? for the first image, and text generated in response to the question what color is the picture? for the second image.

Generate embeddings for multimodal data by using the predict method of the MultimodalEmbeddingGenerator class:

1. In the notebook, create a code cell and copy the following code into it:

2. Click Run play_circle_filled.

   The code returns the embeddings generated by a call to an embedding model.

Chunk PDF objects by using the pdf_chunk method of the Series.BlobAccessor class:

1. In the notebook, create a code cell and copy the following code into it:

2. Click Run play_circle_filled.

   The code returns the chunked PDF data.

Caution: Deleting a project has the following effects:
• Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Google Cloud console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4