Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://www.mlflow.org/docs/latest/python_api/mlflow.entities.html below:

mlflow.entities

The mlflow.entities module defines entities returned by the MLflow REST API.

class mlflow.entities.Assessment(name: str, source: AssessmentSource, trace_id: Optional[str] = None, run_id: Optional[str] = None, rationale: Optional[str] = None, metadata: Optional[dict[str, str]] = None, span_id: Optional[str] = None, create_time_ms: Optional[int] = None, last_update_time_ms: Optional[int] = None, assessment_id: Optional[str] = None, error: Optional[AssessmentError] = None, expectation: Optional[mlflow.entities.assessment.ExpectationValue] = None, feedback: Optional[mlflow.entities.assessment.FeedbackValue] = None, overrides: Optional[str] = None, valid: Optional[bool] = None)[source]

Note

Experimental: This class may change or be removed in a future release without warning.

Base class for assessments that can be attached to a trace. An Assessment should be one of the following types:

• Expectations: A label that represents the expected value for a particular operation.

  For example, an expected answer for a user question from a chatbot.

• Feedback: A label that represents the feedback on the quality of the operation.

  Feedback can come from different sources, such as human judges, heuristic scorers, or LLM-as-a-Judge.

assessment_id: str | None = None

create_time_ms: int | None = None

error: AssessmentError | None = None

expectation: mlflow.entities.assessment.ExpectationValue | None = None

feedback: mlflow.entities.assessment.FeedbackValue | None = None

classmethod from_dictionary(d: dict[str, typing.Any]) → Assessment[source]

classmethod from_proto(proto)[source]

last_update_time_ms: int | None = None

metadata: dict[str, str] | None = None

name: str

overrides: str | None = None

rationale: str | None = None

run_id: str | None = None

source: AssessmentSource

span_id: str | None = None

to_dictionary()[source]

to_proto()[source]

trace_id: str | None = None

valid: bool | None = None

class mlflow.entities.AssessmentError(error_code: str, error_message: Optional[str] = None, stack_trace: Optional[str] = None)[source]

Note

Experimental: This class may change or be removed in a future release without warning.

Error object representing any issues during generating the assessment.

For example, if the LLM-as-a-Judge fails to generate an feedback, you can log an error with the error code and message as shown below:

from mlflow.entities import AssessmentError

error = AssessmentError(
    error_code="RATE_LIMIT_EXCEEDED",
    error_message="Rate limit for the judge exceeded.",
    stack_trace="...",
)

mlflow.log_feedback(
    trace_id="1234",
    name="faithfulness",
    source=AssessmentSourceType.LLM_JUDGE,
    error=error,
    # Skip setting value when an error is present
)

Parameters

• error_code – The error code.

• error_message – The detailed error message. Optional.

• stack_trace – The stack trace of the error. Truncated to 1000 characters before being logged to MLflow. Optional.

error_code: str

error_message: str | None = None

classmethod from_dictionary(error_dict)[source]

classmethod from_proto(proto)[source]

stack_trace: str | None = None

to_dictionary()[source]

to_proto()[source]

class mlflow.entities.AssessmentSource(source_type: str, source_id: str = 'default')[source]

Note

Experimental: This class may change or be removed in a future release without warning.

Source of an assessment (human, LLM as a judge with GPT-4, etc).

When recording an assessment, MLflow mandates providing a source information to keep track of how the assessment is conducted.

Parameters

• source_type – The type of the assessment source. Must be one of the values in the AssessmentSourceType enum or an instance of the enumerator value.

• source_id – An identifier for the source, e.g. user ID or LLM judge ID. If not provided, the default value “default” is used.

Note:

The legacy AssessmentSourceType “AI_JUDGE” is deprecated and will be resolved as “LLM_JUDGE”. You will receive a warning if using this deprecated value. This legacy term will be removed in a future version of MLflow.

Example:

Human annotation can be represented with a source type of “HUMAN”:

import mlflow
from mlflow.entities.assessment import AssessmentSource, AssessmentSourceType

source = AssessmentSource(
    source_type=AssessmentSourceType.HUMAN,  # or "HUMAN"
    source_id="bob@example.com",
)

LLM-as-a-judge can be represented with a source type of “LLM_JUDGE”:

import mlflow
from mlflow.entities.assessment import AssessmentSource, AssessmentSourceType

source = AssessmentSource(
    source_type=AssessmentSourceType.LLM_JUDGE,  # or "LLM_JUDGE"
    source_id="gpt-4o-mini",
)

Heuristic evaluation can be represented with a source type of “CODE”:

import mlflow
from mlflow.entities.assessment import AssessmentSource, AssessmentSourceType

source = AssessmentSource(
    source_type=AssessmentSourceType.CODE,  # or "CODE"
    source_id="repo/evaluation_script.py",
)

To record more context about the assessment, you can use the metadata field of the assessment logging APIs as well.

classmethod from_dictionary(source_dict: dict[str, typing.Any]) → AssessmentSource[source]

classmethod from_proto(proto)[source]

source_id: str = 'default'

source_type: str

to_dictionary() → dict[str, typing.Any][source]

to_proto()[source]

class mlflow.entities.AssessmentSourceType(source_type: str)[source]

Note

Experimental: This class may change or be removed in a future release without warning.

Enumeration and validator for assessment source types.

This class provides constants for valid assessment source types and handles validation and standardization of source type values. It supports both direct constant access and instance creation with string validation.

The class automatically handles: - Case-insensitive string inputs (converts to uppercase) - Deprecation warnings for legacy values (AI_JUDGE → LLM_JUDGE) - Validation of source type values

Available source types:

• HUMAN: Assessment performed by a human evaluator

• LLM_JUDGE: Assessment performed by an LLM-as-a-judge (e.g., GPT-4)

• CODE: Assessment performed by deterministic code/heuristics

• SOURCE_TYPE_UNSPECIFIED: Default when source type is not specified

Note

The legacy “AI_JUDGE” type is deprecated and automatically converted to “LLM_JUDGE” with a deprecation warning. This ensures backward compatibility while encouraging migration to the new terminology.

Example

Using class constants directly:

from mlflow.entities.assessment import AssessmentSource, AssessmentSourceType

# Direct constant usage
source = AssessmentSource(source_type=AssessmentSourceType.LLM_JUDGE, source_id="gpt-4")

String validation through instance creation:

# String input - case insensitive
source = AssessmentSource(
    source_type="llm_judge",  # Will be standardized to "LLM_JUDGE"
    source_id="gpt-4",
)

# Deprecated value - triggers warning
source = AssessmentSource(
    source_type="AI_JUDGE",  # Warning: converts to "LLM_JUDGE"
    source_id="gpt-4",
)

AI_JUDGE = 'AI_JUDGE'

CODE = 'CODE'

HUMAN = 'HUMAN'

LLM_JUDGE = 'LLM_JUDGE'

SOURCE_TYPE_UNSPECIFIED = 'SOURCE_TYPE_UNSPECIFIED'

classmethod from_proto(proto_source_type) → str[source]

class mlflow.entities.Dataset(name: str, digest: str, source_type: str, source: str, schema: Optional[str] = None, profile: Optional[str] = None)[source]

Dataset object associated with an experiment.

property digest: str

String digest of the dataset.

classmethod from_proto(proto)[source]

property name: str

String name of the dataset.

property profile: str

String profile of the dataset.

property schema: str

String schema of the dataset.

property source: str

String source of the dataset.

property source_type: str

String source_type of the dataset.

to_dictionary()[source]

to_proto()[source]

class mlflow.entities.DatasetInput(dataset: Dataset, tags: Optional[list[InputTag]] = None)[source]

DatasetInput object associated with an experiment.

property dataset: Dataset

Dataset.

classmethod from_proto(proto)[source]

property tags: list[InputTag]

Array of input tags.

to_dictionary()[source]

to_proto()[source]

class mlflow.entities.Document(page_content: str, metadata: dict[str, typing.Any] = <factory>, id: Optional[str] = None)[source]

An entity used in MLflow Tracing to represent retrieved documents in a RETRIEVER span.

Parameters

• page_content – The content of the document.

• metadata – A dictionary of metadata associated with the document.

• id – The ID of the document.

classmethod from_langchain_document(document)[source]

classmethod from_llama_index_node_with_score(node_with_score)[source]

id: str | None = None

metadata: dict[str, typing.Any]

page_content: str

to_dict()[source]

class mlflow.entities.Expectation(name: str, value: Any, source: Optional[AssessmentSource] = None, trace_id: Optional[str] = None, metadata: Optional[dict[str, str]] = None, span_id: Optional[str] = None, create_time_ms: Optional[int] = None, last_update_time_ms: Optional[int] = None)[source]

Note

Experimental: This class may change or be removed in a future release without warning.

Represents an expectation about the output of an operation, such as the expected response that a generative AI application should provide to a particular user query.

Parameters

• name – The name of the assessment.

• value – The expected value of the operation. This can be any JSON-serializable value.

• source – The source of the assessment. If not provided, the default source is HUMAN.

• trace_id – The ID of the trace associated with the assessment. If unset, the assessment is not associated with any trace yet. should be specified.

• metadata – The metadata associated with the assessment.

• span_id – The ID of the span associated with the assessment, if the assessment should be associated with a particular span in the trace.

• create_time_ms – The creation time of the assessment in milliseconds. If unset, the current time is used.

• last_update_time_ms – The last update time of the assessment in milliseconds. If unset, the current time is used.

Example

from mlflow.entities import AssessmentSource, Expectation

expectation = Expectation(
    name="expected_response",
    value="The capital of France is Paris.",
    source=AssessmentSource(
        source_type=AssessmentSourceType.HUMAN,
        source_id="john@example.com",
    ),
    metadata={"project": "my-project"},
)

classmethod from_dictionary(d: dict[str, typing.Any]) → Expectation[source]

classmethod from_proto(proto) → Expectation[source]

property value: Any

class mlflow.entities.Experiment(experiment_id, name, artifact_location, lifecycle_stage, tags=None, creation_time=None, last_update_time=None)[source]

Experiment object.

DEFAULT_EXPERIMENT_NAME = 'Default'

property artifact_location

String corresponding to the root artifact URI for the experiment.

property creation_time

property experiment_id

String ID of the experiment.

classmethod from_proto(proto)[source]

property last_update_time

property lifecycle_stage

Lifecycle stage of the experiment. Can either be ‘active’ or ‘deleted’.

property name

String name of the experiment.

property tags

Tags that have been set on the experiment.

to_proto()[source]

class mlflow.entities.ExperimentTag(key, value)[source]

Tag object associated with an experiment.

classmethod from_proto(proto)[source]

property key

String name of the tag.

to_proto()[source]

property value

String value of the tag.

class mlflow.entities.Feedback(name: str = 'feedback', value: Optional[Union[float, int, str, bool, dict[str, float | int | str | bool], list[float | int | str | bool]]] = None, error: Optional[Union[Exception, AssessmentError]] = None, source: Optional[AssessmentSource] = None, trace_id: Optional[str] = None, metadata: Optional[dict[str, str]] = None, span_id: Optional[str] = None, create_time_ms: Optional[int] = None, last_update_time_ms: Optional[int] = None, rationale: Optional[str] = None, overrides: Optional[str] = None, valid: bool = True)[source]

Note

Experimental: This class may change or be removed in a future release without warning.

Represents feedback about the output of an operation. For example, if the response from a generative AI application to a particular user query is correct, then a human or LLM judge may provide feedback with the value "correct".

Parameters

• name – The name of the assessment. If not provided, the default name “feedback” is used.

• value – The feedback value. This can be one of the following types: - float - int - str - bool - list of values of the same types as above - dict with string keys and values of the same types as above

• error – An optional error associated with the feedback. This is used to indicate that the feedback is not valid or cannot be processed. Accepts an exception object, or an Expectation object.

• rationale – The rationale / justification for the feedback.

• source – The source of the assessment. If not provided, the default source is CODE.

• trace_id – The ID of the trace associated with the assessment. If unset, the assessment is not associated with any trace yet. should be specified.

• metadata – The metadata associated with the assessment.

• span_id – The ID of the span associated with the assessment, if the assessment should be associated with a particular span in the trace.

• create_time_ms – The creation time of the assessment in milliseconds. If unset, the current time is used.

• last_update_time_ms – The last update time of the assessment in milliseconds. If unset, the current time is used.

Example

from mlflow.entities import AssessmentSource, Feedback

feedback = Feedback(
    name="correctness",
    value=True,
    rationale="The response is correct.",
    source=AssessmentSource(
        source_type="HUMAN",
        source_id="john@example.com",
    ),
    metadata={"project": "my-project"},
)

property error_code: str | None

The error code of the error that occurred when the feedback was created.

property error_message: str | None

The error message of the error that occurred when the feedback was created.

classmethod from_dictionary(d: dict[str, typing.Any]) → Feedback[source]

classmethod from_proto(proto)[source]

property value: float | int | str | bool | dict[str, float | int | str | bool] | list[float | int | str | bool]

class mlflow.entities.FileInfo(path, is_dir, file_size)[source]

Metadata about a file or directory.

property file_size

Size of the file or directory. If the FileInfo is a directory, returns None.

classmethod from_proto(proto)[source]

property is_dir

Whether the FileInfo corresponds to a directory.

property path

String path of the file or directory.

to_proto()[source]

class mlflow.entities.InferenceTableLocation(full_table_name: str)[source]

Represents the location of a Databricks inference table.

Parameters

full_table_name – The fully qualified name of the inference table where the trace is stored, in the format of <catalog>.<schema>.<table>.

classmethod from_dict(d: dict[str, typing.Any]) → InferenceTableLocation[source]

classmethod from_proto(proto) → InferenceTableLocation[source]

full_table_name: str

to_dict() → dict[str, typing.Any][source]

to_proto()[source]

class mlflow.entities.InputTag(key: str, value: str)[source]

Input tag object associated with a dataset.

classmethod from_proto(proto)[source]

property key: str

String name of the input tag.

to_proto()[source]

property value: str

String value of the input tag.

class mlflow.entities.LifecycleStage[source]

ACTIVE = 'active'

DELETED = 'deleted'

classmethod is_valid(lifecycle_stage)[source]

classmethod matches_view_type(view_type, lifecycle_stage)[source]

classmethod view_type_to_stages(view_type=3)[source]

class mlflow.entities.LiveSpan(otel_span: opentelemetry.trace.span.Span, trace_id: str, span_type: str = 'UNKNOWN')[source]

A “live” version of the Span class.

The live spans are those being created and updated during the application runtime. When users start a new span using the tracing APIs within their code, this live span object is returned to get and set the span attributes, status, events, and etc.

add_event(event: SpanEvent)[source]

Add an event to the span.

Parameters

event – The event to add to the span. This should be a SpanEvent object.

from_dict(data: dict[str, typing.Any]) → Span[source]

Create a Span object from the given dictionary.

record_exception(exception: str | Exception)[source]

Record an exception on the span, adding an exception event and setting span status to ERROR.

Parameters

exception – The exception to record. Can be an Exception instance or a string describing the exception.

set_attribute(key: str, value: Any)[source]

Set a single attribute to the span.

set_attributes(attributes: dict[str, typing.Any])[source]

Set the attributes to the span. The attributes must be a dictionary of key-value pairs. This method is additive, i.e. it will add new attributes to the existing ones. If an attribute with the same key already exists, it will be overwritten.

set_inputs(inputs: Any)[source]

Set the input values to the span.

set_outputs(outputs: Any)[source]

Set the output values to the span.

set_span_type(span_type: str)[source]

Set the type of the span.

set_status(status: SpanStatusCode | str)[source]

Set the status of the span.

Parameters

status – The status of the span. This can be a SpanStatus object or a string representing of the status code defined in SpanStatusCode e.g. "OK", "ERROR".

class mlflow.entities.LoggedModel(experiment_id: str, model_id: str, name: str, artifact_location: str, creation_timestamp: int, last_updated_timestamp: int, model_type: Optional[str] = None, source_run_id: Optional[str] = None, status: LoggedModelStatus | int = LoggedModelStatus.READY, status_message: Optional[str] = None, tags: Optional[Union[list[LoggedModelTag], dict[str, str]]] = None, params: Optional[Union[list[LoggedModelParameter], dict[str, str]]] = None, metrics: Optional[list[Metric]] = None)[source]

MLflow entity representing a Model logged to an MLflow Experiment.

property artifact_location: str

String. Location of the model artifacts.

property creation_timestamp: int

Integer. Model creation timestamp (milliseconds since the Unix epoch).

property experiment_id: str

String. Experiment ID associated with this Model.

classmethod from_proto(proto)[source]

property last_updated_timestamp: int

Integer. Timestamp of last update for this Model (milliseconds since the Unix epoch).

property metrics: list[Metric] | None

List of metrics associated with this Model.

property model_id: str

String. Unique ID for this Model.

property model_type: str | None

String. Type of the model.

property model_uri: str

URI of the model.

property name: str

String. Name for this Model.

property params: dict[str, str]

Model parameters.

property source_run_id: str | None

String. MLflow run ID that generated this model.

property status: LoggedModelStatus

String. Current status of this Model.

property status_message: str | None

String. Descriptive message for error status conditions.

property tags: dict[str, str]

Dictionary of tag key (string) -> tag value for this Model.

to_dictionary() → dict[str, typing.Any][source]

to_proto()[source]

class mlflow.entities.LoggedModelInput(model_id: str)[source]

ModelInput object associated with a Run.

classmethod from_proto(proto)[source]

property model_id: str

Model ID.

to_proto()[source]

class mlflow.entities.LoggedModelOutput(model_id: str, step: int)[source]

ModelOutput object associated with a Run.

classmethod from_proto(proto)[source]

property model_id: str

Model ID

property step: str

Step at which the model was logged

to_proto()[source]

class mlflow.entities.LoggedModelParameter(key, value)[source]

MLflow entity representing a parameter of a Model.

classmethod from_proto(proto)[source]

property key

String key corresponding to the parameter name.

to_proto()[source]

property value

String value of the parameter.

class mlflow.entities.LoggedModelStatus(value)[source]

Enum for status of an mlflow.entities.LoggedModel.

FAILED = 'FAILED'

PENDING = 'PENDING'

READY = 'READY'

UNSPECIFIED = 'UNSPECIFIED'

classmethod from_int(status_int: int) → LoggedModelStatus[source]

classmethod from_proto(proto)[source]

static is_finalized(status) → bool[source]

Determines whether or not a LoggedModelStatus is a finalized status. A finalized status indicates that no further status updates will occur.

to_int() → int[source]

to_proto()[source]

class mlflow.entities.LoggedModelTag(key, value)[source]

Tag object associated with a Model.

classmethod from_proto(proto)[source]

property key

String name of the tag.

to_proto()[source]

property value

String value of the tag.

class mlflow.entities.Metric(key, value, timestamp, step, model_id: Optional[str] = None, dataset_name: Optional[str] = None, dataset_digest: Optional[str] = None, run_id: Optional[str] = None)[source]

Metric object.

property dataset_digest: str | None

String. Digest of the dataset associated with the metric.

property dataset_name: str | None

String. Name of the dataset associated with the metric.

classmethod from_dictionary(metric_dict)[source]

Create a Metric object from a dictionary.

Parameters

metric_dict (dict) – Dictionary containing metric information.

Returns

The Metric object created from the dictionary.

Return type

Metric

classmethod from_proto(proto)[source]

property key

String key corresponding to the metric name.

property model_id

ID of the Model associated with the metric.

property run_id: str | None

String. Run ID associated with the metric.

property step

Integer metric step (x-coordinate).

property timestamp

Metric timestamp as an integer (milliseconds since the Unix epoch).

to_dictionary()[source]

Convert the Metric object to a dictionary.

Returns

The Metric object represented as a dictionary.

Return type

dict

to_proto()[source]

property value

Float value of the metric.

class mlflow.entities.MlflowExperimentLocation(experiment_id: str)[source]

Represents the location of an MLflow experiment.

Parameters

experiment_id – The ID of the MLflow experiment where the trace is stored.

experiment_id: str

classmethod from_dict(d: dict[str, typing.Any]) → MlflowExperimentLocation[source]

classmethod from_proto(proto) → MlflowExperimentLocation[source]

to_dict() → dict[str, typing.Any][source]

to_proto()[source]

class mlflow.entities.NoOpSpan[source]

No-op implementation of the Span interface.

This instance should be returned from the mlflow.start_span context manager when span creation fails. This class should have exactly the same interface as the Span so that user’s setter calls do not raise runtime errors.

E.g.

with mlflow.start_span("span_name") as span:
    # Even if the span creation fails, the following calls should pass.
    span.set_inputs({"x": 1})
    # Do something

add_event(event: SpanEvent)[source]

property context

end(outputs: Optional[Any] = None, attributes: Optional[dict[str, typing.Any]] = None, status: Optional[Union[SpanStatus, str]] = None, end_time_ns: Optional[int] = None)[source]

property end_time_ns

The end time of the span in nanosecond.

property name

The name of the span.

property parent_id

The span ID of the parent span.

record_exception(exception: str | Exception)[source]

set_attribute(key: str, value: Any)[source]

set_attributes(attributes: dict[str, typing.Any])[source]

set_inputs(inputs: dict[str, typing.Any])[source]

set_outputs(outputs: dict[str, typing.Any])[source]

set_status(status: SpanStatus)[source]

property span_id

The ID of the span. This is only unique within a trace.

property start_time_ns

The start time of the span in nanosecond.

property status

The status of the span.

property trace_id

No-op span returns a special trace ID to distinguish it from the real spans.

class mlflow.entities.Param(key, value)[source]

Parameter object.

classmethod from_proto(proto)[source]

property key

String key corresponding to the parameter name.

to_proto()[source]

property value

String value of the parameter.

class mlflow.entities.Prompt(name: str, description: Optional[str] = None, creation_timestamp: Optional[int] = None, tags: Optional[dict[str, str]] = None)[source]

Entity representing a prompt in the MLflow Model Registry.

This contains prompt-level information (name, description, tags) but not version-specific content. To access version-specific content like the template, use PromptVersion.

property creation_timestamp: int | None

The creation timestamp of the prompt.

property description: str | None

The description of the prompt.

property name: str

The name of the prompt.

property tags: dict[str, str]

Prompt-level metadata as key-value pairs.

class mlflow.entities.Run(run_info: RunInfo, run_data: RunData, run_inputs: Optional[RunInputs] = None, run_outputs: Optional[RunOutputs] = None)[source]

Run object.

property data: RunData

The run data, including metrics, parameters, and tags.

Return type

mlflow.entities.RunData

classmethod from_proto(proto)[source]

property info: RunInfo

The run metadata, such as the run id, start time, and status.

Return type

mlflow.entities.RunInfo

property inputs: RunInputs

The run inputs, including dataset inputs.

Return type

mlflow.entities.RunInputs

property outputs: RunOutputs

The run outputs, including model outputs.

Return type

mlflow.entities.RunOutputs

to_dictionary() → dict[typing.Any, typing.Any][source]

to_proto()[source]

class mlflow.entities.RunData(metrics=None, params=None, tags=None)[source]

Run data (metrics and parameters).

classmethod from_proto(proto)[source]

property metrics

Dictionary of string key -> metric value for the current run. For each metric key, the metric value with the latest timestamp is returned. In case there are multiple values with the same latest timestamp, the maximum of these values is returned.

property params

Dictionary of param key (string) -> param value for the current run.

property tags

Dictionary of tag key (string) -> tag value for the current run.

to_dictionary()[source]

to_proto()[source]

class mlflow.entities.RunInfo(run_id, experiment_id, user_id, status, start_time, end_time, lifecycle_stage, artifact_uri=None, run_name=None)[source]

Metadata about a run.

property artifact_uri[source]

String root artifact URI of the run.

property end_time[source]

End time of the run, in number of milliseconds since the UNIX epoch.

property experiment_id

String ID of the experiment for the current run.

classmethod from_proto(proto)[source]

classmethod get_orderable_attributes()[source]

classmethod get_searchable_attributes()[source]

property lifecycle_stage

One of the values in LifecycleStage describing the lifecycle stage of the run.

property run_id[source]

String containing run id.

property run_name[source]

String containing run name.

property start_time[source]

Start time of the run, in number of milliseconds since the UNIX epoch.

property status[source]

One of the values in mlflow.entities.RunStatus describing the status of the run.

to_proto()[source]

property user_id[source]

String ID of the user who initiated this run.

class mlflow.entities.RunInputs(dataset_inputs: list[DatasetInput], model_inputs: Optional[list[LoggedModelInput]] = None)[source]

RunInputs object.

property dataset_inputs: list[DatasetInput]

Array of dataset inputs.

classmethod from_proto(proto)[source]

property model_inputs: list[LoggedModelInput]

Array of model inputs.

to_dictionary() → dict[str, typing.Any][source]

to_proto()[source]

class mlflow.entities.RunOutputs(model_outputs: list[LoggedModelOutput])[source]

RunOutputs object.

classmethod from_proto(proto)[source]

property model_outputs: list[LoggedModelOutput]

Array of model outputs.

to_dictionary() → dict[typing.Any, typing.Any][source]

to_proto()[source]

class mlflow.entities.RunStatus[source]

Enum for status of an mlflow.entities.Run.

FAILED = 4

FINISHED = 3

KILLED = 5

RUNNING = 1

SCHEDULED = 2

static all_status()[source]

static from_string(status_str)[source]

static is_terminated(status)[source]

static to_string(status)[source]

class mlflow.entities.RunTag(key, value)[source]

Tag object associated with a run.

classmethod from_proto(proto)[source]

property key

String name of the tag.

to_proto()[source]

property value

String value of the tag.

class mlflow.entities.SourceType[source]

Enum for originating source of a mlflow.entities.Run.

JOB = 2

LOCAL = 4

NOTEBOOK = 1

PROJECT = 3

SOURCETYPE_TO_STRING = {1: 'NOTEBOOK', 2: 'JOB', 3: 'PROJECT', 4: 'LOCAL', 5: 'UNKNOWN'}

UNKNOWN = 5

static from_string(status_str)[source]

static to_string(status)[source]

class mlflow.entities.Span(otel_span: opentelemetry.sdk.trace.ReadableSpan)[source]

A span object. A span represents a unit of work or operation and is the building block of Traces.

This Span class represents immutable span data that is already finished and persisted. The “live” span that is being created and updated during the application runtime is represented by the LiveSpan subclass.

property attributes: dict[str, typing.Any]

Get all attributes of the span.

Returns

A dictionary of all attributes of the span.

property end_time_ns: int | None

The end time of the span in nanosecond.

property events: list[SpanEvent]

Get all events of the span.

Returns

A list of all events of the span.

classmethod from_dict(data: dict[str, typing.Any]) → Span[source]

Create a Span object from the given dictionary.

classmethod from_dict_v2(data: dict[str, typing.Any]) → Span[source]

Create a Span object from the given dictionary in v2 schema.

get_attribute(key: str) → Optional[Any][source]

Get a single attribute value from the span.

Parameters

key – The key of the attribute to get.

Returns

The value of the attribute if it exists, otherwise None.

property inputs: Any

The input values of the span.

property name: str

The name of the span.

property outputs: Any

The output values of the span.

property parent_id: str | None

The span ID of the parent span.

property request_id: str

Deprecated. Use trace_id instead.

property span_id: str

The ID of the span. This is only unique within a trace.

property span_type: str

The type of the span.

property start_time_ns: int

The start time of the span in nanosecond.

property status: SpanStatus

The status of the span.

to_dict() → dict[str, typing.Any][source]

to_proto()[source]

Convert into OTLP compatible proto object to sent to the Databricks Trace Server.

property trace_id: str

The trace ID of the span, a unique identifier for the trace it belongs to.

class mlflow.entities.SpanEvent(name: str, timestamp: int = <factory>, attributes: dict[str, typing.Union[str, bool, int, float, typing.Sequence[str], typing.Sequence[bool], typing.Sequence[int], typing.Sequence[float]]] = <factory>)[source]

An event that records a specific occurrences or moments in time during a span, such as an exception being thrown. Compatible with OpenTelemetry.

Parameters

• name – Name of the event.

• timestamp – The exact time the event occurred, measured in nanoseconds. If not provided, the current time will be used.

• attributes – A collection of key-value pairs representing detailed attributes of the event, such as the exception stack trace. Attributes value must be one of [str, int, float, bool, bytes] or a sequence of these types.

attributes: dict[str, typing.Union[str, bool, int, float, typing.Sequence[str], typing.Sequence[bool], typing.Sequence[int], typing.Sequence[float]]]

classmethod from_exception(exception: Exception)[source]

Create a span event from an exception.

json()[source]

name: str

timestamp: int

to_proto()[source]

Convert into OTLP compatible proto object to sent to the Databricks Trace Server.

class mlflow.entities.SpanStatus(status_code: SpanStatusCode, description: str = '')[source]

Status of the span or the trace.

Parameters

• status_code – The status code of the span or the trace. This must be one of the values of the mlflow.entities.SpanStatusCode enum or a string representation of it like “OK”, “ERROR”.

• description – Description of the status. This should be only set when the status is ERROR, otherwise it will be ignored.

description: str = ''

status_code: SpanStatusCode

class mlflow.entities.SpanStatusCode(value)[source]

Enum for status code of a span

ERROR = 'ERROR'

OK = 'OK'

UNSET = 'UNSET'

static from_proto_status_code(status_code: str) → SpanStatusCode[source]

Convert a string status code to the corresponding SpanStatusCode enum value.

class mlflow.entities.SpanType[source]

Predefined set of span types.

AGENT = 'AGENT'

CHAIN = 'CHAIN'

CHAT_MODEL = 'CHAT_MODEL'

EMBEDDING = 'EMBEDDING'

LLM = 'LLM'

MEMORY = 'MEMORY'

PARSER = 'PARSER'

RERANKER = 'RERANKER'

RETRIEVER = 'RETRIEVER'

TOOL = 'TOOL'

UNKNOWN = 'UNKNOWN'

class mlflow.entities.Trace(info: TraceInfo, data: TraceData)[source]

A trace object.

Parameters

• info – A lightweight object that contains the metadata of a trace.

• data – A container object that holds the spans data of a trace.

data: TraceData

classmethod from_dict(trace_dict: dict[str, typing.Any]) → Trace[source]

classmethod from_json(trace_json: str) → Trace[source]

info: TraceInfo

static pandas_dataframe_columns() → list[str][source]

search_assessments(name: Optional[str] = None, *, span_id: Optional[str] = None, all: bool = False, type: Optional[Literal['expectation', 'feedback']] = None) → list['Assessment'][source]

Get assessments for a given name / span ID. By default, this only returns assessments that are valid (i.e. have not been overridden by another assessment). To return all assessments, specify all=True.

Parameters

• name – The name of the assessment to get. If not provided, this will match all assessment names.

• span_id – The span ID to get assessments for. If not provided, this will match all spans.

• all – If True, return all assessments regardless of validity.

• type – The type of assessment to get (one of “feedback” or “expectation”). If not provided, this will match all assessment types.

Returns

A list of assessments that meet the given conditions.

search_spans(span_type: Optional[SpanType] = None, name: Optional[Union[str, re.Pattern]] = None, span_id: Optional[str] = None) → list[Span][source]

Search for spans that match the given criteria within the trace.

Parameters

• span_type – The type of the span to search for.

• name – The name of the span to search for. This can be a string or a regular expression.

• span_id – The ID of the span to search for.

Returns

A list of spans that match the given criteria. If there is no match, an empty list is returned.

import mlflow
import re
from mlflow.entities import SpanType


@mlflow.trace(span_type=SpanType.CHAIN)
def run(x: int) -> int:
    x = add_one(x)
    x = add_two(x)
    x = multiply_by_two(x)
    return x


@mlflow.trace(span_type=SpanType.TOOL)
def add_one(x: int) -> int:
    return x + 1


@mlflow.trace(span_type=SpanType.TOOL)
def add_two(x: int) -> int:
    return x + 2


@mlflow.trace(span_type=SpanType.TOOL)
def multiply_by_two(x: int) -> int:
    return x * 2


# Run the function and get the trace
y = run(2)
trace_id = mlflow.get_last_active_trace_id()
trace = mlflow.get_trace(trace_id)

# 1. Search spans by name (exact match)
spans = trace.search_spans(name="add_one")
print(spans)
# Output: [Span(name='add_one', ...)]

# 2. Search spans by name (regular expression)
pattern = re.compile(r"add.*")
spans = trace.search_spans(name=pattern)
print(spans)
# Output: [Span(name='add_one', ...), Span(name='add_two', ...)]

# 3. Search spans by type
spans = trace.search_spans(span_type=SpanType.LLM)
print(spans)
# Output: [Span(name='run', ...)]

# 4. Search spans by name and type
spans = trace.search_spans(name="add_one", span_type=SpanType.TOOL)
print(spans)
# Output: [Span(name='add_one', ...)]

to_dict() → dict[str, typing.Any][source]

to_json(pretty=False) → str[source]

to_pandas_dataframe_row() → dict[str, typing.Any][source]

to_proto()[source]

Convert into a proto object to sent to the MLflow backend.

NB: The Trace definition in MLflow backend doesn’t include the data field,

but rather only contains TraceInfoV3.

class mlflow.entities.TraceData(spans: Optional[list[Span]] = None, **kwargs)[source]

A container object that holds the spans data of a trace.

Parameters

spans – List of spans that are part of the trace.

classmethod from_dict(d)[source]

property intermediate_outputs: dict[str, typing.Any] | None

Returns intermediate outputs produced by the model or agent while handling the request. There are mainly two flows to return intermediate outputs: 1. When a trace is generate by the mlflow.log_trace API, return intermediate_outputs attribute of the span. 2. When a trace is created normally with a tree of spans, aggregate the outputs of non-root spans.

property request: str | None

property response: str | None

spans: list[Span]

to_dict() → dict[str, typing.Any][source]

class mlflow.entities.TraceInfo(trace_id: str, trace_location: mlflow.entities.trace_location.TraceLocation, request_time: int, state: mlflow.entities.trace_state.TraceState, request_preview: Optional[str] = None, response_preview: Optional[str] = None, client_request_id: Optional[str] = None, execution_duration: Optional[int] = None, trace_metadata: dict[str, str] = <factory>, tags: dict[str, str] = <factory>, assessments: list[mlflow.entities.assessment.Assessment] = <factory>)[source]

Metadata about a trace, such as its ID, location, timestamp, etc.

Parameters

• trace_id – The primary identifier for the trace.

• trace_location – The location where the trace is stored, represented as a TraceLocation object. MLflow currently support MLflow Experiment or Databricks Inference Table as a trace location.

• request_time – Start time of the trace, in milliseconds.

• state – State of the trace, represented as a TraceState enum. Can be one of [OK, ERROR, IN_PROGRESS, STATE_UNSPECIFIED].

• request_preview – Request to the model/agent, equivalent to the input of the root, span but JSON-encoded and can be truncated.

• response_preview – Response from the model/agent, equivalent to the output of the root span but JSON-encoded and can be truncated.

• client_request_id – Client supplied request ID associated with the trace. This could be used to identify the trace/request from an external system that produced the trace, e.g., a session ID in a web application.

• execution_duration – Duration of the trace, in milliseconds.

• trace_metadata – Key-value pairs associated with the trace. They are designed for immutable values like run ID associated with the trace.

• tags – Tags associated with the trace. They are designed for mutable values, that can be updated after the trace is created via MLflow UI or API.

• assessments – List of assessments associated with the trace.

assessments: list[Assessment]

client_request_id: str | None = None

execution_duration: int | None = None

property execution_time_ms: int | None

property experiment_id: str | None

An MLflow experiment ID associated with the trace, if the trace is stored in MLflow tracking server. Otherwise, None.

classmethod from_dict(d: dict[str, typing.Any]) → TraceInfo[source]

Create a TraceInfoV3 object from a dictionary.

classmethod from_proto(proto) → TraceInfo[source]

property request_id: str

Deprecated. Use trace_id instead.

property request_metadata: dict[str, str]

Deprecated. Use trace_metadata instead.

request_preview: str | None = None

request_time: int

response_preview: str | None = None

state: TraceState

property status: mlflow.entities.trace_status.TraceStatus

Deprecated. Use state instead.

tags: dict[str, str]

property timestamp_ms: int

to_dict() → dict[str, typing.Any][source]

Convert the TraceInfoV3 object to a dictionary.

to_proto()[source]

property token_usage: dict[str, int] | None

Returns the aggregated token usage for the trace.

Returns

A dictionary containing the aggregated LLM token usage for the trace. - “input_tokens”: The total number of input tokens. - “output_tokens”: The total number of output tokens. - “total_tokens”: Sum of input and output tokens.

Note

The token usage tracking is not supported for all LLM providers. Refer to the MLflow Tracing documentation for which providers support token usage tracking.

trace_id: str

trace_location: TraceLocation

trace_metadata: dict[str, str]