Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://developers.arcgis.com/python/api-reference/arcgis.learn.toc.html below:

arcgis.learn module | ArcGIS API for Python

Functions for calling the Deep Learning Tools.

arcgis.learn.export_training_data(input_raster, input_class_data=None, chip_format=None, tile_size=None, stride_size=None, metadata_format=None, classvalue_field=None, buffer_radius=None, output_location=None, context=None, input_mask_polygons=None, rotation_angle=0, reference_system='MAP_SPACE', process_all_raster_items=False, blacken_around_feature=False, fix_chip_size=True, additional_input_raster=None, input_instance_data=None, instance_class_value_field=None, min_polygon_overlap_ratio=0, *, gis=None, future=False, estimate=False, **kwargs)

Function is designed to generate training sample image chips from the input imagery data with labeled vector data or classified images. The output of this service tool is the data store string where the output image chips, labels and metadata files are going to be stored.

Note

This function is supported with ArcGIS Enterprise (Image Server)

Parameter

Description

input_raster

Required ImageryLayer/Raster/Item/String (URL). Raster layer that needs to be exported for training.

input_class_data

Labeled data, either a feature layer or image layer. Vector inputs should follow a training sample format as generated by the ArcGIS Pro Training Sample Manager. Raster inputs should follow a classified raster format as generated by the Classify Raster tool.

chip_format

Optional string. The raster format for the image chip outputs.

• TIFF: TIFF format

• PNG: PNG format

• JPEG: JPEG format

• MRF: MRF (Meta Raster Format)

tile_size

Optional dictionary. The size of the image chips.

Example: {“x”: 256, “y”: 256}

stride_size

Optional dictionary. The distance to move in the X and Y when creating the next image chip. When stride is equal to the tile size, there will be no overlap. When stride is equal to half of the tile size, there will be 50% overlap.

Example: {“x”: 128, “y”: 128}

metadata_format

Optional string. The format of the output metadata labels. There are 4 options for output metadata labels for the training data, KITTI Rectangles, PASCAL VOCrectangles, Classified Tiles (a class map) and RCNN_Masks. If your input training sample data is a feature class layer such as building layer or standard classification training sample file, use the KITTI or PASCAL VOC rectangle option. The output metadata is a .txt file or .xml file containing the training sample data contained in the minimum bounding rectangle. The name of the metadata file matches the input source image name. If your input training sample data is a class map, use the Classified Tiles as your output metadata format option.

• KITTI_rectangles: The metadata follows the same format as the Karlsruhe Institute of Technology and Toyota echnological Institute (KITTI) Object Detection Evaluation dataset. The KITTI dataset is a vision benchmark suite. This is the default.The label files are plain text files. All values, both numerical or strings, are separated by spaces, and each row corresponds to one object. This format can be used with FasterRCNN, RetinaNet, SingleShotDetector and YOLOv3 models.

• PASCAL_VOC_rectangles: The metadata follows the same format as the Pattern Analysis, Statistical Modeling and Computational Learning, Visual Object Classes (PASCAL_VOC) dataset. The PASCAL VOC dataset is a standardized image data set for object class recognition.The label files are XML files and contain information about image name, class value, and bounding box(es). This format can be used with FasterRCNN, RetinaNet, SingleShotDetector and YOLOv3 models.

• Classified_Tiles: This option will output one classified image chip per input image chip. No other meta data for each image chip. Only the statistics output has more information on the classes such as class names, class values, and output statistics. This format can be used with BDCNEdgeDetector, DeepLab, HEDEdgeDetector, MultiTaskRoadExtractor, PSPNetClassifier and UnetClassifier models.

• RCNN_Masks: This option will output image chips that have a mask on the areas where the sample exists. The model generates bounding boxes and segmentation masks for each instance of an object in the image. This format can be used with MaskRCNN model.

• Labeled_Tiles: This option will label each output tile with a specific class. This format is used for image classification. This format can be used with FeatureClassifier model.

• MultiLabeled_Tiles: Each output tile will be labeled with one or more classes. For example, a tile may be labeled agriculture and also cloudy. This format is used for object classification. This format can be used with FeatureClassifier model.

• Export_Tiles: The output will be image chips with no label. This format is used for image enhancement techniques such as Super Resolution and Change Detection. This format can be used with ChangeDetector, CycleGAN, Pix2Pix and SuperResolution models.

• CycleGAN: The output will be image chips with no label. This format is used for image translation technique CycleGAN, which is used to train images that do not overlap.

• Imagenet: Each output tile will be labeled with a specific class. This format is used for object classification; however, it can also be used for object tracking when the Deep Sort model type is used during training.

• Panoptic_Segmentation: The output will be one classified image chip and one instance per input image chip. The output will also have image chips that mask the areas where the sample exists; these image chips will be stored in a different folder. This format is used for both pixel classification and instance segmentation, therefore there will be two output labels folders.

classvalue_field

Optional string. Specifies the field which contains the class values. If no field is specified, the system will look for a ‘value’ or ‘classvalue’ field. If this feature does not contain a class field, the system will presume all records belong the 1 class.

buffer_radius

Optional integer. Specifies a radius for point feature classes to specify training sample area.

output_location

This is the output location for training sample data. It can be the server data store path or a shared file system path.

Example:

Server datastore path -

• /fileShares/deeplearning/rooftoptrainingsamples

• /rasterStores/rasterstorename/rooftoptrainingsamples

File share path -

• \\servername\deeplearning\rooftoptrainingsamples

context

Optional dictionary. Context contains additional settings that affect task execution. Dictionary can contain value for following keys:

• exportAllTiles - Choose if the image chips with overlapped labeled data will be exported.

  • True - Export all the image chips, including those that do not overlap labeled data.

  • False - Export only the image chips that overlap the labelled data. This is the default.

• startIndex - Allows you to set the start index for the sequence of image chips. This lets you append more image chips to an existing sequence. The default value is 0.

• cellSize - cell size can be set using this key in context parameter

• extent - Sets the processing extent used by the function

Setting context parameter will override the values set using arcgis.env variable for this particular function.(cellSize, extent)

Example:

{“exportAllTiles” : False, “startIndex”: 0 }

input_mask_polygons

Optional FeatureLayer. The feature layer that delineates the area where image chips will be created. Only image chips that fall completely within the polygons will be created.

rotation_angle

Optional float. The rotation angle that will be used to generate additional image chips.

An image chip will be generated with a rotation angle of 0, which means no rotation. It will then be rotated at the specified angle to create an additional image chip. The same training samples will be captured at multiple angles in multiple image chips for data augmentation. The default rotation angle is 0.

reference_system

Optional string. Specifies the type of reference system to be used to interpret the input image. The reference system specified should match the reference system used to train the deep learning model.

• MAP_SPACE : The input image is in a map-based coordinate system. This is the default.

• IMAGE_SPACE : The input image is in image space, viewed from the direction of the sensor that captured the image, and rotated such that the tops of buildings and trees point upward in the image.

• PIXEL_SPACE : The input image is in image space, with no rotation and no distortion.

process_all_raster_items

Optional bool. Specifies how all raster items in an image service will be processed.

• False : all raster items in the image service will be mosaicked together and processed. This is the default.

• True : all raster items in the image service will be processed as separate images.

blacken_around_feature

Optional bool. Specifies whether to blacken the pixels around each object or feature in each image tile. This parameter only applies when the metadata format is set to Labeled_Tiles and an input feature class or classified raster has been specified.

• False : Pixels surrounding objects or features will not be blackened. This is the default.

• True : Pixels surrounding objects or features will be blackened.

fix_chip_size

Optional bool. Specifies whether to crop the exported tiles such that they are all the same size. This parameter only applies when the metadata format is set to Labeled_Tiles and an input feature class or classified raster has been specified.

• True : Exported tiles will be the same size and will center on the feature. This is the default.

• False : Exported tiles will be cropped such that the bounding geometry surrounds only the feature in the tile.

additional_input_raster

Optional ImageryLayer/Raster/Item/String (URL). An additional input imagery source that will be used for image translation methods.

This parameter is valid when the metadata_format parameter is set to Classified_Tiles, Export_Tiles, or CycleGAN.

input_instance_data

Optional. The training sample data collected that contains classes for instance segmentation.

The input can also be a point feature without a class value field or an integer raster without any class information.

This parameter is only valid when the metadata_format parameter is set to Panoptic_Segmentation.

instance_class_value_field

Optional string. The field that contains the class values for instance segmentation. If no field is specified, the tool will use a value or class value field, if one is present. If the feature does not contain a class field, the tool will determine that all records belong to one class.

This parameter is only valid when the metadata_format parameter is set to Panoptic_Segmentation.

min_polygon_overlap_ratio

Optional float. The minimum overlap percentage for a feature to be included in the training data. If the percentage overlap is less than the value specified, the feature will be excluded from the training chip, and will not be added to the label file.

The percent value is expressed as a decimal. For example, to specify an overlap of 20 percent, use a value of 0.2. The default value is 0, which means that all features will be included.

This parameter improves the performance of the tool and also improves inferencing. The speed is improved since less training chips are created. Inferencing is improved since the model is trained to only detect large patches of objects and ignores small corners of features.

This parameter is honoured only when the input_class_data parameter value is a feature service.

gis

Optional GIS . The GIS on which this tool runs. If not specified, the active GIS is used.

estimate

Keyword only parameter. Optional Boolean. If True, the number of credits needed to run the operation will be returned as a float. Available only on ArcGIS Online

future

Keyword only parameter. Optional boolean. If True, the result will be a GPJob object and results will be returned asynchronously.

Returns:

Output string containing the location of the exported training data

arcgis.learn.export_point_dataset(data_path, output_path, block_size=50.0, max_points=8192, extra_features=[], **kwargs)

Note: This function has been deprecated starting from ArcGIS API for Python version 1.9.0. Export data using Prepare Point Cloud Training Data tool available in 3D Analyst Extension from ArcGIS Pro 2.8 onwards.

arcgis.learn.estimate_batch_size(model, mode='train', **kwargs)

Function to calculate estimated batch size based on GPU capacity, size of model and data.

Parameter

Description

model

Required arcgis.learn imagery model. Model instance for which batch size should be estimated. Not supported for text, tabular, timeseries or tracking models such as FullyConnectedNetwork, MLModel, TimeSeriesModel, SiamMask, PSETAE and EfficientDet models.

mode

Optional string. Default train. The mode for which batch size is estimated. Supported ‘train’ and ‘eval’ mode for calculating batch size in training mode and evaluation mode respectively. Note: max_batchsize is capped at 1024 for train and eval mode and recommended_batchsize is capped at 64 for train mode.

Returns:

Named tuple of recommended_batchsize and max_batchsize

arcgis.learn.prepare_data(path, class_mapping=None, chip_size=224, val_split_pct=0.1, batch_size=64, transforms=None, collate_fn=<function _bb_pad_collate>, seed=42, dataset_type=None, resize_to=None, working_dir=None, **kwargs)

Prepares a data object from training sample exported by the Export Training Data tool in ArcGIS Pro or Image Server, or training samples in the supported dataset formats. This data object consists of training and validation data sets with the specified transformations, chip size, batch size, split percentage, etc.

Parameter

Description

path

Required string. Path to data directory or a list of paths in case of multi-folder training.

class_mapping

Optional dictionary. Mapping from id to its string label. Not supported for MaskRCNN model.

chip_size

Optional integer, default 224. Size of the image to train the model. Images are cropped to the specified chip_size. If image size is less than chip_size, the image size is used as chip_size. A chip size that is a multiple of 32 pixels is recommended. Not supported for SuperResolution, SiamMask, WNet_cGAN, Pix2Pix and CycleGAN.

val_split_pct

Optional float. Percentage of training data to keep as validation.

batch_size

Optional integer. Default 64. Batch size for mini batch gradient descent (Reduce it if getting CUDA Out of Memory Errors). Batch size is required to be greater than 1. If None is provided, a recommended batch size is used. This is estimated based on GPU capacity, size of model and data. To explicitly find the recommended batch_size, use arcgis.learn.estimate_batch_size() method.

transforms

Optional tuple. Fast.ai transforms for data augmentation of training and validation datasets respectively (We have set good defaults which work for satellite imagery well). If transforms is set to False no transformation will take place and chip_size parameter will also not take effect. If the dataset_type is ‘PointCloud’ and ‘PointCloudOD’, use Transform3d.

collate_fn

Optional function. Passed to PyTorch to collate data into batches(usually default works).

seed

Optional integer. Random seed for reproducible train-validation split.

dataset_type

Optional string. prepare_data() function will infer the dataset_type on its own if it contains a map.txt file. If the path does not contain the map.txt file pass one of ‘PASCAL_VOC_rectangles’, ‘KITTI_rectangles’, ‘Imagenet’. This parameter is mandatory for dataset ‘PointCloud’, ‘PointCloudOD’, ‘ImageCaptioning’, ‘ChangeDetection’, ‘WNet_cGAN’ and ‘ObjectTracking’. Note: For details on dataset_type please refer to this link.

resize_to

Optional integer or tuple of integers. A tuple should be of the form (height, width). Resize the images to a given size. Works only for “PASCAL_VOC_rectangles”, “Labelled_Tiles”, “superres” and “Imagenet”.First resizes the image to the given size and then crops images of size equal to chip_size. Note: If resize_to is less than chip_size, the resize_to is used as chip_size.

working_dir

Optional string. Sets the default path to be used as a prefix for saving trained models and checkpoints.

Keyword Arguments

Parameter

Description

n_masks

Optional int. Default value is 30. Required for MaXDeepLab panoptic segmentation model. It represents the max number of class labels and instances any image can contain. To compute the exact value for your dataset, use the compute_n_masks() method available with MaXDeepLab model.

downsample_factor

Optional float. Factor to downsample the images for image SuperResolution. for example: if value is 2 and image size 256x256, it will create label images of size 128x128. Default is 4

min_points

For dataset_type=’PointCloud’ and ‘PointCloudOD’: Optional int. Filtering based on minimum number of points in a block. Set min_points=1000 to filter out blocks with less than 1000 points.

For dataset_type=’PSETAE’: Optional int. Number of pixels equal to or multiples of 64 to sample from the each masked region of training data i.e. 64, 128 etc.

extra_features

Optional List. Contains a list of strings which mentions extra features to be used for training, applicable with dataset_type ‘PointCloud’ and ‘PointCloudOD’. By default only x, y, and z are considered for training irrespective of what features were exported. For example: [‘intensity’, ‘numberOfReturns’, ‘returnNumber’, ‘red’, ‘green’, ‘blue’, ‘nearInfrared’].

remap_classes

Optional dictionary {int:int}. Mapping from class values to user defined values, in both training and validation data.

For dataset_type=’PointCloud’: It will remap LAS classcode structure. For example: {1:3, 2:4} will remap LAS classcode 1 to 3 and classcode 2 to 4.

For dataset_type=’PointCloudOD’: It will remap object class ids. When this parameter is set as remap_classes={5:3, 2:4}, then ‘5’ and 2 class values will be considered as ‘3’, and ‘4’, respectively.

classes_of_interest

Optional list of int.

For dataset_type=’PointCloud’: This will filter training blocks based on classes_of_interest. If we have “1, 3, 5, 7” LAS classcodes in our dataset, but we are mainly interested in 1 and 3 classcodes, Set classes_of_interest=[1,3]. Only those blocks will be considered for training which either have 1 or 3 LAS classcodes in them, rest of the blocks will be filtered out. If remapping of rest of the classcodes is required, set background_classcode to some value.

For dataset_type=’PointCloudOD’: This will filter training blocks based on classes_of_interest. If we have “2, 3, 10, 16” object classes in the 3d feature class, but we are mainly interested in 2 and 10 object classes, Set classes_of_interest=[2,10]. Only those blocks will be considered for training which either have 2 or 10 object classes in them, the rest of the blocks will be filtered out. Set background_classcode as True to discard other classes.

Note: classes_of_interest is applied on the remapped class structure, if remap_classes is also used.

background_classcode

This parameter is only applicable when classes_of_interest is specified.

For dataset_type=’PointCloud’: Optional int. Default: None. This will remap other class values, except classes_of_interest to background_classcode.

For dataset_type=’PointCloudOD’: Optional Bool. Default: False. If set to ‘True’, only classes_of_interest class values will be considered and rest of the class values will be discarded.

stratify

Optional boolean, default False. If True, prepare_data will try to maintain the class proportion in train and validation data according to the val_split_pct. Default value feature classification is True. Default value pixel classification is False.

Note: Applies to single label feature classification, object detection and pixel classification.

timesteps_of_interest

Optional list. List of time steps of interest. This will filter multi-temporal timesereis based on timesteps_of_interest. If the dataset have time-steps [0, 1, 2, 3], but we are mainly interested in 0, 1 and 2, Set timesteps_of_interest=[0,1,2]. Only those time-steps will be considered for training, rest of the time-steps will be filtered out. Applicable only for dataset_type=’PSETAE’.

channels_of_interest

Optional list. List of spectral bands/channels of interest. This will filter out bands from rasters of multi-temporal timeseries based on channels_of_interest list. If we have bands [0,1,2,3,4] in our dataset, but we are mainly interested in 0, 1 and 2, Set channels_of_interest=[0,1,2]. Only those spectral bands will be considered for training. Applicable only for dataset_type=’PSETAE’.

n_temporal

Required int. Number of temporal observations or time steps. Applicable only for dataset_type=’PSETAE’.

n_temporal_dates

Required list of strings. The dates of that observations will be used for the positional encoding and should be stored as a list of dates strings in YYYY-MM-DD format. For example, If we have stacked imagery of n bands each from two dates then, [‘YYYY-MM-DD’,’YYYY-MM-DD’]. Applicable only for dataset_type=’PSETAE’.

num_workers

Optional int. Default 0. number of subprocesses to use for data loading on the Windows operating system. 0 means that the data will be loaded in the main process.

forecast_timesteps

Required int. Default set to 1. How far the model should forecast into the future. A forecast timestep is the interval at which predictions are made, For example, If we have 8-hourly data point and we want to make a 8 hr, 16 hr, 24 hr forecast, forecast timesteps is set to 1, 2, 3 respectively and so on. In case of hourly and monthly data point, for forecasts of 1, 2, 3 hr/month, forecast timestep is set to 1, 2, 3 respectively and so on. Applicable only for climaX model architecuture.

hrs_each_step

Optional int. Default set to 1 (hrs). Number of hours in which data is collected, for example, if you have 8-hourly, hourly, montly, daily then, hrs_each_step is to be set to 8, 1, 720 (30 days * 24), 24 hrs respectively. Applicable only for climaX model architecuture.

Returns:

data object

arcgis.learn.prepare_tabulardata(input_features=None, variable_predict=None, explanatory_variables=None, explanatory_rasters=None, date_field=None, cell_sizes=[3, 4, 5, 6, 7], distance_features=None, preprocessors=None, val_split_pct=0.1, seed=42, batch_size=64, index_field=None, working_dir=None, **kwargs)

Prepares a tabular data object from input_features and optionally rasters.

Parameter

Description

input_features

Optional FeatureLayer Object or spatially enabled dataframe. This contains features denoting the value of the dependent variable. Leave empty for using rasters with MLModel.

variable_predict

Optional String or List, denoting the field_names of the variable to predict. Keep none for unsupervised training using ML Model. For timeseries it will work for continuous variable. As of now we support only binary classification in fairness evaluation.

explanatory_variables

Optional list containing field names from input_features By default the field type is continuous. To override field type to categorical, pass a 2-sized tuple in the list containing:

1. field to be taken as input from the input_features.

2. True/False denoting Categorical/Continuous variable. If the field is text, the value should be ‘text’

and if the field is image path, the value should be ‘image’.

For example:

[“Field_1”, (“Field_2”, True)] or [“Field_1”, (“Field_3”, ‘text’)]

Here Field_1 is treated as continuous and Field_2 as categorical and Field_3 as Text

explanatory_rasters

Optional list containing Raster objects. By default the rasters are continuous. To mark a raster categorical, pass a 2-sized tuple containing:

1. Raster object.

2. True/False denoting Categorical/Continuous variable.

For example:

[raster_1, (raster_2, True)]

Here raster_1 is treated as continuous and raster_2 as categorical. To select only specific bands of raster, pass 2/3 sized tuple containing:

1. Raster object.

2. True/False denoting Categorical/Continuous variable.

3. Tuple holding the indexes of the bands to be used.

For example:

[raster_1, (raster_2, True,(0,)),(raster_3, (0,1,2))]

Here bands with indexes 0 will be chosen from raster_2 and it will be treated as categorical variable, bands with indexes 0,1,2 will be chosen from raster_3 and they will be treated as continuous.

date_field

Optional field_name. This field contains the date in the input_features. The field type can be a string or date time field. If specified, the field will be split into Year, month, week, day, dayofweek, dayofyear, is_month_end, is_month_start, is_quarter_end, is_quarter_start, is_year_end, is_year_start, hour, minute, second, elapsed and these will be added to the prepared data as columns. All fields other than elapsed and dayofyear are treated as categorical.

cell_sizes

Size of H3 cells (specified as H3 resolution) for spatially aggregating input features and passing in the cell ids as additional explanatory variables to the model. If a spatial dataframe is passed as input_features, ensure that the spatial reference is 4326, and the geometry type is Point. Not applicable when explanatory_rasters are provided. Not applicable for MLModel.

distance_features

Optional list of FeatureLayer objects. Distance is calculated from features in these layers to features in input_features. Nearest distance to each feature is added in the prepared data. Field names in the prepared data added are “NEAR_DIST_1”, “NEAR_DIST_2” etc.

preprocessors

For FullyConnectedNetworks: All the transforms are applied by default and hence users need not pass any additional transforms/preprocessors. For MLModel which uses Scikit-learn transforms:

1. Supply a column transformer object.

2. Supply a list of tuple,

For example:

[(‘Col_1’, ‘Col_2’, Transform1()), (‘Col_3’, Transform2())]

Categorical data is by default encoded. If nothing is specified, default transforms are applied to fill missing values and normalize categorical data. For Raster use raster.name for the first band, raster.name_1 for 2nd band, raster.name_2 for 3rd and so on.

val_split_pct

Optional float. Percentage of training data to keep as validation. By default 10% data is kept for validation.

seed

Optional integer. Random seed for reproducible train-validation split. Default value is 42.

batch_size

Optional integer. Batch size for mini batch gradient descent (Reduce it if getting CUDA Out of Memory Errors). Default value is 64.

index_field

Optional string. Field Name in the input features which will be used as index field for the data. Used for Time Series, to visualize values on the x-axis.

working_dir

Optional string. Sets the default path to be used as a prefix for saving trained models and checkpoints.

Keyword Arguments

Parameter

Description

stratify

Optional boolean. If True, prepare_tabulardata will try to maintain the class proportion in train and validation data according to the val_split_pct. Default value is False.

Note

Applies to classification problems.

random_split

Optional boolean. sets the behaviour of train and validation split to random or last n steps. If set to True then random sampling will be performed. Otherwise, last n steps will be used as validation. val_split_pct will determine the number the records for validation. Default value is True

Note

Applies to timeseries

Returns:

TabularData object

arcgis.learn.prepare_textdata(path, task, text_columns=None, label_columns=None, train_file='train.csv', valid_file=None, val_split_pct=0.1, seed=42, batch_size=8, process_labels=False, remove_html_tags=False, remove_urls=False, working_dir=None, dataset_type=None, class_mapping=None, **kwargs)

Prepares a text data object from the files present at data folder

Parameter

Description

path

Required directory path. The directory path where the training and validation files are present.

task

Required string. The task for which the dataset is prepared. Available choice at this point is “classification”, “sequence_translation” or “entity_recognition”.

text_columns

Optional string. This parameter is mandatory when task is “classification” or “sequence_translation”. This parameter is mandatory when task is entity_recognition task with input dataset_type as csv. The column that will contain the input text.

label_columns

Optional list. This parameter is mandatory when task is “classification” or “sequence_translation”. The list of columns denoting the class label/translated text to predict. Provide a list of columns in case of multi-label classification problem.

train_file

Optional string. The file name containing the training data. Supported file formats/extensions are .csv and .tsv Default value is train.csv

valid_file

Optional string. The file name containing the validation data. Supported file formats/extensions are .csv and .tsv. Default value is None. If None then some portion of the training data will be kept for validation (based on the value of val_split_pct parameter)

val_split_pct

Optional float. Percentage of training data to keep as validation. By default 10% data is kept for validation.

seed

Optional integer. Random seed for reproducible train-validation split. Default value is 42.

batch_size

Optional integer. Batch size for mini batch gradient descent (Reduce it if getting CUDA Out of Memory Errors). Default value is 16.

process_labels

Optional boolean. If true, default processing functions will be called on label columns as well. Default value is False.

remove_html_tags

Optional boolean. If true, remove html tags from text. Default value is False.

remove_urls

Optional boolean. If true, remove urls from text. Default value is False.

working_dir

Optional string. Sets the default path to be used as a prefix for saving trained models and checkpoints.

dataset_type

Optional list. This parameter is mandatory when task is “entity_recognition” Accepted data format for this model are - ‘ner_json’,’BIO’ or ‘LBIOU’, ‘csv’ For csv dataset type. If an entity has multiple values. It should be separated by ,.

class_mapping

Optional dictionary. This parameter is optional and can only be used when the task is entity recognition. The dictionary specifies the location entity. Use the format: class_mapping={‘address_tag’: ‘location’}. The value linked to the ‘address_tag’ key will be identified as a location entity. If the model extracts multiple location entities from a single document, each location will be listed separately in the results.

Keyword Arguments

Parameter

Description

stratify

Optional boolean. If True, prepare_textdata will try to maintain the class proportion in train and validation data according to the val_split_pct. The default value is True.

Note

Applies only to single-label text classification.

encoding

Optional string. Applicable only when task is entity_recognition: The encoding to read the csv/json file. Default is ‘UTF-8’

Returns:

TextData object

class arcgis.learn.Transform3d(rotation=[2.5, 2.5, 45], scaling=5, jitter=0.0, **kwargs)

Create transformations for 3D datasets, that can be used in prepare_data() to apply data augmentation with a 50% probability. Applicable for dataset_type=’PointCloud’ and dataset_type=’PointCloudOD’.

Parameter

Description

rotation

An optional list of float. It defines a value in degrees for each X, Y, and Z, dimensions which will be used to rotate a block around the X, Y, and Z, axes.

Example: A value of [2, 3, 180] means a random value for each X, Y, and Z will be selected between, [-2, 2], [-3, 3], and [-180, 180], respectively. The block will rotate around the respective axis as per the selected random value.

Note: For dataset_type=’PointCloudOD’, rotation around the X and Y axes will not be considered. Default: [2.5, 2.5, 45]

scaling

An optional float. It defines a percentage value, that will be used to apply scaling transformation to a block.

Example: A value of 5 means, for each X, Y, and Z, dimensions a random value will be selected within the range of [0, 5], where the block might be scaled up or scaled down randomly, in the respective dimension.

Note: For dataset_type=’PointCloudOD’, the same scale percentage in all three directions is considered. Default: 5

jitter

Optional float within [0, 1]. It defines a value in meters, which is used to add random variations in X, Y, and Z of all points.

Example: if the value provided is 0.1 then within the range of [-0.1, 0.1] a random value is selected, The selected value is then added to the point’s X coordinate. Similarly, it is applied for Y and Z coordinates.

Note: Only applicable for dataset_type=’PointCloud’. Default: 0.0.

Returns:

Transform3d object

class arcgis.learn.AutoML(data=None, total_time_limit=3600, mode='Basic', algorithms=None, eval_metric='auto', n_jobs=1, ml_task='auto', **kwargs)

Automates the process of model selection, training and hyperparameter tuning of machine learning models within a specified time limit. Based upon MLJar(https://github.com/mljar/mljar-supervised/) and scikit-learn.

Note that automated machine learning support is provided only for supervised learning. Refer https://supervised.mljar.com/

Parameter

Description

data

Required TabularDataObject. Returned data object from prepare_tabulardata() function.

total_time_limit

Optional Int. The total time limit in seconds for AutoML training. Default is 3600 (1 Hr)

mode

Optional Str. Can be {Basic, Intermediate, Advanced}. This parameter defines the goal of AutoML and how intensive the AutoML search will be.

Basic : To to be used when the user wants to explain and understand the data. Uses 75%/25% train/test split. Uses the following models: Baseline, Linear, Decision Tree, Random Trees, XGBoost, Neural Network, and Ensemble. Has full explanations in reports: learning curves, importance plots, and SHAP plots. Intermediate : To be used when the user wants to train a model that will be used in real-life use cases. Uses 5-fold CV (Cross-Validation). Uses the following models: Linear, Random Trees, LightGBM, XGBoost, CatBoost, Neural Network, and Ensemble. Has learning curves and importance plots in reports.

Advanced : To be used for machine learning competitions (maximum performance). Uses 10-fold CV (Cross-Validation). Uses the following models: Decision Tree, Random Trees, Extra Trees, XGBoost, CatBoost, Neural Network, Nearest Neighbors, Ensemble, and Stacking.It has only learning curves in the reports. Default is Basic

algorithms

Optional. List of str. The list of algorithms that will be used in the training. The algorithms can be: Linear, Decision Tree, Random Trees, Extra Trees, LightGBM, Xgboost, Neural Network

eval_metric

Optional Str. The metric to be used to compare models. Possible values are: For binary classification - logloss (default), auc, f1, average_precision, accuracy. For multiclass classification - logloss (default), f1, accuracy For regression - rmse (default), mse, mae, r2, mape, spearman, pearson

Note - If there are only 2 unique values in the target, then binary classification is performed, If number of unique values in the target is between 2 and 20 (included), then multiclass classification is performed, In all other cases, regression is performed on the dataset.

n_jobs

Optional. Int. Number of CPU cores to be used. By default, it is set to 1.Set it to -1 to use all the cores.

kwargs

sensitive_variables

Optional. List of strings. Variables in the feature class/dataframe which are sensitive and prone to model bias. Ex - [‘sex’,’race’] or [‘nationality’]

fairness_metric

Optional. String. Name of fairness metric based on which fairness optimization should be done on the evaluated models. Available metrics for binary classification are ‘demographic_parity_difference’ , ‘demographic_parity_ratio’, ‘equalized_odds_difference’, ‘equalized_odds_ratio’. ‘demographic_parity_ratio’ is the default. Available metrics for regression are ‘group_loss_ratio’ (Default) and ‘group_loss_difference’.

fairness_threshold

Optional. Float. Required when the chosen metric is group_loss_difference The threshold value for fairness metric. Default values are as follows: - for demographic_parity_difference the metric value should be below 0.25, - for demographic_parity_ratio the metric value should be above 0.8, - for equalized_odds_difference the metric value should be below 0.25, - for equalized_odds_ratio the metric value should be above 0.8. - for group_loss_ratio the metric value should be above 0.8. - for group_loss_difference the metric value should be below 0.25,

privileged_groups

Optional. List. List of previleged groups in the sensitive attribute. For example, in binary classification task, a privileged group is the one with the highest selection rate. Example value: [{“sex”: “Male”}]

underprivileged_groups

Optional. List. List of underprivileged groups in the sensitive attribute. For example, in binary classification task, an underprivileged group is the one with the lowest selection rate. Example value: [{“sex”: “Female”}]

Returns:

AutoML Object

copy_and_overwrite(from_path, to_path)

fairness_score(sensitive_feature, fairness_metrics=None, visualize=False)

Shows sample results for the model.

Returns:

tuple/dataframe

fit(sample_weight=None)

Fits the AutoML model.

classmethod from_model(emd_path)

Creates an AutoML Model Object from an Esri Model Definition (EMD) file. The model object created can only be used for inference on a new dataset and cannot be retrained.

Parameter

Description

emd_path

Required string. Path to Esri Model Definition file.

Returns:

AutoML Object

get_ml_task(all_labels)

predict(input_features=None, explanatory_rasters=None, datefield=None, distance_features=None, output_layer_name='Prediction Layer', gis=None, prediction_type='features', output_raster_path=None, match_field_names=None, cell_sizes=[3, 4, 5, 6, 7], confidence=True, get_local_explanations=False, **kwargs)

Predict on data from feature layer, dataframe and or raster data.

Parameter

Description

input_features

Optional FeatureLayer or spatial dataframe. Required if prediction_type=’features’. Contains features with location and some or all fields required to infer the dependent variable value.

explanatory_rasters

Optional list. Required if prediction_type=’raster’. Contains a list of raster objects containing some or all fields required to infer the dependent variable value.

datefield

Optional string. Field name from feature layer that contains the date, time for the input features. Same as prepare_tabulardata().

cell_sizes

Size of H3 cells (specified as H3 resolution) for spatially aggregating input features and passing in the cell ids as additional explanatory variables to the model. If a spatial dataframe is passed as input_features, ensure that the spatial reference is 4326, and the geometry type is Point. Not applicable when explanatory_rasters are provided.

distance_features

Optional List of FeatureLayer objects. These layers are used for calculation of field “NEAR_DIST_1”, “NEAR_DIST_2” etc in the output dataframe. These fields contain the nearest feature distance from the input_features. Same as prepare_tabulardata() .

output_layer_name

Optional string. Used for publishing the output layer.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

prediction_type

Optional String. Set ‘features’ or ‘dataframe’ to make output feature layer predictions. With this feature_layer argument is required.

Set ‘raster’, to make prediction raster. With this rasters must be specified.

output_raster_path

Optional path. Required when prediction_type=’raster’, saves the output raster to this path.

match_field_names

Optional dictionary. Specify mapping of field names from prediction set to training set. For example:

{

“Field_Name_1”: “Field_1”,

“Field_Name_2”: “Field_2”

}

confidence

Optional Bool. Set confidence to True to get prediction confidence for classification use cases.Default is True.

Returns:

FeatureLayer if prediction_type=’features’, dataframe for prediction_type=’dataframe’ else creates an output raster.

predict_proba()

Returns:

output from AutoML’s model.predict_proba() with prediction probability for the training data

report()

Returns:

a report of the different models trained by AutoML along with their performance.

save(path)

Saves the model in the path specified. Creates an Esri Model and a dlpk. Uses pickle to save the model and transforms.

Returns:

path

score()

Returns:

output from AutoML’s model.score(), R2 score in case of regression and Accuracy in case of classification.

show_results(rows=5)

Shows sample results for the model.

Returns:

dataframe

class arcgis.learn.AutoDL(data=None, total_time_limit=2, mode='basic', network=None, verbose=True, **kwargs)

Automates the process of model selection, training and hyperparameter tuning of arcgis.learn supported deep learning models within a specified time limit.

Parameter

Description

data

Required ImageryDataObject. Returned data object from prepare_data() function.

total_time_limit

Optional Int. The total time limit in hours for AutoDL training. Default is 2 Hr.

mode

Optional String. Can be “basic” or “advanced”.

• basic : To be used when the user wants to train all selected networks.

• advanced : To be used when the user wants to tune hyper parameters of two

best performing models from basic mode.

network

Optional List of str. The list of models that will be used in the training. For eg: Supported Object Detection models: [“SingleShotDetector”, “RetinaNet”, “FasterRCNN”, “YOLOv3”, “MaskRCNN”, “DETReg” ,”RTDetrV2”,”ATSS”, “CARAFE”, “CascadeRCNN”, “CascadeRPN”, “DCN”, ‘Detectors’, ‘DoubleHeads’, ‘DynamicRCNN’, ‘EmpiricalAttention’, ‘FCOS’, ‘FoveaBox’, ‘FSAF’, ‘GHM’, ‘LibraRCNN’, ‘PaFPN’, ‘PISA’, ‘RegNet’,’RepPoints’, ‘Res2Net’, ‘SABL’, ‘VFNet’] Supported Pixel Classification models: [“DeepLab”, “UnetClassifier”, “PSPNetClassifier”, “SamLoRA”, “ANN”, “APCNet”, “CCNet”, “CGNet”, “HRNet”, ‘DeepLabV3Plus’, “Mask2Former”, ‘DMNet’, ‘DNLNet’, ‘FastSCNN’, ‘FCN’, ‘GCNet’, ‘MobileNetV2’, ‘NonLocalNet’, ‘PSANet’, ‘SemFPN’, ‘UperNet’]

verbose

Optional Boolean. To be used to display logs while training the models. Default is True.

Returns:

AutoDL Object

average_precision_score()

Calculates the average of the “average precision score” of all classes for selected networks

fit(**kwargs)

Train the selected networks for the specified number of epochs and using the specified learning rates

mIOU()

Calculates the mIOU of all classes for selected networks

report(allow_plot=False)

returns a HTML report of the different models trained by AutoDL along with their performance.

score(allow_plot=False)

returns output from AutoDL’s model.score(), “average precision score” in case of detection and accuracy in case of classification.

show_results(rows=5, threshold=0.25, **kwargs)

Shows sample results for the model.

Parameter

Description

rows

Optional number of rows. By default, 5 rows are displayed.

Returns:

dataframe

supported_classification_models()

Supported classification models.

supported_detection_models()

Supported detection models.

class arcgis.learn.ImageryModel

Imagery Model is used to fine tune models trained using AutoDL

available_metrics()

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score()

Computes average precision on the validation set for each class.

Parameter

Description

detect_thresh

Optional float. The probability above which a detection will be considered for computing average precision.

iou_thresh

Optional float. The intersection over union threshold with the ground truth labels, above which a predicted bounding box will be considered a true positive.

mean

Optional bool. If False returns class-wise average precision otherwise returns mean average precision.

Returns:

dict if mean is False otherwise float

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', **kwargs)

Train the model for the specified number of epochs while using the specified learning rates

Parameter

Description

epochs

Optional integer. Number of cycles of training on the data. Increase it if the model is underfitting. Default value is 10.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

load(path, data)

Loads a compatible saved model for inferencing or fine tuning from the disk, which can be used to further fine tune the models saved using AutoDL.

Parameter

Description

path

Required string. Path to Esri Model Definition(EMD) or DLPK file.

data

Required ImageryDataObject. Returned data object from prepare_data() function.

lr_find(allow_plot=True)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mIOU()

Computes mean IOU on the validation set for each class.

Parameter

Description

mean

Optional bool. If False returns class-wise mean IOU, otherwise returns mean iou of all classes combined.

show_progress

Optional bool. Displays the progress bar if True.

Returns:

dict if mean is False otherwise float

plot_losses()

Plot validation and training losses after fitting the model.

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector, FeatureClassifier and RetinaNet . torchscript format is supported by SiamMask . For usage of SiamMask model in ArcGIS Pro 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters: Boolean overwrite if True, it will overwrite the item on ArcGIS Online/Enterprise, default False.

show_results(rows=5, **kwargs)

Displays the results of a trained model on a part of the validation set.

rows

Optional int. Number of rows of results to be displayed.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.FeatureClassifier(data, backbone='resnet34', pretrained_path=None, mixup=False, oversample=False, backend='pytorch', *args, **kwargs)

Creates an image classifier to classify the area occupied by a geographical feature based on the imagery it overlaps with.

Parameter

Description

data

Required fastai Databunch. Returned data object from prepare_data() function.

backbone

Optional string. Backbone convolutional neural network model used for feature extraction, which is resnet34 by default. Supported backbones: ResNet family and specified Timm models(experimental support) from backbones().

pretrained_path

Optional string. Path where pre-trained model is saved.

mixup

Optional boolean. If set to True, it creates new training images by randomly mixing training set images.

The default is set to False.

oversample

Optional boolean. If set to True, it oversamples unbalanced classes of the dataset during training. Not supported with MultiLabel dataset.

backend

Optional string. Controls the backend framework to be used for this model, which is ‘pytorch’ by default.

valid options are “pytorch”, “tensorflow”

wavelengths

Optional list. A list of central wavelengths corresponding to each data band (in micrometers).

Returns:

FeatureClassifier Object

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

static backbones()

Supported list of backbones for this model.

categorize_features(feature_layer, raster=None, class_value_field='class_val', class_name_field='prediction', confidence_field='confidence', cell_size=1, coordinate_system=None, predict_function=None, batch_size=64, overwrite=False)

Deprecated since version 1.7.1: Please use classify_objects() instead

Deprecated since version 1.7.1: Please use classify_objects() instead

Categorizes each feature by classifying its attachments or an image of its geographical area (using the provided Imagery Layer) and updates the feature layer with the prediction results in the output_label_field. Deprecated, Use the Classify Objects Using Deep Learning tool or classify_objects()

Parameter

Description

feature_layer

Required. Public FeatureLayer or path of local feature class for classification with read, write, edit permissions.

raster

Optional. ImageryLayer or path of local raster to be used for exporting image chips. (Requires arcpy)

class_value_field

Required string. Output field to be added in the layer, containing class value of predictions.

class_name_field

Required string. Output field to be added in the layer, containing class name of predictions.

confidence_field

Optional string. Output column name to be added in the layer which contains the confidence score.

cell_size

Optional float. Cell size to be used for exporting the image chips.

coordinate_system

Optional. Cartographic Coordinate System to be used for exporting the image chips.

predict_function

Optional list of tuples. Used for calculation of final prediction result when each feature has more than one attachment. The predict_function takes as input a list of tuples. Each tuple has first element as the class predicted and second element is the confidence score. The function should return the final tuple classifying the feature and its confidence.

batch_size

Optional integer. The no of images or tiles to process in a single go.

The default value is 64.

overwrite

Optional boolean. If set to True the output fields will be overwritten by new values.

The default value is False.

Returns:

Boolean : True if operation is successful, False otherwise

classify_features(feature_layer, labeled_tiles_directory, input_label_field, output_label_field, confidence_field=None, predict_function=None)

Deprecated in ArcGIS version 1.9.1 and later: Use the Classify Objects Using Deep Learning tool or classify_objects()

Classifies the exported images and updates the feature layer with the prediction results in the output_label_field. Works with RGB images only.

Parameter

Description

feature_layer

Required. FeatureLayer for classification.

labeled_tiles_directory

Required. Folder structure containing images and labels folder. The chips should have been generated using the export training data tool in the Labeled Tiles format, and the labels should contain the OBJECTIDs of the features to be classified.

input_label_field

Required. Value field name which created the labeled tiles. This field should contain the OBJECTIDs of the features to be classified. In case of attachments this field is not used.

output_label_field

Required. Output column name to be added in the layer which contains predictions.

confidence_field

Optional. Output column name to be added in the layer which contains the confidence score.

predict_function

Optional. Used for calculation of final prediction result when each feature has more than one attachment. The predict_function takes as input a list of tuples. Each tuple has first element as the class predicted and second element is the confidence score. The function should return the final tuple classifying the feature and its confidence

Returns:

Boolean : True/False if operation is successful

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

static foundation_model_backbones()

Supported list of foundation model backbones for this model.

classmethod from_model(emd_path, data=None)

Creates a Feature classifier from an Esri Model Definition (EMD) file.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

FeatureClassifier Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False, **kwargs)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_confusion_matrix(**kwargs)

Plots a confusion matrix of the model predictions to evaluate accuracy kwargs

Parameter

Description

thresh

confidence score threshold for multilabel predictions, defaults to 0.5

plot_hard_examples(num_examples)

Plots the hard examples with their heatmaps.

Parameter

Description

num_examples

Number of hard examples to plot prepare_data() function.

plot_losses()

Plot validation and training losses after fitting the model.

predict(img_path, visualize=False, gradcam=False)

Runs prediction on an Image. Works with RGB images only.

Parameter

Description

img_path

Required. Path to the image file to make the predictions on.

visualize

Optional: Set this parameter to True to visualize the image being predicted.

gradcam

Optional: Set this parameter to True to get gradcam visualization to help with explanability of the prediction. If set to True, visualize parameter must also be set to True.

Returns:

prediction label and confidence

predict_folder_and_create_layer(folder, feature_layer_name, gis=None, prediction_field='predict', confidence_field='confidence')

Predicts on images present in the given folder and creates a feature layer. The images stored in the folder contain GPS information as part of EXIF metadata. Works with RGB images only.

Parameter

Description

folder

Required String. Folder containing images to inference on.

feature_layer_name

Required String. The name of the feature layer used to publish.

gis

Optional GIS Object, the GIS on which this tool runs. If not specified, the active GIS is used.

prediction_field

Optional String. The field name to use to add predictions.

confidence_field

Optional String. The field name to use to add confidence.

Returns:

FeatureCollection Object

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, gradcam=False, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

gradcam

Optional boolean. Setting this to True for labelled tiles will enable the ‘explainability_map’ parameter in the Classify Object Using Deep Learning tool in ArcGIS Pro/Online. The explainability_map parameter can be used to visualize the Grad-CAM from the tool. Setting this to True will also save Explainability Map in the saved folder Default is set to False. This feature works only with RGB images.

kwargs

Optional Parameters.

show_results(rows=5, **kwargs)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

property supported_backbones

Supported list of backbones for this model.

property supported_datasets

Supported dataset types for this model.

static torchgeo_backbones()

Supported list of torchgeo backbones for this model.

static transformer_backbones()

Supported list of transformer backbones for this model.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.FasterRCNN(data, backbone='resnet50', pretrained_path=None, **kwargs)

Model architecture from https://arxiv.org/abs/1506.01497. Creates a FasterRCNN object detection model, based on https://github.com/pytorch/vision/blob/master/torchvision/models/detection/faster_rcnn.py.

Parameter

Description

data

Required fastai Databunch. Returned data object from prepare_data() function.

backbone

Optional string. Backbone convolutional neural network model used for feature extraction, which is resnet50 by default. Supported backbones: ResNet family and specified Timm models(experimental support) from backbones().

pretrained_path

Optional string. Path where pre-trained model is saved.

kwargs

Returns:

FasterRCNN Object

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score(detect_thresh=0.2, iou_thresh=0.1, mean=False, show_progress=True)

Computes average precision on the validation set for each class.

Parameter

Description

detect_thresh

Optional float. The probability above which a detection will be considered for computing average precision.

iou_thresh

Optional float. The intersection over union threshold with the ground truth labels, above which a predicted bounding box will be considered a true positive.

mean

Optional bool. If False returns class-wise average precision otherwise returns mean average precision.

Returns:

dict if mean is False otherwise float

static backbones()

Supported list of backbones for this model.

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

classmethod from_model(emd_path, data=None)

Creates a FasterRCNN object from an Esri Model Definition (EMD) file.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

FasterRCNN Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False, **kwargs)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_losses()

Plot validation and training losses after fitting the model.

predict(image_path, threshold=0.5, nms_overlap=0.1, return_scores=False, visualize=False, resize=False)

Runs prediction on an Image. This method is only supported for RGB images.

Parameter

Description

image_path

Required. Path to the image file to make the predictions on.

threshold

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

return_scores

Optional boolean. Will return the probability scores of the bounding box predictions if True.

visualize

Optional boolean. Displays the image with predicted bounding boxes if True.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

Returns:

Returns a tuple with predictions, labels and optionally confidence scores if return_scores=True. The predicted bounding boxes are returned as a list of lists containing the xmin, ymin, width and height of each predicted object in each image. The labels are returned as a list of class values and the confidence scores are returned as a list of floats indicating the confidence of each prediction.

predict_video(input_video_path, metadata_file, threshold=0.5, nms_overlap=0.1, track=False, visualize=False, output_file_path=None, multiplex=False, multiplex_file_path=None, tracker_options={'assignment_iou_thrd': 0.3, 'detect_frames': 10, 'vanish_frames': 40}, visual_options={'color': (255, 255, 255), 'fontface': 0, 'show_labels': True, 'show_scores': True, 'thickness': 2}, resize=False)

Runs prediction on a video and appends the output VMTI predictions in the metadata file. This method is only supported for RGB images.

Parameter

Description

input_video_path

Required. Path to the video file to make the predictions on.

metadata_file

Required. Path to the metadata csv file where the predictions will be saved in VMTI format.

threshold

Optional float. The probability above which a detection will be considered.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

track

Optional bool. Set this parameter as True to enable object tracking.

visualize

Optional boolean. If True a video is saved with prediction results.

output_file_path

Optional path. Path of the final video to be saved. If not supplied, video will be saved at path input_video_path appended with _prediction.avi. Supports only AVI and MP4 formats.

multiplex

Optional boolean. Runs Multiplex using the VMTI detections.

multiplex_file_path

Optional path. Path of the multiplexed video to be saved. By default a new file with _multiplex.MOV extension is saved in the same folder.

tracking_options

Optional dictionary. Set different parameters for object tracking. assignment_iou_thrd parameter is used to assign threshold for assignment of trackers, vanish_frames is the number of frames the object should be absent to consider it as vanished, detect_frames is the number of frames an object should be detected to track it.

visual_options

Optional dictionary. Set different parameters for visualization. show_scores boolean, to view scores on predictions, show_labels boolean, to view labels on predictions, thickness integer, to set the thickness level of box, fontface integer, fontface value from opencv values, color tuple (B, G, R), tuple containing values between 0-255.

resize

Optional boolean. Resizes the video frames to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the video frames are resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the frame (of the same size as the model was trained on).

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters.

show_results(rows=5, thresh=0.5, nms_overlap=0.1)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

thresh

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

property supported_backbones

Supported list of backbones for this model.

property supported_datasets

Supported dataset types for this model.

static torchgeo_backbones()

Supported list of torchgeo backbones for this model.

static transformer_backbones()

Supported list of transformer backbones for this model.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.RetinaNet(data, scales=None, ratios=None, backbone=None, pretrained_path=None, *args, **kwargs)

Creates a RetinaNet Object Detector with the specified zoom scales and aspect ratios. Based on the Fast.ai notebook

Parameter

Description

data

Required fastai Databunch. Returned data object from prepare_data() function.

scales

Optional list of float values. Zoom scales of anchor boxes.

ratios

Optional list of float values. Aspect ratios of anchor boxes.

backbone

Optional string. Backbone convolutional neural network model used for feature extraction, which is resnet50 by default. Supported backbones: ResNet family and specified Timm models(experimental support) from backbones().

pretrained_path

Optional string. Path where pre-trained model is saved.

Returns:

RetinaNet Object

MIN_BATCH_VAL_AMP = 8

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score(detect_thresh=0.5, iou_thresh=0.1, mean=False, show_progress=True)

Computes average precision on the validation set for each class.

Parameter

Description

detect_thresh

Optional float. The probability above which a detection will be considered for computing average precision.

iou_thresh

Optional float. The intersection over union threshold with the ground truth labels, above which a predicted bounding box will be considered a true positive.

mean

Optional bool. If False returns class-wise average precision otherwise returns mean average precision.

Returns:

dict if mean is False otherwise float

static backbones()

Supported list of backbones for this model.

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

classmethod from_model(emd_path, data=None)

Creates a RetinaNet Object Detector from an Esri Model Definition (EMD) file.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

RetinaNet Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False, **kwargs)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_losses()

Plot validation and training losses after fitting the model.

predict(image_path, threshold=0.5, nms_overlap=0.1, return_scores=True, visualize=False, resize=False, batch_size=1)

Predicts and displays the results of a trained model on a single image. This method is only supported for RGB images.

Parameter

Description

image_path

Required. Path to the image file to make the predictions on.

thresh

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

return_scores

Optional boolean. Will return the probability scores of the bounding box predictions if True.

visualize

Optional boolean. Displays the image with predicted bounding boxes if True.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

batch_size

Optional int. Batch size to be used during tiled inferencing. Default value 1.

Returns:

‘List’ of xmin, ymin, width, height of predicted bounding boxes on the given image

predict_video(input_video_path, metadata_file, threshold=0.5, nms_overlap=0.1, track=False, visualize=False, output_file_path=None, multiplex=False, multiplex_file_path=None, tracker_options={'assignment_iou_thrd': 0.3, 'detect_frames': 10, 'vanish_frames': 40}, visual_options={'color': (255, 255, 255), 'fontface': 0, 'show_labels': True, 'show_scores': True, 'thickness': 2}, resize=False)

Runs prediction on a video and appends the output VMTI predictions in the metadata file. This method is only supported for RGB images.

Parameter

Description

input_video_path

Required. Path to the video file to make the predictions on.

metadata_file

Required. Path to the metadata csv file where the predictions will be saved in VMTI format.

threshold

Optional float. The probability above which a detection will be considered.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

track

Optional bool. Set this parameter as True to enable object tracking.

visualize

Optional boolean. If True a video is saved with prediction results.

output_file_path

Optional path. Path of the final video to be saved. If not supplied, video will be saved at path input_video_path appended with _prediction.avi. Supports only AVI and MP4 formats.

multiplex

Optional boolean. Runs Multiplex using the VMTI detections.

multiplex_file_path

Optional path. Path of the multiplexed video to be saved. By default a new file with _multiplex.MOV extension is saved in the same folder.

tracking_options

Optional dictionary. Set different parameters for object tracking. assignment_iou_thrd parameter is used to assign threshold for assignment of trackers, vanish_frames is the number of frames the object should be absent to consider it as vanished, detect_frames is the number of frames an object should be detected to track it.

visual_options

Optional dictionary. Set different parameters for visualization. show_scores boolean, to view scores on predictions, show_labels boolean, to view labels on predictions, thickness integer, to set the thickness level of box, fontface integer, fontface value from opencv values, color tuple (B, G, R), tuple containing values between 0-255.

resize

Optional boolean. Resizes the video frames to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the video frames are resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the frame (of the same size as the model was trained on).

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters.

show_results(rows=5, thresh=0.5, nms_overlap=0.1)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

thresh

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

property supported_backbones

Supported list of backbones for this model.

property supported_datasets

Supported dataset types for this model.

static transformer_backbones()

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.YOLOv3(data=None, pretrained_path=None, **kwargs)

Creates a YOLOv3 object detector.

Parameter

Description

data

Required fastai Databunch. Returned data object from prepare_data() function. YOLOv3 only supports image sizes in multiples of 32 (e.g. 256, 416, etc.)

pretrained_path

Optional string. Path where pre-trained model is saved.

Returns:

YOLOv3 Object

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score(detect_thresh=0.1, iou_thresh=0.1, mean=False, show_progress=True)

Computes average precision on the validation set for each class.

Parameter

Description

detect_thresh

Optional float. The probability above which a detection will be considered for computing average precision. Defaults to 0.1. To be modified according to the dataset and training.

iou_thresh

Optional float. The intersection over union threshold with the ground truth labels, above which a predicted bounding box will be considered a true positive.

mean

Optional bool. If False returns class-wise average precision otherwise returns mean average precision.

Returns:

dict if mean is False otherwise float

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

classmethod from_model(emd_path, data=None)

Creates a YOLOv3 Object Detector from an Esri Model Definition (EMD) file.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

YOLOv3 Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False, **kwargs)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_losses()

Plot validation and training losses after fitting the model.

predict(image_path, threshold=0.1, nms_overlap=0.1, return_scores=True, visualize=False, resize=False, batch_size=1)

Predicts and displays the results of a trained model on a single image. The image size should at least be 416x416px if using COCO pretrained weights. This method is only supported for RGB images.

Parameter

Description

image_path

Required. Path to the image file to make the predictions on.

threshold

Optional float. The probability above which a detection will be considered valid. Defaults to 0.1. To be modified according to the dataset and training.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

return_scores

Optional boolean. Will return the probability scores of the bounding box predictions if True.

visualize

Optional boolean. Displays the image with predicted bounding boxes if True.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

batch_size

Optional int. Batch size to be used during tiled inferencing. Default value 1.

Returns:

‘List’ of xmin, ymin, width, height of predicted bounding boxes on the given image

predict_video(input_video_path, metadata_file, threshold=0.1, nms_overlap=0.1, track=False, visualize=False, output_file_path=None, multiplex=False, multiplex_file_path=None, tracker_options={'assignment_iou_thrd': 0.3, 'detect_frames': 10, 'vanish_frames': 40}, visual_options={'color': (255, 255, 255), 'fontface': 0, 'show_labels': True, 'show_scores': True, 'thickness': 2}, resize=False)

Runs prediction on a video and appends the output VMTI predictions in the metadata file. This method is only supported for RGB images.

Parameter

Description

input_video_path

Required. Path to the video file to make the predictions on.

metadata_file

Required. Path to the metadata csv file where the predictions will be saved in VMTI format.

threshold

Optional float. The probability above which a detection will be considered. Defaults to 0.1. To be modified according to the dataset and training.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

track

Optional bool. Set this parameter as True to enable object tracking.

visualize

Optional boolean. If True a video is saved with prediction results.

output_file_path

Optional path. Path of the final video to be saved. If not supplied, video will be saved at path input_video_path appended with _prediction.avi. Supports only AVI and MP4 formats.

multiplex

Optional boolean. Runs Multiplex using the VMTI detections.

multiplex_file_path

Optional path. Path of the multiplexed video to be saved. By default a new file with _multiplex.MOV extension is saved in the same folder.

tracking_options

Optional dictionary. Set different parameters for object tracking. assignment_iou_thrd parameter is used to assign threshold for assignment of trackers, vanish_frames is the number of frames the object should be absent to consider it as vanished, detect_frames is the number of frames an object should be detected to track it.

visual_options

Optional dictionary. Set different parameters for visualization. show_scores boolean, to view scores on predictions, show_labels boolean, to view labels on predictions, thickness integer, to set the thickness level of box, fontface integer, fontface value from opencv values, color tuple (B, G, R), tuple containing values between 0-255.

resize

Optional boolean. Resizes the video frames to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the video frames are resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the frame (of the same size as the model was trained on).

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters.

show_results(rows=5, thresh=0.1, nms_overlap=0.1)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

thresh

Optional float. The probability above which a detection will be considered valid. Defaults to 0.1. To be modified according to the dataset and training.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

property supported_backbones

Supported backbones for this model.

property supported_datasets

Supported dataset types for this model.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.SingleShotDetector(data, grids=None, zooms=[1.0], ratios=[[1.0, 1.0]], backbone=None, drop=0.3, bias=-4.0, focal_loss=False, pretrained_path=None, location_loss_factor=None, ssd_version=2, backend='pytorch', *args, **kwargs)

Creates a Single Shot Detector with the specified grid sizes, zoom scales and aspect ratios. Based on Fast.ai MOOC Version2 Lesson 9.

Parameter

Description

data

Required fastai Databunch. Returned data object from prepare_data() function.

grids

Required list. Grid sizes used for creating anchor boxes.

zooms

Optional list. Zooms of anchor boxes.

ratios

Optional list of tuples. Aspect ratios of anchor boxes.

backbone

Optional string. Backbone convolutional neural network model used for feature extraction, which is resnet34 by default. Supported backbones: ResNet, DenseNet, VGG families and specified Timm models(experimental support) from backbones().

dropout

Optional float. Dropout probability. Increase it to reduce overfitting.

bias

Optional float. Bias for SSD head.

focal_loss

Optional boolean. Uses Focal Loss if True.

pretrained_path

Optional string. Path where pre-trained model is saved.

location_loss_factor

Optional float. Sets the weight of the bounding box loss. This should be strictly between 0 and 1. This is default None which gives equal weight to both location and classification loss. This factor adjusts the focus of model on the location of bounding box.

ssd_version

Optional int within [1,2]. Use version=1 for arcgis v1.6.2 or earlier

backend

Optional string. Controls the backend framework to be used for this model, which is ‘pytorch’ by default.

valid options are ‘pytorch’, ‘tensorflow’

wavelengths

Optional list. A list of central wavelengths corresponding to each data band (in micrometers).

Returns:

SingleShotDetector Object

MIN_BATCH_VAL_AMP = 8

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score(detect_thresh=0.2, iou_thresh=0.1, mean=False, show_progress=True)

Computes average precision on the validation set for each class.

Parameter

Description

detect_thresh

Optional float. The probability above which a detection will be considered for computing average precision.

iou_thresh

Optional float. The intersection over union threshold with the ground truth labels, above which a predicted bounding box will be considered a true positive.

mean

Optional bool. If False returns class-wise average precision otherwise returns mean average precision.

Returns:

dict if mean is False otherwise float

static backbones()

Supported list of backbones for this model.

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

classmethod from_emd(data, emd_path)

Creates a Single Shot Detector from an Esri Model Definition (EMD) file.

Parameter

Description

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

emd_path

Required string. Path to Esri Model Definition file.

Returns:

SingleShotDetector Object

classmethod from_model(emd_path, data=None)

Creates a Single Shot Detector from an Esri Model Definition (EMD) file.

Note: Only supported for Pytorch models.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

SingleShotDetector Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False, **kwargs)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_losses()

Plot validation and training losses after fitting the model.

predict(image_path, threshold=0.5, nms_overlap=0.1, return_scores=False, visualize=False, resize=False, batch_size=1)

Runs prediction on an Image. This method is only supported for RGB images.

Parameter

Description

image_path

Required. Path to the image file to make the predictions on.

threshold

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

return_scores

Optional boolean. Will return the probability scores of the bounding box predictions if True.

visualize

Optional boolean. Displays the image with predicted bounding boxes if True.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

batch_size

Optional int. Batch size to be used during tiled inferencing. Default value 1.

Returns:

‘List’ of xmin, ymin, width, height of predicted bounding boxes on the given image

predict_video(input_video_path, metadata_file, threshold=0.5, nms_overlap=0.1, track=False, visualize=False, output_file_path=None, multiplex=False, multiplex_file_path=None, tracker_options={'assignment_iou_thrd': 0.3, 'detect_frames': 10, 'vanish_frames': 40}, visual_options={'color': (255, 255, 255), 'fontface': 0, 'show_labels': True, 'show_scores': True, 'thickness': 2}, resize=False)

Runs prediction on a video and appends the output VMTI predictions in the metadata file. This method is only supported for RGB images.

Parameter

Description

input_video_path

Required. Path to the video file to make the predictions on.

metadata_file

Required. Path to the metadata csv file where the predictions will be saved in VMTI format.

threshold

Optional float. The probability above which a detection will be considered.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

track

Optional bool. Set this parameter as True to enable object tracking.

visualize

Optional boolean. If True a video is saved with prediction results.

output_file_path

Optional path. Path of the final video to be saved. If not supplied, video will be saved at path input_video_path appended with _prediction.avi. Supports only AVI and MP4 formats.

multiplex

Optional boolean. Runs Multiplex using the VMTI detections.

multiplex_file_path

Optional path. Path of the multiplexed video to be saved. By default a new file with _multiplex.MOV extension is saved in the same folder.

tracking_options

Optional dictionary. Set different parameters for object tracking. assignment_iou_thrd parameter is used to assign threshold for assignment of trackers, vanish_frames is the number of frames the object should be absent to consider it as vanished, detect_frames is the number of frames an object should be detected to track it.

visual_options

Optional dictionary. Set different parameters for visualization. show_scores boolean, to view scores on predictions, show_labels boolean, to view labels on predictions, thickness integer, to set the thickness level of box, fontface integer, fontface value from opencv values, color tuple (B, G, R), tuple containing values between 0-255.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters.

show_results(rows=5, thresh=0.5, nms_overlap=0.1)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

thresh

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

property supported_backbones

Supported list of backbones for this model.

property supported_datasets

Supported dataset types for this model.

static torchgeo_backbones()

Supported list of torchgeo backbones for this model.

static transformer_backbones()

Supported list of transformer backbones for this model.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.RTDetrV2(data, backbone='resnet18', pretrained_path=None, **kwargs)

Model architecture from https://arxiv.org/pdf/2407.17140. Creates a RTDetrV2 object detection model, based on https://github.com/lyuwenyu/RT-DETR/tree/main.

Parameter

Description

data

Required fastai Databunch. Returned data object from prepare_data() function.

backbone

Optional string. Backbone convolutional neural network model used for feature extraction, which is resnet50 by default. Supported backbones: ResNet family and specified Timm models(experimental support) from backbones().

pretrained_path

Optional string. Path where pre-trained model is saved.

Returns:

RTDetrV2 Object

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score(detect_thresh=0.2, iou_thresh=0.1, mean=False, show_progress=True)

Computes average precision on the validation set for each class.

Parameter

Description

detect_thresh

Optional float. The probability above which a detection will be considered for computing average precision.

iou_thresh

Optional float. The intersection over union threshold with the ground truth labels, above which a predicted bounding box will be considered a true positive.

mean

Optional bool. If False returns class-wise average precision otherwise returns mean average precision.

Returns:

dict if mean is False otherwise float

static backbones()

Supported list of backbones for this model.

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

classmethod from_model(emd_path, data=None)

Creates a RTDetrV2 object from an Esri Model Definition (EMD) file.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

RTDetrV2 Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_losses()

Plot validation and training losses after fitting the model.

predict(image_path, threshold=0.5, nms_overlap=0.1, return_scores=False, visualize=False, resize=False)

Runs prediction on an Image. This method is only supported for RGB images.

Parameter

Description

image_path

Required. Path to the image file to make the predictions on.

threshold

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

return_scores

Optional boolean. Will return the probability scores of the bounding box predictions if True.

visualize

Optional boolean. Displays the image with predicted bounding boxes if True.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

Returns:

Returns a tuple with predictions, labels and optionally confidence scores if return_scores=True. The predicted bounding boxes are returned as a list of lists containing the xmin, ymin, width and height of each predicted object in each image. The labels are returned as a list of class values and the confidence scores are returned as a list of floats indicating the confidence of each prediction.

predict_video(input_video_path, metadata_file, threshold=0.5, nms_overlap=0.1, track=False, visualize=False, output_file_path=None, multiplex=False, multiplex_file_path=None, tracker_options={'assignment_iou_thrd': 0.3, 'detect_frames': 10, 'vanish_frames': 40}, visual_options={'color': (255, 255, 255), 'fontface': 0, 'show_labels': True, 'show_scores': True, 'thickness': 2}, resize=False)

Runs prediction on a video and appends the output VMTI predictions in the metadata file. This method is only supported for RGB images.

Parameter

Description

input_video_path

Required. Path to the video file to make the predictions on.

metadata_file

Required. Path to the metadata csv file where the predictions will be saved in VMTI format.

threshold

Optional float. The probability above which a detection will be considered.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

track

Optional bool. Set this parameter as True to enable object tracking.

visualize

Optional boolean. If True a video is saved with prediction results.

output_file_path

Optional path. Path of the final video to be saved. If not supplied, video will be saved at path input_video_path appended with _prediction.

multiplex

Optional boolean. Runs Multiplex using the VMTI detections.

multiplex_file_path

Optional path. Path of the multiplexed video to be saved. By default a new file with _multiplex.MOV extension is saved in the same folder.

tracking_options

Optional dictionary. Set different parameters for object tracking. assignment_iou_thrd parameter is used to assign threshold for assignment of trackers, vanish_frames is the number of frames the object should be absent to consider it as vanished, detect_frames is the number of frames an object should be detected to track it.

visual_options

Optional dictionary. Set different parameters for visualization. show_scores boolean, to view scores on predictions, show_labels boolean, to view labels on predictions, thickness integer, to set the thickness level of box, fontface integer, fontface value from opencv values, color tuple (B, G, R), tuple containing values between 0-255.

resize

Optional boolean. Resizes the video frames to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the video frames are resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the frame (of the same size as the model was trained on).

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters.

show_results(rows=5, thresh=0.5, nms_overlap=0.1)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

thresh

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

property supported_backbones

Supported list of backbones for this model.

property supported_datasets

Supported dataset types for this model.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.MaskRCNN(data, backbone=None, pretrained_path=None, pointrend=False, *args, **kwargs)

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score(detect_thresh=0.5, iou_thresh=0.5, mean=False, show_progress=True, tta_prediction=False)

Computes average precision on the validation set for each class.

Returns:

dict if mean is False otherwise float

static backbones()

Supported list of backbones for this model.

fastai = <module 'fastai' from '/opt/conda/envs/arcgis/lib/python3.11/site-packages/fastai/__init__.py'>

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

classmethod from_model(emd_path, data=None, **kwargs)

Creates a MaskRCNN Instance segmentation object from an Esri Model Definition (EMD) file.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

MaskRCNN Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False, **kwargs)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_losses()

Plot validation and training losses after fitting the model.

predict(image_path, threshold=0.5, nms_overlap=0.1, return_scores=True, visualize=False, resize=False, tta_prediction=False, **kwargs)

Predicts and displays the results of a trained model on a single image. This method is only supported for RGB images.

Parameter

Description

image_path

Required. Path to the image file to make the predictions on.

thresh

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

return_scores

Optional boolean. Will return the probability scores of the bounding box predictions if True.

visualize

Optional boolean. Displays the image with predicted bounding boxes if True.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

tta_prediction

Optional bool. Perform test time augmentation while predicting

kwargs

Parameter

Description

batch_size

Optional int. Batch size to be used during tiled inferencing

min_obj_size

Optional int. Minimum object size to be detected.

Returns:

‘List’ of xmin, ymin, width, height, labels, scores, of predicted bounding boxes on the given image

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters.

show_results(rows=4, mode='mask', mask_threshold=0.5, box_threshold=0.7, tta_prediction=False, imsize=5, index=0, alpha=0.5, cmap='tab20', **kwargs)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

mode

Required arguments within [‘bbox’, ‘mask’, ‘bbox_mask’].

• bbox - For visualizing only bounding boxes.

• mask - For visualizing only mask

• bbox_mask - For visualizing both mask and bounding boxes.

mask_threshold

Optional float. The probability above which a pixel will be considered mask.

box_threshold

Optional float. The probability above which a detection will be considered valid.

tta_prediction

Optional bool. Perform test time augmentation while predicting

property supported_backbones

Supported list of backbones for this model.

property supported_datasets

Supported dataset types for this model.

static torchgeo_backbones()

Supported list of torchgeo backbones for this model.

static transformer_backbones()

Supported list of transformer backbones for this model.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.

class arcgis.learn.MMDetection(data, model, model_weight=False, pretrained_path=None, **kwargs)

Parameter

Description

data

Required fastai Databunch. Returned data object from prepare_data() function.

model

Required model name or path to the configuration file from MMDetection repository. The list of the supported models can be queried using supported_models .

model_weight

Optional path of the model weight from MMDetection repository.

pretrained_path

Optional string. Path where pre-trained model is saved.

Returns:

MMDetection Object

property available_metrics

List of available metrics that are displayed in the training table. Set monitor value to be one of these while calling the fit method.

average_precision_score(detect_thresh=0.2, iou_thresh=0.1, mean=False, show_progress=True)

Computes average precision on the validation set for each class.

Parameter

Description

detect_thresh

Optional float. The probability above which a detection will be considered for computing average precision.

iou_thresh

Optional float. The intersection over union threshold with the ground truth labels, above which a predicted bounding box will be considered a true positive.

mean

Optional bool. If False returns class-wise average precision otherwise returns mean average precision.

Returns:

dict if mean is False otherwise float

fit(epochs=10, lr=None, one_cycle=True, early_stopping=False, checkpoint=True, tensorboard=False, monitor='valid_loss', mixed_precision=False, **kwargs)

Train the model for the specified number of epochs and using the specified learning rates

Parameter

Description

epochs

Required integer. Number of cycles of training on the data. Increase it if underfitting.

lr

Optional float or slice of floats. Learning rate to be used for training the model. If lr=None, an optimal learning rate is automatically deduced for training the model.

one_cycle

Optional boolean. Parameter to select 1cycle learning rate schedule. If set to False no learning rate schedule is used.

early_stopping

Optional boolean. Parameter to add early stopping. If set to ‘True’ training will stop if parameter monitor value stops improving for 5 epochs. A minimum difference of 0.001 is required for it to be considered an improvement.

checkpoint

Optional boolean or string. Parameter to save checkpoint during training. If set to True the best model based on monitor will be saved during training. If set to ‘all’, all checkpoints are saved. If set to False, checkpointing will be off. Setting this parameter loads the best model at the end of training.

tensorboard

Optional boolean. Parameter to write the training log. If set to ‘True’ the log will be saved at <dataset-path>/training_log which can be visualized in tensorboard. Required tensorboardx version=2.1

The default value is ‘False’.

Note

Not applicable for Text Models

monitor

Optional string. Parameter specifies which metric to monitor while checkpointing and early stopping. Defaults to ‘valid_loss’. Value should be one of the metric that is displayed in the training table. Use {model_name}.available_metrics to list the available metrics to set here.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision training. If set to True, model training will be done in mixed precision mode. Only Pytorch based models are supported. This feature is experimental. The default value is ‘False’.

classmethod from_model(emd_path, data=None)

Creates a MMDetection object from an Esri Model Definition (EMD) file.

Parameter

Description

emd_path

Required string. Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

data

Required fastai Databunch or None. Returned data object from prepare_data() function or None for inferencing.

Returns:

MMDetection Object

load(name_or_path, **kwargs)

Loads a compatible saved model for inferencing or fine tuning from the disk.

Parameter

Description

name_or_path

Required string. Name or Path to Deep Learning Package (DLPK) or Esri Model Definition(EMD) file.

Keyword Arguments

Parameter

Description

strict

Optional boolean, default True. Whether to strictly enforce the keys of file`s state dict match with the model `Module.state_dict.

lr_find(allow_plot=True, mixed_precision=False, **kwargs)

Runs the Learning Rate Finder. Helps in choosing the optimum learning rate for training the model.

Parameter

Description

allow_plot

Optional boolean. Display the plot of losses against the learning rates and mark the optimal value of the learning rate on the plot. The default value is ‘True’.

mixed_precision

Optional boolean. Parameter to enable/disable mixed precision. If set to True, optimum learning rate will be derived in mixed precision mode. Only Pytorch based models are supported. The default value is ‘False’.

plot_losses()

Plot validation and training losses after fitting the model.

predict(image_path, threshold=0.5, nms_overlap=0.1, return_scores=False, visualize=False, resize=False)

Runs prediction on an Image. This method is only supported for RGB images.

Parameter

Description

image_path

Required. Path to the image file to make the predictions on.

threshold

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

return_scores

Optional boolean. Will return the probability scores of the bounding box predictions if True.

visualize

Optional boolean. Displays the image with predicted bounding boxes if True.

resize

Optional boolean. Resizes the image to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the image is resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the image (of the same size as the model was trained on).

Returns:

Returns a tuple with predictions, labels and optionally confidence scores if return_scores=True. The predicted bounding boxes are returned as a list of lists containing the xmin, ymin, width and height of each predicted object in each image. The labels are returned as a list of class values and the confidence scores are returned as a list of floats indicating the confidence of each prediction.

predict_video(input_video_path, metadata_file, threshold=0.5, nms_overlap=0.1, track=False, visualize=False, output_file_path=None, multiplex=False, multiplex_file_path=None, tracker_options={'assignment_iou_thrd': 0.3, 'detect_frames': 10, 'vanish_frames': 40}, visual_options={'color': (255, 255, 255), 'fontface': 0, 'show_labels': True, 'show_scores': True, 'thickness': 2}, resize=False)

Runs prediction on a video and appends the output VMTI predictions in the metadata file. This method is only supported for RGB images.

Parameter

Description

input_video_path

Required. Path to the video file to make the predictions on.

metadata_file

Required. Path to the metadata csv file where the predictions will be saved in VMTI format.

threshold

Optional float. The probability above which a detection will be considered.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

track

Optional bool. Set this parameter as True to enable object tracking.

visualize

Optional boolean. If True a video is saved with prediction results.

output_file_path

Optional path. Path of the final video to be saved. If not supplied, video will be saved at path input_video_path appended with _prediction.avi. Supports only AVI and MP4 formats.

multiplex

Optional boolean. Runs Multiplex using the VMTI detections.

multiplex_file_path

Optional path. Path of the multiplexed video to be saved. By default a new file with _multiplex.MOV extension is saved in the same folder.

tracking_options

Optional dictionary. Set different parameters for object tracking. assignment_iou_thrd parameter is used to assign threshold for assignment of trackers, vanish_frames is the number of frames the object should be absent to consider it as vanished, detect_frames is the number of frames an object should be detected to track it.

visual_options

Optional dictionary. Set different parameters for visualization. show_scores boolean, to view scores on predictions, show_labels boolean, to view labels on predictions, thickness integer, to set the thickness level of box, fontface integer, fontface value from opencv values, color tuple (B, G, R), tuple containing values between 0-255.

resize

Optional boolean. Resizes the video frames to the same size (chip_size parameter in prepare_data) that the model was trained on, before detecting objects. Note that if resize_to parameter was used in prepare_data, the video frames are resized to that size instead.

By default, this parameter is false and the detections are run in a sliding window fashion by applying the model on cropped sections of the frame (of the same size as the model was trained on).

save(name_or_path, framework='PyTorch', publish=False, gis=None, compute_metrics=True, save_optimizer=False, save_inference_file=True, **kwargs)

Saves the model weights, creates an Esri Model Definition and Deep Learning Package zip for deployment to Image Server or ArcGIS Pro.

Parameter

Description

name_or_path

Required string. Name of the model to save. It stores it at the pre-defined location. If path is passed then it stores at the specified path with model name as directory name and creates all the intermediate directories.

framework

Optional string. Exports the model in the specified framework format (‘PyTorch’, ‘tflite’ ‘torchscript’, and ‘TF-ONXX’ (deprecated)). Only models saved with the default framework (PyTorch) can be loaded using from_model. tflite framework (experimental support) is supported by SingleShotDetector - tensorflow backend only, RetinaNet - tensorflow backend only.``torchscript`` format is supported by SiamMask, MaskRCNN, SingleShotDetector, YOLOv3 and RetinaNet. For usage of SiamMask model in ArcGIS Pro >= 2.8, load the PyTorch framework saved model and export it with torchscript framework using ArcGIS API for Python >= v1.8.5. For usage of SiamMask model in ArcGIS Pro 2.9, set framework to torchscript and use the model files additionally generated inside ‘torch_scripts’ folder. If framework is TF-ONNX (Only supported for SingleShotDetector), batch_size can be passed as an optional keyword argument.

publish

Optional boolean. Publishes the DLPK as an item.

gis

Optional GIS Object. Used for publishing the item. If not specified then active gis user is taken.

compute_metrics

Optional boolean. Used for computing model metrics.

save_optimizer

Optional boolean. Used for saving the model-optimizer state along with the model. Default is set to False

save_inference_file

Optional boolean. Used for saving the inference file along with the model. If False, the model will not work with ArcGIS Pro 2.6 or earlier. Default is set to True.

kwargs

Optional Parameters.

show_results(rows=5, thresh=0.5, nms_overlap=0.1)

Displays the results of a trained model on a part of the validation set.

Parameter

Description

rows

Optional int. Number of rows of results to be displayed.

thresh

Optional float. The probability above which a detection will be considered valid.

nms_overlap

Optional float. The intersection over union threshold with other predicted bounding boxes, above which the box with the highest score will be considered a true positive.

property supported_datasets

Supported dataset types for this model.

supported_models = ['atss', 'carafe', 'cascade_rcnn', 'cascade_rpn', 'dcn', 'detectors', 'dino', 'double_heads', 'dynamic_rcnn', 'empirical_attention', 'fcos', 'foveabox', 'fsaf', 'ghm', 'hrnet', 'libra_rcnn', 'nas_fcos', 'pafpn', 'pisa', 'regnet', 'reppoints', 'res2net', 'sabl', 'vfnet']

List of models supported by this class.

supported_transformer_models = ['dino']

List of transformer models supported by this class.

unfreeze()

Unfreezes the earlier layers of the model for fine-tuning.