Bases: SampledNeighborEmbedding
InfoTSNE algorithm introduced in [Damrich et al., 2022].
It uses a EntropicAffinity as input affinity \(\mathbf{P}\) and a StudentAffinity as output affinity \(\mathbf{Q}\).
The loss function is defined as:
\[-\sum_{ij} P_{ij} \log Q_{ij} + \sum_i \log \Big( \sum_{j \in \mathrm{Neg}(i)} Q_{ij} \Big)\]
where \(\mathrm{Neg}(i)\) is the set of negatives samples for point \(i\).
perplexity (float) – Number of ‘effective’ nearest neighbors. Consider selecting a value between 2 and the number of samples. Different values can result in significantly different results.
n_components (int, optional) – Dimension of the embedding space.
lr (float or 'auto', optional) – Learning rate for the algorithm, by default “auto”.
optimizer (str or torch.optim.Optimizer, optional) – Name of an optimizer from torch.optim or an optimizer class. Default is “SGD”.
optimizer_kwargs (dict or 'auto', optional) – Additional keyword arguments for the optimizer. Default is ‘auto’, which sets appropriate momentum values for SGD based on early exaggeration phase.
scheduler (str or torch.optim.lr_scheduler.LRScheduler, optional) – Name of a scheduler from torch.optim.lr_scheduler or a scheduler class. Default is “LinearLR”.
scheduler_kwargs (dict, optional) – Additional keyword arguments for the scheduler.
init ({'normal', 'pca'} or torch.Tensor of shape (n_samples, output_dim), optional) – Initialization for the embedding Z, default ‘pca’.
init_scaling (float, optional) – Scaling factor for the initialization, by default 1e-4.
min_grad_norm (float, optional) – Precision threshold at which the algorithm stops, by default 1e-7.
max_iter (int, optional) – Number of maximum iterations for the descent algorithm, by default 1000.
device (str, optional) – Device to use, by default “auto”.
backend ({"keops", "faiss", None}, optional) – Which backend to use for handling sparsity and memory efficiency. Default is “faiss”.
verbose (bool, optional) – Verbosity, by default False.
random_state (float, optional) – Random seed for reproducibility, by default None.
max_iter_affinity (int, optional) – Number of maximum iterations for the entropic affinity root search.
metric_in ({'sqeuclidean', 'manhattan'}, optional) – Metric to use for the input affinity, by default ‘sqeuclidean’.
metric_out ({'sqeuclidean', 'manhattan'}, optional) – Metric to use for the output affinity, by default ‘sqeuclidean’.
early_exaggeration_coeff (float, optional) – Factor for the early exaggeration phase, by default 12.
early_exaggeration_iter (int, optional) – Number of iterations for the early exaggeration phase, by default 250.
n_negatives (int, optional) – Number of negative samples for the noise-contrastive loss, by default 300.
sparsity (bool, optional) – Whether to use sparsity mode for the input affinity. Default is True.
check_interval (int, optional) – Interval for checking convergence, by default 50.
discard_NNs (bool, optional) – Whether to discard the nearest neighbors from the negative sampling. Default is False.
compile (bool, optional) – Whether to compile the loss function with torch.compile for faster computation. Default is False.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4