a  d_5@sddlmZmZmZmZddlmZddlmZddl m Z ddl m Z m Z ddlmZmZmZddlmZddlmZdd lmZdd lmZmZesd d gZGd dde ZGddde ZGddde ZdS))AnyOptionalSequenceUnion)Tensor)Literal)_ClassificationTaskWrapper)BinaryConfusionMatrixMulticlassConfusionMatrix)"_binary_cohen_kappa_arg_validation_cohen_kappa_reduce&_multiclass_cohen_kappa_arg_validation)Metric)ClassificationTaskNoMultilabel)_MATPLOTLIB_AVAILABLE)_AX_TYPE_PLOT_OUT_TYPEBinaryCohenKappa.plotMulticlassCohenKappa.plotcseZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed <de e e e ed eed dfdd ZedddZde eeeefe eedddZZS)BinaryCohenKappaa Calculate `Cohen's kappa score`_ that measures inter-annotator agreement for binary tasks. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): A int or float tensor of shape ``(N, ...)``. If preds is a floating point tensor with values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. Addtionally, we convert to int tensor with thresholding using the value in ``threshold``. - ``target`` (:class:`~torch.Tensor`): An int tensor of shape ``(N, ...)``. .. note:: Additional dimension ``...`` will be flattened into the batch dimension. As output to ``forward`` and ``compute`` the metric returns the following output: - ``bck`` (:class:`~torch.Tensor`): A tensor containing cohen kappa score Args: threshold: Threshold for transforming probability to binary (0,1) predictions ignore_index: Specifies a target value that is ignored and does not contribute to the metric calculation weights: Weighting type to calculate the score. Choose from: - ``None`` or ``'none'``: no weighting - ``'linear'``: linear weighting - ``'quadratic'``: quadratic weighting validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example (preds is int tensor): >>> from torch import tensor >>> from torchmetrics.classification import BinaryCohenKappa >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0, 1, 0, 0]) >>> metric = BinaryCohenKappa() >>> metric(preds, target) tensor(0.5000) Example (preds is float tensor): >>> from torchmetrics.classification import BinaryCohenKappa >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0.35, 0.85, 0.48, 0.01]) >>> metric = BinaryCohenKappa() >>> metric(preds, target) tensor(0.5000) Fis_differentiableThigher_is_betterfull_state_updateplot_lower_bound?plot_upper_bound?NZlinearZ quadraticnone) threshold ignore_indexweights validate_argskwargsreturnc s<tj||fddd||r,t|||||_||_dSNF) normalizer#)super__init__r r"r#)selfr r!r"r#r$ __class__p/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torchmetrics/classification/cohen_kappa.pyr)bs  zBinaryCohenKappa.__init__r%cCst|j|jSzCompute metric.r Zconfmatr"r*r-r-r.computepszBinaryCohenKappa.computevalaxr%cCs |||S)a9Plot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure object and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> from torch import rand, randint >>> # Example plotting a single value >>> from torchmetrics.classification import BinaryCohenKappa >>> metric = BinaryCohenKappa() >>> metric.update(rand(10), randint(2,(10,))) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> from torch import rand, randint >>> # Example plotting multiple values >>> from torchmetrics.classification import BinaryCohenKappa >>> metric = BinaryCohenKappa() >>> values = [ ] >>> for _ in range(10): ... values.append(metric(rand(10), randint(2,(10,)))) >>> fig_, ax_ = metric.plot(values) Z_plotr*r5r6r-r-r.plotts(r)rNNT)NN)__name__ __module__ __qualname____doc__rbool__annotations__rrrfloatrrintrrr)rr3rrrrr9 __classcell__r-r-r+r.r#s0 8      rcseZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed <d Z e ed <deeeeedeed dfdd ZedddZdeeeeefeeedddZZS)MulticlassCohenKappaa" Calculate `Cohen's kappa score`_ that measures inter-annotator agreement for multiclass tasks. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): Either an int tensor of shape ``(N, ...)` or float tensor of shape ``(N, C, ..)``. If preds is a floating point we apply ``torch.argmax`` along the ``C`` dimension to automatically convert probabilities/logits into an int tensor. - ``target`` (:class:`~torch.Tensor`): An int tensor of shape ``(N, ...)``. .. note:: Additional dimension ``...`` will be flattened into the batch dimension. As output to ``forward`` and ``compute`` the metric returns the following output: - ``mcck`` (:class:`~torch.Tensor`): A tensor containing cohen kappa score Args: num_classes: Integer specifing the number of classes ignore_index: Specifies a target value that is ignored and does not contribute to the metric calculation weights: Weighting type to calculate the score. Choose from: - ``None`` or ``'none'``: no weighting - ``'linear'``: linear weighting - ``'quadratic'``: quadratic weighting validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example (pred is integer tensor): >>> from torch import tensor >>> from torchmetrics.classification import MulticlassCohenKappa >>> target = tensor([2, 1, 0, 0]) >>> preds = tensor([2, 1, 0, 1]) >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric(preds, target) tensor(0.6364) Example (pred is float tensor): >>> from torchmetrics.classification import MulticlassCohenKappa >>> target = tensor([2, 1, 0, 0]) >>> preds = tensor([[0.16, 0.26, 0.58], ... [0.22, 0.61, 0.17], ... [0.71, 0.09, 0.20], ... [0.05, 0.82, 0.13]]) >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric(preds, target) tensor(0.6364) FrTrrrrrrZClassplot_legend_nameNr) num_classesr!r"r#r$r%c s<tj||fddd||r,t|||||_||_dSr&)r(r)r r"r#)r*rEr!r"r#r$r+r-r.r)s  zMulticlassCohenKappa.__init__r/cCst|j|jSr0r1r2r-r-r.r3szMulticlassCohenKappa.computer4cCs |||S)aPlot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure object and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> from torch import randn, randint >>> # Example plotting a single value >>> from torchmetrics.classification import MulticlassCohenKappa >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric.update(randn(20,3).softmax(dim=-1), randint(3, (20,))) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> from torch import randn, randint >>> # Example plotting a multiple values >>> from torchmetrics.classification import MulticlassCohenKappa >>> metric = MulticlassCohenKappa(num_classes=3) >>> values = [] >>> for _ in range(20): ... values.append(metric(randn(20,3).softmax(dim=-1), randint(3, (20,)))) >>> fig_, ax_ = metric.plot(values) r7r8r-r-r.r9s(r)NNT)NN)r:r;r<r=rr>r?rrrr@rrDstrrArrrr)rr3rrrrr9rBr-r-r+r.rCs0 ;       rCc @sBeZdZdZd edeeeeedeeee e ddd Z dS) CohenKappaa0Calculate `Cohen's kappa score`_ that measures inter-annotator agreement. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. This function is a simple wrapper to get the task specific versions of this metric, which is done by setting the ``task`` argument to either ``'binary'`` or ``'multiclass'``. See the documentation of :class:`~torchmetrics.classification.BinaryCohenKappa` and :class:`~torchmetrics.classification.MulticlassCohenKappa` for the specific details of each argument influence and examples. Legacy Example: >>> from torch import tensor >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0, 1, 0, 0]) >>> cohenkappa = CohenKappa(task="multiclass", num_classes=2) >>> cohenkappa(preds, target) tensor(0.5000) rNT)binaryZ multiclassr)taskr rEr"r!r#r$r%cKst|}||||d|tjkr6t|fi|S|tjkrnt|ts^tdt |dt |fi|Std|ddS)zInitialize task metric.)r"r!r#z+`num_classes` is expected to be `int` but `z was passed.`zTask z not supported!N) rZfrom_strupdateBINARYrZ MULTICLASS isinstancerA ValueErrortyperC)clsrIr rEr"r!r#r$r-r-r.__new__:s    zCohenKappa.__new__)rNNNT) r:r;r<r=rr@rrAr>rrrPr-r-r-r.rGs  rGN)typingrrrrZtorchrZtyping_extensionsrZ torchmetrics.classification.baserZ,torchmetrics.classification.confusion_matrixr r Z2torchmetrics.functional.classification.cohen_kappar r r Ztorchmetrics.metricrZtorchmetrics.utilities.enumsrZtorchmetrics.utilities.importsrZtorchmetrics.utilities.plotrrZ__doctest_skip__rrCrGr-r-r-r.s      |