a  dE@sddlmZmZmZmZmZddlmZddlm Z ddl m Z m Z ddl mZddlmZddlmZddlmZmZesd gZGd d d eZd S) )AnyListOptionalSequenceUnion)Tensor)Literal)_fleiss_kappa_compute_fleiss_kappa_update)Metric) dim_zero_cat)_MATPLOTLIB_AVAILABLE)_AX_TYPE_PLOT_OUT_TYPEFleissKappa.plotcseZdZUdZdZeed<dZeed<dZeed<dZ e ed<e e ed <de d ed d fd d Ze d dddZe dddZdee ee d feeedddZZS) FleissKappaaCalculatees `Fleiss kappa`_ a statistical measure for inter agreement between raters. .. math:: \kappa = \frac{\bar{p} - \bar{p_e}}{1 - \bar{p_e}} where :math:`\bar{p}` is the mean of the agreement probability over all raters and :math:`\bar{p_e}` is the mean agreement probability over all raters if they were randomly assigned. If the raters are in complete agreement then the score 1 is returned, if there is no agreement among the raters (other than what would be expected by chance) then a score smaller than 0 is returned. As input to ``forward`` and ``update`` the metric accepts the following input: - ``ratings`` (:class:`~torch.Tensor`): Ratings of shape ``[n_samples, n_categories]`` or ``[n_samples, n_categories, n_raters]`` depedenent on ``mode``. If ``mode`` is ``counts``, ``ratings`` must be integer and contain the number of raters that chose each category. If ``mode`` is ``probs``, ``ratings`` must be floating point and contain the probability/logits that each rater chose each category. As output of ``forward`` and ``compute`` the metric returns the following output: - ``fleiss_k`` (:class:`~torch.Tensor`): A float scalar tensor with the calculated Fleiss' kappa score. Args: mode: Whether `ratings` will be provided as counts or probabilities. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example: >>> # Ratings are provided as counts >>> import torch >>> from torchmetrics.nominal import FleissKappa >>> _ = torch.manual_seed(42) >>> ratings = torch.randint(0, 10, size=(100, 5)).long() # 100 samples, 5 categories, 10 raters >>> metric = FleissKappa(mode='counts') >>> metric(ratings) tensor(0.0089) Example: >>> # Ratings are provided as probabilities >>> import torch >>> from torchmetrics.nominal import FleissKappa >>> _ = torch.manual_seed(42) >>> ratings = torch.randn(100, 5, 10).softmax(dim=1) # 100 samples, 5 categories, 10 raters >>> metric = FleissKappa(mode='probs') >>> metric(ratings) tensor(-0.0105) Ffull_state_updateis_differentiableThigher_is_betterg?plot_upper_boundcountsrZprobsN)modekwargsreturnc s<tjfi||dvr"td||_|jdgdddS)Nrz5Argument ``mode`` must be one of 'counts' or 'probs'.rcat)defaultZdist_reduce_fx)super__init__ ValueErrorrZ add_state)selfrr __class__j/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torchmetrics/nominal/fleiss_kappa.pyrSs zFleissKappa.__init__)ratingsrcCst||j}|j|dS)z+Updates the counts for fleiss kappa metric.N)r rrappend)r r%rr#r#r$updateZs zFleissKappa.update)rcCst|j}t|S)zComputes Fleiss' kappa.)r rr )r rr#r#r$compute_s zFleissKappa.compute)valaxrcCs |||S)aPlot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> # Example plotting a single value >>> import torch >>> from torchmetrics.nominal import FleissKappa >>> metric = FleissKappa(mode="probs") >>> metric.update(torch.randn(100, 5, 10).softmax(dim=1)) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> # Example plotting multiple values >>> import torch >>> from torchmetrics.nominal import FleissKappa >>> metric = FleissKappa(mode="probs") >>> values = [ ] >>> for _ in range(10): ... values.append(metric(torch.randn(100, 5, 10).softmax(dim=1))) >>> fig_, ax_ = metric.plot(values) )Z_plot)r r)r*r#r#r$plotds&r)r)NN)__name__ __module__ __qualname____doc__rbool__annotations__rrrfloatrrrrrr'r(rrrrrr+ __classcell__r#r#r!r$rs /     rN)typingrrrrrZtorchrZtyping_extensionsrZ,torchmetrics.functional.nominal.fleiss_kappar r Ztorchmetrics.metricr Ztorchmetrics.utilities.datar Ztorchmetrics.utilities.importsr Ztorchmetrics.utilities.plotrrZ__doctest_skip__rr#r#r#r$s