a  d"@sddlmZmZmZmZddlZddlmZm Z m Z m Z m Z ddl mZmZe rjddlmZddlmZe rdd lmZmZmZmZe eZeeefZeeZeeGd d d eZdS) )AnyDictListUnionN)add_end_docstringsis_torch_availableis_vision_availableloggingrequires_backends)PIPELINE_INIT_ARGSPipeline)Image) load_image)$MODEL_FOR_IMAGE_SEGMENTATION_MAPPING'MODEL_FOR_INSTANCE_SEGMENTATION_MAPPING'MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING(MODEL_FOR_UNIVERSAL_SEGMENTATION_MAPPINGcsbeZdZdZfddZddZeeee fdfdd Z dd d Z d d Z dddZ ZS)ImageSegmentationPipelinea Image segmentation pipeline using any `AutoModelForXXXSegmentation`. This pipeline predicts masks of objects and their classes. Example: ```python >>> from transformers import pipeline >>> segmenter = pipeline(model="facebook/detr-resnet-50-panoptic") >>> segments = segmenter("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png") >>> len(segments) 2 >>> segments[0]["label"] 'bird' >>> segments[1]["label"] 'bird' >>> type(segments[0]["mask"]) # This is a black and white mask showing where is the bird on the original image. >>> segments[0]["mask"].size (768, 512) ``` This image segmentation pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"image-segmentation"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=image-segmentation). csftj|i||jdkr.td|jdt|d|tt t t t dS)NtfzThe z is only available in PyTorch.Zvision) super__init__ framework ValueError __class__r Zcheck_model_typedictritemsrrr)selfargskwargsrr/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/transformers/pipelines/image_segmentation.pyrCs  z"ImageSegmentationPipeline.__init__cKsni}i}d|vr(|d|d<|d|d<d|vr<|d|d<d|vrP|d|d<d|vrd|d|d<|i|fS)Nsubtask thresholdmask_thresholdoverlap_mask_area_thresholdr")rr Zpreprocess_kwargsZpostprocess_kwargsr"r"r#_sanitize_parametersSs     z.ImageSegmentationPipeline._sanitize_parameters)returnc stj|fi|S)a Perform segmentation (detect masks & classes) in the image(s) passed as inputs. Args: images (`str`, `List[str]`, `PIL.Image` or `List[PIL.Image]`): The pipeline handles three types of images: - A string containing an HTTP(S) link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the same format: all as HTTP(S) links, all as local paths, or all as PIL images. subtask (`str`, *optional*): Segmentation task to be performed, choose [`semantic`, `instance` and `panoptic`] depending on model capabilities. If not set, the pipeline will attempt tp resolve in the following order: `panoptic`, `instance`, `semantic`. threshold (`float`, *optional*, defaults to 0.9): Probability threshold to filter out predicted masks. mask_threshold (`float`, *optional*, defaults to 0.5): Threshold to use when turning the predicted masks into binary values. overlap_mask_area_threshold (`float`, *optional*, defaults to 0.5): Mask overlap threshold to eliminate small, disconnected segments. Return: A dictionary or a list of dictionaries containing the result. If the input is a single image, will return a list of dictionaries, if the input is a list of several images, will return a list of list of dictionaries corresponding to each image. The dictionaries contain the mask, label and score (where applicable) of each detected object and contains the following keys: - **label** (`str`) -- The class label identified by the model. - **mask** (`PIL.Image`) -- A binary mask of the detected object as a Pil Image of shape (width, height) of the original image. Returns a mask filled with zeros if no object is found. - **score** (*optional* `float`) -- Optionally, when the model is capable of estimating a confidence of the "object" described by the label and the mask. )r__call__)rimagesr r!r"r#r*bs'z"ImageSegmentationPipeline.__call__NcCst|}|j|jfg}|jjjjdkr~|dur4i}n d|gi}|jf|gdd|}|j|dd|jjj |j dd|d<n|j|gdd}||d<|S) NZOneFormerConfigZ task_inputspt)r+return_tensors max_length)paddingr.r-Z input_ids target_size) rheightwidthmodelconfigr__name__image_processor tokenizerZ task_seq_lenr)rimager$r0r inputsr"r"r# preprocesss$  z$ImageSegmentationPipeline.preprocesscCs&|d}|jfi|}||d<|S)Nr0)popr3)rZ model_inputsr0 model_outputsr"r"r#_forwards z"ImageSegmentationPipeline._forward??cCsd}|dvr"t|jdr"|jj}n|dvr>t|jdr>|jj}|dur||||||ddd}g}|d} |d D]Z} | | d kd } tj| tj d d } |j j j | d} | d} | | | | dqtn|dvrht|jdrh|jj||ddd}g}|} t| }|D]H} | | kd } tj| tj d d } |j j j | } | d| | dqntd|dt|j |S)N>ZpanopticN"post_process_panoptic_segmentation>instanceN"post_process_instance_segmentationr0)r%r&r' target_sizesr segmentationZ segments_infoidL)modeZlabel_idscore)rIlabelmask>ZsemanticN"post_process_semantic_segmentation)rCzSubtask z is not supported for model )hasattrr6r@rBrZ fromarraynumpyZastypenpZuint8r3r4Zid2labelappendrLuniquertype)rr<r$r%r&r'fnoutputs annotationrDsegmentrKrJrIlabelsr"r"r# postprocesssN    z%ImageSegmentationPipeline.postprocess)N)Nr>r?r?)r5 __module__ __qualname____doc__rr(r Predictionsr Predictionr*r:r=rX __classcell__r"r"r!r#rs# ) r) typingrrrrrNrOutilsrrr r r baser rZPILrZ image_utilsrZmodels.auto.modeling_autorrrrZ get_loggerr5loggerstrr]r\rr"r"r"r#s