Plugins#

Plugins are Python modules that extend the functionality of OpenPifPaf. Plugins are developed independently of OpenPifPaf (i.e. you don’t need to clone and modify OpenPifPaf’s core code).

This is an overview of OpenPifPaf’s main components and which are extendible with plugins: plugin overview

Discovery#

Plugins are modules whose name starts with openpifpaf_. Some plugins are also included in openpifpaf.plugins. During the discovery process, every plugin’s register() function is called. A simple plugin that does nothing, the nothingplugin would have this file structure:

|- openpifpaf_nothingplugin  # project directory (root directory for version control)
  |- Readme.md
  |- openpifpaf_nothingplugin  # Python module
    |- __init__.py

where the __init__.py file would contain an empty register() function:

def register():
    pass

Registrations#

A plugin contains new implementations of base classes provided by OpenPifPaf. The register() function adds these implementations to OpenPifPaf’s lists of implementations. Those lists are:

openpifpaf.DATAMODULES

{'animal': openpifpaf.plugins.animalpose.animal_kp.AnimalKp,
 'apollo': openpifpaf.plugins.apollocar3d.apollo_kp.ApolloKp,
 'cifar10': openpifpaf.plugins.cifar10.datamodule.Cifar10,
 'cocodet': openpifpaf.plugins.coco.cocodet.CocoDet,
 'cocokp': openpifpaf.plugins.coco.cocokp.CocoKp,
 'crowdpose': openpifpaf.plugins.crowdpose.module.CrowdPose,
 'nuscenes': openpifpaf.plugins.nuscenes.nuscenes.NuScenes,
 'posetrack2018': openpifpaf.plugins.posetrack.posetrack2018.Posetrack2018,
 'posetrack2017': openpifpaf.plugins.posetrack.posetrack2017.Posetrack2017,
 'cocokpst': openpifpaf.plugins.posetrack.cocokpst.CocoKpSt,
 'wholebody': openpifpaf.plugins.wholebody.wholebody.Wholebody}

openpifpaf.DECODERS

{openpifpaf.decoder.cifcaf.CifCaf,
 openpifpaf.decoder.cifcaf.CifCafDense,
 openpifpaf.decoder.cifdet.CifDet,
 openpifpaf.decoder.pose_similarity.PoseSimilarity,
 openpifpaf.decoder.tracking_pose.TrackingPose}

openpifpaf.BASE_TYPES

{openpifpaf.network.basenetworks.CLIPConvNeXt,
 openpifpaf.network.basenetworks.ConvNeXtV2,
 openpifpaf.network.basenetworks.EffNetV2,
 openpifpaf.network.basenetworks.HRFormer,
 openpifpaf.network.basenetworks.MobileNetV2,
 openpifpaf.network.basenetworks.MobileNetV3,
 openpifpaf.network.basenetworks.Resnet,
 openpifpaf.network.basenetworks.ShuffleNetV2,
 openpifpaf.network.basenetworks.ShuffleNetV2K,
 openpifpaf.network.basenetworks.SqueezeNet,
 openpifpaf.network.basenetworks.SwinTransformer,
 openpifpaf.network.basenetworks.XCiT,
 openpifpaf.network.tracking_base.TrackingBase,
 openpifpaf.plugins.cifar10.basenet.Cifar10Net}

openpifpaf.BASE_FACTORIES

{'mobilenetv2': <function openpifpaf.network.factory.<lambda>()>,
 'mobilenetv3large': <function openpifpaf.network.factory.<lambda>()>,
 'mobilenetv3small': <function openpifpaf.network.factory.<lambda>()>,
 'resnet18': <function openpifpaf.network.factory.<lambda>()>,
 'resnet50': <function openpifpaf.network.factory.<lambda>()>,
 'resnet101': <function openpifpaf.network.factory.<lambda>()>,
 'resnet152': <function openpifpaf.network.factory.<lambda>()>,
 'resnext50': <function openpifpaf.network.factory.<lambda>()>,
 'resnext101': <function openpifpaf.network.factory.<lambda>()>,
 'shufflenetv2x1': <function openpifpaf.network.factory.<lambda>()>,
 'shufflenetv2x2': <function openpifpaf.network.factory.<lambda>()>,
 'shufflenetv2k16': <function openpifpaf.network.factory.<lambda>()>,
 'shufflenetv2k20': <function openpifpaf.network.factory.<lambda>()>,
 'shufflenetv2kx5': <function openpifpaf.network.factory.<lambda>()>,
 'shufflenetv2k30': <function openpifpaf.network.factory.<lambda>()>,
 'shufflenetv2k44': <function openpifpaf.network.factory.<lambda>()>,
 'squeezenet': <function openpifpaf.network.factory.<lambda>()>,
 'swin_t': <function openpifpaf.network.factory.<lambda>()>,
 'swin_s': <function openpifpaf.network.factory.<lambda>()>,
 'swin_b': <function openpifpaf.network.factory.<lambda>()>,
 'swin_b_window_12': <function openpifpaf.network.factory.<lambda>()>,
 'swin_l': <function openpifpaf.network.factory.<lambda>()>,
 'swin_l_window_12': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_nano_12_p16': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_tiny_12_p16': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_tiny_24_p16': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_small_12_p16': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_small_24_p16': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_medium_24_p16': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_large_24_p16': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_nano_12_p8': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_tiny_12_p8': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_tiny_24_p8': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_small_12_p8': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_small_24_p8': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_medium_24_p8': <function openpifpaf.network.factory.<lambda>()>,
 'xcit_large_24_p8': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_s': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_m': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_l': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_xl': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_s16_s': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_s16_m': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_s16_l': <function openpifpaf.network.factory.<lambda>()>,
 'effnetv2_s16_xl': <function openpifpaf.network.factory.<lambda>()>,
 'botnet': <function openpifpaf.network.factory.<lambda>()>,
 'hrformersmall': <function openpifpaf.network.factory.<lambda>()>,
 'hrformersmallcat': <function openpifpaf.network.factory.<lambda>()>,
 'hrformerbase': <function openpifpaf.network.factory.<lambda>()>,
 'hrformerbasecat': <function openpifpaf.network.factory.<lambda>()>,
 'convnextv2base': <function openpifpaf.network.factory.<lambda>()>,
 'clipconvnextbase': <function openpifpaf.network.factory.<lambda>()>,
 'tshufflenetv2k16': <function openpifpaf.network.factory.<lambda>()>,
 'tshufflenetv2k30': <function openpifpaf.network.factory.<lambda>()>,
 'tresnet50': <function openpifpaf.network.factory.<lambda>()>,
 'cifar10net': <function openpifpaf.plugins.cifar10.register.<locals>.<lambda>()>}

openpifpaf.HEADS

{openpifpaf.headmeta.Cif: openpifpaf.network.heads.CompositeField4,
 openpifpaf.headmeta.Caf: openpifpaf.network.heads.CompositeField4,
 openpifpaf.headmeta.CifDet: openpifpaf.network.heads.CompositeField4,
 openpifpaf.headmeta.TSingleImageCif: openpifpaf.network.tracking_heads.TBaseSingleImage,
 openpifpaf.headmeta.TSingleImageCaf: openpifpaf.network.tracking_heads.TBaseSingleImage,
 openpifpaf.headmeta.Tcaf: openpifpaf.network.tracking_heads.Tcaf}

openpifpaf.LOSSES

{'a': openpifpaf.network.losses.composite.CompositeLossByComponent,
 openpifpaf.headmeta.Cif: openpifpaf.network.losses.composite.CompositeLoss,
 openpifpaf.headmeta.Caf: openpifpaf.network.losses.composite.CompositeLoss,
 openpifpaf.headmeta.CifDet: openpifpaf.network.losses.composite.CompositeLoss,
 openpifpaf.headmeta.TSingleImageCif: openpifpaf.network.losses.composite.CompositeLoss,
 openpifpaf.headmeta.TSingleImageCaf: openpifpaf.network.losses.composite.CompositeLoss,
 openpifpaf.headmeta.Tcaf: openpifpaf.network.losses.composite.CompositeLoss}

openpifpaf.MODEL_MIGRATION

{<function openpifpaf.network.model_migration.fix_feature_cache(model)>,
 <function openpifpaf.network.model_migration.subscribe_cache_reset(model)>,
 <function openpifpaf.network.model_migration.tcaf_shared_preprocessing(model)>}

openpifpaf.CHECKPOINT_URLS

{'shufflenetv2k30-animalpose': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.12.9/shufflenetv2k30-210511-120906-animal.pkl.epoch400',
 'shufflenetv2k16-apollo-24': 'http://github.com/DuncanZauss/openpifpaf_assets/releases/download/v0.1.0/shufflenetv2k16-201113-135121-apollo.pkl.epoch290',
 'shufflenetv2k16-apollo-66': 'http://github.com/DuncanZauss/openpifpaf_assets/releases/download/v0.1.0/sk16_apollo_66kp.pkl',
 'shufflenetv2k30-apollo-66': 'http://github.com/DuncanZauss/openpifpaf_assets/releases/download/v0.1.0/sk30_apollo_66kp.pkl',
 'mobilenetv2': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.12a5/mobilenetv2-201112-193315-cocokp-1728a9f5.pkl',
 'mobilenetv3small': 'http://github.com/openpifpaf/torchhub/releases/download/v0.13/mobilenetv3small-210822-213409-cocokp-slurm726252-edge513-o10s-803b24ae.pkl',
 'mobilenetv3large': 'http://github.com/openpifpaf/torchhub/releases/download/v0.13/mobilenetv3large-210820-184901-cocokp-slurm725985-edge513-o10s-6c76cbfb.pkl',
 'resnet18': <object at 0x7cda51fd4e80>,
 'resnet50': 'http://github.com/openpifpaf/torchhub/releases/download/v0.13/resnet50-210830-150728-cocokp-slurm728641-edge513-o10s-ecd30da4.pkl',
 'resnet101': <object at 0x7cda51fd4e80>,
 'resnet152': <object at 0x7cda51fd4e80>,
 'shufflenetv2x1': <object at 0x7cda51fd4e80>,
 'shufflenetv2x2': <object at 0x7cda51fd4e80>,
 'shufflenetv2k16': 'http://github.com/openpifpaf/torchhub/releases/download/v0.13/shufflenetv2k16-210820-232500-cocokp-slurm726069-edge513-o10s-7189450a.pkl',
 'shufflenetv2k16-withdense': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.12b4/shufflenetv2k16-210221-131426-cocokp-o10s-627d901e.pkl',
 'shufflenetv2k30': 'http://github.com/openpifpaf/torchhub/releases/download/v0.13/shufflenetv2k30-210821-003923-cocokp-slurm726072-edge513-o10s-5fe1c400.pkl',
 'shufflenetv2k30*': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.14.2/cocokp-shufflenetv2k30.pt.epoch350',
 'shufflenetv2k44': <object at 0x7cda51fd4e80>,
 'swin_s': 'http://github.com/dmizr/openpifpaf/releases/download/v0.12.14/swin_s_fpn_lvl_3_lr_5e-5_resumed-d286d41a.pkl',
 'swin_b': 'http://github.com/dmizr/openpifpaf/releases/download/v0.12.14/swin_b_fpn_lvl_3_lr_5e-5_resumed-fa951ce0.pkl',
 'swin_t_input_upsample': 'http://github.com/dmizr/openpifpaf/releases/download/v0.12.14/swin_t_input_upsample_no_fpn_lr_5e-5_resumed-e0681112.pkl',
 'swin_l_input_upsample': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.14.2/cocokp-swinl-inputupsample.pt.epoch250',
 'hrformerbasecat': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.14.2/cocokp-hrformerbasecat.pt.epoch250',
 'clipconvnextbase': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.14.2/cocokp-clipconvnextbase.pt.epoch250',
 'convnextv2base': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.14.2/cocokp-convnextv2base.pt.epoch250',
 'mobilenetv3small-cocodet': 'http://github.com/openpifpaf/torchhub/releases/download/v0.13/mobilenetv3small-210822-215020-cocodet-slurm726253-5f2c894f.pkl',
 'resnet18-cocodet': 'http://github.com/openpifpaf/torchhub/releases/download/v0.12.10/resnet18-210526-031303-cocodet-slurm610002-1faf5801.pkl',
 'resnet50-crowdpose': 'http://github.com/vita-epfl/openpifpaf-torchhub/releases/download/v0.12a7/resnet50-201005-100758-crowdpose-d978a89f.pkl',
 'shufflenetv2k16-nuscenes': 'http://github.com/DuncanZauss/openpifpaf_assets/releases/download/v0.1.0/nuscenes_sk16.pkl.epoch150',
 'tshufflenetv2k16': <object at 0x7cda51fd4e80>,
 'tshufflenetv2k30': 'http://github.com/openpifpaf/torchhub/releases/download/v0.12.10/tshufflenetv2k30-210628-075118-posetrack2018-cocokpst-slurm668247-o25-3d734bb8.pkl',
 'shufflenetv2k16-wholebody': 'http://github.com/DuncanZauss/openpifpaf_assets/releases/download/v0.1.0/sk16_wholebody.pkl',
 'shufflenetv2k30-wholebody': 'http://github.com/DuncanZauss/openpifpaf_assets/releases/download/v0.1.0/sk30_wholebody.pkl'}

openpifpaf.PAINTERS

{'Annotation': openpifpaf.show.painters.KeypointPainter,
 'AnnotationCrowd': openpifpaf.show.painters.CrowdPainter,
 'AnnotationDet': openpifpaf.show.painters.DetectionPainter}

The relevant base classes that must be derived from to extend the above lists are DataModule (in openpifpaf.datasets), Decoder (in openpifpaf.decoder), BaseNetwork and HeadNetwork (both in openpifpaf.network).

OpenPifPaf Guide

Plugins

On this page

Plugins#

Discovery#

Registrations#