Skip to content

Potential enhancements for DropBlock #188

@theabhirath

Description

@theabhirath

DropBlock is a type of regularisation that tries to replace dropout. The original paper describes it as best used with a linear scaling rate across blocks in a model, as has been implemented in #174. However, timm adds several experimental features, including:

  • a scaling value for the gamma calculated (already included in Overhaul of ResNet API #174).
  • a certain configuration of DropBlock rates picked by experimentation to give better results on ResNets - these however extend only to four-stage ResNets i.e. those similar to the models in the original ResNet paper.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestlayersRelated to the Layers module - generic layers for reuse by models

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions