Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

T-Few

This repository contains the official code for the paper: "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning".

This method outperforms in-context learning with GPT-3 and achieves state-of-the-art on "RAFT".

Setup

First, create a virtual environment for the project and install all the requirments. (We use conda to manage environments. Be sure to install and initialize conda first.)

  1. Create a virtual environment with python 3.7 conda create -n tfew python==3.7, then activate the environment conda activate tfew.
  2. Install other dependencies. pip install -r requirements.txt -f https://download.pytorch.org/whl/cu113/torch_stable.html
  3. If you plan to run SAID, then install dependencies with python src/intrinsic_said_setup.py develop. Otherwise, skip this step.

The steps above only needs to be done once. In addition, every time you start a new session, you will need to run . bin/start.sh

Run your first experiment

Once you finished setting up the environment, you can try running CUDA_VISIBLE_DEVICES=3 python -m src.pl_train -c t0.json+rte.json -k save_model=False exp_name=first_exp The outputs of this run will be saved to ${OUTPUT_PATH}/first_exp/, which is usually /t-few/exp_out/first_exp/. Here, first_exp is the experiment name, you can run more experiments with different expeirment names. The code will automatically skip finished experiments. (However, if you wish to rerun a finished experiment under the same experiment name, you will need to manually remove the corresponding files in the output directory.)

There are two ways to control an experiment.

  1. You can specify config files with -c. Multiple config files can be combined with +. (When there are conflits, config terms from the config file on the right will have greater power.) This will be convinient when you have multiple terms that forms a fixed group.
  2. You can override values with -k. This will be convinient when you need to change a small number of terms.

It is recommended to use GPUs with 40GB to train T0(3B) and 80GB to train T0

Run an array of experiments

In this project, we often need to run a large number of experiments. Here is an example bash script bin/few-shot-pretrained-3b-100k.sh to fine-tune 3B pre-trained (IA)3 on all datasets.

This should take a few hours. After that, you can use scripts/get_results_table.py to generate a csv summary.

Citation

If you find this repo helpful, welcome to cite our work:

@article{liu2020tfew,
  title={Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning},
  author={Liu, Haokun and Tam, Derek and Muqeeth, Mohammed and Mohta, Jay and Huang, Tenghao and Bansal, Mohit and Raffel, Colin},
  journal={arXiv preprint arXiv:2205.05638},
  year={2022}
}

We use the following code in our works:

@article{mahabadi2021compacter,
  title={Compacter: Efficient low-rank hypercomplex adapter layers},
  author={Mahabadi, Rabeeh Karimi and Henderson, James and Ruder, Sebastian},
  journal={arXiv preprint arXiv:2106.04647},
  year={2021}
}

@article{sung2021training,
  title={Training Neural Networks with Fixed Sparse Masks},
  author={Sung, Yi-Lin and Nair, Varun and Raffel, Colin},
  journal={arXiv preprint arXiv:2111.09839},
  year={2021}
}

@article{aghajanyan2020intrinsic,
  title={Intrinsic dimensionality explains the effectiveness of language model fine-tuning},
  author={Aghajanyan, Armen and Zettlemoyer, Luke and Gupta, Sonal},
  journal={arXiv preprint arXiv:2012.13255},
  year={2020}
}
Comments
  • Multi-GPU Support

    Multi-GPU Support

    Hello,

    Have you tried training on Multi-GPU setup? I tried running your fine-tuning example like so:

    export CUDA_VISIBLE_DEVICES=0,1
    python -m src.pl_train -c t03b.json+ia3.json+rte.json -k load_weight="pretrained_checkpoints/t03b_ia3_finish.pt" exp_name=t03b_rte_seed42_ia3_pretrained100k few_shot_random_seed=42 seed=42
    

    But I get errors in the lightning data loaders.

    Any Ideas? Thank you

  • Can't run 11 billion model on A100 with 80GB

    Can't run 11 billion model on A100 with 80GB

    Hi @craffel @muqeeth @HaokunLiu,

    We're trying to reproduce T-Few results for a paper, but we're getting 'CUDA out of memory' using an A100 with 80GB (your recommended setup).

    This is what we're running:

    python -m src.pl_train -c t011b.json+ia3.json+rte.json -k load_weight="pretrained_checkpoints/t011b_ia3_finish.pt" exp_name=t011b_rte_seed42_ia3_pretrained few_shot_random_seed=42 seed=42
    

    We installed according to the README instructions and are using the default settings in the config files. We are able to run the 3 billion model using the command above, just not the 11 billion. Is there anything we are doing wrong?

    This is the exception:

    CUDA out of memory

    Thank you

  • KeyError: 'HF_HOME'

    KeyError: 'HF_HOME'

    Hi! I was trying to run the example in README, but it says KeyError: 'HF_HOME' This is the script I used: python -m src.pl_train -c t03b.json+rte.json -k save_model=False exp_name=first_exp I can't find anywhere in the code that sets the value of this environment variable.

    Mark experiment first_exp as claimed
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    Using bfloat16 Automatic Mixed Precision (AMP)
    GPU available: False, used: False
    TPU available: False, using: 0 TPU cores
    IPU available: False, using: 0 IPUs
    Traceback (most recent call last):
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/Users/weiqiuyou/Documents/codebases/t-few/src/pl_train.py", line 86, in <module>
        main(config)
      File "/Users/weiqiuyou/Documents/codebases/t-few/src/pl_train.py", line 57, in main
        trainer.fit(model, datamodule)
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 741, in fit
        self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
        return trainer_fn(*args, **kwargs)
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 777, in _fit_impl
        self._run(model, ckpt_path=ckpt_path)
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1131, in _run
        self._data_connector.prepare_data()
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 154, in prepare_data
        self.trainer.datamodule.prepare_data()
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/core/datamodule.py", line 474, in wrapped_fn
        fn(*args, **kwargs)
      File "/Users/weiqiuyou/Documents/codebases/t-few/src/data/data_module.py", line 17, in prepare_data
        _ = self.dataset_reader.read_few_shot_dataset()
      File "/Users/weiqiuyou/Documents/codebases/t-few/src/data/dataset_readers.py", line 164, in read_few_shot_dataset
        orig_data = self.read_orig_dataset("train")
      File "/Users/weiqiuyou/Documents/codebases/t-few/src/data/dataset_readers.py", line 146, in read_orig_dataset
        orig_data = load_dataset(*self.dataset_stash, split=split, cache_dir=os.environ["HF_HOME"])
      File "/Users/weiqiuyou/opt/miniconda3/envs/tfew/lib/python3.7/os.py", line 678, in __getitem__
        raise KeyError(key) from None
    KeyError: 'HF_HOME'
    
  • To which epoch/training step does the finish.pt checkpoint belong to?

    To which epoch/training step does the finish.pt checkpoint belong to?

    Hi everyone!

    When I run the experiments after eval_epoch_interval's the model is validated and a checkpoint is written out as global_stepXXXXX.pt. At the end there is also a final checkpoint written out named finish.pt. I assumed this one either belongs to the best intermediate validation performance or the last epoch. However, from comparing it with the other checkpoints that were created it seems that finish.pt differs from all global_stepXXXXX.pt checkpoints, so I am wondering to which point in training does the finish.pt belong to?

    Sorry if I miss something obvious here.

    Best, Stefan

  • AttributeError: Can't pickle local object 'create_collate_fn.<locals>.collate_fn'

    AttributeError: Can't pickle local object 'create_collate_fn..collate_fn'

    When I tried to run the demo, I found this error! @dptam @jmohta @muqeeth

    Using bfloat16 Automatic Mixed Precision (AMP)
    GPU available: False, used: False
    TPU available: False, using: 0 TPU cores
    IPU available: False, using: 0 IPUs
    HPU available: False, using: 0 HPUs
    WARNING:datasets.builder:Reusing dataset super_glue (/Users/caffrey/Documents/research/t-few-master/cache/super_glue/rte/1.0.2/d040c658e2ddef6934fdd97deb45c777b6ff50c524781ea434e7219b56a428a7)
    Missing logger folder: exp_out/first_exp/log
    WARNING:datasets.builder:Reusing dataset super_glue (/Users/caffrey/Documents/research/t-few-master/cache/super_glue/rte/1.0.2/d040c658e2ddef6934fdd97deb45c777b6ff50c524781ea434e7219b56a428a7)
    Train size 32
    Eval size 277
    
      | Name  | Type                       | Params
    -----------------------------------------------------
    0 | model | T5ForConditionalGeneration | 2.8 B 
    -----------------------------------------------------
    2.8 B     Trainable params
    0         Non-trainable params
    2.8 B     Total params
    11,399.029Total estimated model params size (MB)
    Sanity Checking: 0it [00:00, ?it/s]Traceback (most recent call last):
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/runpy.py", line 197, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/Users/caffrey/Documents/paper/t-few-master/src/pl_train.py", line 86, in <module>
        main(config)
      File "/Users/caffrey/Documents/paper/t-few-master/src/pl_train.py", line 57, in main
        trainer.fit(model, datamodule)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 770, in fit
        self._call_and_handle_interrupt(
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 723, in _call_and_handle_interrupt
        return trainer_fn(*args, **kwargs)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 811, in _fit_impl
        results = self._run(model, ckpt_path=self.ckpt_path)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1236, in _run
        results = self._run_stage()
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1323, in _run_stage
        return self._run_train()
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1345, in _run_train
        self._run_sanity_check()
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1413, in _run_sanity_check
        val_loop.run()
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 204, in run
        self.advance(*args, **kwargs)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 155, in advance
        dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 199, in run
        self.on_run_start(*args, **kwargs)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 88, in on_run_start
        self._data_fetcher = iter(data_fetcher)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/pytorch_lightning/utilities/fetching.py", line 178, in __iter__
        self.dataloader_iter = iter(self.dataloader)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 443, in __iter__
        return self._get_iterator()
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 389, in _get_iterator
        return _MultiProcessingDataLoaderIter(self)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1062, in __init__
        w.start()
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/multiprocessing/process.py", line 121, in start
        self._popen = self._Popen(self)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
        return Popen(process_obj)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
        super().__init__(process_obj)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
        self._launch(process_obj)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
        reduction.dump(process_obj, fp)
      File "/Users/caffrey/miniforge3/envs/tongji/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
        ForkingPickler(file, protocol).dump(obj)
    AttributeError: Can't pickle local object 'create_collate_fn.<locals>.collate_fn'
    
    
  • Where are performance results of experiments stored

    Where are performance results of experiments stored

    Hi,

    thank you very much for sharing your code!

    I ran the example from the readme and the parts of the few-shot-pretrained-3b-100k.sh script. However, the dev_scores.json for the readme example only contains the line:

    {"accuracy": 0.6101083032490975, "score_gt": 0.3983679488032303, "score_cand": 0.6958685107394676}

    And for t03b_copa_seed42_ia3_pretrained100k (the first experiment of few-shot-pretrained-3b-100k.sh):

    {"accuracy": 0.85, "score_gt": 0.06061243396921782, "score_cand": 0.4640417302213609}

    Those are just the results of the "Validation sanity check" right at the beginning, so I wondered where the validation results after each epoch are stored or am I missing something here?

    Thanks!

  • Clarification about IA^3

    Clarification about IA^3

    Hi :)

    I was reading your interesting paper https://arxiv.org/pdf/2205.05638.pdf.

    In Section 3.3, you specify that IA^3 adds a total of d_k + d_v + d_ff parameters.

    However, if I look at this line, you seem to be allocating 2 * d vectors for each linear layer (multi_lora_a, multi_lora_b) and multiplying multi_lora_a with the input and multi_lora_b with the transformed input.

    https://github.com/r-three/t-few/blob/9dbc9cc429888a0c27fc22188b4e9549e0e83f40/src/models/lora.py#L43

    Am I missing something?

    Thank you for your clarification :-)

  • Does pl_train.py support TPU training?

    Does pl_train.py support TPU training?

    Hello, I am interested in using T-Few recipe for some experiments with Google Cloud TPUs. I am wondering whether the pl_train.py script supports TPU already? I read in the Acknowledgments section of the paper the authors cite that TPU cloud was utilised, however in this script I can see that gpu is directly supported. Any pointers will be appreciated, particularly I would like to use T0–11b

  • Accuracy could not match with the log when load_model

    Accuracy could not match with the log when load_model

    Hi, @muqeeth @dptam @craffel , when I set the eval_epoch_interval=1. I have some accuracy in my log, and I save my model and checkpoint. But when I tried to reload the model, its accuracy did not match the accuracy. image image

  • ImportError: cannot import name 'fast_walsh_hadamard_transform' from 'src.models.fwh_cuda' (unknown location)

    ImportError: cannot import name 'fast_walsh_hadamard_transform' from 'src.models.fwh_cuda' (unknown location)

    I tried running the example from the README and got this error. Can you help?

    $ CUDA_VISIBLE_DEVICES=3 python -m src.pl_train -c t0.json+rte.json -k save_model=False exp_name=first_exp
    Traceback (most recent call last):
      File "/home/james/.conda/envs/tfew/lib/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/home/james/.conda/envs/tfew/lib/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/home/james/github/t-few/src/pl_train.py", line 10, in <module>
        from src.models.EncoderDecoder import EncoderDecoder
      File "/home/james/github/t-few/src/models/EncoderDecoder.py", line 11, in <module>
        from .intrinsic import intrinsic_plugin_on_step
      File "/home/james/github/t-few/src/models/intrinsic.py", line 10, in <module>
        from .fwh_cuda import fast_walsh_hadamard_transform as fast_walsh_hadamard_transform_cuda
    ImportError: cannot import name 'fast_walsh_hadamard_transform' from 'src.models.fwh_cuda' (unknown location)
    
  • Releasing evaluation log probabilites

    Releasing evaluation log probabilites

    Hi, thanks for open-sourcing model code! Could you release the log probabilities for evaluation tasks (i.e., the model probabilities for valid answers for each prompt on each question for all evaluated datasets)? This data would allow for for fine-grained evaluation of models and comparing against other LLMs.

    cf. https://github.com/facebookresearch/metaseq/issues/25

  • Missing config.split_option_flag?

    Missing config.split_option_flag?

    Hi, thanks for the code!

    When I run:

    CUDA_VISIBLE_DEVICES=0 python -m src.pl_train -c t03b.json+rte.json -k save_model=False exp_name=first_exp3
    

    I get:

    Reusing dataset super_glue (/localdata/hjl/hf/super_glue/rte/1.0.2/d040c658e2ddef6934fdd97deb45c777b6ff50c524781ea434e7219b56a428a7)
    Train size 32
    Eval size 277
    LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
    Missing logger folder: /home/hjl/t-few/exp_out/first_exp3/log
    
      | Name  | Type                       | Params
    -----------------------------------------------------
    0 | model | T5ForConditionalGeneration | 2.8 B
    -----------------------------------------------------
    2.8 B     Trainable params
    0         Non-trainable params
    2.8 B     Total params
    11,399.029Total estimated model params size (MB)
    Validation sanity check:   0%|                                                                                                                                                                                                                             | 0/18 [00:00<?, ?it/s]Traceback (most recent call last):
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/home/hjl/t-few/src/pl_train.py", line 86, in <module>
        main(config)
      File "/home/hjl/t-few/src/pl_train.py", line 57, in main
        trainer.fit(model, datamodule)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 741, in fit
        self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
        return trainer_fn(*args, **kwargs)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 777, in _fit_impl
        self._run(model, ckpt_path=ckpt_path)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1199, in _run
        self._dispatch()
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1279, in _dispatch
        self.training_type_plugin.start_training(self)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 202, in start_training
        self._results = trainer.run_stage()
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1289, in run_stage
        return self._run_train()
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1311, in _run_train
        self._run_sanity_check(self.lightning_module)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1375, in _run_sanity_check
        self._evaluation_loop.run()
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 145, in run
        self.advance(*args, **kwargs)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 110, in advance
        dl_outputs = self.epoch_loop.run(dataloader, dataloader_idx, dl_max_batches, self.num_dataloaders)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 145, in run
        self.advance(*args, **kwargs)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 122, in advance
        output = self._evaluation_step(batch, batch_idx, dataloader_idx)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 217, in _evaluation_step
        output = self.trainer.accelerator.validation_step(step_kwargs)
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 236, in validation_step
        return self.training_type_plugin.validation_step(*step_kwargs.values())
      File "/opt/conda/hjl/envs/tfew/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 219, in validation_step
        return self.model.validation_step(*args, **kwargs)
      File "/home/hjl/t-few/src/models/EncoderDecoder.py", line 229, in validation_step
        batch_output = self.predict(batch)
      File "/home/hjl/t-few/src/models/EncoderDecoder.py", line 139, in predict
        if not self.config.split_option_flag:
    AttributeError: 'Config' object has no attribute 'split_option_flag'
    

    I can't find a reference to split_option_flag in any of the config files. Should I manually set it?

    Thanks!

Related tags
Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166
Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166

Region Proportion Regularized Inference (RePRI) for Few-Shot Segmentation In this repo, we provide the code for our paper : "Few-Shot Segmentation Wit

Jul 19, 2022
Ready-to-use code and tutorial notebooks to boost your way into few-shot image classification.

Easy Few-Shot Learning Ready-to-use code and tutorial notebooks to boost your way into few-shot image classification. This repository is made for you

Jul 28, 2022
All the essential resources and template code needed to understand and practice data structures and algorithms in python with few small projects to demonstrate their practical application.

Data Structures and Algorithms Python INDEX 1. Resources - Books Data Structures - Reema Thareja competitiveCoding Big-O Cheat Sheet DAA Syllabus Inte

Jul 31, 2022
SparseML is a libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models
SparseML is a libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

SparseML is a toolkit that includes APIs, CLIs, scripts and libraries that apply state-of-the-art sparsification algorithms such as pruning and quantization to any neural network. General, recipe-driven approaches built around these algorithms enable the simplification of creating faster and smaller models for the ML performance community at large.

Aug 3, 2022
Code and data of the ACL 2021 paper: Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision

MetaAdaptRank This repository provides the implementation of meta-learning to reweight synthetic weak supervision data described in the paper Few-Shot

Jun 16, 2022
Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)'

SCL Introduction Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)' We evaluated our approach using two baseline

Jul 15, 2022
This repository contains the code for using the H3DS dataset introduced in H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction

H3DS Dataset This repository contains the code for using the H3DS dataset introduced in H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction Access

Jul 13, 2022
Official code for "Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021".
Official code for

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021. Introduction We proposed a novel model training paradi

Jul 18, 2022
Official code release for "Learned Spatial Representations for Few-shot Talking-Head Synthesis" ICCV 2021
Official code release for

Official code release for "Learned Spatial Representations for Few-shot Talking-Head Synthesis" ICCV 2021

Jun 22, 2022
This repository is the code of the paper "Sparse Spatial Transformers for Few-Shot Learning".

?? Sparse Spatial Transformers for Few-Shot Learning This code implements the Sparse Spatial Transformers for Few-Shot Learning(SSFormers). Our code i

Aug 3, 2022
Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code.

Jul 27, 2022
The code is for the paper "A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation"
The code is for the paper

SD-AANet The code is for the paper "A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation" [arxiv] Overview confi

Apr 12, 2022
The Pytorch code of "Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification", CVPR 2022 (Oral).
The Pytorch code of

DeepBDC for few-shot learning        Introduction In this repo, we provide the implementation of the following paper: "Joint Distribution Matters: Dee

Aug 5, 2022
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Aug 3, 2022
CharacterGAN: Few-Shot Keypoint Character Animation and Reposing
CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

CharacterGAN Implementation of the paper "CharacterGAN: Few-Shot Keypoint Character Animation and Reposing" by Tobias Hinz, Matthew Fisher, Oliver Wan

Aug 5, 2022
Face2webtoon - Despite its importance, there are few previous works applying I2I translation to webtoon.
Face2webtoon - Despite its importance, there are few previous works applying I2I translation to webtoon.

Despite its importance, there are few previous works applying I2I translation to webtoon. I collected dataset from naver webtoon 연애혁명 and tried to transfer human faces to webtoon domain.

Jun 15, 2022
Few-shot Learning of GPT-3

Few-shot Learning With Language Models This is a codebase to perform few-shot "in-context" learning using language models similar to the GPT-3 paper.

Jul 21, 2022
git《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》(CVPR 2021) GitHub: [fig8]
git《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》(CVPR 2021) GitHub: [fig8]

FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (CVPR 2021) This repo contains the implementation of our state-of-the-art fewshot ob

Jul 15, 2022
Library of various Few-Shot Learning frameworks for text classification

FewShotText This repository contains code for the paper A Neural Few-Shot Text Classification Reality Check Environment setup # Create environment pyt

Jun 30, 2022