Home Explore Help
Sign In
hugh
/
waifu2x
1
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Tree: 1b946a5484
Branches Tags
waifu2x
/
appendix
/
benchmark.md

benchmark.md 4.0 KB
History Raw

Benchmarks

Photo

Note: waifu2x's photo models was trained on the blending dataset of kou's photo collection and ukbench.

Note: PSNR in this benchmark uses a MATLAB's rgb2ycbcr compatible function (dynamic range [16 235], not [0 255]) for converting grayscale image. I think it's not correct PSNR. But many paper used this metric.

command: th tools/benchmark.lua -dir <dataset_dir> -model1_dir <model_dir> -method scale -filter Catrom -color y -range_bug 1 -tta <0|1> -force_cudnn 1

Datasets

BSD100: https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/ (100 test images in BSDS300) Urban100: https://github.com/jbhuang0604/SelfExSR

2x - PSNR

| Dataset/Model | Bicubic | vgg_7/photo | upconv_7/photo | upconv_7l/photo | resnet_14l/photo | |---------------|---------------|---------------|------------------|------------------|--------------------| | BSD100 | 29.558 | 31.427 | 31.640 | 31.749 | 31.847 | | Urban100 | 26.852 | 30.057 | 30.477 | 30.759 | 31.016 |

2x with TTA - PSNR

Note: TTA is an ensemble technique that is supported by waifu2x. TTA method is 8x slower than non TTA method but it improves PSNR (~+0.1 on photo, ~+0.4 on art).

| Dataset/Model | Bicubic | vgg_7/photo | upconv_7/photo | upconv_7l/photo | resnet_14l/photo | |---------------|---------------|---------------|------------------|------------------|--------------------| | BSD100 | 29.558 | 31.474 | 31.705 | 31.812 | 31.915 | | Urban100 | 26.852 | 30.140 | 30.599 | 30.868 | 31.162 |

2x - benchmark elapsed time (sec)

Dataset/Model vgg_7/photo upconv_7/photo upconv_7l/photo resnet_14l/photo
BSD100 4.057 2.509 4.947 6.86
Urban100 16.349 7.083 14.178 27.87

2x with TTA - benchmark elapsed time (sec)

Dataset/Model vgg_7/photo upconv_7/photo upconv_7l/photo resnet_14l/photo
BSD100 36.611 20.219 42.486 60.38
Urban100 132.416 65.125 129.916 255.20

Art

command: th tools/benchmark.lua -dir <dataset_dir> -model1_dir <model_dir> -method scale -filter Lanczos -color y -range_bug 1 -tta <0|1> -force_cudnn 1

Dataset

art_test: This dataset contains 85 various fan-arts. Sorry, This dataset is private.

2x - PSNR

| Dataset/Model | Bicubic | vgg_7/art | upconv_7/art | upconv_7l/art | |---------------|---------------|-------------|----------------|----------------| | art_test | 31.022 | 37.495 | 38.330 | 39.140 |

2x with TTA - PSNR

| Dataset/Model | Bicubic | vgg_7/art | upconv_7/art | upconv_7l/art | |---------------|---------------|-------------|----------------|----------------| | art_test | 31.022 | 37.777 | 38.677 | 39.510 |

2x - benchmark elapsed time (sec)

| Dataset/Model | vgg_7/art | upconv_7/art | upconv_7l/art | |---------------|-------------|----------------|----------------| | art_test | 20.681 | 7.683 | 17.667 |

2x with TTA - benchmark elapsed time (sec)

| Dataset/Model | vgg_7/art | upconv_7/art | upconv_7l/art | |---------------|-------------|----------------|----------------| | art_test | 174.674 | 77.716 | 163.932 |