IEDB Analysis Resource

MHC I Automated Server Benchmarks


This is a live ranking of MHC I servers based on performance, which continues to be reevaluated over time. The weekly IEDB releases are automatically checked for datasets large enough to add to the benchmarks. The benchmark metrics in the table below will only be updated on releases where such new data is becoming available.

Accumulated overall ranking scores

Ranking scores based on data sets submitted to the IEDB for the last at least 5 references.

Server 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 4.0 74 78 77 80 66 71 68 68 70 74 69 74 70 74 77 69 78 72 65 65 54 55 60 52 47 64 70 62 66 64 60 57 56 65 - 55 54 62 -
ANN 4.0 72 72 76 73 67 51 46 40 41 37 53 53 75 78 85 86 91 77 66 60 54 56 71 66 65 68 76 58 55 58 59 56 58 56 - 78 72 68 -
mhcflurry 1.2.0 70 54 60 60 63 18 0 0 0 0 - - - 71 90 85 80 80 70 63 61 61 70 69 80 88 88 90 92 92 92 93 93 83 - 90 88 86 -
NetMHCcons 67 69 70 58 62 70 67 69 72 73 67 73 81 84 76 68 59 60 67 73 78 80 78 73 70 61 68 66 68 69 71 67 65 64 - 56 63 64 -
NetMHCpan 2.8 61 62 62 52 56 64 61 63 68 72 56 63 61 60 57 47 42 46 53 66 68 66 67 63 66 67 72 55 56 53 54 50 52 51 - 65 67 61 -
IEDB Consensus 61 54 61 59 64 54 48 42 44 36 45 45 65 48 40 48 51 44 45 51 51 59 62 56 36 24 55 46 48 48 47 37 39 37 - 52 48 59 -
SMM 53 42 50 66 74 65 62 51 46 41 43 43 42 32 38 38 39 37 39 40 45 51 49 49 36 26 51 54 58 51 50 44 43 42 - 61 50 40 -
SMMPMBEC 41 31 41 63 80 68 65 61 57 50 73 73 38 29 34 32 35 47 52 58 65 65 72 91 77 76 81 54 48 46 44 37 36 35 - 59 46 40 -
ARB 37 38 68 51 36 50 45 51 51 56 - - - 0 50 38 26 26 19 9 10 10 30 25 25 33 55 31 34 29 30 28 29 30 18 52 52 24 -
PickPocket 23 36 43 45 51 54 49 63 54 60 59 53 20 25 12 21 23 35 35 47 50 44 57 52 47 47 53 33 36 32 33 27 24 29 - 64 62 46 -
DeepSeqPa 15 40 80 63 63 46 38 51 40 40 60 60 53 73 27 21 21 21 8 - - - - - - - - - - - - - - - - - - - -
NetMHCpan 3.0 - - - - - 65 65 61 66 66 68 57 59 58 57 64 70 63 64 65 58 62 62 64 66 81 83 70 65 63 59 56 56 60 - 58 53 59 -
ANN 3.4 - - - - - 66 66 66 66 66 78 78 88 78 62 62 54 58 68 75 80 81 78 82 74 65 77 67 69 65 68 64 63 59 - 58 65 46 -

Weekly ranking scores

Ranking scores based on data sets submitted to the IEDB within a single week.

Server 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 4.0 75 100 57 86 100 95 50 61 25 82 - 60 70 85 100 47 67 96 75 80 50 56 70 70 16 - 95 49 55 88 89 60 52 58 62 64 9 100 61
ANN 4.0 75 60 69 82 70 90 75 39 63 38 - 30 - 75 - 75 100 67 94 93 42 35 89 40 55 - 95 100 9 - 89 52 59 57 55 100 73 36 69
mhcflurry 1.2.0 85 70 - 56 35 90 - - - 0 - - - - - - - 71 90 68 - 50 89 10 59 - 95 87 86 - 89 93 92 95 55 91 95 91 85
NetMHCcons 65 80 74 50 80 90 38 39 63 80 - 60 70 75 100 86 78 79 49 57 72 89 74 65 88 - 95 60 41 50 89 69 70 63 71 18 82 82 63
NetMHCpan 2.8 65 90 69 34 70 90 63 22 38 73 - 70 70 0 100 64 72 63 22 45 75 74 70 60 56 - 95 60 77 50 89 46 54 52 43 64 86 82 60
IEDB Consensus 85 50 55 63 40 100 - 28 88 44 - 0 - 90 - 53 28 17 63 61 14 36 61 85 73 - 9 27 36 - 100 36 44 33 48 36 36 0 58
SMM 95 30 48 52 15 90 100 89 75 47 - 10 - 75 - 25 11 25 61 25 19 39 35 85 68 - 0 9 68 - 89 49 41 42 53 64 55 18 40
SMMPMBEC 90 20 33 40 20 90 100 89 100 46 - 70 - 75 - 19 0 33 51 30 64 61 49 90 82 - 95 100 32 - 89 40 36 35 53 91 55 18 41
ARB 10 0 - 100 15 90 0 50 25 56 - - - - - - - 0 50 2 - 0 26 0 0 - 100 0 0 - 89 25 20 37 18 9 36 82 21
PickPocket 0 40 36 6 90 90 25 50 0 64 - 100 10 75 25 0 44 8 32 7 81 27 47 85 24 - 95 34 82 0 70 27 31 24 25 0 77 82 42
DeepSeqPa 20 10 - - 70 90 13 78 - 33 - 80 30 100 - - - 46 8 - - - - - - - - - - - - - - - - - - - -
NetMHCpan 3.0 - - - - - - - - - 65 - 53 80 75 0 58 67 88 60 77 42 73 65 65 36 - 95 72 77 100 89 54 58 56 56 91 9 82 59
ANN 3.4 - - - - - - - - - 66 - 70 - 85 - 89 67 63 43 61 75 92 59 85 100 - 95 46 55 - 95 64 66 62 59 32 36 36 48

Benchmark results get recalculated as new servers are added to the benchmarks. Numbers in light grey represent scores on datasets before the server was incorporated into the benchmarks. Previous versions of the benchmark results can be downloaded below