IEDB Analysis Resource

MHC I Automated Server Benchmarks


This is a live ranking of MHC I servers based on performance, which continues to be reevaluated over time. The weekly IEDB releases are automatically checked for datasets large enough to add to the benchmarks. The benchmark metrics in the table below will only be updated on releases where such new data is becoming available.

Accumulated overall ranking scores

Ranking scores based on data sets submitted to the IEDB for the last at least 5 references.

Server 2022-05-20 2022-04-29 2021-12-24 2021-12-10 2021-11-05 2021-10-08 2021-10-01 2021-09-03 2021-08-20 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 4.1 87 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NetMHCpan 4.0 84 83 83 69 70 62 60 71 53 58 63 53 49 62 62 69 76 74 76 68 72 70 71 71 74 69 74 70 74 66 69 78 72 65 65 54 55 60 52 47 64 70 62 66 64 60 57 56 65 62 55 54 62 61
NetMHCpan 3.0 82 78 77 66 63 56 55 57 47 60 59 59 57 65 68 69 71 70 68 59 58 55 56 56 66 68 57 59 58 54 64 70 63 64 65 58 62 62 64 66 81 83 70 65 63 59 56 56 60 56 58 53 59 59
ANN 4.0 67 64 58 65 53 51 53 47 52 61 57 60 58 62 67 68 65 70 67 64 51 47 41 41 37 53 53 75 78 82 86 91 77 66 60 54 56 71 66 65 68 76 58 55 58 59 56 58 56 55 78 72 68 69
NetMHCcons 60 73 71 66 57 48 53 56 62 72 67 73 68 72 73 62 66 69 57 60 70 68 70 73 73 67 73 81 84 79 68 59 60 67 73 78 80 78 73 70 61 68 66 68 69 71 67 65 64 71 56 63 64 63
IEDB Consensus 53 49 52 59 58 63 59 53 52 40 49 66 74 66 68 54 43 57 62 68 59 53 43 44 36 45 45 65 48 49 48 51 44 45 51 51 59 62 56 36 24 55 46 48 48 47 37 39 37 48 52 48 59 58
NetMHCpan 2.8 53 72 71 68 62 47 52 57 58 75 73 51 51 55 49 52 51 52 48 55 65 63 65 69 72 56 63 61 60 66 47 42 46 53 66 68 66 67 63 66 67 72 55 56 53 54 50 52 51 43 65 67 61 60
SMM 51 37 40 44 50 64 60 60 61 48 46 54 63 56 63 51 35 47 63 74 66 63 52 47 41 43 43 42 32 26 38 39 37 39 40 45 51 49 49 36 26 51 54 58 51 50 44 43 42 53 61 50 40 40
mhcflurry 1.2.0 48 55 58 60 55 56 64 90 85 84 84 53 59 52 35 79 29 58 58 39 15 0 0 0 0 - - - 71 71 85 80 80 70 63 61 61 70 69 80 88 88 90 92 92 92 93 93 83 55 90 88 86 85
SMMPMBEC 47 41 44 47 53 64 62 67 73 66 55 71 79 65 71 56 41 55 73 78 67 64 61 57 50 73 73 38 29 19 32 35 47 52 58 65 65 72 91 77 76 81 54 48 46 44 37 36 35 53 59 46 40 41
ANN 3.4 43 47 44 53 50 51 54 53 61 68 65 78 76 80 84 70 68 76 58 61 62 58 59 67 66 78 78 88 78 74 62 54 58 68 75 80 81 78 82 74 65 77 67 69 65 68 64 63 59 59 58 65 46 48
PickPocket 32 31 34 40 54 54 52 53 36 36 25 26 20 28 36 26 43 53 51 53 55 51 66 67 60 59 53 20 25 28 21 23 35 35 47 50 44 57 52 47 47 53 33 36 32 33 27 24 29 25 64 62 46 42
ARB 15 16 13 22 28 34 34 29 23 24 42 73 74 60 62 36 38 67 51 38 51 46 53 53 56 - - - 0 0 38 26 26 19 9 10 10 30 25 25 33 55 31 34 29 30 28 29 30 18 52 52 24 21
DeepSeqPa - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Weekly ranking scores

Ranking scores based on data sets submitted to the IEDB within a single week.

Server 2022-05-20 2022-04-29 2021-12-24 2021-12-10 2021-11-05 2021-10-08 2021-10-01 2021-09-03 2021-08-20 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 4.1 87 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NetMHCpan 4.0 87 78 86 86 82 64 36 82 46 41 100 92 0 62 52 67 100 51 75 100 92 46 68 36 82 - 60 70 85 100 47 67 96 75 80 50 56 70 70 16 - 95 49 55 88 89 60 52 58 62 64 9 100 61
NetMHCpan 3.0 73 85 84 88 63 73 55 66 30 55 73 96 17 54 65 79 91 54 65 92 88 36 59 18 65 - 53 80 75 0 58 67 88 60 77 42 73 65 65 36 - 95 72 77 100 89 54 58 56 56 91 9 82 59
ANN 4.0 73 80 61 81 52 9 75 47 40 46 73 83 42 42 71 63 64 66 80 50 88 73 45 64 38 - 30 - 75 - 75 100 67 94 93 42 35 89 40 55 - 95 100 9 - 89 52 59 57 55 100 73 36 69
NetMHCcons 28 59 72 73 79 18 49 51 48 91 73 92 33 69 83 54 73 68 50 71 88 32 45 64 80 - 60 70 75 100 86 78 79 49 57 72 89 74 65 88 - 95 60 41 50 89 69 70 63 71 18 82 82 63
IEDB Consensus 100 46 47 59 46 91 73 53 64 27 - 33 58 54 83 79 46 33 60 33 100 82 36 91 44 - 0 - 90 - 53 28 17 63 61 14 36 61 85 73 - 9 27 36 - 100 36 44 33 48 36 36 0 58
NetMHCpan 2.8 28 51 62 75 89 27 49 55 35 91 100 58 50 78 31 54 82 56 10 50 88 59 32 46 73 - 70 70 0 100 64 72 63 22 45 75 74 70 60 56 - 95 60 77 50 89 46 54 52 43 64 86 82 60
SMM 96 32 43 45 26 82 64 55 78 23 73 29 67 32 73 92 27 52 30 13 88 100 86 82 47 - 10 - 75 - 25 11 25 61 25 19 39 35 85 68 - 0 9 68 - 89 49 41 42 53 64 55 18 40
mhcflurry 1.2.0 69 9 47 83 58 46 48 73 86 100 100 50 88 100 14 79 - - - 29 88 - - 0 0 - - - - - - - 71 90 68 - 50 89 10 59 - 95 87 86 - 89 93 92 95 55 91 95 91 85
SMMPMBEC 73 47 38 45 39 100 51 57 84 73 73 29 88 42 98 88 18 49 75 17 88 100 86 100 46 - 70 - 75 - 19 0 33 51 30 64 61 49 90 82 - 95 100 32 - 89 40 36 35 53 91 55 18 41
ANN 3.4 16 68 44 42 46 36 60 47 52 73 73 100 25 64 98 83 55 78 55 75 96 64 0 73 66 - 70 - 85 - 89 67 63 43 61 75 92 59 85 100 - 95 46 55 - 95 64 66 62 59 32 36 36 48
PickPocket 62 15 28 29 41 55 55 62 35 23 73 42 8 2 42 0 36 44 5 88 88 18 55 18 64 - 100 10 75 25 0 44 8 32 7 81 27 47 85 24 - 95 34 82 0 70 27 31 24 25 0 77 82 42
ARB 14 32 11 26 10 0 36 45 21 0 0 0 96 57 100 8 0 - 100 13 88 0 55 36 56 - - - - - - - 0 50 2 - 0 26 0 0 - 100 0 0 - 89 25 20 37 18 9 36 82 21
DeepSeqPa - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Benchmark results get recalculated as new servers are added to the benchmarks. Numbers in light grey represent scores on datasets before the server was incorporated into the benchmarks. Previous versions of the benchmark results can be downloaded below