IEDB Analysis Resource

MHC I Automated Server Benchmarks


This is a live ranking of MHC I servers based on performance, which continues to be reevaluated over time. The weekly IEDB releases are automatically checked for datasets large enough to add to the benchmarks. The benchmark metrics in the table below will only be updated on releases where such new data is becoming available.

Accumulated overall ranking scores

Ranking scores based on data sets submitted to the IEDB for the last at least 5 references.

Server 2023-08-10 2023-08-03 2023-06-15 2023-05-11 2023-04-20 2022-11-24 2022-10-28 2022-10-06 2022-07-22 2022-05-20 2022-04-29 2021-12-24 2021-12-10 2021-11-05 2021-10-08 2021-10-01 2021-09-03 2021-08-20 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 4.1 72 73 64 66 66 65 71 76 76 87 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NetMHCcons 68 63 64 59 56 58 53 44 56 60 73 71 66 57 48 53 56 62 72 67 73 68 72 73 62 66 69 57 60 70 68 70 73 73 67 73 81 84 79 68 59 60 67 73 78 80 78 73 70 61 68 66 68 69 71 67 65 64 71 56 63 64 63
ANN 3.4 65 60 75 63 62 59 46 39 42 43 47 44 53 50 51 54 53 61 68 65 78 76 80 84 70 68 76 58 61 62 58 59 67 66 78 78 88 78 74 62 54 58 68 75 80 81 78 82 74 65 77 67 69 65 68 64 63 59 59 58 65 46 48
NetMHCpan 4.0 64 58 48 59 59 59 66 72 75 84 83 83 69 70 62 60 71 53 58 63 53 49 62 62 69 76 74 76 68 72 70 71 71 74 69 74 70 74 66 69 78 72 65 65 54 55 60 52 47 64 70 62 66 64 60 57 56 65 62 55 54 62 61
NetMHCpan 2.8 63 55 40 59 56 62 66 41 51 53 72 71 68 62 47 52 57 58 75 73 51 51 55 49 52 51 52 48 55 65 63 65 69 72 56 63 61 60 66 47 42 46 53 66 68 66 67 63 66 67 72 55 56 53 54 50 52 51 43 65 67 61 60
NetMHCpan 3.0 63 63 59 70 66 71 72 69 73 82 78 77 66 63 56 55 57 47 60 59 59 57 65 68 69 71 70 68 59 58 55 56 56 66 68 57 59 58 54 64 70 63 64 65 58 62 62 64 66 81 83 70 65 63 59 56 56 60 56 58 53 59 59
ANN 4.0 54 58 56 67 67 69 72 73 64 67 64 58 65 53 51 53 47 52 61 57 60 58 62 67 68 65 70 67 64 51 47 41 41 37 53 53 75 78 82 86 91 77 66 60 54 56 71 66 65 68 76 58 55 58 59 56 58 56 55 78 72 68 69
mhcflurry 1.2.0 54 47 56 58 61 56 58 49 35 48 55 58 60 55 56 64 90 85 84 84 53 59 52 35 79 29 58 58 39 15 0 0 0 0 - - - 71 71 85 80 80 70 63 61 61 70 69 80 88 88 90 92 92 92 93 93 83 55 90 88 86 85
SMMPMBEC 53 72 76 59 61 61 61 72 69 47 41 44 47 53 64 62 67 73 66 55 71 79 65 71 56 41 55 73 78 67 64 61 57 50 73 73 38 29 19 32 35 47 52 58 65 65 72 91 77 76 81 54 48 46 44 37 36 35 53 59 46 40 41
IEDB Consensus 52 58 66 71 72 69 70 62 55 53 49 52 59 58 63 59 53 52 40 49 66 74 66 68 54 43 57 62 68 59 53 43 44 36 45 45 65 48 49 48 51 44 45 51 51 59 62 56 36 24 55 46 48 48 47 37 39 37 48 52 48 59 58
SMM 48 52 55 55 56 54 60 57 59 51 37 40 44 50 64 60 60 61 48 46 54 63 56 63 51 35 47 63 74 66 63 52 47 41 43 43 42 32 26 38 39 37 39 40 45 51 49 49 36 26 51 54 58 51 50 44 43 42 53 61 50 40 40
PickPocket 36 42 54 57 57 67 68 51 43 32 31 34 40 54 54 52 53 36 36 25 26 20 28 36 26 43 53 51 53 55 51 66 67 60 59 53 20 25 28 21 23 35 35 47 50 44 57 52 47 47 53 33 36 32 33 27 24 29 25 64 62 46 42
ARB 31 46 53 48 48 51 37 28 24 15 16 13 22 28 34 34 29 23 24 42 73 74 60 62 36 38 67 51 38 51 46 53 53 56 - - - 0 0 38 26 26 19 9 10 10 30 25 25 33 55 31 34 29 30 28 29 30 18 52 52 24 21
DeepSeqPa - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Weekly ranking scores

Ranking scores based on data sets submitted to the IEDB within a single week.

Server 2023-08-10 2023-08-03 2023-06-15 2023-05-11 2023-04-20 2022-11-24 2022-10-28 2022-10-06 2022-07-22 2022-05-20 2022-04-29 2021-12-24 2021-12-10 2021-11-05 2021-10-08 2021-10-01 2021-09-03 2021-08-20 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 4.1 72 75 69 - 71 53 69 75 55 87 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NetMHCcons 72 59 73 - 54 59 60 8 82 28 59 72 73 79 18 49 51 48 91 73 92 33 69 83 54 73 68 50 71 88 32 45 64 80 - 60 70 75 100 86 78 79 49 57 72 89 74 65 88 - 95 60 41 50 89 69 70 63 71 18 82 82 63
ANN 3.4 67 51 73 - 60 88 55 50 18 16 68 44 42 46 36 60 47 52 73 73 100 25 64 98 83 55 78 55 75 96 64 0 73 66 - 70 - 85 - 89 67 63 43 61 75 92 59 85 100 - 95 46 55 - 95 64 66 62 59 32 36 36 48
NetMHCpan 4.0 68 65 45 - 56 45 65 50 55 87 78 86 86 82 64 36 82 46 41 100 92 0 62 52 67 100 51 75 100 92 46 68 36 82 - 60 70 85 100 47 67 96 75 80 50 56 70 70 16 - 95 49 55 88 89 60 52 58 62 64 9 100 61
NetMHCpan 2.8 64 59 60 - 29 21 79 8 82 28 51 62 75 89 27 49 55 35 91 100 58 50 78 31 54 82 56 10 50 88 59 32 46 73 - 70 70 0 100 64 72 63 22 45 75 74 70 60 56 - 95 60 77 50 89 46 54 52 43 64 86 82 60
NetMHCpan 3.0 62 68 61 - 42 68 77 17 82 73 85 84 88 63 73 55 66 30 55 73 96 17 54 65 79 91 54 65 92 88 36 59 18 65 - 53 80 75 0 58 67 88 60 77 42 73 65 65 36 - 95 72 77 100 89 54 58 56 56 91 9 82 59
ANN 4.0 50 59 61 - 50 52 74 75 55 73 80 61 81 52 9 75 47 40 46 73 83 42 42 71 63 64 66 80 50 88 73 45 64 38 - 30 - 75 - 75 100 67 94 93 42 35 89 40 55 - 95 100 9 - 89 52 59 57 55 100 73 36 69
mhcflurry 1.2.0 60 36 54 - 67 52 58 100 0 69 9 47 83 58 46 48 73 86 100 100 50 88 100 14 79 - - - 29 88 - - 0 0 - - - - - - - 71 90 68 - 50 89 10 59 - 95 87 86 - 89 93 92 95 55 91 95 91 85
SMMPMBEC 43 65 79 - 79 69 51 92 100 73 47 38 45 39 100 51 57 84 73 73 29 88 42 98 88 18 49 75 17 88 100 86 100 46 - 70 - 75 - 19 0 33 51 30 64 61 49 90 82 - 95 100 32 - 89 40 36 35 53 91 55 18 41
IEDB Consensus 49 56 55 - 71 78 69 88 27 100 46 47 59 46 91 73 53 64 27 - 33 58 54 83 79 46 33 60 33 100 82 36 91 44 - 0 - 90 - 53 28 17 63 61 14 36 61 85 73 - 9 27 36 - 100 36 44 33 48 36 36 0 58
SMM 45 50 58 - 46 57 56 75 9 96 32 43 45 26 82 64 55 78 23 73 29 67 32 73 92 27 52 30 13 88 100 86 82 47 - 10 - 75 - 25 11 25 61 25 19 39 35 85 68 - 0 9 68 - 89 49 41 42 53 64 55 18 40
PickPocket 28 35 74 - 4 61 68 50 100 62 15 28 29 41 55 55 62 35 23 73 42 8 2 42 0 36 44 5 88 88 18 55 18 64 - 100 10 75 25 0 44 8 32 7 81 27 47 85 24 - 95 34 82 0 70 27 31 24 25 0 77 82 42
ARB 19 56 43 - 25 85 40 50 - 14 32 11 26 10 0 36 45 21 0 0 0 96 57 100 8 0 - 100 13 88 0 55 36 56 - - - - - - - 0 50 2 - 0 26 0 0 - 100 0 0 - 89 25 20 37 18 9 36 82 21
DeepSeqPa - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Benchmark results get recalculated as new servers are added to the benchmarks. Numbers in light grey represent scores on datasets before the server was incorporated into the benchmarks. Previous versions of the benchmark results can be downloaded below