MHC I Automated Server Benchmarks
This is a live ranking of MHC I servers based on performance, which continues to be reevaluated over time. The weekly IEDB releases are automatically checked for datasets large enough to add to the benchmarks. The benchmark metrics in the table below will only be updated on releases where such new data is becoming available.
Accumulated overall ranking scores
Ranking scores based on data sets submitted to the IEDB for the last at least 5 references.
Server | 2023-08-10 | 2023-08-03 | 2023-06-15 | 2023-05-11 | 2023-04-20 | 2022-11-24 | 2022-10-28 | 2022-10-06 | 2022-07-22 | 2022-05-20 | 2022-04-29 | 2021-12-24 | 2021-12-10 | 2021-11-05 | 2021-10-08 | 2021-10-01 | 2021-09-03 | 2021-08-20 | 2021-06-11 | 2021-05-21 | 2021-04-16 | 2021-03-05 | 2021-02-19 | 2021-01-29 | 2021-01-15 | 2021-01-01 | 2020-12-25 | 2020-12-18 | 2020-12-11 | 2020-11-13 | 2020-10-02 | 2020-09-11 | 2020-07-10 | 2020-04-24 | 2020-03-13 | 2019-11-01 | 2019-10-04 | 2019-08-30 | 2019-06-21 | 2019-06-07 | 2019-05-24 | 2019-03-15 | 2019-02-01 | 2018-05-11 | 2018-01-26 | 2017-09-01 | 2017-03-23 | 2016-12-09 | 2016-05-03 | 2016-02-19 | 2015-08-07 | 2015-07-31 | 2015-07-17 | 2015-06-26 | 2015-06-19 | 2015-05-15 | 2015-02-06 | 2015-01-16 | 2014-10-31 | 2014-06-20 | 2014-05-23 | 2014-03-28 | 2014-03-21 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
NetMHCpan 4.1 | 72 | 73 | 64 | 66 | 66 | 65 | 71 | 76 | 76 | 87 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
NetMHCcons | 68 | 63 | 64 | 59 | 56 | 58 | 53 | 44 | 56 | 60 | 73 | 71 | 66 | 57 | 48 | 53 | 56 | 62 | 72 | 67 | 73 | 68 | 72 | 73 | 62 | 66 | 69 | 57 | 60 | 70 | 68 | 70 | 73 | 73 | 67 | 73 | 81 | 84 | 79 | 68 | 59 | 60 | 67 | 73 | 78 | 80 | 78 | 73 | 70 | 61 | 68 | 66 | 68 | 69 | 71 | 67 | 65 | 64 | 71 | 56 | 63 | 64 | 63 |
ANN 3.4 | 65 | 60 | 75 | 63 | 62 | 59 | 46 | 39 | 42 | 43 | 47 | 44 | 53 | 50 | 51 | 54 | 53 | 61 | 68 | 65 | 78 | 76 | 80 | 84 | 70 | 68 | 76 | 58 | 61 | 62 | 58 | 59 | 67 | 66 | 78 | 78 | 88 | 78 | 74 | 62 | 54 | 58 | 68 | 75 | 80 | 81 | 78 | 82 | 74 | 65 | 77 | 67 | 69 | 65 | 68 | 64 | 63 | 59 | 59 | 58 | 65 | 46 | 48 |
NetMHCpan 4.0 | 64 | 58 | 48 | 59 | 59 | 59 | 66 | 72 | 75 | 84 | 83 | 83 | 69 | 70 | 62 | 60 | 71 | 53 | 58 | 63 | 53 | 49 | 62 | 62 | 69 | 76 | 74 | 76 | 68 | 72 | 70 | 71 | 71 | 74 | 69 | 74 | 70 | 74 | 66 | 69 | 78 | 72 | 65 | 65 | 54 | 55 | 60 | 52 | 47 | 64 | 70 | 62 | 66 | 64 | 60 | 57 | 56 | 65 | 62 | 55 | 54 | 62 | 61 |
NetMHCpan 2.8 | 63 | 55 | 40 | 59 | 56 | 62 | 66 | 41 | 51 | 53 | 72 | 71 | 68 | 62 | 47 | 52 | 57 | 58 | 75 | 73 | 51 | 51 | 55 | 49 | 52 | 51 | 52 | 48 | 55 | 65 | 63 | 65 | 69 | 72 | 56 | 63 | 61 | 60 | 66 | 47 | 42 | 46 | 53 | 66 | 68 | 66 | 67 | 63 | 66 | 67 | 72 | 55 | 56 | 53 | 54 | 50 | 52 | 51 | 43 | 65 | 67 | 61 | 60 |
NetMHCpan 3.0 | 63 | 63 | 59 | 70 | 66 | 71 | 72 | 69 | 73 | 82 | 78 | 77 | 66 | 63 | 56 | 55 | 57 | 47 | 60 | 59 | 59 | 57 | 65 | 68 | 69 | 71 | 70 | 68 | 59 | 58 | 55 | 56 | 56 | 66 | 68 | 57 | 59 | 58 | 54 | 64 | 70 | 63 | 64 | 65 | 58 | 62 | 62 | 64 | 66 | 81 | 83 | 70 | 65 | 63 | 59 | 56 | 56 | 60 | 56 | 58 | 53 | 59 | 59 |
ANN 4.0 | 54 | 58 | 56 | 67 | 67 | 69 | 72 | 73 | 64 | 67 | 64 | 58 | 65 | 53 | 51 | 53 | 47 | 52 | 61 | 57 | 60 | 58 | 62 | 67 | 68 | 65 | 70 | 67 | 64 | 51 | 47 | 41 | 41 | 37 | 53 | 53 | 75 | 78 | 82 | 86 | 91 | 77 | 66 | 60 | 54 | 56 | 71 | 66 | 65 | 68 | 76 | 58 | 55 | 58 | 59 | 56 | 58 | 56 | 55 | 78 | 72 | 68 | 69 |
mhcflurry 1.2.0 | 54 | 47 | 56 | 58 | 61 | 56 | 58 | 49 | 35 | 48 | 55 | 58 | 60 | 55 | 56 | 64 | 90 | 85 | 84 | 84 | 53 | 59 | 52 | 35 | 79 | 29 | 58 | 58 | 39 | 15 | 0 | 0 | 0 | 0 | - | - | - | 71 | 71 | 85 | 80 | 80 | 70 | 63 | 61 | 61 | 70 | 69 | 80 | 88 | 88 | 90 | 92 | 92 | 92 | 93 | 93 | 83 | 55 | 90 | 88 | 86 | 85 |
SMMPMBEC | 53 | 72 | 76 | 59 | 61 | 61 | 61 | 72 | 69 | 47 | 41 | 44 | 47 | 53 | 64 | 62 | 67 | 73 | 66 | 55 | 71 | 79 | 65 | 71 | 56 | 41 | 55 | 73 | 78 | 67 | 64 | 61 | 57 | 50 | 73 | 73 | 38 | 29 | 19 | 32 | 35 | 47 | 52 | 58 | 65 | 65 | 72 | 91 | 77 | 76 | 81 | 54 | 48 | 46 | 44 | 37 | 36 | 35 | 53 | 59 | 46 | 40 | 41 |
IEDB Consensus | 52 | 58 | 66 | 71 | 72 | 69 | 70 | 62 | 55 | 53 | 49 | 52 | 59 | 58 | 63 | 59 | 53 | 52 | 40 | 49 | 66 | 74 | 66 | 68 | 54 | 43 | 57 | 62 | 68 | 59 | 53 | 43 | 44 | 36 | 45 | 45 | 65 | 48 | 49 | 48 | 51 | 44 | 45 | 51 | 51 | 59 | 62 | 56 | 36 | 24 | 55 | 46 | 48 | 48 | 47 | 37 | 39 | 37 | 48 | 52 | 48 | 59 | 58 |
SMM | 48 | 52 | 55 | 55 | 56 | 54 | 60 | 57 | 59 | 51 | 37 | 40 | 44 | 50 | 64 | 60 | 60 | 61 | 48 | 46 | 54 | 63 | 56 | 63 | 51 | 35 | 47 | 63 | 74 | 66 | 63 | 52 | 47 | 41 | 43 | 43 | 42 | 32 | 26 | 38 | 39 | 37 | 39 | 40 | 45 | 51 | 49 | 49 | 36 | 26 | 51 | 54 | 58 | 51 | 50 | 44 | 43 | 42 | 53 | 61 | 50 | 40 | 40 |
PickPocket | 36 | 42 | 54 | 57 | 57 | 67 | 68 | 51 | 43 | 32 | 31 | 34 | 40 | 54 | 54 | 52 | 53 | 36 | 36 | 25 | 26 | 20 | 28 | 36 | 26 | 43 | 53 | 51 | 53 | 55 | 51 | 66 | 67 | 60 | 59 | 53 | 20 | 25 | 28 | 21 | 23 | 35 | 35 | 47 | 50 | 44 | 57 | 52 | 47 | 47 | 53 | 33 | 36 | 32 | 33 | 27 | 24 | 29 | 25 | 64 | 62 | 46 | 42 |
ARB | 31 | 46 | 53 | 48 | 48 | 51 | 37 | 28 | 24 | 15 | 16 | 13 | 22 | 28 | 34 | 34 | 29 | 23 | 24 | 42 | 73 | 74 | 60 | 62 | 36 | 38 | 67 | 51 | 38 | 51 | 46 | 53 | 53 | 56 | - | - | - | 0 | 0 | 38 | 26 | 26 | 19 | 9 | 10 | 10 | 30 | 25 | 25 | 33 | 55 | 31 | 34 | 29 | 30 | 28 | 29 | 30 | 18 | 52 | 52 | 24 | 21 |
DeepSeqPa | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
Weekly ranking scores
Ranking scores based on data sets submitted to the IEDB within a single week.
Server | 2023-08-10 | 2023-08-03 | 2023-06-15 | 2023-05-11 | 2023-04-20 | 2022-11-24 | 2022-10-28 | 2022-10-06 | 2022-07-22 | 2022-05-20 | 2022-04-29 | 2021-12-24 | 2021-12-10 | 2021-11-05 | 2021-10-08 | 2021-10-01 | 2021-09-03 | 2021-08-20 | 2021-06-11 | 2021-05-21 | 2021-04-16 | 2021-03-05 | 2021-02-19 | 2021-01-29 | 2021-01-15 | 2021-01-01 | 2020-12-25 | 2020-12-18 | 2020-12-11 | 2020-11-13 | 2020-10-02 | 2020-09-11 | 2020-07-10 | 2020-04-24 | 2020-03-13 | 2019-11-01 | 2019-10-04 | 2019-08-30 | 2019-06-21 | 2019-06-07 | 2019-05-24 | 2019-03-15 | 2019-02-01 | 2018-05-11 | 2018-01-26 | 2017-09-01 | 2017-03-23 | 2016-12-09 | 2016-05-03 | 2016-02-19 | 2015-08-07 | 2015-07-31 | 2015-07-17 | 2015-06-26 | 2015-06-19 | 2015-05-15 | 2015-02-06 | 2015-01-16 | 2014-10-31 | 2014-06-20 | 2014-05-23 | 2014-03-28 | 2014-03-21 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
NetMHCpan 4.1 | 72 | 75 | 69 | - | 71 | 53 | 69 | 75 | 55 | 87 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
NetMHCcons | 72 | 59 | 73 | - | 54 | 59 | 60 | 8 | 82 | 28 | 59 | 72 | 73 | 79 | 18 | 49 | 51 | 48 | 91 | 73 | 92 | 33 | 69 | 83 | 54 | 73 | 68 | 50 | 71 | 88 | 32 | 45 | 64 | 80 | - | 60 | 70 | 75 | 100 | 86 | 78 | 79 | 49 | 57 | 72 | 89 | 74 | 65 | 88 | - | 95 | 60 | 41 | 50 | 89 | 69 | 70 | 63 | 71 | 18 | 82 | 82 | 63 |
ANN 3.4 | 67 | 51 | 73 | - | 60 | 88 | 55 | 50 | 18 | 16 | 68 | 44 | 42 | 46 | 36 | 60 | 47 | 52 | 73 | 73 | 100 | 25 | 64 | 98 | 83 | 55 | 78 | 55 | 75 | 96 | 64 | 0 | 73 | 66 | - | 70 | - | 85 | - | 89 | 67 | 63 | 43 | 61 | 75 | 92 | 59 | 85 | 100 | - | 95 | 46 | 55 | - | 95 | 64 | 66 | 62 | 59 | 32 | 36 | 36 | 48 |
NetMHCpan 4.0 | 68 | 65 | 45 | - | 56 | 45 | 65 | 50 | 55 | 87 | 78 | 86 | 86 | 82 | 64 | 36 | 82 | 46 | 41 | 100 | 92 | 0 | 62 | 52 | 67 | 100 | 51 | 75 | 100 | 92 | 46 | 68 | 36 | 82 | - | 60 | 70 | 85 | 100 | 47 | 67 | 96 | 75 | 80 | 50 | 56 | 70 | 70 | 16 | - | 95 | 49 | 55 | 88 | 89 | 60 | 52 | 58 | 62 | 64 | 9 | 100 | 61 |
NetMHCpan 2.8 | 64 | 59 | 60 | - | 29 | 21 | 79 | 8 | 82 | 28 | 51 | 62 | 75 | 89 | 27 | 49 | 55 | 35 | 91 | 100 | 58 | 50 | 78 | 31 | 54 | 82 | 56 | 10 | 50 | 88 | 59 | 32 | 46 | 73 | - | 70 | 70 | 0 | 100 | 64 | 72 | 63 | 22 | 45 | 75 | 74 | 70 | 60 | 56 | - | 95 | 60 | 77 | 50 | 89 | 46 | 54 | 52 | 43 | 64 | 86 | 82 | 60 |
NetMHCpan 3.0 | 62 | 68 | 61 | - | 42 | 68 | 77 | 17 | 82 | 73 | 85 | 84 | 88 | 63 | 73 | 55 | 66 | 30 | 55 | 73 | 96 | 17 | 54 | 65 | 79 | 91 | 54 | 65 | 92 | 88 | 36 | 59 | 18 | 65 | - | 53 | 80 | 75 | 0 | 58 | 67 | 88 | 60 | 77 | 42 | 73 | 65 | 65 | 36 | - | 95 | 72 | 77 | 100 | 89 | 54 | 58 | 56 | 56 | 91 | 9 | 82 | 59 |
ANN 4.0 | 50 | 59 | 61 | - | 50 | 52 | 74 | 75 | 55 | 73 | 80 | 61 | 81 | 52 | 9 | 75 | 47 | 40 | 46 | 73 | 83 | 42 | 42 | 71 | 63 | 64 | 66 | 80 | 50 | 88 | 73 | 45 | 64 | 38 | - | 30 | - | 75 | - | 75 | 100 | 67 | 94 | 93 | 42 | 35 | 89 | 40 | 55 | - | 95 | 100 | 9 | - | 89 | 52 | 59 | 57 | 55 | 100 | 73 | 36 | 69 |
mhcflurry 1.2.0 | 60 | 36 | 54 | - | 67 | 52 | 58 | 100 | 0 | 69 | 9 | 47 | 83 | 58 | 46 | 48 | 73 | 86 | 100 | 100 | 50 | 88 | 100 | 14 | 79 | - | - | - | 29 | 88 | - | - | 0 | 0 | - | - | - | - | - | - | - | 71 | 90 | 68 | - | 50 | 89 | 10 | 59 | - | 95 | 87 | 86 | - | 89 | 93 | 92 | 95 | 55 | 91 | 95 | 91 | 85 |
SMMPMBEC | 43 | 65 | 79 | - | 79 | 69 | 51 | 92 | 100 | 73 | 47 | 38 | 45 | 39 | 100 | 51 | 57 | 84 | 73 | 73 | 29 | 88 | 42 | 98 | 88 | 18 | 49 | 75 | 17 | 88 | 100 | 86 | 100 | 46 | - | 70 | - | 75 | - | 19 | 0 | 33 | 51 | 30 | 64 | 61 | 49 | 90 | 82 | - | 95 | 100 | 32 | - | 89 | 40 | 36 | 35 | 53 | 91 | 55 | 18 | 41 |
IEDB Consensus | 49 | 56 | 55 | - | 71 | 78 | 69 | 88 | 27 | 100 | 46 | 47 | 59 | 46 | 91 | 73 | 53 | 64 | 27 | - | 33 | 58 | 54 | 83 | 79 | 46 | 33 | 60 | 33 | 100 | 82 | 36 | 91 | 44 | - | 0 | - | 90 | - | 53 | 28 | 17 | 63 | 61 | 14 | 36 | 61 | 85 | 73 | - | 9 | 27 | 36 | - | 100 | 36 | 44 | 33 | 48 | 36 | 36 | 0 | 58 |
SMM | 45 | 50 | 58 | - | 46 | 57 | 56 | 75 | 9 | 96 | 32 | 43 | 45 | 26 | 82 | 64 | 55 | 78 | 23 | 73 | 29 | 67 | 32 | 73 | 92 | 27 | 52 | 30 | 13 | 88 | 100 | 86 | 82 | 47 | - | 10 | - | 75 | - | 25 | 11 | 25 | 61 | 25 | 19 | 39 | 35 | 85 | 68 | - | 0 | 9 | 68 | - | 89 | 49 | 41 | 42 | 53 | 64 | 55 | 18 | 40 |
PickPocket | 28 | 35 | 74 | - | 4 | 61 | 68 | 50 | 100 | 62 | 15 | 28 | 29 | 41 | 55 | 55 | 62 | 35 | 23 | 73 | 42 | 8 | 2 | 42 | 0 | 36 | 44 | 5 | 88 | 88 | 18 | 55 | 18 | 64 | - | 100 | 10 | 75 | 25 | 0 | 44 | 8 | 32 | 7 | 81 | 27 | 47 | 85 | 24 | - | 95 | 34 | 82 | 0 | 70 | 27 | 31 | 24 | 25 | 0 | 77 | 82 | 42 |
ARB | 19 | 56 | 43 | - | 25 | 85 | 40 | 50 | - | 14 | 32 | 11 | 26 | 10 | 0 | 36 | 45 | 21 | 0 | 0 | 0 | 96 | 57 | 100 | 8 | 0 | - | 100 | 13 | 88 | 0 | 55 | 36 | 56 | - | - | - | - | - | - | - | 0 | 50 | 2 | - | 0 | 26 | 0 | 0 | - | 100 | 0 | 0 | - | 89 | 25 | 20 | 37 | 18 | 9 | 36 | 82 | 21 |
DeepSeqPa | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
Benchmark results get recalculated as new servers are added to the benchmarks. Numbers in light grey represent scores on datasets before the server was incorporated into the benchmarks. Previous versions of the benchmark results can be downloaded below