I'm excited about this new model and architecture, and I'm especially curious about what led to the rather hugely improved performance on MTEB compared to existing models and architectures.
At first glance, it seems like the authors here did not train for as long for the MTEB Experiments. For reference, the retrieval section of MTEB(Eng, v1) corresponds with BEIR, and the authors here report scores ranging from 21.0 to 31.6 after 2000 training steps for base size BERT, RoBERTa, NomicBERT, and ModernBERT, whereas the ModernBERT paper reports scores between 37.7 to 41.6 for those same models.
It's a little surprising that they only trained with 2000 training steps, considering they used 19 different datasets.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4