Since a few day, the SocialMinds lab has a new mini GPU cluster at its disposal: two NVidia RTX6000 Blackwell nodes, with 96GB of VRAM each: enough to run large inference jobs, fine-tune open-source models or even train from scratch smaller models.

As a small exercice, we’ve trained from scratch the 2020 Vision Transformers from Google on the ImageNet dataset. In 3 days and 300 epochs, we’ve reached a top-1 accuracy of 76% on the validation set: without performing any specific optimization, we land 10 points below the best pre-trained versions from Google: not too bad! ViT is a pretty old model, but still the backbone of many vision models today.

Stay tuned for more results in the coming months!