Head over to our on-demand library to view sessions from VB Transform 2023. Register Here
MosaicML has unveiled MPT-7B-8K, an open-source large language model (LLM) with 7 billion parameters and an 8k context length.
According to the company, the model is trained on the MosaicML platform and underwent a pretraining process commencing from the MPT-7B checkpoint. The pretraining phase was conducted using Nvidia H100s, with an additional three days of training on 256 H100s,…
