Databricks Open Sources DBRX AI to Rival GPT Models and Spur Innovation
-
Databricks spent $10 million and months of work to create DBRX, an AI language model similar to ChatGPT. Tests show it surpasses all other open source AI models and rivals private models like GPT-4.
-
Databricks acquired startup MosaicML last year, bringing in experts who used recent innovations to train DBRX efficiently at scale using over 3,000 GPUs.
-
DBRX has 136 billion parameters but only activates 36 billion on average per query, making it efficient. Curriculum learning in the final training stage further improved it.
-
By open sourcing DBRX, Databricks aims to spur AI innovation by putting powerful technology into more hands. The company also seeks greater transparency.
-
Databricks plans scientific studies of how DBRX gained abilities, contributing to understanding of AI. It also hopes the model helps companies apply AI to their own data.