What is performance vs prediction speed in this context? Training vs inference?

…— meaning they are using a binary tree of depth six, which breaks the layer in half six times — yet they still retain 94% of the original performance while cutting down costs and increasing prediction speed by orders of magnitude.
A 78x Faster Frontier Model, the New Amazing AI Breakthrough
1.1K
9
Ignacio de Gregorio
Lannie Rose
·Follow
Dec 7, 2023
--
What is performance vs prediction speed in this context? Training vs inference?
--
--
Written by Lannie Rose1.6K Followers
·104 Following
Nice to have a place where my writing can be ignored by millions
Responses (1)
Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams