THE AI INTERVIEW
it would not be enough. You need to break the compute boundary beyond a single chip.”
Like any factory, an AI factory is judged on how efficiently it turns inputs into outputs. Its inputs are data and energy, while its outputs are tokens, the small units of text and other data that AI models process and generate.
There is also a growing distinction between training, where a model learns from vast datasets, and inference, where a finished model responds to real-world requests.
Michael adds:“ Training is basically building the machine, and it’ s usually a batch operation. Once you deploy the model for inference, the interaction is very different, because you depend on the timeliness of the response and on availability.”
Choosing where and how to run AI Enterprises building their own AI capacity can outsource it to a cloud provider or run it on-premises, but cost, data confidentiality and latency can all favour keeping AI in-house.
Proprietary foundation models, such as Google’ s Gemini and those built by ChatGPT developer OpenAI, are powerful general-purpose systems whose inner workings are not shared.
“ They are trained on the data they were trained on, which is not necessarily your data,” Michael notes.“ If you want to tune a model to fit your needs, you need to go to open models. You can post-train them, run them on-premises and build specialised versions with your own data.”
aimagazine. com 25