Various readily available AI foundation models can be applied to myriad use cases. These models allow even those with relatively little technical background or experience with AI to build AI products. AI Engineering will help people understand the distinction between AI engineering and classical machine learning models and how to develop an AI model and navigate challenges that might arise during that process. Learn how to adapt a model to your specific needs and, ultimately, how to choose a model that works for you. Datasets are important, and so are ways to evaluate where you are on your journey.
AI engineering differs from traditional machine learning engineering.
What distinguishes AI today from earlier incarnations is its sheer scale. Applications like ChatGPT and Google’s Gemini and Midjourney swallow up significant amounts of electricity and are trained on massive amounts of data. The amount of publicly available data to train them on may actually run out. These powerful AI models can run myriad applications, increasing their economic value — and, ultimately, improving people’s lives.
Training large language model (LLM) AIs requires huge amounts of data and computational power. Few companies can meet this demand. The difference between today’s LLMs and earlier language models is “self-supervision”: Older language model algorithms needed specifically labeled data — which can take a great deal of time and resources to gather. Supervision involves tagging data with behavior and other features you want a model to learn about and then use to inform its output. Once training is complete, the model can apply what it learned from the tagged datasets to analyze data in general. Labeling becomes more difficult...
Chip Huyen is a writer and computer scientist who works at the intersection of AI, data, and storytelling. She has worked with Snorkel AI and NVIDIA, founded an AI infrastructure start-up, and taught machine learning systems design at Stanford University.
Comment on this summary