Jeffrey Jiao- AI (Large Language Model) Technology Principles and Applications E

AI (Large Language Model) Technology Principles and Applications: Empowering ITBP Team Capability Enhancement

Jeffrey Jiao, AI and Digital Transformation Practitioner, Former McKinsey Consultant

Update memo of requirements:

2025/10/09 10:00 Meeting update on requirements:

Add retail FMCG (Fast-Moving Consumer Goods) cases, expected to deliver insights for drug-to-consumer (to C) sales;

Basic concept study materials are deliverd one week before class so that students can enter the classroom quickly.

2025/09/18 10:30 Meeting update on requirements:

Training objectives:

Align the ITBP crew (9 people) on AI cognition and conceptual language system;

To become Merck's pioneer in China in the field of AI;

Demand grasping:

Establish a global perspective: study the differences and comparisons between Chinese and American representative companies in AI grasping, buildment direction, offering buildment route and methodology;

Industry Practical Experience: Analyze lessons learned from AI applications across different industries (domestic, international, and foreign-funded); examine the AI strategies of major Chinese tech companies (Baidu, Alibaba, Tencent, ByteDance, etc.) and central state-owned enterprises; and deliver reference frameworks for large model selection in B2B enterprises (particularly healthcare).

Other requirements:

Hope to be know-how intensive, theoretical and interesting;

In the morning, everyone is energetic and has a higher know-how density, and in the afternoon, there is more interaction;

From 15:00 to 15:30, invite German colleagues to share for half an hour on planning, game plan, etc.;

Materials should be in English, with bilingual (Chinese-English) format recommended.

program objective

Help the crew to deepen the grasping of AI technology, realize the transformation from theory to hands-on application, and be able to use AI to solve work pain points; cultivate technical capabilities beyond theory, and non-technical colleagues can also master basic hands-on coding competencies.

Build a systematic AI technology capability framework for the ITBP crew to facilitate the implementation of enterprise digital transformation.

Course Objectives

The entire ITBP crew (both technical and non-technical backgrounds, friendly to beginners)

Duration of instruction

A total of 6 hours (theoretical explanation 35%+ hands-on operation 45%+ interactive discussion 20%)

course content

Part 1: Global Vision and Technology Frontiers

Global vision for AI buildment

Global buildment of AI and China-US comparison

Global AI buildment status and trends

Technological buildment speed: The evolution speed of artificial intelligence is far faster than expected. Experts predict that general artificial intelligence (AGI) may be realized in the next 2-6 years, and the technological progress cycle will be shortened from the century of the industrial age to the month and week level of the AI era.

Performance breakby way of: The performance of the AI system continues to break by way of in the complex benchmarking, and the scores of MMMU, GPQA and SWE-bench increased by 18.8, 48.9 and 67.3 percentage points respectively within one year.

Market size: The total size of China's AI market is expected to reach 51.13 billion US dollars in 2025, and the market size of AI-related markets will exceed 3.8 trillion yuan.

Comparison of AI buildment paths between China and the US

The "elite" route vs the "industrialist" route

dimension

America

China

idea of buildment

Lab elitism, technology-driven

Industrial pragmatism, scenario driven

technology roadmap

Closed source monopolies, dominated by private companies

Multiple open source, government guidance and industrial integration

Areas of strength

Base model, underlying architecture, open source ecology

Application scenarios, data resources, industrialization speed

Representative enterprises

OpenAI，Google、Anthropic

Alibaba, Huawei, Baidu, Tencent

International game plan

Technical protection, export restrictions, building alliances

Open source sharing, promoting offerings and open source models

The changing technology gap and the rise of open source power

The performance gap between China and the US has narrowed significantly: in benchmarks such as MMLU, MMMU, MATH and HumanEval, the performance gap between China and the US has narrowed significantly from 17.5, 13.5, 24.3 and 31.6 percentage points at the end of 2023 to only 0.3, 8.1, 1.6 and 3.7 percentage points at the end of 2024.

The rise of open-source models: China is accelerating the construction of an "independently controllable and open collaborative" industrial system, pooling resources by way of a new national system for technological breakby way ofs. AliTongyi Qianwen has become the world's largest open-source large model, with its derivative models surpassing 100,000 in number, outperforming the American llama.

Breakby way of of DeepSeek: On January 20,2025, DeepSeek released R1 reasoning model, which achieved performance equivalent to OpenAI o1 at a very low cost (only $0.55 per million input tags, while OpenAI o1 is up to $15), which was called "the Android moment in AI".

Advances and breakby way ofs in large model technology

Three major changes in technological buildment

From "aesthetics of violence" to "meticulous cultivation": the competition of large models shifts from the comparison of parameter scale to the enhancement of efficiency and application;

From "heavy learning and development" to "heavy reasoning": The reasoning efficiency is significantly enhanced by way of reinforcement study and know-how distillation technology;

From "single mode" to "multi-mode fusion": seamless fusion of text, image, audio and video.

Major technological breakby way ofs in 2024-2025

Mixed expert architecture (MoE): GPT-5 has 512 experts, and the activated parameters account for only 7%, greatly improving efficiency;

Breakby way of in reasoning ability: OpenAI's o3 outperforms some human experts in fields such as mathematics and programming; Google's Gemini 2.0 Lightning Thinking demonstrates amazing problem-solving ability in physics and engineering problems;

Multimodal fusion: CLIP++ model achieves four-mode joint representation of text, image, audio and video, and the cross-mode retrieval accuracy is 91.3%;

Edge AI buildment: The global edge AI market size will reach 121.204 billion yuan in 2024 and is expected to grow by 29.49% CAGR to 571.486 billion yuan in 2030.

The rise of AI agents (Agents)

Autonomous enhancement: AI agents can handle complex multi-step tasks without human intervention, from "passive response" to "active service";

Commercial Year: 2025 will be the "commercial year" of AI Agent. By 2028,15% of daily work decisions are expected to be completed by Agentic AI;

Enterprise applications: Deloitte predicts that agents will be able to support the work of supply chain managers, software builders, financial analysts and others;

Computing power infrastructure and hardware ecology

China's computing power scale: China's intelligent computing power scale will reach 725.3 EFLOPS in 2024 and is expected to reach 1037.3 EFLOPS in 2025, an increase of 43% compared with 2024.

Cost of computing power: Training costs for a single model plunged from $12 million in 2022 to $850,000.

Green computing power buildment: With the popularization of large models, green solutions such as liquid cooling technology and edge computing are building rapidly.

Comparison of hardware ecology between China and the US:

The United States has a clear advantage: The NVIDIA H100 achieves 1,979 TFLOPS in half-precision (FP16) computing power, which is 7.7 times that of Huawei's Ascend 910B, and its video memory bandwidth is 336% higher.

CUDA ecological barrier: Nvidia A100/H100 series occupies 90% of the global AI learning and development chip market, and the CUDA ecosystem forms an insurmountable technical stickiness.

China's Breakby way of Strategy: Focusing on "Scenario Customization + Process Innovation", Huawei Ascend 910B achieves 14nm workflow mass offeringion by way of Chiplet technology, with an energy efficiency ratio exceeding 30% in smart city video analysis.

Core know-how of large models

Core concepts (the concepts and methods involved in the data set to the pre-learning and development phase)

It covers all the conceptual principles and hands-on workflowes involved from data set construction to base model formation, and is expected to achieve the level of "mastering first principles" for large models.

Concepts: What is a data set? How does it differ from algorithms or computing power? Where are the data set scarcity issues? Why did Transformers emerge? What is the self-attention mechanism? What exactly does pre-learning and development aim to achieve?...

Part 2: Industry practice and interactive discussion

Core know-how points of large models

Core concepts (emphasis on the concepts and methods involved in "vertical model" learning and development)

Note: The afternoon session will feature more interactive activities, where attendees can apply concepts, principles, and methodologies from large models to engage in open discussions. Topics will include insights from industry leaders, future commercial trends, healthcare industry strategies, and the positioning and value delivery of IT-BP solutions...

The time was set around 3:00PM and a German colleague was introduced to share.

Introduction to large model integration and Agent Design

What is an AI Agent

Popular definition, attributes and classification of intelligent body

Perceive the environment and act to achieve your objectives

Attribute: autonomy, objective orientation, perception, initiative

Classification: simple reflex, model reflex, based on target, based on utility, study type

General architecture of intelligent agents

AI Agent= large model + memory + planning + tools

Intelligent agent buildment framework

Select buildment platform: Internet platform, local deployment

Model selection: based on professional attributes, data attributes, multimodal attributes, etc

Service orchestration: prompt engineering, single Agent/multi Agent, component invocation, etc

Core competency buildment: workflow, memory, know-how, conversational experience, etc

Release and maintenance

Application of prompts in the workflow of building agents

Understand the task prompts

Task decomposition, task definition, introduction of examples

Task planning and execution

Task chain design, multiple rounds of task execution, execution feedback adjustment

Match information retrieval

Limited information sources, multi-level retrieval and dynamic retrieval

User interaction design

Multimodality, context

NL2SQL prompt application

definition

NL2SQL, also known as Text2SQL, is the conversion of a user's natural language into SQL statements so that the user can retrieve the data from the database.

Challenges in hands-on use of NL2SQL

The ambiguity and uncertainty of natural semantics, the complexity of relationships between data tables, the integrity of data, and similarity.

Zero-shot and Few-Shot

NL2SQL prompt element

Tasks, reference information, output requirements, examples, input and output items for this time

Common prompt cases

Text prompt method, position play method, code prompt method, instruction fine-tuning prompt method, context study

Cutting-edge exploration

C3，CoT、DIN-SQL

RAG prompt application (vs context project)

RAG architecture

Question analysis and continuation

Information retrieval and integration

Optimization strategies for generating answers

Limit and guide the generation model

Industry pioneers

AI's iPhone Moment: From Cloud Native to AI Primitive

Stanford 2025 Artificial Intelligence Index Report

Cloud native vs. AI native

The connotation of AI native: AI reshapes software and enables intelligent industry

Huawei, Tencent and other enterprises' AI architecture evolution ideas

Sierra's innovation (founded by OpenAI's board chairman): A new way to 'connect' with clients

Zhong Mobile AI Lingxi: 1 billion users have an AI assistant

Industry application practice and lessons learned

Cross-industry application case sharing

manufacturing industry：

A steel plant used a large model to analyze equipment data, predicting machine failures three days in advance and reducing maintenance costs by 40%.

China has built 1,200 smart factories, with 72 "lighthouse factories" accounting for 42% of the global total. AI has optimized offeringion workflowes, reducing energy consumption by 15%.

Tesla Optimus Gen2 robot has been applied in factory offeringion, and man-machine collaboration will become an crucial mode of intelligent manufacturing in the future

banking commercial：

Ant Group's risk control brain 3.0 achieved a credit fraud identification accuracy of 99.993% (an increase of 3 orders of magnitude compared with 2022)

An insurance company used a large model to automatically identify injury photos, reducing the time to settle claims from three days to 10 minutes, and client satisfaction doubled

Cross-border payment and clearing efficiency has been enhanced by 22 times, supporting RCEP regional second-level arrival

Medical care:

The diagnostic accuracy of IBM Watson has reached 98.7%, surpassing the average for senior physicians

The AI-assisted cervical cytology diagnosis system builded by Tencent and Ambipin has an accuracy rate of more than 95 percent and has been deployed in more than 7,000 hospitals

Hengrui Pharmaceutical applied AI to new drug research and buildment by way of the "Bai Cheng Intelligent Drug Platform", shortening the research and buildment cycle by 30%

Educational field：

The introduction of AI tools in primary and secondary schools in Beijing has increased the efficiency of English classes by 30%

A middle school used a large model to analyze students' mistakes and automatically generate targeted exercises, raising the class average by 15 points

Duolingo, the gamified language study app in the US, has surpassed 100 million monthly active users, while Khan Academy's AI tutors guide inquiry-based study

Retail field：

Meituan, China Mobile's掌厅to C, Sierra (OpenAI board of directors) and other retail innovation achievements

Summary of application experience and lessons learned

Avoid blindly pursuing "big and complete": A food factory spent millions of yuan to train a general model, only to find that the industry terms were not recognized accurately. We should first build small models in vertical fields and then expand gradually.

Data is more crucial than algorithms: A logistics company trained its path planning model with publicly available online data, but the actual results were not satisfactory. Real commercial data should be accumulated, even if it is only Excel tables.

Employees are not rivals, but assisters: a bank pushed smart client service, which was resisted, and then trained tellers to use AI to answer professional questions, improving service efficiency by 50%.

The red line of security should not be touched: A hospital was punished by the regulatory authorities for directly uploading patients' CT images to the public cloud. Sensitive data must be deployed on-premises.

Analysis of AI planning by China's major companies and state-owned enterprises

AI game plan of Internet giants

Ali: Tongyi Qianwen has become the world's first open source large model, and the number of derivative models has exceeded 100,000. It is expected to invest 380 billion yuan in AI and cloud computing in the next three years.

Baidu: Wenxin Yiyang is commercialized by way of search, e-commerce recommendation and other scenarios.

Tencent: The average reasoning delay of the hybrid model in the field of game NPC interaction is reduced by 35%.

ByteDance: Ocean Engine enables automatic generation of advertising scripts, increasing e-commerce conversion rates by 35%.

Characteristics of AI applications in central state-owned enterprises

Focus on security and control: Prioritize local deployment to make certain data security

Scenario-driven: Focus on ROI validation starting from specific commercial pain points

Integration of industry, academia, and research: Collaborating with research institutions to build industry-specific large models, such as the 'Xingyu 3.0' astronomical model builded by the National Astronomical Observatories of the Chinese Academy of Sciences based on Qwen.

Reference for Model Selection of Enterprise B (Medical)

Points to consider:

In-house vs. Outsourcing: Internal and external collaboration models

Base model + vertical category: How high is the barrier to medical professional learning and development models?

Ownership interests, security and compliance of data exchange: data privacy computing, federated study, GDPR and other regulations

Ensuring interpretability, localization (localization), AI for Science...

老师简介

Introduction to lecturers

弄懂更多

博闻智咨询|焦波老师