Domain 1: Fundamentals of Large Language Models (20%)

Domain 1 of the 1Z0-1127-25 Oracle Cloud Infrastructure 2025 Generative AI Professional exam covers the theoretical and practical foundations of large language models. This domain represents approximately 10 questions on the exam. It is the conceptual bedrock for the remaining three domains -- every question about OCI Generative AI service, RAG, and agents assumes you understand these fundamentals.

The exam tests six areas within this domain:

1. Transformer architecture and LLM types
2. Tokenization and embeddings
3. Prompt engineering techniques
4. Decoding strategies and inference parameters
5. Fine-tuning methods (with emphasis on T-Few and LoRA)
6. Emerging LLM topics (code models, multi-modal models, language agents, hallucination)

1. Transformer Architecture and LLM Types

The Transformer

The Transformer architecture, introduced in the 2017 paper "Attention Is All You Need," is the foundation of every modern LLM. The exam expects you to understand its core mechanisms, not implement them.

Key components:

Three Model Architectures

The Transformer spawned three architectural patterns. This distinction is heavily tested:

Exam trap: BERT is an encoder -- it does not generate text. GPT and Llama are decoders -- they generate text but are not designed to produce embeddings natively (though embeddings can be extracted). If a question asks which model type produces embeddings for semantic search, the answer is encoder models.

What Makes a Model "Large"

A large language model (LLM) is a probabilistic model of text with a very large number of parameters. There is no agreed-upon threshold for "large." The exam focuses on the practical implication: LLMs can perform tasks they were not explicitly trained on through in-context learning. (OCI GenAI Concepts)

2. Tokenization and Embeddings

Tokenization

Tokens are the fundamental input units for LLMs -- words, subwords, or individual characters/punctuation. The model never sees raw text; it sees token IDs. (OCI GenAI Concepts)

Examples from Oracle documentation:

• "apple" = 1 token
• "friendship" = 2 tokens ("friend" + "ship")
• "don't" = 2 tokens ("don" + "'t")
• Rule of thumb: ~4 characters per token

Tokenization algorithms (know the names and distinctions):

Exam trap: The exam may ask about the relationship between context window size (measured in tokens) and model capability. Larger context windows allow more input but increase computational cost quadratically with self-attention.

Embeddings

Embeddings are numerical vector representations that capture the semantic meaning of text. They transform words, phrases, or entire documents into arrays of numbers (typically 384 or 1024 dimensions in OCI). (OCI GenAI Concepts)

Similarity measurement methods:

OCI embedding models: Cohere Embed 4 (multimodal -- text + images), Cohere Embed English 3, Cohere Embed Multilingual 3, plus Light variants for lower latency. (OCI Pretrained Models)

Exam trap: Embedding-based search is semantic (meaning-based), not keyword-based. A keyword search for "car" will not match "automobile." Embedding search will, because their vectors are close in semantic space.

3. Prompt Engineering

Prompt engineering is the iterative process of crafting input text to optimize LLM outputs. It modifies the model's output probability distribution without changing any model parameters. This distinction between prompting (no parameter changes) and fine-tuning (parameter changes) is fundamental to the exam. (OCI GenAI Concepts)

In-Context Learning Techniques

Soft Prompting

Soft prompting (also called prompt tuning) is a hybrid between prompting and fine-tuning. It adds a small number of trainable parameters (soft prompts) to the model's input layer while keeping all original model weights frozen. (Jvikraman GenAI Notes)

Exam trap: Soft prompting is NOT the same as prompt engineering. Prompt engineering uses natural language text as input. Soft prompting uses learned continuous vectors prepended to the input. Soft prompting modifies parameters (the soft prompt vectors); prompt engineering does not.

Preamble (System Prompt)

In OCI Generative AI, the preamble is the initial context or guiding message sent to a chat model. It sets the model's role, tone, and behavioral constraints. The default preamble for Cohere Command R+ is: "You are Command. You are an extremely capable large language model built by Cohere." This is customizable per API call. (OCI GenAI Concepts)

Prompt Security

Two attack vectors the exam covers:

• Prompt Injection / Jailbreaking: Malicious input designed to override system instructions or produce harmful outputs. OCI Guardrails includes a dedicated prompt injection defense. (OCI GenAI Concepts)
• Memorization Attacks: Attempts to coerce the model into repeating its training data or system prompt verbatim.

4. Decoding Strategies and Inference Parameters

Decoding is the process of generating output text from an LLM. It happens iteratively, one token at a time: the model predicts a probability distribution over the vocabulary, selects a token, appends it, and repeats. (Jvikraman GenAI Notes)

Decoding Methods

Inference Parameters (Critical for Exam)

These parameters are directly configurable in the OCI Generative AI Playground and API:

(OCI GenAI Concepts)

Exam trap: Temperature = 0 produces deterministic output (use with a seed parameter for identical results). High temperature (approaching 1.0+) introduces hallucinations. The exam frequently tests which parameter combination is most likely to cause hallucinations -- the answer is high temperature + high top-p + low penalties.

Exam trap: Frequency penalty and presence penalty are different. Frequency penalty scales with how many times a token appeared. Presence penalty applies equally to all tokens that appeared at least once, regardless of count. Know this distinction.

How top-k and top-p interact: When both are set, the model first selects the top-k tokens, then from those, keeps only tokens whose cumulative probability reaches p. For example, if k=20 but the top 10 tokens already sum to 0.75 (and p=0.75), only those 10 are considered.

5. Fine-Tuning Methods

Fine-tuning modifies model parameters using task-specific data. This is the key distinction from prompting, which leaves parameters unchanged. The exam heavily tests the differences between fine-tuning approaches. (Jvikraman GenAI Notes)

Fine-Tuning Methods Comparison

T-Few in Depth (Heavily Tested)

T-Few is an additive Parameter-Efficient Fine-Tuning (PEFT) technique based on the (IA)^3 method from the paper "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning". Key facts:

• Mechanism: Inserts learned vectors that rescale inner activations within attention and feed-forward modules. The original pre-trained weights remain frozen.
• Parameter efficiency: Adds approximately 0.01% of the baseline model's size in new parameters.
• Performance: Outperforms GPT-3 in-context learning on benchmarks while using over 1,000x fewer FLOPs during inference.
• OCI usage: OCI Generative AI uses T-Few for Cohere Command R (08-2024) models. (OCI Fine-Tuning Methods)
• Training: Requires a dedicated AI cluster with 2 model units. Minimum 1 unit-hour per job.

T-Few hyperparameters in OCI:

(OCI Fine-Tuning Parameters)

LoRA in Depth

LoRA adds small trainable rank-decomposition matrices (low-rank matrices) that transform inputs and outputs, rather than updating all original parameters. The weight matrices are scaled by LoRA alpha / LoRA r.

• OCI usage: OCI uses LoRA for Meta Llama 3.3 70B, Llama 3.1 70B, and Cohere Command R (08-2024). (OCI Fine-Tuning Methods)

LoRA hyperparameters in OCI:

(OCI Fine-Tuning Parameters)

Training steps formula: totalTrainingSteps = (totalTrainingEpochs x size(trainingDataset)) / trainingBatchSize

OCI Model-to-Method Mapping (Memorize This)

(OCI Fine-Tuning Methods)

Exam trap: Cohere Command R supports both T-Few and LoRA. Meta Llama models support only LoRA. If a question asks which method is available for Llama fine-tuning, T-Few is wrong.

Fine-Tuning Evaluation Metrics

Exam trap: The exam tests the difference. Accuracy is "how many right." Loss is "how wrong." A model can have improving loss but fluctuating accuracy during early training. Both are reported on the OCI model details page under "Model Performance."

6. Emerging LLM Topics

Code Models

Code models are LLMs trained on code, comments, and documentation. They assist with code generation, completion, debugging, and explanation. Key examples: Codex (OpenAI), Code Llama (Meta), StarCoder (BigCode). In OCI, xAI Grok Code Fast 1 is available as a coding-focused model supporting TypeScript, Python, Java, Rust, C++, and Go. (OCI Pretrained Models)

Multi-Modal Models

Multi-modal LLMs process and generate across multiple data types (text, images, audio, video). In OCI, several multi-modal models are available:

(OCI Pretrained Models)

Diffusion models (DALL-E, Stable Diffusion) produce complex outputs simultaneously rather than token-by-token. They work well for images because image data is continuous. They are difficult to apply to text generation because text is categorical/discrete. (DBExam Sample Questions)

Exam trap: The exam asks why diffusion models are hard to apply to text. The answer is that text representation is categorical (discrete tokens), unlike continuous image pixel data.

Language Agents

Language agents use LLMs as reasoning engines that can take actions in the real world. Key concepts:

In OCI, agentic capabilities are supported by models like Cohere Command A Reasoning, Grok 4.1 Fast, and Llama 4 Maverick/Scout. (OCI Pretrained Models)

Hallucination

Hallucination is the phenomenon where an LLM generates factually incorrect information or unrelated content as if it were true. There is no known methodology to reliably eliminate hallucination entirely. (Jvikraman GenAI Notes)

Causes:

• Training data gaps, biases, or errors
• High temperature during decoding (flattened probability distribution)
• Lack of grounding in source documents
• Ambiguous or poorly constructed prompts

Mitigation strategies:

Exam trap: RAG does not eliminate hallucination. It reduces it by providing context. The model can still hallucinate information not present in the retrieved documents. The RAG Triad evaluation framework (Context Relevance, Groundedness, Answer Relevance) tests whether the full pipeline is working correctly.

Model Evaluation Metrics

7. OCI Generative AI Service -- Domain 1 Context

While OCI service details are primarily tested in Domain 2, Domain 1 questions may reference these service concepts:

(OCI GenAI Overview)

Quick-Reference: Decision Tree for Customization

Use this to answer "which technique should you use" questions:

Is the model performing well enough with standard prompting?
  YES --> Use zero-shot or few-shot prompting (no cost, no training)
  NO --> Do you have labeled training data?
    NO --> Use few-shot prompting with more examples, or RAG for grounding
    YES --> Is the dataset small (<100K samples)?
      YES --> Use T-Few (Cohere) or LoRA (Llama) for parameter-efficient fine-tuning
      NO --> Consider vanilla fine-tuning (if compute budget allows)
Does the data change frequently?
  YES --> Use RAG (retrieves current information, no retraining needed)
  NO --> Fine-tuning is acceptable

References

• OCI Generative AI Service Overview
• OCI Generative AI Concepts
• Choosing a Fine-Tuning Method
• Fine-Tuning Hyperparameters
• OCI Pretrained Models
• K21 Academy OCI GenAI Certification Guide
• T-Few Paper: "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"
• Jvikraman: Generative AI Concepts Study Notes
• DBExam: OCI 1Z0-1127-25 Sample Questions