Skip to content
Linear Horizon Training

Example · 3 days

Applied NLP and the maths behind it

Classical NLP through transformers in Python, from TF-IDF to Hugging Face, with the linear algebra, probability and statistics that explain why it works.

For data scientists, analysts and ML engineers.

  1. Day 1

    • Maths: linear algebra for embeddings and attention

      Quarter day · NLP and the maths behind models · Intermediate

      • Vectors, dot products, norms and cosine similarity.
      • Matrices as transformations; matrix multiplication by hand and in NumPy.
      • Projections and dimensionality reduction: PCA and what it keeps.
      • Attention written as matrix operations.
    • Maths: probability and information theory

      Quarter day · NLP and the maths behind models · Intermediate

      • Distributions, conditional probability and Bayes’ rule.
      • Softmax, temperature and sampling.
      • Entropy, cross-entropy, KL divergence and perplexity.
      • Exercise: compute a model’s loss on a sentence by hand.
    • NLP: Python for text and preprocessing

      Quarter day · NLP and the maths behind models · Foundation

      • Strings, Unicode, regular expressions and reading text at scale.
      • Tokenisation, normalisation, stemming and lemmatisation.
      • Stop words, and when removing them hurts.
      • Lab: clean and tokenise a real corpus.
    • NLP: features, bag of words and TF-IDF

      Quarter day · NLP and the maths behind models · Foundation

      • Bag of words, n-grams and vocabularies.
      • TF-IDF and what it rewards.
      • Sparse matrices and similarity between documents.
      • Lab: find similar documents with TF-IDF.
  2. Day 2

    • NLP: text classification

      Quarter day · NLP and the maths behind models · Intermediate

      • Naive Bayes, logistic regression and linear SVMs on text.
      • Imbalanced classes, precision, recall and thresholds.
      • Error analysis: reading the mistakes.
      • Lab: build and evaluate a classifier for a real labelling task.
    • NLP: topic modelling

      Quarter day · NLP and the maths behind models · Intermediate

      • LDA and NMF.
      • Embedding-based topics with BERTopic.
      • Choosing the number of topics and judging coherence.
      • Lab: find the themes in a set of customer comments.
    • Maths: gradients and optimisation

      Quarter day · NLP and the maths behind models · Intermediate

      • Derivatives, the chain rule and backpropagation.
      • Gradient descent, momentum and Adam.
      • Learning rates, overfitting and regularisation.
      • Lab: train a tiny model with gradients you wrote yourself.
    • NLP: word and sentence embeddings

      Quarter day · NLP and the maths behind models · Intermediate

      • Word2vec and GloVe: meaning from context.
      • Analogies, bias and the limits of static vectors.
      • Sentence embeddings and semantic search.
      • Lab: compare TF-IDF and embeddings on the same search task.
  3. Day 3

    • NLP: linguistic structure with spaCy

      Quarter day · NLP and the maths behind models · Intermediate

      • Part-of-speech tagging and dependency parsing.
      • Named entity recognition, and training a custom entity.
      • Rule-based matching alongside models.
      • Lab: extract structured records from free text.
    • NLP: transformers with Hugging Face

      Quarter day · NLP and the maths behind models · Intermediate

      • Pretrained encoders and the Hugging Face ecosystem.
      • Fine-tuning a small transformer for classification or NER.
      • When a small fine-tuned model beats an LLM, on cost and accuracy.
      • Lab: fine-tune and compare against the classical baseline.
    • Statistics for evaluating AI systems

      Quarter day · NLP and the maths behind models · Intermediate

      • Sampling error and how big a test set needs to be.
      • Confidence intervals and bootstrapping for accuracy and judge scores.
      • Paired comparisons: is version B really better than version A?
      • Exercise: decide whether a reported improvement is real.
    • NLP: capstone pipeline

      Quarter day · NLP and the maths behind models · Intermediate

      • Put the pieces together: clean, extract, classify and search.
      • Choosing classical, transformer or LLM for each step.
      • Packaging the pipeline and evaluating it end to end.
      • Present the pipeline and its numbers to the group.
Contact