REVISION 5 — Everything Technical

0. How to read this, and the honest framing

1. The map — how the fields nest

Start here. Almost everyone muddles these four words, and getting the nesting right in your first sentence buys you real credibility. They are not competing things — they are boxes inside boxes, plus one that sits sideways.

The nesting

  +---------------------------------------------------+
  |  ARTIFICIAL INTELLIGENCE                          |
  |  Any technique that makes a machine do something  |
  |  we would call intelligent.                       |
  |  INCLUDES HAND-WRITTEN RULES. A 1980s tax         |
  |  calculator with 400 IF-statements is AI.         |
  |                                                   |
  |   +---------------------------------------------+ |
  |   |  MACHINE LEARNING                           | |
  |   |  The machine finds the rules ITSELF, from   | |
  |   |  examples, instead of being given them.     | |
  |   |                                             | |
  |   |   +---------------------------------------+ | |
  |   |   |  DEEP LEARNING                        | | |
  |   |   |  Machine learning using neural        | | |
  |   |   |  networks with MANY LAYERS.           | | |
  |   |   |                                       | | |
  |   |   |   +-------------------------------+   | | |
  |   |   |   |  LARGE LANGUAGE MODELS        |   | | |
  |   |   |   |  Very large deep networks     |   | | |
  |   |   |   |  trained on text.             |   | | |
  |   |   |   +-------------------------------+   | | |
  |   |   +---------------------------------------+ | |
  |   +---------------------------------------------+ |
  +---------------------------------------------------+

  AND SITTING ACROSS ALL OF IT, NOT INSIDE IT:

  DATA SCIENCE -- the craft of getting data into a
  state where any of the above can be trusted, and
  deciding what the result is allowed to mean.

  STATISTICS -- is this pattern real, or chance?

Every large language model is deep learning, every deep-learning system is machine learning, and all of it is artificial intelligence — but never the reverse. Rule-based systems are AI and are not machine learning, which matters enormously, because most tools competition agencies actually deploy are rule-based.

2. The vocabulary spine — every term, one line each

Term	Plain definition	Concrete example
Model	A function with adjustable numbers in it, which you tune until its outputs match known answers	Given a tender's features, output the probability it was rigged
Parameter / weight	One of those adjustable numbers. Learned, not chosen by you	A 1.1-billion-parameter model has 1.1 billion of them
Hyperparameter	A setting you choose before training, which is not learned	How fast to learn; how many layers; the `r = 16` in your own fine-tune
Feature	One input the model is allowed to look at	Firm size, sector, number of bidders, winning margin
Label / target	The known answer you are training it to reproduce	This tender was found to be rigged — and note that is a record of enforcement, not of conduct
Training	Adjusting the parameters until outputs fit labels	What ran on the rented GPU in your thesis
Inference	Using the finished model on new data. No learning happens	Every request your Astroformer chat makes
Training / test set	Data it learns from / data held back to check it	Skip the split and you cannot tell learning from memorising
Overfitting	It memorised the training data and fails on anything new	Perfect on 200 past cases, useless on the 201st
Underfitting	It is too simple to capture the pattern at all	Fitting a straight line to something curved
Generalisation	Performing well on data never seen in training. The entire goal	—
Neural network / ANN	A model made of layers of simple units passing numbers forward. Artificial neural network	Section 4
Deep	*Just means many layers.* There is no deeper meaning	Two layers is shallow; fifty is deep
RNN	Recurrent neural network — reads a sequence one step at a time, carrying a memory	Section 6
LSTM	Long short-term memory — an RNN with gates that let it hold information longer	Section 6
Transformer	The architecture behind every modern language model. Reads a whole sequence at once	Section 7
Attention	The mechanism letting a model decide which other words matter for the word it is on	Section 7
Encoder / decoder	An encoder reads; a decoder writes. Models use one or both	Section 8
Token	The chunk of text a model actually processes — a word or part of one	competition → `compet` + `ition`
Embedding	A list of numbers representing meaning, positioned so similar meanings sit close together	Section 11
Context window	The maximum number of tokens it can consider at once	Yours was 512, which is small
Pretraining	The first, enormous, general training run. Costs millions; you will never do it	Done by others for TinyLlama
Fine-tuning	Taking a trained model and adjusting it on your own smaller, specific data	Your thesis
RAG	Retrieval-augmented generation — look it up first, then answer using what you found	Section 9
Hallucination	Fluent, confident falsehood	Inventing a case citation
Temperature	A randomness dial. Low is repetitive and safe; high is creative and unreliable	For legal work, low — and say why: reproducibility
Loss function	A number measuring how wrong the model currently is. Training minimises it	Section 5
Gradient descent	The method: nudge every parameter in whichever direction lowers the loss	Section 5
Backpropagation	The bookkeeping that works out which direction each parameter should be nudged	Section 5
Epoch	One complete pass through all the training data	Yours ran two
Batch	How many examples it sees before updating the parameters	Yours: 4, accumulated 8 times, so effectively 32
Quantisation	Storing parameters at lower numeric precision to save memory	*The Q* in QLoRA — 4-bit instead of 16-bit**
Data leakage	Information about the answer sneaking into the inputs, inflating performance	Section 10 — the most important failure on this page
Drift	The world changes, so a model trained on the past gets worse over time	A screen calibrated before a market restructured

3. The four ways a machine can learn

Paradigm	What it needs	What it gives	The catch
Supervised	Labelled examples — inputs and known answers	Prediction and classification. The workhorse	Labels are expensive, and in law they are usually records of enforcement rather than of conduct
Unsupervised	Just the data, no answers	Clusters, groups, anomalies, structure	*No ground truth, so you cannot call it right* — only coherent.** Interpretation is on you
Self-supervised	Raw data only — it invents its own task, like predicting the next word	How every LLM is built. Turns the whole internet into training data with nobody labelling anything	You get general capability, not the specific thing you wanted
Reinforcement	An environment and a reward signal	Behaviour learned by trial and error	Needs enormous numbers of trials. Rare in legal settings

4. The neuron and the neural network

y = f\!\left(\sum_{i=1}^{n} w_i x_i + b\right)

The output of a neuron is a function applied to the sum of each input multiplied by its own weight, plus a bias term.Term by term. The x values are the inputs. The w values are the weights — how much each input counts, and these are what training learns. b is the bias, a standing nudge up or down. **f is the *activation function*, and it is the only reason any of this works: it is a simple non-linear step, and without it a stack of layers would collapse mathematically into one straight line however many you added. So the activation function is what lets depth buy you anything at all.**

One neuron, then a network

  ONE NEURON
    x1 --w1--\
    x2 --w2---> [ sum  + b ] -> [ f ] -> y
    x3 --w3--/

  A NETWORK ("feed-forward", or ANN)

   INPUT        HIDDEN       HIDDEN      OUTPUT
    (o)----------(o)----------(o)
       \   X   /    \   X   /    \
    (o)----------(o)----------(o)-------(o)  0.87
       /   X   \    /   X   \    /       "probability
    (o)----------(o)----------(o)          rigged"
       \       /    \       /    /
    (o)----------(o)----------(o)

   firm size   every neuron    every neuron   one number
   sector      sees every      sees every
   n bidders   input           layer-1 output
   margin

   "HIDDEN" just means "not the input or the
   output". It does NOT mean secret.

   "DEEP" just means many hidden layers.

   Each line is ONE WEIGHT. This toy network has
   about 52. GPT-scale models have hundreds of
   billions.

A neural network is a stack of layers of these units where every connection is one learned number. Hidden is unfortunate jargon for the middle layers — it means structurally internal, not unknowable. Depth is simply how many middle layers there are.

Neural network against graph network — the distinction that catches people

**Both are described using the word *node*, which is the whole problem. In a neural network the nodes are computational units — the committee members above. The structure is fixed by the designer, and data flows through it. In a graph the nodes are the data — firms, tenders, people — and the structure *is* the thing you are studying. *The one-line test: if the nodes are things in the world, it is a graph; if the nodes are arithmetic, it is a neural network.* A *graph neural network* is a neural network that operates on graph-shaped data, which is a third thing. And agencies overwhelmingly use plain graph analysis, because it needs no training data and the output can be explained to a lawyer in one sentence.**

5. How it actually learns

w \;\leftarrow\; w \;-\; \eta \, \frac{\partial L}{\partial w}

Each weight is replaced by itself, minus the learning rate times the slope of the loss with respect to that weight.In words: find out whether nudging this weight up would make things worse, and if so nudge it down instead. L is the loss. The fraction is the slope — how much the loss changes if this one weight changes. **η (*eta*) is the learning rate, the step size, and you choose it. The minus sign is the entire algorithm: move against the slope.** Repeat a few million times.

The training loop, and the only graph that matters

   take a BATCH -> FORWARD PASS (predict)
        ^                    |
        |                    v
        |              LOSS (one number:
        |                    how wrong?)
        |                    |
        |                    v
        |           BACKWARD PASS
        |          (backpropagation:
        |           which way to nudge
        |           every weight?)
        |                    |
        |                    v
        '------------ UPDATE weights

   One full pass over all the data = one EPOCH.

   AND THE THING TO WATCH WHILE IT RUNS:

   loss
    ^
    |\
    | \        training loss (keeps falling)
    |  \___
    |      \_____
    |   ___________------ validation loss
    |  /                  (starts RISING here)
    | /         ^
    +-----------|-----------------> time
                |
         STOP HERE. Past this point the model
         is MEMORISING, not learning.
         That gap is OVERFITTING.

The loop is mechanical; the judgement is in the second graph. Training loss falling while held-out loss rises is the signature of overfitting, and the moment they diverge is when to stop. This is why a held-out set is not optional — without one you cannot see this picture at all.

6. Sequences — RNNs, LSTMs, and why they lost

The network in section 4 takes a fixed set of inputs. Language is not fixed — it is a sequence of varying length where order changes meaning. *The Commission fined the firm* and *The firm fined the Commission* contain identical words. So sequence models were the next step, and the story of how they failed is why transformers exist.

An RNN reads one word at a time, carrying a memory

   "The"   "firm"  "fined"  "the"  "Commission"
     |        |       |       |         |
     v        v       v       v         v
   [RNN] -> [RNN] -> [RNN] -> [RNN] -> [RNN]
        h1       h2      h3       h4      |
                                          v
   the SAME box reused at every step.   output
   h = the "hidden state" = its memory of
       everything so far, squeezed into one
       FIXED-SIZE list of numbers.

   THE PROBLEM, IN TWO PARTS:

   1. THE BOTTLENECK
      Everything the sentence said must fit in h.
      By word 200, word 1 has been overwritten.

   2. VANISHING GRADIENTS
      To learn that word 1 mattered for word 200,
      the training signal must travel back through
      200 multiplications. Numbers below 1,
      multiplied 200 times, go to roughly zero.

        signal
         1.0 |*
             | *
             |  **
             |    ****
         0.0 |        ********************
             +----------------------------->
              200   150   100    50     1
                 distance back through the sentence

      So it CANNOT learn long-range links.
      Not "learns them poorly" -- cannot.

The recurrent design reuses one box at every step, carrying a fixed-size memory forward. Both failures follow from that: the memory has a fixed budget, and the training signal decays as it travels back. The second has a name worth knowing — the vanishing gradient problem.

LSTM — the patch, and what the gates actually do

**An LSTM is *long short-term memory*: an RNN with three learned gates controlling its memory. The analogy — a notebook with a highlighter, an eraser and a redactor. The input gate decides what is worth writing down. The forget gate decides what to erase. The output gate decides what to reveal at this step. Because these are learned, the network can choose to keep one fact for 300 words and discard another immediately — which plain RNNs could not do. It genuinely helped and was the state of the art for years. But it did not fix the real problem: it still reads strictly one step at a time, so it cannot be parallelised across a sentence, which means it cannot be trained on enormous amounts of text. That is what killed it — not accuracy, but throughput.**

7. Transformers and attention

Sequential reading against attention

  RNN / LSTM -- information travels STEP BY STEP
   The -> agency -> fined -> the -> firm -> because -> it
                                    \_______________/
                            4 hops for "it" to reach
                            "firm", degrading each hop

  TRANSFORMER -- every word looks at every word

           The agency fined the firm because  it
            |     |     |    |    |      |    |
   "it" <---+-----+-----+----+----+------+----+
            .05   .08   .04  .03  .62   .06  .12
                              ^
                        the heaviest weight
                        goes to "firm"

   These weights are COMPUTED, not fixed. They
   are what "attention" means. Each word's
   meaning is rebuilt as a blend of the words
   it attends to.

   "MULTI-HEAD" attention runs several of these
   in parallel, each free to learn a different
   kind of relationship:

     head 1: subject    <-> verb
     head 2: pronoun    <-> what it refers to
     head 3: negation   <-> what is negated

   Nobody assigns those roles. They emerge.

Attention replaces a walk along the sentence with a direct weighted lookup from every position to every other. Multi-head attention runs several such lookups in parallel so different relationship types are captured at once. This is the single architectural idea behind every modern language model.

\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right) V

Attention compares each query against every key to get scores, scales them down, turns them into weights that sum to one, and uses those weights to average the values.The library analogy makes this readable. The query is what the current word is looking for. The keys are the labels on every other word — what each offers. The values are the actual content. So: match your question against every label, see which fit best, then take a blend of the contents weighted by fit. softmax just turns raw scores into weights adding to 1. Dividing by the square root of the dimension is housekeeping — without it the scores grow as the model widens and the weights collapse onto one word. You do not need this formula in the interview. You need the library analogy.

8. Encoder, decoder, and the three model families

The three families

  1. ENCODER-ONLY -- "the reader"
     Sees the whole input at once, BOTH
     directions. Produces understanding,
     not text.

       [the firm concealed the agreement]
                    |
             ENCODER (bidirectional)
                    |
          one rich representation
                    |
         +----------+----------+
         |                     |
     classify            extract entities
    ("is this a           ("firm","agreement")
     cartel case?")

     Example: BERT.
     USE FOR: classification, tagging, search,
     similarity. Anything where you UNDERSTAND
     existing text rather than write new text.
     ** This is what most agency work needs. **


  2. DECODER-ONLY -- "the writer"
     Generates one token at a time, each
     conditioned on everything so far.
     Can only look LEFT.

       "The firm concealed the" -> "agreement"

     Example: GPT. And TinyLlama -- YOUR thesis.
     USE FOR: generation, chat, summarising, QA.


  3. ENCODER-DECODER -- "the translator"
     Read it all, then write it all.

       [long judgment] -> ENC -> DEC -> [summary]

     Example: BART (your summarisation paper AND
     your thesis coherence step), T5.
     USE FOR: translation, summarisation -- any
     input-text-to-output-text transformation.

Encoder-only models read and classify. Decoder-only models write. Encoder-decoder models transform one text into another. You have used all three: BERT-family thinking in your NLP work, TinyLlama as a decoder in your thesis, and BART as an encoder-decoder in both your thesis and your summarisation paper.

9. LLMs and RAG

Term	Meaning	Your own instance
Pretraining	The vast general run using next-word prediction. Costs millions	Done by others
Fine-tuning	Adjusting a trained model on your own smaller data	346,920 rows on TinyLlama
LoRA / parameter-efficient tuning	Freeze the original, train a small set of extra parameters alongside. Far cheaper, and you can keep several task add-ons for one base model	`r = 16`, `alpha = 32`
Quantisation	Lower numeric precision to fit in less memory	*The Q* in QLoRA — and it needs CUDA, which is why it could not run on your MacBook**
Context window	How many tokens it can consider at once	512 — which is why stuffing in three documents plus three web snippets was too much
Temperature	Randomness dial	*Say low, for anything legal, and say why: reproducibility*
System prompt	Standing instructions the user does not see	Your Astroformer chat: use only the chart data above
Instruction tuning / RLHF	Extra training that makes a model follow instructions. Reinforcement learning from human feedback	Why a chat model answers rather than continuing your sentence
Sycophancy	The tendency to agree with the user rather than hold a position — a known side effect of training on human approval	Directly relevant to your Mens Rea result
Emergence	Two different meanings. Be careful. In ML: abilities appearing suddenly with scale. In complexity science: system behaviour not predictable from the parts	Schrepel means the second. The first is contested — one well-known paper argues the apparent jumps are largely an artefact of all-or-nothing metrics

RAG — the pipeline

  BUILD IT ONCE (offline)
   documents -> CHUNK into passages
                  |   <-- where most of the
                  |       quality is decided
                  v
                EMBED each chunk into numbers
                  |
                  v
                store in a VECTOR INDEX

  USE IT (per question)
   question -> EMBED it the same way
                  |
                  v
             find the NEAREST chunks
             (nearest = closest in MEANING)
                  |
                  v
             top 3-5 passages
                  |
                  v
             build a PROMPT:
               "Using ONLY the passages below,
                answer. Cite which you used.
                [p1] [p2] [p3]
                Question: ..."
                  |
                  v
             MODEL writes the answer
                  |
                  v
             answer + the passages it came from
                          ^
                          '-- THIS is what matters
                              legally: PROVENANCE

Retrieval-augmented generation separates the factual layer from the writing layer. The facts come from documents you control and can show; the model only phrases them. Chunking is the step that decides retrieval quality — and splitting a holding away from its reasoning is the classic way to ruin a legal corpus.

The four ways RAG fails — naming them is the credibility move

One — retrieval failure. If the right passage is not retrieved, the model answers from memory and sounds equally confident. A false reassurance is worse than no tool. Two — chunking failure. Split a provision from its exception and the retrieved passage is actively misleading. For legal text, chunk on structure — article, paragraph, recital — not a fixed token count. Three — the model ignores the passages and uses pretrained memory anyway. This is testable: swap in a deliberately wrong passage and see whether the answer changes. Four — stale index. The law changed; your corpus did not. And the evaluation that follows is recall-oriented, not accuracy-oriented: for questions where you know which passages are relevant, what fraction does the system surface?

10. Evaluation — and why accuracy lies

This is the most important section on the page for your interview, because the project's framing is about accuracy and safeguards — and the sharpest thing you can demonstrate is knowing when a number is meaningless.

The confusion matrix — everything comes from four boxes

                    WHAT WAS ACTUALLY TRUE
                     rigged         clean
                  +--------------+--------------+
        flagged   | TRUE         | FALSE        |
   WHAT           | POSITIVE     | POSITIVE     |
   THE            |              |              |
   MODEL          | correctly    | INNOCENT     |
   SAID           | caught       | FIRM         |
                  |              | INVESTIGATED |
                  +--------------+--------------+
        not       | FALSE        | TRUE         |
        flagged   | NEGATIVE     | NEGATIVE     |
                  |              |              |
                  | CARTEL       | correctly    |
                  | MISSED       | left alone   |
                  +--------------+--------------+

   The two errors are NOT symmetric, and which
   you would rather make is a LEGAL question,
   not a technical one:

     false positive -> an innocent undertaking
                       is investigated: cost,
                       reputation, disruption
     false negative -> a cartel continues and
                       consumers keep overpaying

   Choosing the threshold between them is a
   POLICY DECISION currently made by typing
   a number.
   ** Saying that out loud is the whole point. **

Every evaluation metric is built from these four counts. The asymmetry between the two errors is where law enters, because the trade-off between investigating the innocent and missing the guilty is a normative choice that a default parameter should not be making.

\text{Precision} = \frac{TP}{TP + FP} \qquad \text{Recall} = \frac{TP}{TP + FN} \qquad F_1 = 2\cdot\frac{P \cdot R}{P + R}

Precision is the share of things you flagged that really were positive. Recall is the share of actual positives you managed to flag. F1 is the harmonic mean of the two.In plain words. Precision: when it raises the alarm, how often is it right? — the question an investigated firm cares about. Recall: of all the real cartels, how many did it find? — the question the agency cares about. They trade off: flag everything and recall is perfect while precision collapses. F1 combines them, and the *harmonic* mean is used because it punishes imbalance — you cannot score well by being excellent at one and terrible at the other.

Data leakage — the one failure to explain cold

Leakage is when information about the answer sneaks into the inputs, so the model looks brilliant in testing and fails in the world. The clearest documented example is in legal prediction. Aletras and colleagues (2016) predicted European Court of Human Rights outcomes at about 79 percent accuracy from case text — but the text was the published judgment's facts section, written by the court, after the court decided. **When Medvedeva and colleagues (2020) redid it properly — training on past cases and predicting *future* ones — accuracy fell to roughly 58 to 68 percent. They also got high accuracy from the judges' surnames alone. The competition-law version is exact: a model trained on Commission decision text to *predict infringement* is reading the Commission's own framing of a case it already decided. The fix is a strict temporal split and pre-decision features only — and report both numbers, because the gap between them measures how much the system learned the institution rather than the law.**

Three more traps worth naming

Baseline. Always say what you beat. *84 percent* is meaningless until you know that always guessing the commonest class gives 80. Validating on your own benchmark. If you wrote the test items the score is optimistic — and that applies to your own Mens Rea work, which is why you volunteer it. And explanation methods are not the answer. SHAP and LIME routinely disagree with each other and are unstable under resampling; recent work names explanation multiplicity — explanations differing across repeated runs with the model, input and prediction all fixed. So do not offer them as your audit method. Offer counterfactual explanations instead — *had your turnover been below X you would not have been flagged* — because that form is testable and actionable by a court.

11. NLP and embeddings

Term	Definition	Worked instance
Corpus	The body of text you are working with	All Commission decisions 1977–2025
Document	One item in the corpus	One decision
Token	One processed unit of text	self-preferencing → `self`, `-`, `prefer`, `encing`
Tokenisation	Splitting text into tokens	*Where legal text breaks: Art 102 TFEU* can shatter into five meaningless pieces**
Stop words	Very common words often removed — the, of, and	*Careful: not* is a stop word in some lists, and removing it inverts legal meaning**
Lemmatisation	Reducing words to a base form	concealed, concealing, conceals → conceal
Named entity recognition	Finding and typing names in text	Google Ireland Ltd → organisation
TF-IDF	Scores a word highly if frequent here and rare elsewhere, so it finds distinctive words. Order is discarded	Fast, interpretable, and a genuinely hard baseline to beat on legal text
Static embedding	One vector per word, always the same	*GloVe — so consideration* is identical in contract doctrine and ordinary speech. The limitation in your summarisation paper**
Contextual embedding	The vector changes with the surrounding sentence	BERT-family. Solves exactly that
Cosine similarity	The angle between two vectors, ignoring length	1 = same direction, 0 = unrelated

\cos(\theta) = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|}

Cosine similarity is the dot product of two vectors divided by the product of their lengths.Why divide by the lengths: it removes document size from the comparison. A long decision and a short one on the same subject should count as similar; without this division the long one looks different merely for being long. And a precise point you can make: if you use squared Euclidean distance on vectors that have not been normalised, you are not computing cosine similarity, and the two rank results differently. That exact mismatch is in your own thesis — the text says cosine, the code builds a distance index on unnormalised vectors.

What an embedding space actually is

   A 2-D sketch of a space that really has
   384 or 768 dimensions:

        ^
        |        cartel
        |     collusion  *  *
        |   price-fixing *
        |                        merger
        |                      *   *  concentration
        |                              * acquisition
        |
        |  GDPR *
        |  consent *  * personal data
        |
        +-------------------------------------->

   Similar MEANINGS end up near each other,
   because they appeared in similar CONTEXTS
   during training. Nobody put them there.

   This is why retrieval works WITHOUT keyword
   match: a query about "bid rigging" lands
   near documents saying "collusive tendering"
   with no words in common.

   AND THE LIMIT, which matters legally:
   "nearby" means "used in similar contexts",
   NOT "legally equivalent". "Agreement" and
   "concerted practice" sit close together and
   are DIFFERENT legal concepts with different
   proof requirements. The geometry does not
   know that.

An embedding places text in a space where proximity reflects contextual similarity. That is what makes semantic search work without shared keywords, and it is also the limitation: distributional similarity is not legal equivalence, and two concepts can sit adjacent while requiring entirely different things to be proved.

12. Data-science craft — the ten things that break projects

About sixty percent of the work happens before any model, and cleaning is the largest block. Saying this is a credibility signal — it says you have run a project rather than read about one.
The unit of analysis decides what your result can mean. One row = one firm, one case, or one tender are three different studies. **And *case* measures enforcement, not conduct.**
The ecological fallacy: a correlation at district or country level licenses no claim about individuals. **Say *at district level* inside the sentence and the fallacy cannot sneak in.**
Plot rows per time period first, before anything else. Real-world processes change gradually; data-collection processes change abruptly. A step in the series is a change in administration, not in the world.
Three kinds of missing data. *Completely at random* — safe. *At random* — means random given what you already observe, so fixable by imputation or weighting. *Not at random* — depends on the missing value itself, and no fix exists from the data alone. **Your own crime data is the third kind: the dark figure *is* the selection.**
Rates or counts? Counts scale with population, so the biggest unit always looks worst. Per-capita rates are unstable with small denominators. And changing the denominator can reverse the finding with the numerator untouched — so declare it.
Simpson's paradox: a pattern can hold in every subgroup and reverse when pooled, with every number correct. Check the obvious subgroups before reporting any pooled comparison.
Entity resolution is a legal question, not a cleaning step — competition law acts on the *undertaking*, a single economic unit, not the legal person. So your name-matching threshold affects attribution of liability and the lawful size of a fine.
Measurement validity: name the construct, name the proxy, name the direction of the error. *I am measuring detected cartels, which is a sample selected on concealment, so my prevalence estimate is a lower bound.* Three clauses, and it closes off most attacks.
The design is the argument, not the regression. Difference-in-differences rests on parallel trends; regression discontinuity on nothing else jumping at the threshold; synthetic control on no spillover. **And all give *average* effects, which cannot carry individual attribution — that gap is the real frontier.**

13. The twelve sentences that cover almost anything

*Every LLM is deep learning, every deep-learning system is machine learning, all of it is AI — but never the reverse. Rule-based systems are AI and are not machine learning, and most tools agencies actually deploy are rule-based.*
*Statistics asks whether a pattern is real. Machine learning asks whether I can predict the next one. Data science is the craft around both — and in practice the hard part is usually that the same firm appears under five different names.*
*Nobody labelled the training data for a language model. It was shown text with the next word hidden and asked to predict it, billions of times — which is why the amount of data stopped being the constraint.*
*RNNs and LSTMs read a sentence strictly left to right, carrying what they can remember. Transformers look at the whole page at once. And the reason that mattered was not mainly accuracy but parallelism, which is what made internet-scale training possible.*
*Attention means that for every word, the model computes how much every other word matters to it and rebuilds that word's meaning as a weighted blend. But attention weights are not an explanation — they can be changed substantially while the output stays the same.*
*Encoder-only models read and classify; decoder-only models write; encoder-decoder models transform one text into another. Reaching for a generative model when you need a classifier is the most common and most expensive mistake in applied legal NLP.*
*Ground and generate: compute or retrieve the facts in a way you can show and reproduce, and let the model only do the prose. Never let it invent the input. I have built that twice.*
*Accuracy is the wrong metric for anything rare. If one tender in a thousand is rigged, a model that flags nothing is 99.9 percent accurate and worthless. Report precision and recall separately.*
*The two errors are not symmetric — investigating an innocent undertaking and missing a live cartel are different kinds of harm — so the threshold between them is a legal decision currently made by typing a number. Making that choice visible is most of the safeguard question.*
*The clearest documented leakage failure is in legal prediction: about 79 percent accuracy from judgment text written by the court after it decided, falling to roughly 58 to 68 percent when redone as genuine forecasting — and high accuracy from the judges' names alone. Competition law is about to make the same mistake.*
*Police records measure reporting, not offending. A cartel screen measures detection, not collusion. Same defect, and I have already built the toolkit for separating the two.*
*These methods can tell a regulator where to look and whether a rule worked. They cannot carry the burden of proof against an individual undertaking — and being clear about which of those two jobs a tool is doing is most of the safeguard question.*