Reading and shifting the microbiome

Here is a problem that stalled microbiology for roughly a century. Most of the bacteria in your gut cannot be grown in a lab dish. They evolved for the exact conditions of the colon, no oxygen, a fixed temperature, a specific chemical soup, surrounded by hundreds of species they lean on, and pulled onto a Petri plate they simply die. If your only tool for studying a microbe is culturing it (growing a pure colony you can poke at), then the majority of the gut community is invisible, and for a hundred years it mostly was. MIC.1 and MIC.2 told you what the microbiome is and what it does. This lesson is about how we read it, and how, honestly, we can change it. It is the most computer-science-flavored of the three, because the fix for the reading problem is the one you already understand: stop trying to grow the organism and read its DNA instead.

The reading problem, and the sequencing fix

The move that broke the stall is called a culture-independent method, and it is what it sounds like: study the microbes without growing them. You cannot coax the organism onto a plate, so you skip the organism and go straight for its information. Take a raw sample (a smear of gut contents), extract all the DNA in it, and sequence that DNA directly.

If you took the bioinformatics track, this is the payoff of its opening idea. BIO-1 framed a biological sequence as data, a plain string of letters, and BIO-2 and BIO-3 turned reading and comparing those strings into a discipline. A sequencing read is exactly that: a short string of A, C, G, and T copied off one fragment of DNA in your sample. You do not need the cell alive or even intact, only enough of its DNA and a machine that reads letters. That single shift, from culturing cells to reading strings, made the dark majority of the microbiome legible. Two flavors of it do most of the work, and they answer different questions.

16S rRNA: a molecular barcode

Start with the cheap, targeted one. Say you want a fast census, a list of who is present and in what proportion. You need one gene that meets two demands at once: it must be present in every bacterium, so you can find it in any species, and it must differ just enough between species to tell them apart. Those demands pull in opposite directions, and one gene threads the needle, the 16S ribosomal RNA gene, usually just called 16S.

Derive why it works. This gene encodes a piece of the bacterial ribosome, the machine that builds every protein (the translation step of the central dogma from S5). Because the ribosome is essential and ancient, every bacterium carries the gene, meeting the first demand. Part of it is load-bearing, so change those regions and the ribosome breaks, and they barely differ across all of bacterial life. They are conserved, a fixed frame you can always grab onto. Other stretches tolerate change and have drifted apart species by species over evolutionary time. They are the variable regions, and they carry a pattern unique to each lineage. Conserved enough to always find, variable enough to identify, which is exactly what a barcode is.

So the census writes itself. Sequence the 16S gene from everything in the sample and you get a pile of 16S reads. For each read, ask which known organism it came from by comparing it against a reference database of 16S sequences from named species (curated collections like SILVA or Greengenes). Count the matches and you have your census: who is present, and in what proportion. One honest limit lives in that word identify. A short-read 16S survey usually pins down the genus, and often the species, but it frequently cannot separate very closely related species, whose variable regions differ by only a letter or two.

That matching step is the alignment idea the bioinformatics track built in BIO-3: line a read up against a reference and read off where they agree and where they diverge. In the widget below, treat the top strand as a slice of a reference 16S sequence and the bottom as a read pulled from a gut sample. Where they match, the read belongs to that organism. Where they mismatch, it is from something else, perhaps a close relative whose variable region has drifted a few letters. Predict where the differences will fall before you run the alignment.

matrix[i][j] = max(
  matrix[i-1][j-1] + s(a[i-1], b[j-1]),   // diagonal: align the two residues
  matrix[i-1][j]   + gap,                 // up: a gap in sequence b
  matrix[i][j-1]   + gap                  // left: a gap in sequence a
)
s(x, y) = match when x == y, else mismatch

// A thin sketch of needlemanWunsch (global). Local alignment is the same
// loop with a 0 floor and a traceback that starts at the largest cell.
function fill(a: string, b: string, match: number, mismatch: number, gap: number) {
  const m = a.length;
  const n = b.length;
  const matrix: number[][] = [];
  for (let i = 0; i <= m; i++) matrix.push(new Array(n + 1).fill(0));

  // Seed the borders: a run of k gaps costs k * gap. (Local leaves these at 0.)
  for (let i = 1; i <= m; i++) matrix[i][0] = i * gap;
  for (let j = 1; j <= n; j++) matrix[0][j] = j * gap;

  // Fill. Row i indexes a, column j indexes b.
  for (let i = 1; i <= m; i++) {
    for (let j = 1; j <= n; j++) {
      const s = a[i - 1] === b[j - 1] ? match : mismatch;
      const diag = matrix[i - 1][j - 1] + s;   // align a[i-1] with b[j-1]
      const up = matrix[i - 1][j] + gap;        // a gap in b
      const left = matrix[i][j - 1] + gap;      // a gap in a
      matrix[i][j] = Math.max(diag, up, left);
      // Smith-Waterman (local) instead: Math.max(0, diag, up, left)
    }
  }

  // The global score is the bottom-right corner.
  return matrix[m][n];
}

Shotgun metagenomics: not just who, but what

16S gives you a guest list and little else, because you read only one gene. It can say a relative of a known species is present, but not what that bacterium can do. To learn capability, you read everything. Shotgun metagenomics shears all the DNA in the sample into countless fragments and sequences the lot, from every organism at once. Now you are not naming members, you are cataloguing their genes: which metabolic pathways the community carries, which vitamin-making genes, and, critically, which antibiotic-resistance genes. It also reaches members that a 16S survey (bacteria and archaea) misses, eukaryotes like fungi and the viruses, which carry no 16S gene at all, and can often separate strains, not just species. The cost is far more sequencing and much heavier computation. 16S is a quick head count of who showed up. Shotgun is a full inventory of what everyone can do.

The data trap: percentages are not counts

Now a pitfall that has burned real studies, and one the statistics lesson BIO-5 prepares you for. Sequencing does not hand you the absolute number of each species. It hands you reads, which you convert into relative abundances, the fraction of reads belonging to each taxon. By definition those fractions sum to 100 percent, and that innocent detail is the whole trap. It has a name, compositional data, and the consequence is that the parts are not independent.

Watch it go wrong. Suppose one species' absolute numbers explode after a treatment. It grabs a bigger share of the reads, so every other species' share drops, even though not one of them lost a single cell. Read the table naively and you will report that a dozen species declined, when in truth one grew and shoved the rest down in the percentages. To tell an absolute change from a relative one you need an absolute anchor, a known amount of DNA spiked in as a ruler, or a direct cell count. This is the BIO-5 warning in textbook form: a number that looks like a measurement of one thing (how much of species A there is) is really a measurement of a ratio (A's share of the whole), and reading it as the wrong quantity produces confident, published nonsense.

A programmer's frame: profiling a system you cannot debug

Here is the model that makes all of this click, followed at once by where it lies to you, because an analogy without its limit is a bug.

Studying the microbiome is profiling a live distributed system you can never attach a debugger to. You cannot pause it or inspect a running process directly. All you can do is sample its logs and infer, from the log lines, which services are up and what they are doing. In that frame, 16S sequencing is a lightweight service-discovery ping: cheap and fast, it tells you which services are running but little about them. Shotgun metagenomics is a full process-and-capability dump: heavy, but it reports what each process can actually do. And the compositional-data trap is the familiar bug of reading percentages instead of absolute counts, mistaking "this service is 20 percent of the log lines" for "this service did 20 percent as much work as it used to," when the totals underneath have shifted.

Now the failure edges, three of them. The analogy oversells how clean the data is. Sequencing reads are noisy, carrying real errors in the letters. The reference databases you match against are badly incomplete, because a large fraction of gut organisms have never been named, so many reads match nothing or the wrong nearest neighbor and get mislabeled. And, worst of all, relative abundance is not absolute abundance. A profiler on a system you own gives you ground truth. Here you infer the run state of a system nobody fully catalogued, from a lossy sample, through an incomplete lookup table. Keep the picture for intuition, and keep those three holes in view.

Shifting the microbiome: four levers, honestly

Reading it is half the story. Can we deliberately change it? Yes, with four tools, from the bluntest to the most complete, and for each the honest evidence rather than the marketing.

Antibiotics: the sledgehammer

An antibiotic kills or halts bacteria by sabotaging machinery only bacteria have, such as building a cell wall or running the bacterial ribosome. That specificity is why, as MIC.1 stressed, antibiotics do nothing to a virus, which owns none of that machinery. They are one of the great lifesaving inventions, and they are blunt. A broad-spectrum antibiotic cannot aim: it hits the helpful residents along with the target, which is how a course can tip a gut into the dysbiosis MIC.2 described and let an opportunist like Clostridioides difficile bloom in the cleared space. That collateral damage is one cost. The larger cost is what selection does next.

Here the fast-evolution engine from MIC.1 turns against us. A bacterial population is enormous and divides in minutes, so before you ever apply a drug, a handful of cells almost certainly already carry a mutation or a borrowed gene that blunts it. Apply the antibiotic and you do exactly what selection does best (the mutation-and-selection engine from S7): kill the susceptible majority and hand the rare survivors an empty field to multiply into. Within days the population can be mostly resistant. Worse, because bacteria trade genes sideways (the horizontal gene transfer from MIC.1), a resistance gene that arose in one species can spread to others without being reinvented. Every dose, in medicine and in agriculture, is a selection pressure breeding the very bacteria that defeat it. This is antibiotic resistance, one of the most serious slow-moving problems in medicine: we are running short of drugs that still work, and evolution disarms the new ones almost as fast as we deploy them. It is also why taking an antibiotic for a viral cold, where it cannot help, is not harmless: it breeds resistance for no benefit.

Probiotics: live bacteria you swallow

A probiotic is a dose of live bacteria you ingest hoping they help. The honest verdict is modest, strain-specific, and often wildly oversold. Many swallowed strains are transient: they pass through without ever taking up residence, because the seats are full, and the colonization resistance from MIC.2 works against a newcomer just as well as against a pathogen. Where a benefit is real, it tends to be tied to one particular strain and one situation (easing some cases of antibiotic-associated diarrhea), not the sweeping "supports gut health" on the label. Run every probiotic claim through the evidence standard MIC.2 built from LON-1.2 and BIO-5: the right organism, a real outcome, a proper control, cause rather than mere correlation. Held to that bar, most capsule claims deflate.

Prebiotics: feed the bacteria you already have

A prebiotic flips the strategy. Instead of adding new bacteria, you feed the helpful ones already living in you, which in practice means fermentable fiber, the same fiber the fermenters in MIC.2 turn into short-chain fatty acids. Rather than parachuting in strangers who cannot get a foothold, you hand your resident community more of what it eats and let it grow. This is why the best-supported piece of microbiome advice is also the least glamorous: eat a wide variety of plant fiber. It is a prebiotic strategy, and it works on the community you already have instead of betting on colonists who mostly wash out.

Fecal microbiota transplant: move the whole community

The most complete lever, and the most surprising, is a fecal microbiota transplant (FMT): transferring stool from a healthy donor into a patient. It sounds crude, but it does what no capsule can, moving the entire community at once, hundreds of species in their real proportions, instead of one isolated strain. For one condition it is strikingly effective, recurrent C. difficile infection, the post-antibiotic bloom from MIC.2. When the resident community has been wiped out and C. difficile keeps returning, re-seeding a whole healthy community restores colonization resistance, with cure rates well above what more antibiotics achieve. For almost everything else (obesity, inflammatory bowel disease, mood, metabolic disease) FMT is experimental, with no established therapy yet. The pattern is familiar: one solid, controlled result surrounded by promising correlations that have not earned the same confidence.

The beautiful callback: a bacterial immune system we borrowed

End with the most elegant loop in the whole track. One of the most powerful tools in modern biology, CRISPR, was not invented by anyone. It was discovered already running, inside bacteria, doing a job you have a name for by now: it is an immune system.

Work through it. As MIC.1 noted, viruses hijack host cells to copy themselves. The ones that specifically infect bacteria are called phages. Bacteria fight back with CRISPR. When a bacterium survives an attack, it snips out a short piece of the invader's DNA and files it in a dedicated stretch of its own genome, a lineup of snippets from past attackers. That stored snippet is a molecular memory of an infection the cell lived through. On a later attack by the same virus, the bacterium reads the matching snippet out into a small guide molecule, and a cutting protein (called Cas) follows the guide to the virus's DNA and slices it in two, before it can take over.

Now read those properties back. The response is specific: it targets one sequence, not everything in reach. It is learned from past exposure: the cell only stores snippets of viruses it has actually met and survived. And it is remembered: the snippets persist and pass to daughter cells. Specific, learned, remembered, the exact defining traits of an adaptive immune system. Bacteria, the "simple" prokaryotes from MIC.1, evolved genuine adaptive immunity, complete with memory, long before any animal did.

Here is the twist that earned a Nobel Prize. The bacterial system aims its cutter wherever the stored guide tells it to, so researchers asked the engineer's question: what if we supply our own guide, one we design, pointing the cutter at any DNA sequence we choose, in any organism we like? Do that and CRISPR becomes a programmable pair of molecular scissors. Type in a target sequence, and Cas cuts the genome exactly there, after which the cell's own repair machinery can be nudged into disabling a gene or pasting in a new stretch of DNA. That is CRISPR gene editing: a defense bacteria evolved to survive viruses, turned into the most precise, cheapest, most widely used gene-editing tool we have ever held. We did not design it. We found it running in the microbial world and borrowed it, a fitting note to end a track on microbes.

That is the shape of this whole track, and of the enrichment phase it closes. The spine taught you the cell, DNA, the central dogma, and the mutation-and-selection engine. The systems track (SYS) zoomed out to show those cells cooperating into tissues, organs, and a body that holds itself steady. The immunity track (IMM) showed a body that defends itself and, remarkably, learns. And this microbiome track (MIC) showed that you were never a single organism to begin with, but an ecosystem, one we can finally read by sequencing and cautiously shift with the four levers above, using tools like CRISPR that the microbes wrote first. Keep the evidence standard you built along the way. It is the most durable thing this track gave you, and it will outlast every headline.