← All posts

How to Pick the Right LoRA Epoch (Not the Last One)

Wavemaker saves every training epoch with comparable sample grids—here is how to choose a checkpoint that generalizes instead of overfitting your dataset.

Illustration for: How to Pick the Right LoRA Epoch (Not the Last One)
Conceptual illustration — product screenshots appear in the guide below where they help you click through.

Pick the right LoRA epoch by comparing Wavemaker’s per-epoch sample grids (same prompts, same seeds), choosing the checkpoint that generalizes, and promoting it as a version—never assuming the final epoch is best.

Training Studio Train tab checkpoint grids for epoch comparison

Train tab sample grids — compare epochs before promoting a version.

The overfitting trap

Local training folders encourage “use the last .safetensors.” That habit exists because manual comparison is tedious. Wavemaker automates comparison visuals but refuses to automate the decision—because the last epoch routinely:

  • Memorizes exact training backgrounds
  • Locks wardrobe from captioned photos
  • Sharpening teeth/skin into uncanny territory
  • Collapses when the trigger token appears without other cues

Training Studio’s opinionated default: human or evaluation-gated pick.

How sample grids stay apples-to-apples

Each epoch checkpoint renders a grid using:

  • Fixed prompt pack (character/style/product appropriate)
  • Locked seeds per cell

So differences reflect weights, not RNG luck. This is the fairness layer that makes epoch choice faster than loading ten files into Comfy blind.

When grids look noisy, fix dataset preparation before retraining—no grid magic saves bad captions.

Step-by-step epoch review

1. Wait for awaiting_checkpoint

Training jobs emit progress until epochs complete. API consumers receive training_job.awaiting_checkpoint webhooks—pause automation until a pick happens.

2. Scan early vs late epochs

SignalOften best epoch regionWarning sign
Character identityMid trainingLate epoch skin/textures
Style LoRASlightly earlierLate epoch burns palette
Product SKUMid-lateEarly epoch too weak

There is no universal epoch number—only grids for your dataset.

3. Stress-test the trigger token

Read prompts in the grid cells. Does the character appear when the token stands alone? If only full training sentences work, you may be under-trained or over-captioned with duplicate phrasing.

4. Pick → version pin

Selecting a grid cell creates the asset version workflows bind. Unpicked epochs countdown 14 days—set calendar reminders on long approval chains.

5. Optional blind A/B

Feelings lie; pairwise votes do not. Blind A/B testing LoRA checkpoints compares versions and weights with identities hidden until you commit.

Relationship to pricing and credits

Training holds cover epoch production; grids are part of training UX. Re-training because you picked wrong costs another ~60 credits for image LoRA at default steps—epoch discipline saves money.

Video jobs (~200 credits base) hurt more when you skip grid review—see video LoRA training for Wan.

When the “best” epoch still fails in workflow

Epoch pick optimizes training objectives; inference adds node weight, base model, and prompt structure. If grids looked great but workflows drift:

  1. Tune LoRA strength
  2. Confirm binding uses the intended version pin
  3. Verify base model compatibility (Flux/SDXL vs krea2 honesty)

API: pick_checkpoint

MCP and REST expose explicit pick operations—agents must not default to max(epoch). Document your org policy: e.g., “always require human click on grid” or “promote only after Evaluate winner.”

Myths

“Higher epoch = smarter model.” Training loss can keep improving while generalization reverses.

“One grid cell looked bad—discard epoch.” Check multiple cells; occasional seed interactions happen—look for patterns.

“I’ll fix it in post with img2img.” Fix epoch + dataset; post hoc editing does not scale to video pipelines.

Casting and multi-character projects

Epoch picks are per asset. Multi-role productions should pick each role, bundle via cast packs, and keep general video consistency patterns in /blog/consistent-characters-across-scenes/ without conflating epoch science with casting logistics.

Side-by-side review ritual

Block thirty minutes with no other task. Open three epochs maximum per pass—early, middle, late—to avoid decision fatigue. Zoom grids to faces or SKU labels depending on preset; thumbnail view hides overcook artifacts that export failures reveal.

Screenshot your pick with epoch number for audit trails. When legal asks “which model generated this ad,” version pin plus epoch metadata answers.

Unpicked epochs expire at fourteen days—calendar it. If compliance requires retaining every checkpoint, download artifacts through supported export paths before expiry rather than expecting infinite host storage.

Re-training vs keeping an old epoch

When dataset v2 adds only five photos, compare via blind A/B before retiring v1. Small dataset deltas sometimes help; sometimes hurt. Grids tell you which without narrative debate.

Teaching stakeholders why last epoch loses

Producers unfamiliar with LoRA expect monotonic improvement. Share a one-slide story: training loss ↓ while generalization ↑ then ↓. Wavemaker’s UX encodes that story visually—use grids in stakeholder reviews instead of final PNGs alone.

Grid literacy by preset

Character grids — scan eyes, teeth, ears, hairline; overcook shows waxy skin and asymmetric pupils. Product grids — scan label edges, cap geometry, logo kerning; overcook sharpens fake microtext. Style grids — scan palette drift and texture repetition; undercook looks like base model with faint filter. Video motion samples — scan temporal stability on hair and fabric; a pretty first frame may still flicker.

If one cell in a grid fails but others pass, note the prompt for that cell—it may be out-of-distribution rather than epoch failure.

Regulated industries may require retaining which checkpoint shipped in market. Wavemaker version pins satisfy that when workflows are frozen at publish. Document epoch number in your asset changelog when brand asks “what changed between v2 and v3.”

Automation boundaries

Scripts may poll job status and download grid thumbnails via API, but epoch selection should remain an explicit pick_checkpoint call with human or evaluation approval. Auto-max-epoch violates platform design and voids the fairness grids exist to provide.

When grids lie (rare)

Bad fixed prompt packs that never appear in production can mis-rank epochs. Customize prompt packs in Evaluate after grid pick if production prompts differ systematically. Dataset caption leakage can make mid epochs look perfect on grid prompts that repeat training text—hold-out phrasing matters.

Pairing with train LoRA online economics

Each retrain costs credits; picking wrong epoch and discovering late wastes generation spend too. Spend review time before scaling slug-runs—epoch review is cheaper than remediating a live campaign.

Field notes for checkpoint reviewers

Build a habit: early epoch, mid epoch, late epoch—never more than three grids per sitting. Drink water; overcook detection is visual and fatiguing. When two epochs tie, run blind A/B instead of flipping a coin. If your org uses cast packs, epoch pick is still per asset—do not copy epoch numbers across roles. API integrators: log pick_checkpoint responses in your asset registry; slug-runs should refuse if pick missing. When retraining after client adds photos, compare v1 and v2 grids on the same prompt pack—change one variable. Educate finance that last epoch auto would create support debt; manual pick is product policy aligned with quality. Store screenshots of winning grid in ticket system for regulated ads. If grids uniformly bad, return to dataset preparation—no epoch rescues empty captions.

Workshop scenarios

Late epoch looked sharp in meeting but failed Evaluate — demote, pick mid epoch, promote after blind win. Two epochs tied on grids — run Evaluate with held-out captions, not training text. Client added 5 photos mid-project — retrain v2, A/B against v1 before swapping cast pack pin. API automation almost picked max epoch — block policy; human pick or evaluation promote only. Video grid pretty, playback flickers — reject epoch, adjust clip dataset captions, retrain before bind. In each case, version pin + changelog beats oral tradition.

Extended production checklist

Calendar epoch review within twenty-four hours of grid availability so unpicked checkpoints are not lost to fourteen-day expiry. Screenshot the winning grid with epoch label for regulated industries. When stakeholders insist “use the last one because training finished,” show mid vs late grid on a held-out prompt—visual beats argument. If API automation orchestrates training, hard-block pick_checkpoint until a human or Evaluate promotion event fires—never max epoch. Pair epoch decisions with strength tuning smoke tests before cast pack updates. Document pick rationale in Library changelog entries. When retraining, run blind A/B between old and new versions rather than assuming improvement. For video LoRAs, watch full clip playback, not first-frame thumbnails. Link broad consistency questions to /blog/consistent-characters-across-scenes/. Keep /lora-training pricing in budget docs when retrain loops multiply.

Epoch review is a quality gate, not bureaucracy—it protects spend on generation and brand reputation after training completes.

Where to go next

Frequently asked questions

Why doesn't Wavemaker auto-select the final LoRA epoch?
Because the last epoch often overfits—perfect on training photos, brittle on new prompts. Every epoch saves a checkpoint with identical grid prompts and locked seeds so you can compare fairly and pick a version explicitly.
How long do unpicked checkpoints stay available?
Fourteen days. After you pick a winner, unpicked epoch checkpoints expire to control storage; your picked version remains in Library as a durable asset version.
What should I look for in a sample grid?
Identity stability across grid cells, absence of plastic skin or melted features, trigger token working without copying exact training outfits, and no sudden style drift between adjacent epochs.
Can I change my pick later?
Yes—train produces multiple epochs until expiry. You can pick a different epoch as a new version, then run blind A/B evaluations to promote a default for workflows.