Agent-based model
Controls actors, resources, networks, constraints, timing, state transitions and aggregate outcomes. The mechanisms are explicit, testable and reproducible.
Methodology
Make the assumptions visible. Then test them.
The Scenario Lab turns an uncertain, consequential decision into a governed computational experiment. Its purpose is not to produce a theatrical forecast. It is to compare interventions consistently, reveal failure paths and identify choices that remain acceptable when the world changes.
Lab charter
Fitness before sophistication.
A simulation is useful only when interaction, adaptation, feedback or path dependence materially affect a real decision. If a simpler analysis, forecast or facilitated workshop can answer the question, we use that instead.
Mechanism before spectacleA plausible animation is not a causal explanation.
Evidence before synthetic plausibilityGenerated behaviour is never promoted to observed fact.
Ranges before point predictionsUncertainty is part of the result.
Distribution before averagesEffects on different groups and ecosystems stay visible.
Participation is not simulationModelled stakeholders do not replace affected stakeholders.
Reproducibility by defaultInputs, versions, prompts, parameters and runs are traceable.
The suitability gate
First decide whether the decision should be modelled.
Before development, we screen for decision value, evidence fitness and possible harm. The result may be proceed, proceed as exploratory only, use a simpler method or do not model.
- Is there a defined decision and accountable owner?The options, deadline, consequences and authority to act must be explicit.
- Do interaction and feedback materially change the answer?Simulation must add something a spreadsheet or static forecast cannot.
- Is the evidence adequate for the intended claim?Exploratory questions can tolerate more uncertainty than high-impact quantitative claims.
- Could modelling create false legitimacy or material harm?We reject coercive, discriminatory or manipulative applications and restrict sensitive targeting.
- Can domain experts and affected stakeholders challenge the design?Real participation is required where legitimacy, lived experience or contested values matter.
Hybrid architecture
Rules where rules are reliable. AI where context matters.
Each layer has a distinct job. No component gets authority simply because it is sophisticated.
LLM layer
May interpret language, choose from an approved action set, negotiate inside limits or generate hypotheses. Outputs are structured, logged, versioned and benchmarked against simpler rules.
Evidence and people
Observed data, research, experts and stakeholder participation anchor the design and challenge the interpretation. Simulated people never replace real people.
Behavioural and psychometric layer
Model variation without turning people into labels.
Behaviour matters when trust, loss sensitivity, social influence, readiness or capacity change how a system responds. We represent those differences as uncertain group distributions—not fixed personality types. The public experience does not assess, diagnose or infer the personality of the website visitor.
Name the construct first
Every behavioural variable needs an intended meaning, use, population, observable mechanism and evidence source before it enters the model.
Use evidence in context
Where appropriate, combine licensed or open validated measures with observed behaviour, interviews and participation. Translation alone does not establish cultural equivalence.
Validate comparisons
Check reliability, construct validity, measurement invariance, subgroup error and whether the variables improve out-of-sample behaviour—not merely model fit.
What the public demo does
It samples illustrative behavioural parameters for 180 synthetic agents and applies the same population and shocks to three strategies. It asks no personal questions, creates no profile and does not transmit simulation controls to Consilience.
What commissioned work adds
Joint population distributions, empirical calibration, cultural adaptation, sensitivity across constructs and model structures, and real stakeholder challenge. Any psychometric instrument is reviewed for validity, permission and permitted use.
LLM boundaryLanguage models can expand hypotheses and simulate bounded interactions, but recent evidence shows that synthetic participants may compress heterogeneity, exaggerate effects and perform worse on sensitive social questions. We benchmark them against simpler rules and observed evidence; they do not stand in for a population.
Current practice notes · 2025–26
- OECD + World Economic Forum: AI in Strategic Foresight (2025)AI is increasingly used for scanning and scenario development, while reliability, bias and transparency remain material constraints.
- Nature Computational Science: replication of 156 scenario-based experiments (2025)LLMs can support pilots, but showed inflated effects and weaker results for socially sensitive studies; human validation remains necessary.
- ICML: uncertainty quantification for LLM survey simulation (2025)More synthetic responses can produce intervals that are narrower yet poorly calibrated when simulated and human populations differ.
- PNAS: SSDataBench evaluation of synthetic social data (2026)Model size alone does not guarantee population fidelity; joint distributions and heterogeneity require explicit evaluation.
- AERA · APA · NCME Standards for Educational and Psychological TestingValidity and fairness belong to a score interpretation for a defined use and population—not to a generic “validated test” label.
Evidence architecture
Every assumption carries a label.
A versioned evidence ledger records the source, date, owner, confidence, sensitivity and update requirement for important rules and parameters. LLM-generated material remains hypothetical unless independently supported.
Model claim ruleA model is decision-grade only for the question, context and permitted use for which it has been validated.
Observed
Directly measured, documented or reliably recorded for the system in scope.
Estimated
Statistically inferred or calibrated from declared data and methods.
Elicited
Provided by named experts or stakeholders with context and confidence captured.
Hypothesized
Plausible but not yet substantiated; tested transparently rather than presented as fact.
Experiment design
Keep worlds, choices and assumptions separate.
Scenarios are external worlds the decision-maker does not control. Strategies are available actions. Assumptions are uncertain mechanisms or parameters. Separating them makes comparisons interpretable.
What could surround us?
Current trajectory, credible shocks, compound events, tail risks, institutional change and alternative external conditions.
What can we choose?
Interventions, sequences, policies, operating modes, contingency actions and reversible first moves.
What might work differently?
Behavioural rules, adoption rates, network effects, thresholds, elasticities and alternative model structures.
Repeated stochastic runs reveal outcome distributions, pathways and failure conditions—not one polished number pretending to be the future.
Verification and validation
A ladder of checks before a claim.
The validation standard rises with the impact of the decision. A failed gate lowers the claim, changes the route or stops the study.
1. Conceptual validityDoes the structure reflect the relevant theory and domain understanding?
2. Code verificationDoes the implementation behave as the documented design requires?
3. Data validationAre sources, transformations, coverage and limitations correct?
4. Micro-validationAre agent-level behaviours plausible and evidence-supported?
5. Macro-validationCan the model reproduce relevant aggregate patterns?
6. CalibrationDo selected parameters fit agreed empirical targets?
7. BacktestingCan the model reproduce withheld or historical periods where appropriate?
8. Sensitivity and uncertaintyWhich inputs, mechanisms or structures drive the conclusion?
9. Stress and adversarial testingWhere does the model break, mislead or become unsafe?
10. Independent challengeCan domain experts and stakeholders identify important omissions?
Delivery gates
Human accountability at every consequential handoff.
- G0
Suitability and harm screen
Set the permitted claim level and governance needs—or decide not to model.
- G1
Decision contract
The sponsor approves the decision, system boundary, options, consequences, permitted use and stop conditions.
- G2
Evidence and model specification
Domain and data owners approve assumptions, access, unresolved gaps and the minimum credible structure.
- G3
Validation and claim review
The evidence supports the proposed claim—or the claim is reduced before scenario experiments proceed.
- G4
Decision rehearsal
The accountable owner reviews counter-evidence, trade-offs, fragile findings and monitoring triggers before acting.
- G5
Operation, change and retirement
Production models require monitoring, version control, review dates, rollback and an explicit end of life.
Implementation routes
From a portable study to a governed living lab.
The delivery surface follows the decision, security boundary and need for reuse.
| Route | Best suited to | Implementation | Primary outcome |
|---|---|---|---|
| Scenario Sprint | Early or urgent decisions with limited data | Portable prototype, bounded scenarios and facilitated review | Decision map and initial robust or fragile options |
| Decision Model | Material one-off strategy, policy or transition | Calibration and validation proportionate to available evidence, with a defined experiment programme | Auditable decision evidence and trigger plan |
| Living Lab | Recurring decisions in a changing environment | Approved data refresh, monitoring, model registry and operating cadence | A governed, repeatable capability refreshed on an agreed cadence |
| Participatory Lab | Public, contested or distributionally sensitive decisions | Co-design workshops, explicit contested assumptions and accessible explorer | Legitimate trade-off analysis and shared understanding |
| Independent Review | An existing internal or third-party model | Verification, sensitivity, bias and governance assessment | Assurance evidence and remediation priorities |
Responsible-use boundary
The model advises. People remain accountable.
Scientific and ethical caveatResults are conditional on the data, mechanisms and uncertainties represented. They must not be the sole basis for decisions affecting rights, safety, access to essential services or protected groups. Models of stakeholder behaviour do not replace consent, participation or lived experience.
We do not offer guaranteed forecasts, legal or regulatory conclusions, accredited assurance, automated high-impact individual decisions or evidence that does not exist. Sensitive projects require proportionate privacy, security, ethics and domain review.
A decision worth modelling?
Start with the question, not the technology.
Try the public model, then describe the real decision, deadline, consequences and affected system. We will identify the smallest credible route—or tell you when another method is more appropriate.