DisInforMeter Scale Development and Validation

Authors

Masked for Peer Review

███████ ████████

██████ ██████████

██████████ ████████

██████ ███████

What this report is

This site is the annotated analysis log for the DisInforMeter — a multidimensional psychometric instrument that measures vulnerability to Foreign Information Manipulation and Interference (FIMI). It accompanies the manuscript and is built so that every fit index, table, and figure in the paper can be traced back to the exact data and code that produced it.

The instrument is being developed across five studies spanning Lithuania, Germany, and a multinational wave. Each study contributes evidence to a shared psychometric pipeline — from initial item generation and first-order measurement, through higher-order construct representation and harmonised FIMI prediction, to deployment adaptation, invariance, and external nomological-network validation. This landing page is the map: it explains the eight-stage pipeline, shows which study contributes evidence to which stage, documents the different operationalisations of the FIMI outcome and of the DisInforMeter scale itself, and links each stage to the page that performs it.

The construct in one figure

The working model treats FIMI receptivity as the downstream criterion predicted by four broader DisInforMeter dimensions: ideological grievances (THREAT), abandonment by the state, media, and allies (ABAND), pragmatic accommodation of authoritarian foreign powers (FEAR), and admiration of those powers (SUPF). Each broader dimension is built from first-order subscales established in Study 1 and refined in subsequent waves. How those broader dimensions should be represented is an empirical construct-modelling question — correlated first-order factors, reflective second-order factors (=~), pure formative composites (<~), or hybrid formative–reflective / MIMIC-style representations (=~ + ~). Whether a chosen representation predicts FIMI well is a separate structural-prediction question answered only after the FIMI criterion has been operationalised and harmonised.

Figure 1: DisInforMeter conceptual model. Four higher-order predictors of FIMI receptivity, each composed of first-order subscales. FIMI itself is measured by four planned operationalisations (full, Russian-origin, Chinese-origin, and the 4-item source-balanced Short FIMI) — see the FIMI operationalisations section below.

Two things are worth emphasising right away, because they are the two areas where the project departs from a textbook scale-development paper and where the rest of this site spends most of its effort:

  1. The DisInforMeter scale (the predictors) exists in several versions. A long item pool was fielded in Study 1 (whose higher-order structure could only be explored as correlated first-order factors, a reflective second-order model, or — without dedicated affective indicators — an exploratory pure formative composite); consolidated into a long instrument in Studies 2 / 3 / 5 that adds direct higher-order indicators and therefore admits hybrid formative–reflective (MIMIC-style) higher-order models; then reduced to a short form in Study 2 and deployed multinationally in Study 4. The Study 4 deployment merged three pairs of items into single items, so the multinational short scale is not strictly identical to the S2/S3 short scale — those combined items need their own validation step.
  2. The FIMI outcome (the DV) also exists in several versions. The full battery is 15 items in S1 and 8 items in S2–S4, but the S2+ 8-item battery is a strict subset of the S1 battery, simply renamed — news1 in S2+ is the same content as news3 in S1, and so on. All items split by origin into Russian FIMI and Chinese FIMI: in S2–S4 that gives 5 Russian + 3 Chinese items, while in S1 the seven dropped items are all Russian-origin too, giving 12 Russian + 3 Chinese. The Short FIMI uses the same four source-balanced items in every wave, just renamed across waves (S1: news4, news9, news14, news15 = S2+: news2, news5, news7, news8).

Both points are documented in dedicated sections further down, and both have their own step in the analytic pipeline.

The five studies

Table 1: Studies contributing to the DisInforMeter development pipeline.
Study Country Fielded N (planned / target) Primary contribution
S1 Lithuania Dec 2024 ~700 Initial item pool, exploratory dimensionality, first-order and exploratory higher-order construct models, and the long 15-item FIMI criterion
S2 Lithuania Mar 2025 ~600 Consolidated first-order measurement, higher-order construct modelling, harmonised 8-item FIMI criterion, structural prediction, and short-form prototype
S3 Germany May 2025 ~800 German replication of S2 first-order measurement, higher-order representation, harmonised FIMI prediction, and short-form performance
S4 Cross-national (16) Dec 2025 ~8,000 Completed first-pass multinational short-form deployment analysis with harmonised FIMI prediction, cross-country invariance, combined-item bridge checks, COND effects, and external-anchor probes
S5 Lithuania (VU SONA) Mar-Apr 2026 ~250 Completed first-pass validation wave for long- and brief-form measurement, external nomological-network anchors, and exploratory S2/S5 invariance; no FIMI items administered

All five studies have been ingested and cleaned in 01-data-preparation.qmd and described in 02-descriptives.qmd. S1–S5 now have first-pass modelling or validation pages. S4 provides the multinational short-form deployment analysis, cross-country invariance, harmonised FIMI prediction, combined-item bridge checks, COND effects, and external-anchor probes. S5 provides long-/brief-form measurement, external validation, and exploratory S2/S5 invariance; it deliberately has no FIMI outcome. The completed cross-study synthesis pages under 04-cross-study/ now collect these per-wave results into the five stage-owner analyses described below.

The eight-stage psychometric pipeline

End-to-end the project runs eight conceptual stages. The order matters: the project first establishes the item and measurement structure, then evaluates higher-order construct representations, then defines the FIMI criterion that structural prediction will consume. This keeps the construct-representation question separate from the criterion-prediction question.

  1. Item generation, item pool assessment, and exploratory dimensionality. Generate the candidate item set from theory and qualitative work; clean the item pool; inspect distributions, redundancy, and missingness; and estimate exploratory dimensionality analyses to identify plausible first-order and higher-order patterns. This stage can suggest candidate architectures but does not commit the project to one final SEM representation. Owner pages: 01 — Data Import and Preparation, 02 — Descriptives, and 03a — Study 1.

  2. First-order measurement validation. For each first-order construct block (THREAT, EXPL, GAY, MIGR, ABAND, ABANH, ABANM, ABANF, FEAR, PRAGR, PRAGCH, SUPR, SUPCH, SUPF, SUPFCH, GEN, CET; legacy S5 scale-registry names retained for Layer-2 columns: ABAN_STATE, CEN, UNP, SUP_GEN), estimate confirmatory measurement models and document loadings, reliabilities, AVE, item refinements, and local fit. Higher-order models are not interpretable if these first-order blocks are unstable. Owner pages: 03a, 03b, 03c, 03d, and 03e; cross-study synthesis in 4a Measurement architecture.

  3. Higher-order construct modelling. Compare how broader DisInforMeter dimensions should be represented: correlated first-order factors, reflective second-order factors (THREAT =~ EXPL + GAY + MIGR), pure formative composites (THREAT <~ EXPL + GAY + MIGR), or hybrid formative–reflective / MIMIC-style specifications (THREAT =~ th9 + th4 + th6; THREAT ~ EXPL + GAY + MIGR). Throughout this stage, <~ is the formative/composite operator and ~ is a structural regression operator. This stage is about construct representation, not FIMI prediction. Owner pages: 03 — Models Overview for the syntax catalogue and 03a through 03e where study-specific evidence exists; cross-study synthesis in 4a Measurement architecture.

  4. Outcome operationalisation and cross-study harmonisation. Define the primary FIMI criterion and the planned comparability/sensitivity scores before interpreting prediction results. The named primary criterion is Full FIMI (FIMI): the full news-battery receptivity score available in each FIMI wave. Three planned alternatives — Russian-origin FIMI, Chinese-origin FIMI, and Short FIMI — are used to test source-specific and cross-study comparability, not to replace the named primary criterion. This stage documents item identity across S1 and S2+ naming, the Russian vs. Chinese FIMI items, and the minimal short version across studies. Owner pages: 03a through 03d, with synthesis in 4b FIMI DV operationalisation.

  5. Structural prediction models. Estimate whether first-order or higher-order representations predict the harmonised FIMI criterion(s). This includes first-order paths such as FIMI ~ CET + EXPL + MOR + ..., higher-order paths such as FIMI ~ THREAT + ABAND + FEAR + SUPF, and sensitivity of paths and R² across the planned FIMI scores. Good measurement fit is not by itself evidence of predictive utility, and strong FIMI prediction is not by itself evidence that a representation is psychometrically best. Owner pages: 03a through 03d; S5 is excluded because it does not administer FIMI; cross-study synthesis in 4c FIMI prediction.

  6. Scale reduction and deployment adaptation. Derive and evaluate a shorter DisInforMeter form for field deployment, including the S2 short-form prototype, S3 replication, and the combined items deployed in S4 (pa2_S4 = pa2 + pa3, pa4_S4 = pa4 + pa5, gen3_S4 = gen3 + gen4). The target is not simply “short-form validation”; it is transportable measurement under survey-length and multinational deployment constraints. Owner pages: scale-reduction derivation on 03b, replication on 03c, deployment adaptation on 03d, and long-form checks on 03e; cross-study synthesis in 4d Deployment and invariance.

  7. Measurement invariance. Test configural, metric, scalar, and where possible strict invariance across time and countries. First-order invariance is the priority because higher-order invariance is only meaningful after the lower-level measurement blocks are stable; higher-order invariance is treated as a separate follow-on question when the required first-order evidence is adequate. Owner pages: 03d fits the S4 country-level deployment ladder; 03e fits the exploratory S2/S5 temporal ladder; 4d Deployment and invariance synthesises S2/S3, S2/S5, S4 country-level, and staged S1/S2 evidence.

  8. External validity and nomological network validation. Relate the DisInforMeter to theoretically adjacent and distinct constructs. S4 provides civic and political anchors such as institutional trust, EU support, democratic importance, and political efficacy; S5 provides the deeper preregistered validation network covering CMQ, populist attitudes, RWA, ITT, I-PANAS-SF, institutional trust, and feeling thermometers. Owner pages: 03d, 03e, and synthesis in 4e Nomological network.

Coverage matrix: study × conceptual stage

The matrix below shows where each of the eight stages is performed, and at what status. This is the fastest way to find the answer to “where do I find X?” — pick a stage in the row, find the study in the column, follow the cell to the corresponding analysis page.

Figure 2: Where each conceptual stage is performed across the five studies after the first-pass S4 and cross-study analyses. ● = completed, ○ = planned, blank = not in scope for that study.

The methodological pipeline

Read through the figure as a staged pipeline: item-pool and dimensionality work precede first-order validation; higher-order construct representation is evaluated before the FIMI criterion is harmonised; prediction then consumes those harmonised outcome definitions; scale adaptation, invariance, and external nomological validation test whether the instrument can be deployed and interpreted beyond the original setting.

Figure 3: Methodological pipeline grouped by conceptual sequence. Outcome harmonisation precedes structural prediction, and higher-order construct representation is separated from FIMI prediction.

The DisInforMeter scale: from long form to multinational short form

The scale itself has gone through four versions, summarised below. The Study 4 short form is not identical to the Study 2/3 short form: three of its items are mergers of two S2/S3 items into one, so they need a dedicated validation step before the multinational evidence can be interpreted as evidence on the same construct.

Figure 4: DisInforMeter scale versions. The long S1 item pool is consolidated into a long instrument in S2/S3/S5 (which adds direct higher-order indicators and therefore admits hybrid MIMIC higher-order specifications, not available on S1), then reduced to a short draft in S2 and replicated in S3. The multinational deployment in S4 carries three combined items relative to S2/S3, which need to be validated against the original split items in S2/S3 and S5.
Table 2: DisInforMeter scale versions used in this project.
Version Studies Description
Long (S1 formative pool) S1 Broad formative pool, canonical labels: EXPL, GAY, MIGR, THREAT, ABANH, ABANF, ABAND, PRAGR, PRAGCH, FEAR, SUPR, SUPCH, RULE (legacy S1 names USE/LONE/SAFE/LEFT/ANXR/ANXK/SUPK are now retired in the sem_specs catalogue).
Long (consolidated reflective) S2, S3, S5 Consolidated reflective measurement model with canonical labels: GEN, THREAT, EXPL, GAY, MIGR, ABAND (+ ABANF, ABANH, ABANM), FEAR, PRAGR, PRAGCH, SUPR, SUPCH, SUPF, SUPFCH, CET.
Short (S2 derivation) S2, S3 First short-form draft derived in S2 and replicated in S3. Selects 2–3 highly diagnostic indicators per higher-order construct (e.g. THREAT = th9/th4/th6; ABAND = ab2/ab4/ab7; FEAR = pa2/pa6).
Short (S4 deployed, with combined items) S4 Multinational deployment of the S2 short form, with three indicators merged into single items to compress survey length: pa2_S4 = pa2+pa3_S2/3; pa4_S4 = pa4+pa5_S2/3; gen3_S4 = gen3+gen4_S2/3.

Combined items deployed in S4 — why they need validation

The Study 4 short form was designed for the multinational wave and trimmed three pairs of overlapping items into single anchors. The mapping is documented on the per-page (03d) but listed here for cross-reference:

Table 3: Combined items in the Study 4 multinational deployment.
S4 item Construct Combines (in S2 / S3) Rationale
pa2 FEAR (fear / anxiety) pa2 + pa3 Two near-identical worry items in S2/S3 were merged into a single anchor for the S4 short form.
pa4 FEAR (fear / anxiety) pa4 + pa5 Two anxiety-about-conflict items in S2/S3 were merged into a single anchor for the S4 short form.
gen3 GEN (general DisInforMeter) gen3 + gen4 Two overlapping general-receptivity items in S2/S3 were merged into one item for S4.

The combined-item validation is part of Stage 6 (scale reduction and deployment adaptation) and lives on two pages. Inside Study 2 / Study 3 we score the row-mean of each combined pair (e.g. mean(pa2, pa3)) and check (a) that its loading on the parent factor is statistically equivalent to that of the original items, and (b) that its correlation with FIMI is preserved. Inside Study 5, where every item is administered separately, we additionally check whether the merged surface form behaves the same way the row-mean does.

The FIMI outcome: full, origin-split, and short

The primary dependent variable is Full FIMI (FIMI), the full news-battery receptivity score available in each FIMI wave. Three planned alternatives — Russian-origin, Chinese-origin, and Short FIMI — are sensitivity and comparability operationalisations around that primary criterion. The same 8 items appear in every wave from S2 onward; in S1 those same 8 items appear inside a longer 15-item battery, just under different names. The figure and the table below treat content identity as the anchor (rather than the raw news# labels), so cells in the same column always refer to the same item content. The mapping is derived from data/item_mapping_mini.csv (rows where scale == "FIMI News Items").

Important caveat on item naming. news1 in S1 is not the same item as news1 in S2. When the battery was trimmed from 15 to 8 items for S2/S3/S4, the surviving items were renumbered. Specifically, S2+ news1–news8 correspond, in order, to S1 news3, news4, news7, news8, news9, news12, news14, news15. The figure and tables below align on this canonical content order, with both naming conventions printed on every x-axis label.

Table 4: Cross-study FIMI item identity. Items 1–8 are administered in every wave; items 9–15 are S1-only.
Canonical ID S1 name S2+ name Origin Content
1 news3 news1 Russian Zelensky bought £20m UK mansion
2 news4 news2 Russian France asked Russia not to touch French military
3 news7 news3 Russian 2026 Milan–Cortina Olympics will be openly LGBT+
4 news8 news4 Russian Ukrainian army wants 'tolerance mentor' at front
5 news9 news5 Russian Kyiv invites gays to join LGBT+ brigades
6 news12 news6 Chinese Li-Meng Yan is a 'known rumour-maker'
7 news14 news7 Chinese CGTN: China–Vietnam friendship
8 news15 news8 Chinese Li Hongzhi: extremist religious leader
9 news1 — Russian Russian hackers vs Finnish Parliament
10 news2 — Russian Ukraine + Poland: joint drone production
11 news5 — Russian Global Times: US the exploiter
12 news6 — Russian Lenin / Kyiv electrification
13 news10 — Russian KillNet vs NATO on gay dating site
14 news11 — Russian Bremerhaven: Islamic family / LGBT+
15 news13 — Russian USA running human experiments on Thai border

The four operationalisations are then defined directly on top of this canonical mapping:

Figure 5: FIMI outcome operationalisations across studies (S5 is excluded because it does not administer FIMI). Facets stack vertically by version — Full at the top, Russian and Chinese in the middle, Short at the bottom. Within each facet, rows = studies and columns = canonical item content; the x-axis labels show the S1 name (top line) and the S2+ name (bottom line) for every item. Items shared across waves line up vertically; the seven S1-only items appear only on the right of the Full FIMI row for S1.
Table 5: FIMI outcome operationalisations: item composition by study and version. Item names are study-specific, but the *content* under each version is identical across studies (cross-reference the canonical-ID table above).
Study Version N items Items (study-specific names, in canonical order)
S1 Full FIMI 15 news3, news4, news7, news8, news9, news12, news14, news15, news1, news2, news5, news6, news10, news11, news13
S1 Russian FIMI 12 news3, news4, news7, news8, news9, news1, news2, news5, news6, news10, news11, news13
S1 Chinese FIMI 3 news12, news14, news15
S1 Short FIMI 4 news4, news9, news14, news15
S2 Full FIMI 8 news1, news2, news3, news4, news5, news6, news7, news8
S2 Russian FIMI 5 news1, news2, news3, news4, news5
S2 Chinese FIMI 3 news6, news7, news8
S2 Short FIMI 4 news2, news5, news7, news8
S3 Full FIMI 8 news1, news2, news3, news4, news5, news6, news7, news8
S3 Russian FIMI 5 news1, news2, news3, news4, news5
S3 Chinese FIMI 3 news6, news7, news8
S3 Short FIMI 4 news2, news5, news7, news8
S4 Full FIMI 8 news1, news2, news3, news4, news5, news6, news7, news8
S4 Russian FIMI 5 news1, news2, news3, news4, news5
S4 Chinese FIMI 3 news6, news7, news8
S4 Short FIMI 4 news2, news5, news7, news8

Three points to take away from the mapping:

  1. Chinese FIMI is 3 items in every wave; Russian FIMI is 12 items in S1 and 5 items in S2+. The five S2+ Russian items (news1, news2, news3, news4, news5) are the same content as five of S1’s twelve Russian items (news3, news4, news7, news8, news9 in S1 naming). The remaining seven S1 Russian items (news1, news2, news5, news6, news10, news11, news13 in S1 naming) were dropped for S2 to give the multinational battery a more balanced Russian/Chinese composition. For cross-study analyses the 5-item shared Russian-origin core is the safest comparison unit; for S1 stand-alone analyses the 12-item Russian-origin scale is the more powerful measure. Chinese FIMI is identical across waves (S1: news12, news14, news15; S2+: news6, news7, news8).
  2. Short FIMI is the source-balanced four-item DV. 4b FIMI DV operationalisation re-ranks all S2+ item subsets using S2/S3/S4 data and an explicit source-coverage rule, selecting news2, news5, news7, news8 as the canonical short criterion.
  3. The seven items S1 fielded that S2+ does not are all Russian-origin. They appear in S1’s Full FIMI and Russian FIMI scores, but cannot be carried into S2+ comparisons because they were not re-administered. The 15-to-8 retention audit (EFA/PCA, subset rankings, item-total diagnostics, and source-composition checks) is documented on 4b FIMI DV operationalisation.

Concretely, 4b FIMI DV operationalisation owns the FIMI criterion derivation and source-specific DV validation; 4c FIMI prediction consumes those definitions for structural prediction (FIMI version × predictor paths, cross-wave path stability, and explained variance).

How to read this report

The site is organised so that each page answers one question with one dataset / model family. A short Where am I in the pipeline? callout opens every analysis page and links it to its inputs and downstream consumers.

Foundations — common to all studies.

  1. Data Import and Preparation — ingestion, cleaning, exclusion logs, and harmonised Parquet / CSV exports for every wave.
  2. Descriptives — sample profiles, item distributions, scale reliabilities, FIMI ←→ predictor scatters, and cross-study comparisons. 2b. Item-pool dimensionality — Stage 1 owner page: KMO / Bartlett / parallel analysis / Velicer’s MAP / oblimin EFA on the S1 broad pool and the S4 deployed predictor pool. Cross-linked from 03a and 03d.

Modelling and validation — pages are ordered by wave, but each one is read for a specific conceptual role in the eight-stage pipeline.

  1. Models — Overview — cross-cutting model reference for Stage 2 first-order measurement, Stage 3 higher-order construct representation, Stage 5 structural prediction, and Stage 6 deployment models. It catalogues registered lavaan specifications with collapsible model cards, the cross-study item-level mapping, and model-complexity comparisons. Estimation results and data-driven path diagrams live on the per-study pages, not here.
  2. Study 1 — Lithuania — Stages 1–5 for the initial item pool: item pool and EFA; block CFAs; first-order structural prediction; reflective, pure formative (<~), and hybrid formative–reflective (MIMIC) higher-order representation checks; two-step factor-score rescue; and S1 FIMI operationalisation including short FIMI selection.
  3. Study 2 — Lithuania — Stages 2–6 for the consolidated instrument: reflective first-order measurement, higher-order representation, harmonised FIMI operationalisation, FIMI origin split, structural prediction, and scale-reduction prototype.
  4. Study 3 — Germany — German replication of the S2 measurement, higher-order representation, FIMI prediction, scale-reduction, and S2 ←→ S3 invariance evidence.
  5. Study 4 — Cross-national — first-pass multinational short-form deployment analysis for Stage 6 combined-item validation, Stage 7 cross-country invariance, Stage 5 FIMI prediction, Stage 8 external-anchor analyses, and the separate COND experiment.
  6. Study 5 — Lithuania — first-pass validation page for Stages 2, 6, 7, and 8. It covers long-form and brief-form measurement validation, external nomological-network validation against preregistered batteries, exploratory S2/S5 invariance, and per-item checks of the combined items deployed in S4; it has no FIMI outcome by design.

Cross-study analyses. Five synthesis pages take the per-study evidence above and answer the cross-wave questions in dedicated venues.

  1. 4a Measurement architecture — side-by-side S1/S2/S3/S5 first-order measurement, loading and reliability stability, higher-order representation comparison (reflective second-order vs hybrid MIMIC vs pure formative <~), construct-evolution history, and the consolidated sem_fit_summary.csv used by the manuscript pipeline. Owns Stages 2 and 3.
  2. 4b FIMI DV operationalisation — S1 15-to-8 FIMI item retention audit, S2/S3/S4 Full/Russian/Chinese DV measurement checks, cross-study FIMI DV invariance, and data-driven short-FIMI derivation. Owns Stage 4.
  3. 4c FIMI prediction — FIMI version × predictor paths across waves, harmonised FIMI prediction across study × version × family, path-stability comparisons, and the S4 COND treatment-effect appendix. Owns Stage 5.
  4. 4d Deployment and invariance — brief-vs-long predictive equivalence, combined-item bridge checks, and the configural/metric/scalar invariance ladders for S2 ←→ S3 (country), S2 ←→ S5 (time), and S4 countries, with S1 ←→ S2 explicitly staged. Owns Stages 6 and 7.
  5. 4e Nomological network — preregistered S5 nomological-network verdict roll-up (Pearson directionality, Fisher-z TOST equivalence, quadratic u-shape) plus live S4 civic, political, and misinformation anchor probes. Owns Stage 8.

Export & metadata.

  1. Data Export — analysis-ready datasets bundled for OSF release.
  2. About — software, packages, and reproducibility information.

You can navigate using the sidebar on the left, the forward / backward buttons at the bottom of each page, or by following the cross-references inside the Where am I? callouts.