USG Events and Media Salience · Proposed paper method

Measurement validation and event-window analysis

Research question: Which U.S. government events attract attention in foreign media, how is the U.S. evaluated in that coverage, and when do these responses differ across countries and outlets?

The main design: validate the measure, assemble an event catalogue independently of the peaks, and compare the same outlets over three full months before and after each event. Use transparent changes and sensitivity plots as the main results. Interrupted time series is a secondary check where the data support it.

1. A precise, simple main specification

Choice	Primary proposal
Observation	Outlet × publication country × month × topic. The three topics are aid, foreign policy and defense. Publication country, event location and affected country are different fields.
Attention / salience	A = 100 × topic articles / all collected news. Report the change in percentage points. This is attention within the observed feed, not audience exposure. A verified feed with no topic articles has A = 0; an absent feed is missing.
Reported evaluation / tone	T = 100 × (positive − negative) / topic articles. Neutral articles remain in the denominator. Tone ranges from −100 to +100 and is undefined with zero topic articles. A change can range from −200 to +200 points. Quoted evaluation must not automatically be interpreted as the outlet’s editorial position.
Event timing	The anchor is the externally dated policy stage, not the month with the largest observed change. Plot the anchor month separately because it mixes pre-event and post-event days. The primary comparison excludes it.
Main window	Pool article counts within the three full months before the anchor and within months 1–3 after it. Calculate each outlet’s two period ratios, then subtract. Do not average monthly ratios with unequal denominators.
Main estimand	Within each event–country–topic case, average paired outlet changes equally across the fixed eligible set: Δ = mean_outlets(Y_post − Y_pre). This describes the average observed outlet response; article-pooled results answer a different question and are a sensitivity check.
Eligibility	For attention, require verified all-news denominators and continuous feed coverage in both windows, irrespective of topic volume. Retain zero-coverage events. For tone, use outlets with at least one topic article in each pooled period and report that conditional sample separately. Report exclusions and changes in eligible outlets.
Overall summaries	Show every eligible event–country result, including null or negative changes. Summarize countries equally within a shared event and events equally within an event family. Report the number of distinct events, countries and outlets; multiple countries responding to one global event are not independent events.

Compute positive and negative attention as complementary outcomes: positive or negative topic articles per 10,000 collected news articles. These distinguish a shift toward criticism from a fall in neutral coverage. Keep the primary outcome and the contrast fixed before estimating the new catalogue.

2. Use peaks and dips to diagnose measurement

The 48 audited episodes are a purposive diagnostic sample. Read the underlying articles and distinguish a real U.S. event, a mistaken U.S. reference or topic, a mistaken evaluation target, a publication-country error, duplicate content, a feed interruption and sparse coverage. Keep ambiguous judgments as alternative specifications.

Before the next estimation stage, double-code a fresh sample that includes ordinary months, predicted neutral articles and articles excluded by the classifier. A practical starting budget is about 400 articles, stratified by topic and label, with documented sampling probabilities. This is a planning budget, not a power calculation: expand strata with too few gold positives and report wide uncertainty rather than claiming precise language-specific accuracy. Weight overall accuracy estimates back to the sampling frame, and adjudicate disagreements. Human coders should use the article text and codebook before seeing the model label or peak designation.

Code U.S. government relevance, topic, evaluated actor, quoted speaker versus journalist narration, sentiment and publication country separately. Include event linkage where the text supports it. Report precision and recall with their denominators, plus agreement by coding task; the current selectively reviewed cases cannot supply a corpus-wide accuracy rate. Broad topic attention includes unrelated contemporaneous news, so article checks must establish how much coverage actually concerns the dated event.

3. Freeze a bounded event catalogue before looking at responses

Choose a documented source, period and inclusion rule for each event family, then include all qualifying events in that frame, including those without media peaks. The paper can begin with one well-documented family and add others when coverage is complete. The current 17 major episodes remain discovery cases even when their dates are independently verified.

Topic	Feasible event family	Separate stages and context
Aid	Officially documented agreements, suspensions or resumptions within a defined program.	Announcement, legal entry into force, stop-work order, waiver and implementation. An agreement date is not evidence that funds were disbursed.
Foreign policy	Official high-level visits or signed diplomatic agreements in a fixed period.	Announcement, visit and signing; distinguish diplomatic action from election expectations and reactions to third parties.
Defense	Official security agreements or scheduled exercises in a defined series.	Signing, ratification, deployment or exercise start. A report revealing historical conduct is an information event, not a new military intervention.

Each record needs an event ID, official source and date, date certainty, policy stage, affected countries, administration, material exposure, expected media mechanism and competing events. Use administration as descriptive context initially: a Trump–Biden–Obama difference also reflects different events, countries, periods and collection coverage.

Prespecify how to handle overlapping windows. Mark competing shocks in the primary display; repeat the summaries excluding overlaps under a rule defined without consulting the outcome. Late events without a full post-window remain in the catalogue and are marked not yet estimable rather than scored as no response.

4. Country and outlet differences without a large model

Begin with two theoretically motivated moderators measured before the event: material or strategic exposure, and the media environment. Use aid dependence for aid cases and alliance/security ties for defense cases; select one media characteristic with adequate dated coverage, such as government ownership or media freedom. Display event-window changes by these groups before adding interactions.

Retain region, regime, outlet ownership, government ties, language and syndication for description and sensitivity checks. Avoid adding all of them to a small-sample regression. If the catalogue eventually supplies enough independent events, estimate one moderator at a time with event fixed effects for shared events and account for dependence within policy shocks and repeated country/outlet observations. Use formal inference only if the resulting cluster support is adequate. A many-country response to one aid freeze does not provide many independent policy shocks.

5. Uncertainty and sensitivity

Main evidence

Report magnitudes, article and outlet counts, pre-event trajectories, the common outlet set and each outlet’s change. Compare equal-outlet, article-pooled and leave-one-out estimates, duplicate/syndication rules, and three- versus six-month windows.

Use n = 5, 10, 20 and 50 topic-article thresholds as support diagnostics, especially for tone. Do not use a post-event topic-count cutoff to define the primary salience sample.

Statistical tests

The present outlet-resampling bars are sensitivity ranges. Outlets share wire stories and political shocks, and very small outlet counts do not justify a conventional confidence-interval interpretation.

Formal catalogue-level inference requires enough independent events and an explicit dependence structure. With few events, retain event-level estimates and ranges. If testing multiple prespecified outcome/family contrasts, define the test family and adjust for multiple comparisons; do not search for a significant specification.

Secondary interrupted time series already implemented

For supported cases, estimate monthly pooled tone on a linear calendar trend, a post-event indicator and a post-event slope change, excluding the mixed anchor month. The current implementation uses up to 12 months before and after, requires at least nine pre and six post observations and a contiguous observed series, and reports the average deviation in post months 1–3 from the extrapolated pretrend.

Calendar-distance HAC uncertainty uses lag 2, with lag 3 and an annual sine/cosine pair as sensitivity checks. The 11 fitted cases are short, selected series without untreated controls. Their nominal intervals are exploratory and do not resolve selection of peaks or coincident events. A basic ITS is useful only when timing and competing changes are assessed (Lopez Bernal, Cummins and Gasparrini). Few clusters and shared disturbances also constrain conventional clustered inference (Cameron and Miller).

6. What is complete and what the paper still needs

Stage	Status and deliverable
Peak-and-dip audit	Completed diagnostic pass: 48 selected episodes, article context, event interpretations and collection/classification flags. Reviewed material varies in depth and is not a representative validation sample.
Corrected analysis copy	43 clear record decisions; 42 transferable to the frozen aggregate copy and one deferred due to a snapshot mismatch. A confirmed French outlet misassigned to Senegal is removed from the relevant numerators and denominators. Corrections cover audited records, not the entire corpus.
Robustness and exploratory statistics	Completed for 17 major cases; 13 pass the joint n20 and outlet-direction screen, and only one passes n50 after every outlet deletion. Interrupted series fitted for 11 cases. Stability is not statistical significance.
Primary paper event analysis	Proposed. Freeze the event catalogue and codebook; verify outlet-level all-news denominators and feed continuity; complete fresh human validation; lock windows, weights and exclusions; then estimate the main attention and tone contrasts. Current shared-topic outlets are not yet a verified continuous-feed sample.
Interpretive scope	Event-associated media attention and reported evaluation within the sampled outlets. Public opinion, government efficacy, administration effects, UN voting mechanisms and instrumental-variable claims require additional evidence and identification.

Expected paper figures

A validation figure showing true event responses alongside label errors, sparse counts and changing outlet composition.
Event-time plots of attention and tone with counts, common outlets and the official event date shown.
A forest plot of three-month paired changes for every catalogue event, with outlet/window sensitivity and a limited country/media comparison.