General News

General News

Source Vitals

Source Attributes

Description

Source Attributes

Description

Data Collection Method

public Web scraping

Latency

generally under 24 hours; longer for local & niche sites

Geographic Coverage

40,000 global news sites; 170 languages

Key Use Cases

tracking news events and industry trends

Delivery Methods

Full stream; filtered stream; search API

 

Data Dictionary

News field name

Field description

Example data

article

Parent element that contains article details

article: {

article/id

A unique article identifier (e.g. 4f3190c023)

498586173

article/providerid

An integer used to identify Premium content sources (internal use only)

3

ca_nationalots

National Opportunities to See: Estimated number of visitors per day from the home country of news source

41976

ca_newsrank

Numerical rank among 40,000+ online news sources included in the supplier's Media List

1626

ca_printcirc

Circulation of print edition of news source, if available

580000

ca_reachpermil

Number of visitors to site per million Internet users

264

ca_webrank

Numerical rank among all web sites worldwide

6558

description

Parent element containing actual content of the article

description: {

description/author

The name of the article's author

The Associated Press

description/charset

Defines the encoding of the article text. This should always be UTF-8.

UTF-8

description/content

The body copy of the article in plain text format. It contains <p> </p> tags to identify the paragraph borders.

News Nation/World\nFour detainees stabbed during altercation at jail in downtown St. Louis\nBy The Associated PressJuly 25, 2024 11:09 am\nST….

description/hltext_display

The title of the article

Four detainees stabbed during altercation at jail in downtown St. Louis - Newsday.

description/language

The detected language of the article's text. See Appendix A for a list of supported languages.

English

description/language_code

Language identifier based generally on the two-letter codes from ISO 639

en

docurl

A URL representing the article's source, generated by extracting the domain from the article URL.

http://www.newsday.com

harvest_time

The date/time article was harvested (GMT)

2024-##-25T12:31:05+0000

location/country

The country, represented by a value like "Germany", where the article was published or where it's publisher is located; associated with the article's Source

United States

location/country_code

2-letter country code identifier based on ISO 3166

US

location/zip_code

For sources based in US a five digit zip code or ZIP+4 code, when available. In the future postal codes for non-US sources may be included.

60103

original_link

The article's URL

http://www.newsday.com/news/nation/jail-stabbings-st-louis-y79675

source

The name of the media source where the article was found

www.nynewsday.com

source_category

The type classification of the media source. Examples include: Blog, Consumer Magazine, Forum, Message Board, Newspaper, Online News, Portal, Radio/TV, Trade Publication and Wire/Syndicator Service. 

MediaProof_En_usa

 

Sample Article

 

{ "article": { "id": "498586173", "providerid": "3", "description": { "charset": "UTF-8", "language": "english", "language_code": "en", "hltext_display": "Four detainees stabbed during altercation at jail in downtown St. Louis - Newsday", "author": "clicking Sign up, you agree to our", "content": "News Nation/World\nFour detainees stabbed during altercation at jail in downtown St. Louis\nBy The Associated PressJuly 25, 2024 11:09 am\nST. LOUIS — Four detainees at a downtown St. Louis jail were injured when they were stabbed during a fight — the latest of several instances of violence at the jail.\nA guard sent out a distress call about a disturbance around 1:20 p.m. Wednesday at the City Justice Center, spokesman for the St. Louis Department of Public Safety, Monte Chambers, told the St. Louis Post-Dispatch. The injured detainees were taken to a hospital. Their conditions were not immediately made available.\nJail officials are investigating what caused the fight.\nThe jail has struggled to hire enough guards. Officials said earlier this year that fewer than 100 correctional officers were on staff, though the budget calls for 226 positions.\nAdvocates for detainees have long complained about conditions at the jail. It was the site of three uprisings between late 2020 and early 2021.\nGet the Breaking News newsletter!\nGet the latest breaking news as it happens.\n" }, "original_link": "http://www.newsday.com/news/nation/jail-stabbings-st-louis-y79675", "docurl": "http://www.newsday.com", "source": "www.nynewsday.com", "source_category": "MediaProof_En_usa", "harvest_time": "2024-07-25T12:31:05+0000", "ca_webrank": "6558", "ca_newsrank": "1626", "location": { "country": "United States", "country_code": "US", "zip_code": "" }, "ca_reachpermil": "264", "ca_nationalots": "41976", "ca_printcirc": "580000" } }