General News

Source Vitals

Source Attributes	Description

Source Attributes	Description
Data Collection Method	public Web scraping
Latency	generally under 24 hours; longer for local & niche sites
Geographic Coverage	40,000 global news sites; 170 languages
Key Use Cases	tracking news events and industry trends
Delivery Methods	Full stream; filtered stream; search API

Data Dictionary

News field name	Field description	Example data
article	Parent element that contains article details	article: {
article/id	A unique article identifier (e.g. 4f3190c023)	`498586173`
article/providerid	An integer used to identify Premium content sources (internal use only)	3
ca_nationalots	National Opportunities to See: Estimated number of visitors per day from the home country of news source	41976
ca_newsrank	Numerical rank among 40,000+ online news sources included in the supplier's Media List	1626
ca_printcirc	Circulation of print edition of news source, if available	580000
ca_reachpermil	Number of visitors to site per million Internet users	264
ca_webrank	Numerical rank among all web sites worldwide	6558
description	Parent element containing actual content of the article	description: {
description/author	The name of the article's author	The Associated Press
description/charset	Defines the encoding of the article text. This should always be UTF-8.	UTF-8
description/content	The body copy of the article in plain text format. It contains <p> </p> tags to identify the paragraph borders.	News Nation/World\nFour detainees stabbed during altercation at jail in downtown St. Louis\nBy The Associated PressJuly 25, 2024 11:09 am\nST….
description/hltext_display	The title of the article	Four detainees stabbed during altercation at jail in downtown St. Louis - Newsday.
description/language	The detected language of the article's text. See Appendix A for a list of supported languages.	English
description/language_code	Language identifier based generally on the two-letter codes from ISO 639	en
docurl	A URL representing the article's source, generated by extracting the domain from the article URL.	http://www.newsday.com
harvest_time	The date/time article was harvested (GMT)	2024-##-25T12:31:05+0000
location/country	The country, represented by a value like "Germany", where the article was published or where it's publisher is located; associated with the article's Source	United States
location/country_code	2-letter country code identifier based on ISO 3166	US
location/zip_code	For sources based in US a five digit zip code or ZIP+4 code, when available. In the future postal codes for non-US sources may be included.	60103
original_link	The article's URL	http://www.newsday.com/news/nation/jail-stabbings-st-louis-y79675
source	The name of the media source where the article was found	www.nynewsday.com
source_category	The type classification of the media source. Examples include: Blog, Consumer Magazine, Forum, Message Board, Newspaper, Online News, Portal, Radio/TV, Trade Publication and Wire/Syndicator Service.	MediaProof_En_usa

Sample Article

{
    "article": {
        "id": "498586173",
        "providerid": "3",
        "description": {
            "charset": "UTF-8",
            "language": "english",
            "language_code": "en",
            "hltext_display": "Four detainees stabbed during altercation at jail in downtown St. Louis - Newsday",
            "author": "clicking Sign up, you agree to our",
            "content": "News Nation/World\nFour detainees stabbed during altercation at jail in downtown St. Louis\nBy  The Associated PressJuly 25, 2024 11:09 am\nST. LOUIS — Four detainees at a downtown St. Louis jail were injured when they were stabbed during a fight — the latest of several instances of violence at the jail.\nA guard sent out a distress call about a disturbance around 1:20 p.m. Wednesday at the City Justice Center, spokesman for the St. Louis Department of Public Safety, Monte Chambers, told the St. Louis Post-Dispatch. The injured detainees were taken to a hospital. Their conditions were not immediately made available.\nJail officials are investigating what caused the fight.\nThe jail has struggled to hire enough guards. Officials said earlier this year that fewer than 100 correctional officers were on staff, though the budget calls for 226 positions.\nAdvocates for detainees have long complained about conditions at the jail. It was the site of three uprisings between late 2020 and early 2021.\nGet the Breaking News newsletter!\nGet the latest breaking news as it happens.\n"
        },
        "original_link": "http://www.newsday.com/news/nation/jail-stabbings-st-louis-y79675",
        "docurl": "http://www.newsday.com",
        "source": "www.nynewsday.com",
        "source_category": "MediaProof_En_usa",
        "harvest_time": "2024-07-25T12:31:05+0000",
        "ca_webrank": "6558",
        "ca_newsrank": "1626",
        "location": {
            "country": "United States",
            "country_code": "US",
            "zip_code": ""
        },
        "ca_reachpermil": "264",
        "ca_nationalots": "41976",
        "ca_printcirc": "580000"
    }
}