General News
Source Vitals
Source Attributes | Description |
---|---|
Data Collection Method | public Web scraping |
Latency | generally under 24 hours; longer for local & niche sites |
Geographic Coverage | 40,000 global news sites; 170 languages |
Key Use Cases | tracking news events and industry trends |
Delivery Methods | Full stream; filtered stream; search API |
Data Dictionary
News field name | Field description | Example data |
article | Parent element that contains article details | article: { |
article/id | A unique article identifier (e.g. 4f3190c023) |
|
article/providerid | An integer used to identify Premium content sources (internal use only) | 3 |
ca_nationalots | National Opportunities to See: Estimated number of visitors per day from the home country of news source | 41976 |
ca_newsrank | Numerical rank among 40,000+ online news sources included in the supplier's Media List | 1626 |
ca_printcirc | Circulation of print edition of news source, if available | 580000 |
ca_reachpermil | Number of visitors to site per million Internet users | 264 |
ca_webrank | Numerical rank among all web sites worldwide | 6558 |
description | Parent element containing actual content of the article | description: { |
description/author | The name of the article's author | The Associated Press |
description/charset | Defines the encoding of the article text. This should always be UTF-8. | UTF-8 |
description/content | The body copy of the article in plain text format. It contains <p> </p> tags to identify the paragraph borders. | News Nation/World\nFour detainees stabbed during altercation at jail in downtown St. Louis\nBy The Associated PressJuly 25, 2024 11:09 am\nST…. |
description/hltext_display | The title of the article | Four detainees stabbed during altercation at jail in downtown St. Louis - Newsday. |
description/language | The detected language of the article's text. See Appendix A for a list of supported languages. | English |
description/language_code | Language identifier based generally on the two-letter codes from ISO 639 | en |
docurl | A URL representing the article's source, generated by extracting the domain from the article URL. | |
harvest_time | The date/time article was harvested (GMT) | 2024-##-25T12:31:05+0000 |
location/country | The country, represented by a value like "Germany", where the article was published or where it's publisher is located; associated with the article's Source | United States |
location/country_code | 2-letter country code identifier based on ISO 3166 | US |
location/zip_code | For sources based in US a five digit zip code or ZIP+4 code, when available. In the future postal codes for non-US sources may be included. | 60103 |
original_link | The article's URL | http://www.newsday.com/news/nation/jail-stabbings-st-louis-y79675 |
source | The name of the media source where the article was found | |
source_category | The type classification of the media source. Examples include: Blog, Consumer Magazine, Forum, Message Board, Newspaper, Online News, Portal, Radio/TV, Trade Publication and Wire/Syndicator Service. | MediaProof_En_usa |
Sample Article
{
"article": {
"id": "498586173",
"providerid": "3",
"description": {
"charset": "UTF-8",
"language": "english",
"language_code": "en",
"hltext_display": "Four detainees stabbed during altercation at jail in downtown St. Louis - Newsday",
"author": "clicking Sign up, you agree to our",
"content": "News Nation/World\nFour detainees stabbed during altercation at jail in downtown St. Louis\nBy The Associated PressJuly 25, 2024 11:09 am\nST. LOUIS — Four detainees at a downtown St. Louis jail were injured when they were stabbed during a fight — the latest of several instances of violence at the jail.\nA guard sent out a distress call about a disturbance around 1:20 p.m. Wednesday at the City Justice Center, spokesman for the St. Louis Department of Public Safety, Monte Chambers, told the St. Louis Post-Dispatch. The injured detainees were taken to a hospital. Their conditions were not immediately made available.\nJail officials are investigating what caused the fight.\nThe jail has struggled to hire enough guards. Officials said earlier this year that fewer than 100 correctional officers were on staff, though the budget calls for 226 positions.\nAdvocates for detainees have long complained about conditions at the jail. It was the site of three uprisings between late 2020 and early 2021.\nGet the Breaking News newsletter!\nGet the latest breaking news as it happens.\n"
},
"original_link": "http://www.newsday.com/news/nation/jail-stabbings-st-louis-y79675",
"docurl": "http://www.newsday.com",
"source": "www.nynewsday.com",
"source_category": "MediaProof_En_usa",
"harvest_time": "2024-07-25T12:31:05+0000",
"ca_webrank": "6558",
"ca_newsrank": "1626",
"location": {
"country": "United States",
"country_code": "US",
"zip_code": ""
},
"ca_reachpermil": "264",
"ca_nationalots": "41976",
"ca_printcirc": "580000"
}
}