Message Boards
- Jay Krall
- Kaylin Linke
Source Vitals
Source Attributes | Description |
|---|---|
Data Collection Method | public Web scraping |
Geographic Coverage | global, 170 languages |
Key Use Cases | product & market research; company reputation analysis; influencer identification |
Delivery Methods | full stream; filtered stream; search API |
Datasets
The Message Boards service offers access to several datasets which are licensed separately:
General Boards, covering more than 400,000 sites
Discuz, a collection of 700 primarily Chinese-language sites using the Tencent Discuz message board framework
Imageboards including 4chan.org and 5ch.net
VerticalScope, a group of 400 enthusiast message boards focused on automotive topics
Data Dictionary
Field | Definition | Data Example |
anchor | If available, a value that points to the location of an individual post within a page. | This field is no longer populated. |
boardname | The name the message board site is known by. | Range Rover Evoque Forums |
categories | The site category | General Talk |
content | The parent element for data associated with the post | content: { |
content/date | The date/time a post was published as it appeared on the message board at the time of harvesting (nothing can be implied about the time zone by the provided date) | ######## |
content/age | If available, the author's age (can be a number or a range like 5-10) | This field is no longer populated. |
content/author | The identifier for the post's author | Spurds |
content/AuthorURL | If available, a link to author's profile | |
content/avatarurl | If available, the URL to the author's avatar | This field is no longer populated. |
content/htmltext | The full text of the post including the HTML tags such as href tags. This is an optional element and must be specified when selecting a data feed configuration. If included, this element will increase the size of the data feed files that Effyis delivers. | Morning. I'm back again with same fault on my 2016 Evoque with 2l Ingenium diesel . Last week it was at dealer as Blocked filter was on for the 2rd time once again a regen and updated software from JLR . But after 400 mls the lights back . Is there a answer to this . Dont get me wrong my Rover dealer is doing his best for me .... do I go for a new DPF ... could this light be on because of another fault . The only fault code is DPF full .... any advice would be helpful please .... |
content/location | If available, the author's geographic location as found on board/posting (i.e. the text string is not validated or normalized in any way and may contain anything the author choose to divulge) | GB |
content/registered | Where available, the date author registered on the message board | This field is no longer populated. |
content/sex | If available, the author's sex | This field is no longer populated. |
content/subject | The post's title (i.e. Thread Title and/or Post Title) | This field is no longer populated. Please use the threadtitle field. |
content/text | The full text of the post | Morning. I'm back again with same fault on my 2016 Evoque with 2l Ingenium diesel . Last week it was at dealer as Blocked filter was on for the 2rd time once again a regen and updated software from JLR . But after 400 mls the lights back . Is there a answer to this . Dont get me wrong my Rover dealer is doing his best for me .... do I go for a new DPF ... could this light be on because of another fault . The only fault code is DPF full .... any advice would be helpful please .... |
countrycode | If available, the 2-letter country code from ISO 3166 that specifies the source country of the message board. | GB |
Crawled | The date/time a post was retrieved by our web harvester. | ######## |
forumid | An ID generated to uniquely identify the Forum. | 5c5932a72 |
forumname | Forum name (from Message Board) | https://www.evoqueownersclub.co.uk/forum/forumdisplay.php?f=49 |
forumurl | Specific Forum home page (URL) | https://www.evoqueownersclub.co.uk/forum/forumdisplay.php?f=49 |
gmt | The GMT offset +/- from the source site, if provided | 0 |
language | The language of the post. | English |
languageCode | The language identifier associated with the post. These identifiers generally match the two-letter language code from ISO 639. If a language does not have a 2-letter code we use the 3-letter code (e.g. Filipino is 'fil'). Text identified as Chinese is output as either 'zh-cn' (Chinese - Simplified) or 'zh-tw' (Chinese – Traditional). The identifier 'unknown' means we are not able to reliably identify the language. See the table below for the full list of supported languages. | en |
MainUrl | The URL associated with the post's source (i.e. Site URL) | |
parentid | The "native" ID of the parent post on sites where our harvesting supports hierarchical threads. |
|
postid | A unique identifier assigned to each post by Socialgist. The value is generated as follows, dependent on the type of site uniqueness check the harvester uses for de-duplication: forumid + “.” + sitepostid (for sites that have sitepostid) The field is a String with max length=64 | 54f626f63bc.291174 |
providerid | An internal identifier used to identify the source of premium content. | 120 |
recommendation | If available, a value generated by other users representing their rating of an author/post. The value is only gathered for a very small number of message boards that target investors (e.g. Investor Village). | This field is no longer populated. |
report | The parent element containing information about a message board post. | report: { |
sentiment | If available, a value (e.g. Strong Sell, Buy) generated by the post's author representing their enthusiasm for a specific security or investment. The value is only gathered for a very small number of message boards that target investors (e.g. Investor Village). | This field is no longer populated. |
signature | If available, the author's signature text, which can include hobbies, URLs, etc… | This field is no longer populated. |
siteid | An ID generated to uniquely identify the Site. | 54f626f63bc |
sitepostid | The "native" ID of the post as assigned by the message board site, where available. In rare cases, this value is synthetically created by our harvester based on data available in the parsed post. The value is normally unique across the entire site, but this is not guaranteed. Uniquess is guaranteed at the forum level. | 291174 |
ThreadID | This identifier represents a unique Thread on a given forum. It can contain alphanumeric characters | 17211 |
threadstarter | A value to indicate if the post started the thread (value is 1) or was in response to another post (value is 0) | 1 |
threadtitle | The title of the Thread. | Dpf warning light |
threadurl | A link to the first page of a Thread. | |
ticker | If available, a value assigned by Socialgist at the site or forum level that indicates an post is likely related to a specific stock market symbol (e.g. MSFT, GOOG). The value is only gathered for a small number of message boards that target investors (e.g. Investor Village, Raging Bull, Yahoo Finance). | This field is no longer populated. |
topics | The site topic | Social |
Url | The URL where post was found (i.e. Thread page where post found). In some cases the URL provided may link to the first page of the Thread regardless of which page the post is found on. | |
Urlwithanchor | The URL where the post was found plus the anchor to locate the individual post on the page. |