Videos

Overview

Socialgist supports multiple video sources based on historical customer demand and content popularity. Our data extraction process aims to retrieve all available videos—and comments for select sources—using author subscriptions or query-based filters. While 100% coverage cannot be guaranteed due to provider-imposed limitations, our goal is to deliver the most relevant content with acceptable latency.

Source Vitals

Source Attributes	Description

Source Attributes	Description
Data Collection Method	public Web scraping & public APIs
Geographic Coverage	global; coverage includes YouTube, Dailymotion, Acfun, Vimeo, Rumble and many more.
Key Use Cases	product & market research; company reputation analysis; influencer identification
Delivery Methods	Full stream; filtered stream; search APIs

REST API Data Dictionary

Element or Attribute Name	Description	Included in Response? (based on mode parameter)
Element or Attribute Name	Description	Basic	Full
Id	Unique ID of the Video (or comment)	x	x
Title	The Video's title. For Video comments, the title is the same as the comment's text		x
Text	Video description or the text of the comment	x	x
Text/@truncated	When present, a value of 'true' indicates that the contents in the Text element have been abbreviated.	x	x
TitleHtml	The HTML representation of the Post's title (or Comment's text). This element is only displayed if body=html or both.		x
ThreadId	A unique identifier of the Thread (i.e. a video post and its related comments)	x	x
VideoTitle	Same as Title. Deprecated and included only for backwards compatibility.		x
Published	The date/time the Post was published (GMT)	x	x
Inserted	The date/time the Post was inserted into the BoardReader database (GMT)	x	x
Crawled	The date/time the Post was harvested by our crawler (GMT).		x
AuthorInfo/Id	The unique identifier for the author		x
AuthorInfo/Url	A link to the author's profile		x
AuthorInfo/Name	The author's name		x
AuthorInfo/SiteAuthorId	The video site's unique identifier for the author		x
AuthorInfo/FirstName	The author's first name		x
AuthorInfo/LastName	The author's last name		x
AuthorInfo/ProfilePicture	Link to the author's profile picture		x
Url	The URL where the Video or Comment was found (i.e. permalink page).	x	x
ThumbnailUrl	The URL of the image with Video thumbnail		x
Language	The language of the video or comment text	x	x
Category	Category of the Video as it appeared on the source site		x
Tags	A string containing the terms used by the video author to "tag" the post.		x
Duration	Duration of the video in seconds		x
ExtKey	A reference ID for the site.
IsComment	A Boolean value that specifies if a Post is Comment or not. Values are '0' (false) and '1' (true).		x
SiteInfo/Id	The unique identifier for the Video site where the Post or Comment appeared		x
SiteInfo/Name	The code name of the Video site used by our harvesting system		x
SiteInfo/SiteUrl	The base Url of the Video site.		x
PostSize	The size of the Post in characters	x	x
VideoStats	The VideoStats object contain statistics about the video. This object is only displayed for API Keys that have Premium Video enabled.		X
VideoStats/ <FavouriteCount>	The favorite count at the time the video’s stats were checked		X
VideoStats/ <NumLikes>	The number of likes at the time the video’s stats were checked		X
VideoStats/ <NumDislikes>	The number of dislikes at the time the video’s stats were checked		X
VideoStats/ <CommentCount>	The number of comments as reported by the video site at the time the video’s stats were checked.		X
VideoStats/ <StatsUpdated>	The date/time when the video's stats were last checked. The value is specified in ISO 8601 (YYYY-MM-DDThh:mm:ss.Z) format.		X
CommentsInThread	Obsolete	x	x

Streaming Data Dictionary

Videos field name	Field description	Data example
video	The parent element that contains information about an individual video	"video": {
id	An ID generated by the crawler to uniquely identify each video.	yt.FauOVXgJ0zU
siteid	A unique text value that identifies the video site (e.g. youtube).	youtube
site_videoid	An identifier created by the video site to uniquely identify each video	FauOVXgJ0zU
title	The title of the video.	maaf jelek
description	The description of the video which is not always included.
thumb_url_code	The image of the screenshot of the actual video.	https://i.ytimg.com/vi/FauOVXgJ0zU/default.jpg
tags	Tags that describe the content of the video. Not always included.	[]
category	The category the video was assigned	People and Blogs
published	The date/time the video was published (UTC)	2024-08-13T13:11:01Z
crawled	The date/time the video was harvested (UTC)	2024-08-16T14:30:11Z
duration	The length of the video in seconds	17
videourl	The URL where the video was found.	https://www.youtube.com/watch?v=FauOVXgJ0zU
author	The parent element that contains author information	author: {
author/name	The name of the author who submitted the video	Prinsca gaming
author/site_authorid	An identifier created by the video site to uniquely identify each author	UCwmB2IO7xlu4fmTdH6Ul36A
author/authorurl	The URL to the author's profile page	http://youtube.com/channel/UCwmB2IO7xlu4fmTdH6Ul36A
lang	The language code to identify the language used in comment.	id8
langid	The socialgist internal mapped id for this language	40
comment	The parent element that contains information about an individual comment	comment: {
id	An ID generated by the crawler to uniquely identify each video or comment.	yt.Ugxb8X5GgmyTo0M24FF4AaABAg
videoid	The ID of the Video that the comment is associated with. This is used by BoardReader as the Thread ID.	yt.9IHwqdz8Xhw
siteid	A unique text value that identifies the video site (e.g. youtube).	youtube
site_commentid	An identifier created by the video site to uniquely identify each comment	UgxPGSMDXNWXumpJw8d4AaABAg
title	Some video sites include a title for each comment. Most cases we just add part of the first sentence of the comment text.	অতীত এর ভুল থেকে শিক্ষা নিবেন। এ দেশ হতে আ লিগ শব্দ টা চিরতরে
content	The text of the comment.	অতীত এর ভুল থেকে শিক্ষা নিবেন। এ দেশ হতে আ লিগ শব্দ টা চিরতরে কবর দিতে হবে জনগনের কাছে আইডল হতে হলে পরানতিক পর্যায়ে রিদয়ের ভিতর ডুকতে হলে জিয়াউর রহমানের আদর্শ উপলব্ধি করবেন। নির্বাচন নিয়ে মাথা…..
published	The date/time the video was published (UTC)	2024-08-15T18:56:03Z
crawled	The date/time the video was harvested (UTC)	2024-08-16T13:50:42Z
videourl	The URL where the video was found.	https://www.youtube.com/watch?v=PH-sbPSv3M8
commentsurl	A URL that displays all the comments for the video. For YouTube this is limited to the most recent 1,000 comments.	https://www.youtube.com/watch?v=PH-sbPSv3M8&lc=Ugxb8X5GgmyTo0M24FF4AaABAg
author	The author object element that contains information about the author of the comment	"author": {
author/name	The comment author’s name or video account name	@kingtexzone5051
author/site_authorid	The video site specific id for the author	UCUb5_lzoVTJ67OienqU5FLQ
author/authorurl	The url for the author of the comment. In some cases it can also be the author of the video.	http://www.youtube.com/@kingtexzone5051
author/profile_picture	The author profile avatar or picture.	https://yt3.ggpht.com/ytc/AIdro_kvrMWUD2Ptl9SASYrYT9RvDS24j2RSUw7ieikxndaDiHo=s48-c-k-c0x00ffffff-no-rj
lang	The language code to identify the language used in comment.	bn
langid	The socialgist internal mapped id for this language	8

Example Messages

Streaming Video

{
  "video": {
    "id": "dm.x7vh6r3",
    "siteid": "dailymotion",
    "site_videoid": "x7vh6r3",
    "title": "Barbie cooking toys- Barbie make a cake-",
    "description": "Barbie cooking toys- Barbie make a cake-",
    "thumb_url_code": "https://s1.dmcdn.net/v/SOJTl1VC2czVCsHjr",
    "tags": [
      "Barbie cooking toys- Barbie make a cake-"
    ],
    "category": "kids",
    "published": "2020-08-09T15:25:06Z",
    "crawled": "2020-08-11T12:50:36Z",
    "duration": "229",
    "videourl": "https://www.dailymotion.com/video/x7vh6r3",
    "author": {
      "name": "bookidstoys",
      "site_authorid": "x2a5wbk",
      "authorurl": "https://www.dailymotion.com/bookidstoys"
    },
    "lang": "en",
    "langid": "22"
  }
}

Streaming Comment

{
  "comment": {
    "match_info": {
      "rule": "https://www.youtube.com/channel/UC3IZKseVpdzPSBaWxBxundA",
      "rule_type": "channel",
      "source_rule_id": "UC3IZKseVpdzPSBaWxBxundA",
      "rule_id": 109524,
      "datastream_id": 988
    },
    "id": "yt.UgxPGSMDXNWXumpJw8d4AaABAg",
    "videoid": "yt.9IHwqdz8Xhw",
    "siteid": "youtube",
    "site_commentid": "UgxPGSMDXNWXumpJw8d4AaABAg",
    "title": "Bts you’re my world forever Bts stay Gold 💜💜💜💜💜💜💜",
    "content": "Bts you’re my world forever Bts stay Gold 💜💜💜??💜💜💜",
    "published": "2020-08-04T16:22:27Z",
    "crawled": "2020-08-11T12:50:36Z",
    "videourl": "https://www.youtube.com/watch?v=9IHwqdz8Xhw",
    "commentsurl": "http://www.youtube.com/all_comments?v=9IHwqdz8Xhw",
    "author": {
      "name": "Abdullah Haytham",
      "site_authorid": "UCeqdyLPFlq7sZJcdtBjTIew",
      "authorurl": "http://www.youtube.com/channel/UCeqdyLPFlq7sZJcdtBjTIew",
      "profile_picture": "https://yt3.ggpht.com/a/AATXAJyKDFYyLpJDmLKqmQayHAXgI8p5PMewyR-E9Q=s48-c-k-c0xffffffff-no-rj-mo"
    },
    "lang": "en",
    "langid": "22"
  }
}

REST API Video

{
            "Id": "71330637727320943",
            "Title": "Charli xcx - party 4 u (official video)",
            "Text": "Charli xcx - party 4 u (official video)\nStream: https:\/\/charlixcx.lnk.to\/hifn-5yearsAY\n\nDirector: Mitch Ryan\n \nManagement: Sam Pringle, Twiggy Rowley, Brandon Creed\n \nCreative Director: Imogene Strauss \n\nAtlantic Records \nVP, Creative: Andrew Reid\nEVP, Marketing: Marisa Aron\nCoordinator Creative: Caroline DeFranco\n \nProduction Company: Cadence Films\nEP\/Co-Founder: Lorenzo Ragioneri\nEP\/HOP: Jeff ...",
            "ThreadId": "yt.agu22bqGHto",
            "VideoTitle": "Charli xcx - party 4 u (official video)",
            "Published": "2025-05-15 16:00:04",
            "Inserted": "2025-05-15 18:54:39",
            "Crawled": "2025-05-15 18:54:39",
            "AuthorInfo": {
                "Id": "",
                "SiteAuthorId": "UCOvEMf00DnwdkpOCRWdc6uA",
                "Url": "http:\/\/youtube.com\/channel\/UCOvEMf00DnwdkpOCRWdc6uA",
                "Name": "Charli xcx",
                "FirstName": "",
                "LastName": "",
                "ProfilePicture": ""
            },
            "Url": "https:\/\/www.youtube.com\/watch?v=agu22bqGHto",
            "ThumbnailUrl": "",
            "Language": "English",
            "Tags": "charli xcx,party 4 u,charli xcx - party 4 u,party 4 u - charli xcx,charli xcx party 4 u,party 4 u charli xcx,charli xcx party for you,party for you,party for you charli xcx,charli xcx - party for you,party for you - charli xcx,charli xcx party on you,party on you charli xcx,charli xcx brat,brat charli xcx,guess charli xcx,charli xcx guess,charli xcx lorde,girl so confusing charli xcx,charli xcx girl so confusing,how i'm feeling now charli xcx",
            "Category": "Music",
            "Duration": "297",
            "ExtKey": "yt.agu22bqGHto",
            "IsComment": 0,
            "SiteInfo": {
                "Id": "",
                "Name": "youtube",
                "SiteUrl": "https:\/\/www.youtube.com"
            },
            "PostSize": 2545,
            "VideoStats": {},
            "CommentsInThread": 0
        }