Scraping Facebook, Twitter & Instagram, at scale
In the event of finding a mention, they wanted to extract the post content along with the details like post URL, profile username, number of likes, comments, retweets and hashtags used.
Media Social Scraping,
one of the things we do Best
Facebook, Twitter and Instagram were the social media platforms to be monitored.
Since the requirement was brand monitoring, the sites had to be crawled in a frequency of daily. Our team programmed web crawlers to crawl and find instances of the keywords provided by the client and extract the required data points upon finding them.
For Facebook, having access to their API was too complex in the expected timeframe (they required at this time a quite long clearence process), then we go though Scraping.
This particular use case comes under site specific crawl and extraction since the setup is specific to the site to be crawled. The client chose to get the data delivered in JSON format.
The initial setup was complete within 4 days and the data started flowing in.
As per the client’s preference, the data was directly being uploaded to their Dropbox account. We started delivering all records of Facebook/Twitter/Instagram posts with mentions of client’s brand and product names on a daily basis.
Benefits to the Client
The initial setup was completed in just 4 days and there was a steady flow of data thereafter
Monitoring mechanisms were set up in order to spot any changes in the source websites
Large amounts of data was handled effortlessly by our extensive tech stask
The client was able to gain deep customer sentiments from the social media data
The cost of the whole process came out cheaper than an in house crawling setup
Get immediate Quotation with your specific Requirements
Scraped Items / Day
Happy Client
Crawled Pages / Day
Let us know your Needs
Latest Articles
Scrapy Tutorials
Nothing brand new under the sun, just good documented video tutorials if you want to…
Machine Intelligence, which tool for which usage ?
It's starting to be complex to select a Machine Intelligence solution. Not only because of…
Google Launches reCAPTCHA v3
Google has just launched it's latest version of reCAPTCHA, which changes the way it tests…