Case Study: Content Generation Automation
As a website that aggregates and produces news, we need to automate how we capture, and publish articles from our partners.
AllAfrica Global Media
Project Requirements
- AllAfrica platform should extract approved content from our media partners.
- AllAfrica platform should save extracted content in its database.
- Allafrica platform should publish saved data from our media partners in the website.
Project Overview
To meet the growing demand for African news, we need to automate the extraction, aggregation, and publishing of content from media organizations to AllAfrica.com, to increase site usage, and deepen user engagement.
The Challenge
Provide access to over 700 stories daily in English and French.
The Approach & Solution
In our analysis, we identified the metadata for each article like link signature, categories, title, date, author, link, and content body. We then used our core technologies to build the "extract-transform-publish" engine which included Linux, Apache, MySQL/PostgreSQL, and Perl as well as extensions such as XML::Comma and Mason. The technology enabled us to scan approved sites, identify news on Africa within those sites, scrap content, save information into the database, and then publish 500+ daily articles.
